Performance comparison between different server machines

[L2]

In your benchmark results (https://aka.ms/aspnet/benchmarks), there are many machines listed like Intel Linux, AMD Linux, Azure Arm64 Linux, etc. Apart from the cpu, I'm assuming all of these machines have different parts like networking chipsets, drivers, motherboards.

Currently, I believe you are tracking performance within a single machine and not making comparisons between different machines. I'm curious what would be the best way to make cpu performance comparisons between different machines while running ASP.NET if they're using different parts, especially network chipsets? Ie, does server machine A perform better than server machine B on a given ASP.NET benchmark?

Would it be recommended to run the whole benchmark (client and server) on the same machine? That way, the effects of different networking parts could be eliminated, but this also introduces other factors such as resource contention between the client and server.

Thanks

[sebastienros]

These machine are on the same switch. Also when you see INTEL (28) Win/Lin it means the exact same physical hardware is used with Windows and Intel. And in that case the same other machine is used as a load generator or database. In the end the only differences are really when comparing two distinct chipset vendors (AMD vs. INTEL vs. AMPERE ...). But we think that the delta implied by the network cars is minimal on these CPU-bound benchmarks, or less than what their corresponding network drivers and other system settings could incur themselves.

Also we are more interested by the variation of the results overtime with dotnet changes. If you cared more about machine to machine comparison your point would make more sense than in our case.

[L2]

These machine are on the same switch. Also when you see INTEL (28) Win/Lin it means the exact same physical hardware is used with Windows and Intel. And in that case the same other machine is used as a load generator or database.

In your setup, are each of the server, load generator, and database run on separate machines? Or does one machine have a combined function? For example the load generator and database run on the same machine or server and database on the same machine?

In the end the only differences are really when comparing two distinct chipset vendors (AMD vs. INTEL vs. AMPERE ...). But we think that the delta implied by the network cars is minimal on these CPU-bound benchmarks, or less than what their corresponding network drivers and other system settings could incur themselves.

When comparing performance between different machines, what would be a good delta? Would you say that +-10% is normal based on your experience?

Thanks again