You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been keeping an eye on these. Usually the deviation does not go over 50%, but here's a PR in which we had 94% in one message, 16% in the next one #1368.
Change the benchmark to count instruction instead of using time
Instruction count is is a useful metric for a compiler but not for what we're building. We could introduce severe performance degradation without noticeably increasing instruction count (think e.g. replacing a linear-time query string parsing function with a quadratic one).
Since #1230 we run a benchmark using wrk against
origin/main
and any pull request and calculate the deviation.We need to validate if the benchmark is reliable when run by the Github action runner. If it is not reliable we can
The text was updated successfully, but these errors were encountered: