Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SDTEST-878] Optimise LocalRepository.relative_to_root helper to make test impact analysis faster #244

Merged

Conversation

anmarchenko
Copy link
Member

@anmarchenko anmarchenko commented Oct 1, 2024

What does this PR do?
Applies some heuristics to avoid calling Pathname#relative_path_from function when determining what would be path relative to the git root for the given filepath:

  • If given filepath is absolute, we use the fact that absolute paths are pre-filtered already and they always contain root as prefix, so we only need to slice the string to get the suffix (path[prefix_length..])
  • If given filepath is relative, we compute Pathname#relative_path_from once and then we store the difference between the relative filepath and the computed relative path from root in the @prefix_to_root cached variable. We can do it because all relative paths are computed relative to the Dir.pwd which doesn't change in the lifetime of the process. After we have @prefix_to_root cache, we prepend it to every filepath we get (or just return the filepath if the tests are running from the git root).

Motivation
Profiling rubocop test suite with Datadog continuous profiler showed that big part of CPU time is spent in Datadog::CI::Git::LocalRepository.relative_to_root function:
profile-before

How to test the change?
Unit tests are provided.

Benchmark results:

  • Middleman overhead 34,1% (down from 40%)
  • Rubocop overhead 64% (down from 128%) - 2x improvement

@codecov-commenter
Copy link

codecov-commenter commented Oct 1, 2024

Codecov Report

Attention: Patch coverage is 90.62500% with 3 lines in your changes missing coverage. Please review.

Project coverage is 98.86%. Comparing base (7d381d5) to head (777af47).

Files with missing lines Patch % Lines
lib/datadog/ci/git/local_repository.rb 93.33% 1 Missing ⚠️
lib/datadog/ci/transport/http.rb 50.00% 1 Missing ⚠️
...atadog/ci/test_optimisation/coverage/event_spec.rb 93.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #244      +/-   ##
==========================================
- Coverage   98.88%   98.86%   -0.02%     
==========================================
  Files         279      279              
  Lines       13575    13603      +28     
  Branches      625      637      +12     
==========================================
+ Hits        13424    13449      +25     
- Misses        151      154       +3     
Flag Coverage Δ
?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@anmarchenko anmarchenko changed the title first pass on optimisation of relative_to_root helper [SDTEST-878] Optimise LocalRepository.relative_to_root helper to make test impact analysis faster Oct 1, 2024
@anmarchenko anmarchenko marked this pull request as ready for review October 1, 2024 12:25
@anmarchenko anmarchenko requested review from a team as code owners October 1, 2024 12:25
@@ -149,6 +149,10 @@ def code
nil
end

def response_size
0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated change, I noticed that library started to fail if certain backend API is not available because for ErrorResponse there is no http_response

@anmarchenko anmarchenko merged commit c12ad86 into main Oct 1, 2024
30 checks passed
@anmarchenko anmarchenko deleted the anmarchenko/code_coverage_performance_optimisation branch October 1, 2024 14:54
@github-actions github-actions bot added this to the 1.8.0 milestone Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants