Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIVIS-8549] better skipped tests and suites support #113

Merged
merged 13 commits into from
Jan 25, 2024

Conversation

anmarchenko
Copy link
Member

@anmarchenko anmarchenko commented Jan 23, 2024

What does this PR do?
Solves multiple issues around skipped tests and test suites:

  • all frameworks now correctly report test suite as skipped if there are no passed and no failed tests
  • Cucumber now correctly reports undefined scenarios as skipped (if not running in strict mode)
  • Cucumber strict mode support: in strict mode undefined scenarios are marked as failed
  • RSpec: previously pending tests with failures were reported without any status, now they are reported as skipped

Motivation
Wrapping up last bugs and problems before GA for Ruby

Additional Notes
As Ruby test frameworks mostly do not provide statuses of test suites (and do not care much about the concept of a test suite), now we track count of passed/failed/skipped tests per test suite in Datadog::CI::TestSuite object

@codecov-commenter
Copy link

codecov-commenter commented Jan 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (d1034f0) 98.96% compared to head (a637995) 99.04%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #113      +/-   ##
==========================================
+ Coverage   98.96%   99.04%   +0.07%     
==========================================
  Files         151      151              
  Lines        6469     6676     +207     
  Branches      299      296       -3     
==========================================
+ Hits         6402     6612     +210     
+ Misses         67       64       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@anmarchenko anmarchenko marked this pull request as ready for review January 24, 2024 18:05
@anmarchenko anmarchenko requested review from a team as code owners January 24, 2024 18:05
@anmarchenko anmarchenko merged commit 9acd580 into main Jan 25, 2024
26 checks passed
@anmarchenko anmarchenko deleted the anmarchenko/skipped_tests_and_suites branch January 25, 2024 12:38
@github-actions github-actions bot added this to the 0.7.0 milestone Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants