diff --git a/CHANGELOG.md b/CHANGELOG.md index 72a8ebf..0a9e956 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.5.0] - 2022-01-21 + +### Added + +- `BLINKA_APPEND` allows multiple test-runs to be appended to the same JSON-file. + ## [0.4.0] - 2021-03-27 ### Changed diff --git a/README.md b/README.md index 56dd463..d8f2ff9 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,5 @@ # Blinka reporter -- [What does this gem do?](#what-does-this-gem-do) -- [How do I install the gem?](#how-do-i-install-the-gem) -- [Which ruby testing frameworks are supported?](#which-ruby-testing-frameworks-are-supported) -- [What is Blinka?](#what-is-blinka) -- [How to send report to Blinka?](#how-to-send-report-to-blinka) -- [How can I report tests in TAP-format?](#how-can-i-report-tests-in-tap-format) - ## What does this gem do? It connects to [supported ruby testing frameworks](#which-ruby-testing-frameworks-are-supported) and outputs a report of all passing, failing and skipped tests into a json-format. This format can be used to report test results using the [ruby client](#how-to-send-report-to-blinka) to [Blinka](#what-is-blinka). @@ -22,7 +15,7 @@ gem install blinka-reporter or add to your Gemfile ```ruby -gem 'blinka-repoter', '~> 0.3.1' +gem 'blinka-repoter', '~> 0.5.0' ``` ## Which ruby testing frameworks are supported? @@ -62,6 +55,30 @@ Add a step to your Github Action Workflow after running tests: `BLINKA_TAG` is optional and can be used to separate different reports, for example when using a build matrix. +## How to make multiple test runs into one report? + +For example when running tests in parallel you might need to run system tests separately. +By using `BLINKA_JSON` first and the `BLINKA_REPORT` with `BLINKA_APPEND` it will keep the results from both runs: + +```yaml +- name: System tests + env: + BLINKA_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }} + BLINKA_JSON: true + BLINKA_REPOSITORY: davidwessman/blinka_reporter + BLINKA_TAG: "" + PARALLEL_WORKERS: 1 + run: bundle exec rails test:system +- name: Tests + env: + BLINKA_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }} + BLINKA_REPORT: true + BLINKA_APPEND: true + BLINKA_REPOSITORY: davidwessman/blinka_reporter + BLINKA_TAG: "" + run: bundle exec rails test +``` + ## How can I report tests in TAP-format? TAP-format ([Test anything protocol](https://testanything.org)) is used to parse tests results on for example Heroku CI. diff --git a/lib/blinka_reporter/version.rb b/lib/blinka_reporter/version.rb index 26062e1..bd0661e 100644 --- a/lib/blinka_reporter/version.rb +++ b/lib/blinka_reporter/version.rb @@ -1,3 +1,3 @@ module BlinkaReporter - VERSION = '0.4.0'.freeze + VERSION = '0.5.0'.freeze end diff --git a/lib/minitest/blinka_plugin.rb b/lib/minitest/blinka_plugin.rb index bf048c8..6ea0bcd 100644 --- a/lib/minitest/blinka_plugin.rb +++ b/lib/minitest/blinka_plugin.rb @@ -11,6 +11,7 @@ def self.plugin_blinka_init(options) def plugin_blinka_options(opts, options); end module BlinkaPlugin + REPORT_PATH = 'blinka_results.json'.freeze TAP_COMMENT_PAD = 8 class Reporter < Minitest::StatisticsReporter attr_accessor :tests @@ -29,7 +30,9 @@ def report super tap_report if ENV['BLINKA_TAP'] - json_report if ENV['BLINKA_JSON'] || ENV['BLINKA_REPORT'] + if ENV['BLINKA_JSON'] || ENV['BLINKA_REPORT'] + json_report(append: !ENV['BLINKA_APPEND'].nil?) + end BlinkaClient.new.report if ENV['BLINKA_REPORT'] rescue BlinkaClient::BlinkaError => error puts(error) @@ -37,7 +40,7 @@ def report private - def json_report + def json_report(append:) result = { total_time: total_time, nbr_tests: count, @@ -48,12 +51,14 @@ def json_report results: tests.map { |test_result| BlinkaMinitest.new(test_result).report } } + result = append_previous(result) if append - File.open('blinka_results.json', 'w+') do |file| + File.open(REPORT_PATH, 'w+') do |file| file.write(JSON.pretty_generate(result)) end + puts - puts('Test results written to `./blinka_results.json`') + puts("Test results written to `#{REPORT_PATH}`") end # Based on https://github.com/kern/minitest-reporters/blob/master/lib/minitest/reporters/progress_reporter.rb @@ -95,6 +100,26 @@ def find_commit ) ) end + + private + + def parse_report + return unless File.exist?(REPORT_PATH) + JSON.parse(File.read(REPORT_PATH)) + end + + def append_previous(result) + previous = parse_report + return if previous.nil? + return if result[:commit] != previous['commit'] + return if result[:tag] != previous['tag'] + + result[:total_time] += previous['total_time'] || 0 + result[:nbr_tests] += previous['nbr_tests'] || 0 + result[:nbr_assertions] += previous['nbr_assertions'] || 0 + result[:results] += previous['results'] || [] + result + end end end end