api/results documentation does not reflect current behaviour

[beaufortfrancois]

In https://github.com/web-platform-tests/wpt.fyi/tree/main/api#apiresults, it says

The summary JSON is in the format
{
  "/path/to/test.html": [1, 1],
}
Where the array contains [number of passes, total tests].

while it is actually something like that "/path/to/test.html": {"s": "O", "c": [1, 1]},

It would be good to update documentation

[jugglinmike]

I'd also like to know what "s", "O", and "c" represent.

This was made more confusing by gh-2920 because the documentation now claims that the old format is "v2":

Example JSON (from the summary_v2.json.gz output)

{
  "/css/css-text/i18n/css3-text-line-break-opclns-213.html": [1, 1],
  "/css/css-writing-modes/table-progression-vrl-001.html": [1, 1],
  // ...
}

Have any maintainers recognized this issue?

[jugglinmike]

The new format was introduced in gh-2858. I'm not an authority on this, but here's my interpretation in case it helps other folks in the interim.

"s" is for "status", and "c" is for "counts":

// SummaryResult is the format of the data from summary files generated with the
// newest aggregation method.
type SummaryResult struct {
    // Status represents the 1-2 character abbreviation for the status of the test.
    Status string `json:"s"`
    // Counts represents the subtest counts (passes and total).
    Counts []int `json:"c"`
}

The status abbreviations are:

STATUS_ABBREVIATIONS = {
    "PASS": "P",
    "OK": "O",
    "FAIL": "F",
    "SKIP": "S",
    "ERROR": "E",
    "NOTRUN": "N",
    "CRASH": "C",
    "TIMEOUT": "T",
    "PRECONDITION_FAILED": "PF"
}

[DanielRyanSmith]

Hey everyone. Apologies for the delay in response. @jugglinmike's description above of the changes here is exactly correct. The intent of the change was to separate the overall test status from the fraction of passes and failures to help display more accurate information on wpt.fyi.

Thank you for pointing out the discrepancy in the documentation. 🙂 I'm submitting a change to the docs to reflect this change.