Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is it standards track? #1181

Open
foolip opened this issue Jan 31, 2024 · 8 comments
Open

Is it standards track? #1181

foolip opened this issue Jan 31, 2024 · 8 comments

Comments

@foolip
Copy link
Member

foolip commented Jan 31, 2024

Feature request from web-platform-dx/web-features#484.

I would like to answer the question "is it standards track" for every feature in web-features, and @tidoust suggested we might be able to improve the status field here in browser-specs to handle that.

The definition we had in the Interop 2024 proposal template was a good start:

Specification for the feature. The feature must defined by a standards-track specification from one of the following organizations:

In web-platform-tests/interop#589 we also concluded that for WebAssembly, Phase 3+ was a reasonable bar.

Zooming out, I think we want to pick a point in the process for each standards organization that crosses the bar of multi-vendor interest. Most processes aren't defined in those terms, but in practice there is roughly such a point.
@foolip foolip mentioned this issue Jan 31, 2024
Open
@tidoust
Copy link
Member

tidoust commented Jan 31, 2024 •
edited
Loading

Thanks for the definition. This made me wonder whether there's enough data in browser-specs already to compute the answer, and whether we would need to override the answer manually in some cases.

On top of status, which currently tries to convey what the spec says its status is, browser-specs has the notion of standing, which is broader than what you're looking at here, and which is used to identify discontinued specs, and to distinguish between specs that are merely unofficial at this stage from specs that are "one step further" (adopted by some group, better-defined scope).

There are 602 specs in browser-specs, of which 572 are in good standing.

Converting the definition to a function that can apply to browser-spec entries, I end up with (note I added AOM and FIDO specs):

function isStandardTrack(spec) {
  if (spec.standing !== 'good') {
    // Spec is in a pending state or has been discontinued
    return false;
  }

  if (spec.organization === 'W3C') {
    if (spec.forkOf && spec.shortname.startsWith('wasm-')) {
      // Note: browser-specs only contains phase 3+ WASM specs
      // Note: a bunch of phase 3+ WASM specs are not in browser-specs yet!
      return true;
    }
    else {
      return spec.release && [
          'First Public Working Draft',
          'Working Draft',
          'Candidate Recommendation Draft',
          'Candidate Recommendation Snapshot',
          'Proposed Recommendation',
          'Recommendation',
          'Proposed Edited Recommendation',
          'Draft Registry',
          'Registry'
        ].includes(spec.release.status);
    }
  }
  else if (spec.organization === 'IETF') {
    return [
      'Proposed Standard',
      'Draft Standard',
      'Internet Standard'
    ].includes(spec.nightly.status);
  }
  else if (spec.organization === 'Khronos Group') {
    // Note: all Khronos Group specs have an "Editor's Draft" status for now.
    return true;
  }
  else if (spec.organization === 'WHATWG') {
    return true;
  }
  else if (spec.organization === 'Alliance for Open Media') {
    // Note: status of AOM specs is either "Draft Deliverable" or
    // "Final Deliverable".
    return true;
  }
  else if (spec.organization === 'Ecma International') {
    // Note: browser-specs only contains stage 3+ TC39 specs
    // Note: all TC39 specs have an "Editor's Draft" status for now
    return true;
  }
  else if (spec.organization === 'FIDO Alliance') {
    // Note: just one spec in browser-specs for now
    return true;
  }
  else {
    return false;
  }
}

Applied to the 572 specs in browser-specs that are in good standing, I end up with 406 standard specs and 166 non-standard specs. See lists below. Do they look correct?

A few entries in the non-standard list are perhaps worth highlighting:

  1. Some MSE and WebCodecs registration specs are on the Note track. That's clumsy but that's correct.
  2. requestVideoFrameCallback ships in all browsers save Firefox, and Mozilla's position about the spec is positive. And yet it is "only" a WICG incubation, so not standard track. [Edit: Same thing for the Compression Streams, which ships in all core browsers. There may be other examples.]
  3. Same thing for the Speech API, whereas synthesis part ships across browsers.
  4. Some specs have been adopted by Working Groups, will get published on the standard track, but only exist as Editor's Draft for the time being. New levels of CSS specs such as css-shapes-2 for instance.
List of 406 standard specs
List of 166 non standard specs

@dontcallmedom
Copy link
Member

a quick note that in my mind, the fact that `rFVC' and the speech API are indeed bug in our incubation process (not bug in the data) - my gut feeling is that we may need a manual override for these situations, but that override shouldn't pretend that it has gone through a proper multi-vendor standardization process either (i.e. it should probably use a different state).

@tidoust
Copy link
Member

tidoust commented Jan 31, 2024

Yes. Looking at the definition and data, I'd say the status key is definitely not the right place to answer the question. I initially thought it could be useful to converge on a more restricted set of values for that key that wouldn't depend on the organization, but it seems good to have a key that tracks the advertised status of the spec (and it's mostly orthogonal to the question raised here).

We could perhaps rather reuse the roughly defined standing key to avoid introducing another roughly defined key for a similar concept. I'm thinking that the "good" value of the standing key could perhaps be split into "good" and "standard-track" in particular.

That may not work for your "shouldn't pretend" clause for cases when we need to override the value. At a minimum, we may want to come up with another name. All names that come to mind so far such as "widely reviewed", "wide support", "consensual", "stable" have overloaded meaning, but I'm sure we'll find something ;)

tidoust added a commit that referenced this issue Jan 31, 2024
@tidoust
Look for new WASM proposal specs at phase 3+
c767e0b 
This updates the find-specs script to also look for new WASM proposals in:
https://github.com/WebAssembly/proposals/blob/main/README.md

Only phase 3+ proposals are being considered, in accordance with:
#1181 (comment)
... which itself derives from:
web-platform-tests/interop#589 (comment)

The script now filters out specs that cannot be fetched (some WASM proposals
at phase 3+ do not have a proper spec).

Fix #529.
@tidoust tidoust mentioned this issue Jan 31, 2024
Merged
tidoust added a commit that referenced this issue Feb 1, 2024
@tidoust
Look for new WASM proposal specs at phase 3+ (#1182)
32e0ddb 
This updates the find-specs script to also look for new WASM proposals in:
https://github.com/WebAssembly/proposals/blob/main/README.md

Only phase 3+ proposals are being considered, in accordance with:
#1181 (comment)
... which itself derives from:
web-platform-tests/interop#589 (comment)

The script now filters out specs that cannot be fetched (some WASM proposals
at phase 3+ do not have a proper spec).

Fix #529.
@tidoust
Copy link
Member

tidoust commented Feb 2, 2024

@Elchi3 Would you be able to shed some light on the standard_track and experimental flags in BCD by any chance? Is there a more precise definition than "part of an active specification or specification process" for standard_track for example?

We're considering implementing the definition at the top of this issue, and wondering whether we might need more nuances, perhaps taking ongoing discussions on specs and possible standard positions into account.

Taking one concrete example, the definition would mean that Background Fetch would be listed as "not standard", since it is a WICG incubation for now. I see that BCD has both flags set to true for interfaces of that spec, e.g., BackgroundFetchManager. Is there a rationale that we could perhaps try to reproduce in browser-specs so that BCD could get that information from there when possible?

Perhaps something like "has a spec, implemented in one browser engine, and no known negative standard positions"?

@tidoust
Copy link
Member

tidoust commented Feb 2, 2024

Perhaps something like "has a spec, implemented in one browser engine, and no known negative standard positions"?

Well, I note that wouldn't work well for something like Web Bluetooth, which also has both flags set in BCD, and that Mozilla and Webkit flag with a negative position.

@foolip
Copy link
Member Author

foolip commented Feb 3, 2024

Thanks for taking a look at this @tidoust and @dontcallmedom! I quite like the idea of using standing for something like this, and perhaps more than two states make sense.

Thanks for writing that code @tidoust, I'll look over the resulting categorization when I'm back at work.

@Elchi3
Copy link

Elchi3 commented Feb 6, 2024

@Elchi3 Would you be able to shed some light on the standard_track and experimental flags in BCD by any chance? Is there a more precise definition than "part of an active specification or specification process" for standard_track for example?

Sure! This touches on the oldest issue still unresolved today in the BCD repo :) mdn/browser-compat-data#1531. Basically, BCD sets standards_track to true if we have a spec_url and BCD would actually like to remove the standards_track property because it doesn't have any additional meaning.

Given BCD's standards_track is mostly based on the fact if we can provide a spec_url property, BCD's rules for valid spec URLs are probably interesting to you as well:

  • spec_url hosts must be in the web-spec package or in our exception list
  • spec_url hosts must have a standing of "good"

See the BCD linter file: https://github.com/mdn/browser-compat-data/blob/main/test/linter/test-spec-urls.ts

Taking one concrete example, the definition would mean that Background Fetch would be listed as "not standard", since it is a WICG incubation for now. I see that BCD has both flags set to true for interfaces of that spec, e.g., BackgroundFetchManager. Is there a rationale that we could perhaps try to reproduce in browser-specs so that BCD could get that information from there when possible?

We discussed changing standards_track from boolean yes/no to a much more nuanced property that could maybe indicate where a spec stands in terms of standardization. For example an enum with the values "incubation", "w3c" etc. but some people were not a fan of this, if I remember correctly. See mdn/browser-compat-data#6103 for the whole discussion. In that issue I also thought if it would be good to derive standards track status from the URL where the spec ist hosted and use that as some sort of indication how to categorize a spec. See my analysis in mdn/browser-compat-data#6103 (comment).

I think in the long-term I would love for BCD to move forward with the removal plans of the standards_track property and let web-specs do any specification classifications.

Standards position data might be related to this classification, too. But here again, I think I would love to see web-specs record that data, if possible. IMO standards positions have more to do with specification data than with compat data (even though the positions itself impact compat data).

@tidoust
Copy link
Member

tidoust commented Feb 6, 2024

Thanks, @Elchi3! So standard_track: true in BCD is essentially the equivalent of standing: "good" in browser-specs, that's good!

What about experimental? I get it that the flag may be set at a fine-grained feature level, e.g., when a feature is added to an otherwise stable spec. I'm more wondering about guidelines for setting the flag at the spec level.

Do you look into standards positions? Do you check whether the spec is at an incubation or standardization phase (a typical example in W3C would be a WICG spec vs. a Working Group spec)?

@tidoust tidoust mentioned this issue Mar 20, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dontcallmedom @Elchi3 @foolip @tidoust