Validate spec_urls based on webref ids by Elchi3 · Pull Request #23958 · mdn/browser-compat-data

Elchi3 · 2024-07-26T10:21:08Z

Draft testing PR for @tidoust :)

Based on w3c/webref#1198 (comment), I wrote a quick test to see if webref ids could be used to (deeply) validate BCD's spec_urls. (that is, we want to check if the fragment ids are valid as well, not just the spec hosts).

It spits out a lot of errors and I would be interested to hear if BCD should be using different fragment ids, or if webref is missing these fragment ids, or if something else is going on. Please see the CI failure for the results.

(This is a draft PR that removes our dependency on web-specs and instead fetches raw webref JSON files, we might not want to fetch the data this way, so consider this PR just a test for now)

Fixes #29065.

tidoust · 2024-07-26T14:15:29Z

It spits out a lot of errors and I would be interested to hear if BCD should be using different fragment ids, or if webref is missing these fragment ids, or if something else is going on. Please see the CI failure for the results.

I'd say that the good news is that, in most cases, it seems that "something else is going on" ;)

Main categories of errors I see:

Fragments in ids extracts are percent-encoded. That does not seem to be the case for URLs used in BCD. If percent-encoding seems wrong, we could perhaps change that (I'm always confused as to when that is needed, or good practice). Otherwise, for comparison purpose, some preprocessing is going to be required, see inline.
The URL used in BCD may have the filename, e.g., index.html in https://webassembly.github.io/spec/js-api/index.html#dom-globaldescriptor-mutable. URLs in the ids extracts don't have the filename (except for multipage specs!). The index.json file in Webref contains a nightly.filename property for each spec that could be used to create the URL variants with the filename if needed. Alternatively, it could perhaps be a good idea to drop that filename in BCD data?
Many URLs in BCD use the series URL, whereas ids extracts in Webref are per specification. For example, you'll have https://drafts.csswg.org/css-logical/#position-properties in BCD, while you'll find https://drafts.csswg.org/css-logical-1/#position-properties in Webref. To find the right level in Webref, you'll need to look at entries in index.json in Webref with the same series.nightlyUrl as the URL (without fragment) used in BCD, then select the entry whose shortname is equal to series.currentSpecification.
The code does not handle the case where a spec does not define any ID. That does happen for some WebGL extensions referenced by BCD, such as WebGL EXT_disjoint_timer_query
For IETF RFCs produced by the HTTP WG, Webref prefers the httpwg.org URL because its rendering is slightly more user-friendly, whereas BCD seems to use www.rfc-editor.org URLs. The latter URL appears as the canonical url in Webref, so it should be relatively easy to find what you want if BCD wants to keep using that origin.

And then there are actual broken links in BCD, such as https://tc39.es/proposal-temporal/#sec-get-temporal.zoneddatetime.prototype.timezone. There are also "outdated" URLs, such as https://tc39.es/ecma262/multipage/additional-ecmascript-features-for-web-browsers.html#sec-object.prototype.__defineGetter__, which redirects to https://tc39.es/ecma262/multipage/fundamental-objects.html#sec-object.prototype.__defineGetter__ that appears in Webref.

There may be a few other error cases to dig into.

Elchi3 · 2024-07-27T08:50:40Z

Fantastique François!! 🎉
Thanks for the very useful review comment! I've updated the script :) Now we're down to just 269 problems found! :)

What I see now:

We should change rfc-editor urls to httpwg urls
Filenames should be omitted
Quite a few legit broken fragment links that need to be fixed in BCD (yay, these are the ones I want to chase with this exercise)

Something I would like for you to take a look:

There are about 22 links to HTML multipage fragments and upon spot checking they work. Maybe these are missing in webref or what am I missing?

tidoust · 2024-07-27T15:45:18Z

There are about 22 links to HTML multipage fragments and upon spot checking they work. Maybe these are missing in webref or what am I missing?

As far as I can tell, all of them are examples of what I called outdated links: they work, but that's because the HTML spec has logic in place to redirect past fragments to their new page. Each time, the content referenced by the link moved to another page of the HTML spec and would better be targeted using the new fragment to avoid a redirect.

For example, clicking on https://html.spec.whatwg.org/multipage/browsing-the-web.html#dom-beforeunloadevent-returnvalue makes you load the browsing-the-web.html page, which includes some JavaScript that detects the fragment, knows it no longer exists in that page, and redirects you to the nav-history-apis.html page where the content was moved. The final URL is https://html.spec.whatwg.org/multipage/nav-history-apis.html#dom-beforeunloadevent-returnvalue. That final URL appears in Webref. Ideally, BCD would always use such final URLs to avoid redirects that consume a bit of time, bandwidth and energy.

github-actions · 2026-06-11T11:06:18Z

This pull request has merge conflicts that must be resolved before it can be merged.

Elchi3 · 2026-06-11T14:15:22Z

30 more to go.

The WebTransport ones are being talked about in WebTransport deprecates out/ingoingHighWaterMark properties #29810
Most CSS ones are valdefs where I'm actually not quite sure why these aren't dfns.
Not sure what changed with CSS random, looks like some things were renamed?
Drafty RFC specs, maybe I'll need to add these to our exception list

Spec URLs - 30 problems (30 errors, 0 warnings, 0 infos):
  ✖ api.WebTransportDatagramDuplexStream.incomingHighWaterMark - Error → Invalid specification URL found: https://w3c.github.io/webtransport/#dom-webtransportdatagramduplexstream-incominghighwatermark
  ✖ api.WebTransportDatagramDuplexStream.outgoingHighWaterMark - Error → Invalid specification URL found: https://w3c.github.io/webtransport/#dom-webtransportdatagramduplexstream-outgoinghighwatermark
  ✖ css.properties.clip-path.path - Error → Invalid specification URL found: https://drafts.csswg.org/css-masking/#ref-for-typedef-basic-shape%E2%91%A0
  ✖ css.properties.font-width.percentage - Error → Invalid specification URL found: https://drafts.csswg.org/css-fonts/#valdef-font-width-percentage-0
  ✖ css.properties.offset-anchor.bottom - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-anchor-position
  ✖ css.properties.offset-anchor.center - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-anchor-position
  ✖ css.properties.offset-anchor.left - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-anchor-position
  ✖ css.properties.offset-anchor.right - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-anchor-position
  ✖ css.properties.offset-anchor.top - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-anchor-position
  ✖ css.properties.offset-path.basic_shape - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-path-basic-shape
  ✖ css.properties.offset-path.url - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-path-url
  ✖ css.properties.offset-position.bottom - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-position-position
  ✖ css.properties.offset-position.center - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-position-position
  ✖ css.properties.offset-position.left - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-position-position
  ✖ css.properties.offset-position.right - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-position-position
  ✖ css.properties.offset-position.top - Error → Invalid specification URL found: https://drafts.csswg.org/motion/#valdef-offset-position-position
  ✖ css.properties.overflow-clip-margin.border-box - Error → Invalid specification URL found: https://drafts.csswg.org/css-overflow/#valdef-overflow-clip-margin-visual-box
  ✖ css.properties.overflow-clip-margin.content-box - Error → Invalid specification URL found: https://drafts.csswg.org/css-overflow/#valdef-overflow-clip-margin-visual-box
  ✖ css.properties.overflow-clip-margin.padding-box - Error → Invalid specification URL found: https://drafts.csswg.org/css-overflow/#valdef-overflow-clip-margin-visual-box
  ✖ css.properties.position-visibility.anchors-valid - Error → Invalid specification URL found: https://drafts.csswg.org/css-anchor-position-1/#valdef-position-visibility-anchors-valid
  ✖ css.properties.position-visibility.anchors-visible - Error → Invalid specification URL found: https://drafts.csswg.org/css-anchor-position-1/#valdef-position-visibility-anchors-visible
  ✖ css.properties.text-fit - Error → Invalid specification URL found: https://drafts.csswg.org/css-text-5/#text-fit-property
  ✖ css.properties.text-fit.grow - Error → Invalid specification URL found: https://drafts.csswg.org/css-text-5/#valdef-text-fit-grow
  ✖ css.properties.text-fit.none - Error → Invalid specification URL found: https://drafts.csswg.org/css-text-5/#valdef-text-fit-none
  ✖ css.properties.text-fit.shrink - Error → Invalid specification URL found: https://drafts.csswg.org/css-text-5/#valdef-text-fit-shrink
  ✖ css.types.random.element-shared - Error → Invalid specification URL found: https://drafts.csswg.org/css-values-5/#valdef-random-element-shared
  ✖ css.types.random.step - Error → Invalid specification URL found: https://drafts.csswg.org/css-values-5/#ref-for-typedef-calc-sum①⑧
  ✖ http.headers.Idempotency-Key - Error → Invalid specification URL found: https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-idempotency-key-header/#name-the-idempotency-key-http-re
  ✖ http.headers.Set-Cookie.SameSite.none_requires_secure - Error → Invalid specification URL found: https://httpwg.org/http-extensions/draft-ietf-httpbis-rfc6265bis.html#section-5.7-3.19
  ✖ http.headers.Unencoded-Digest - Error → Invalid specification URL found: https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-unencoded-digest#name-the-unencoded-digest-field

Elchi3 · 2026-06-15T13:55:34Z

OK, I think we're almost at a stage where this PR will pass tests. Final steps:

Need a review and merge for my fixes to the broken spec_urls in BCD: Fix more spec_urls #29809
Need a resolution and merge to some WebTransport renaming WebTransport deprecates out/ingoingHighWaterMark properties #29810
Need a new web/xref package release that contains https://drafts.csswg.org/css-text-5/.
Need a spec to be evaluated if it is discontinued (and if so, mark the feature as non-standard) draft-ietf-httpapi-idempotency-key-header-07 is expired and archived w3c/browser-specs#2536

github-actions · 2026-06-19T11:06:04Z

This pull request has merge conflicts that must be resolved before it can be merged.

caugner · 2026-07-07T09:58:36Z

Remaining four issues:

Spec URLs - 4 problems (4 errors, 0 warnings, 0 infos):
  ✖ css.properties.overflow-clip-margin.border-box - Error → Invalid specification URL found: https://drafts.csswg.org/css-overflow/#valdef-overflow-clip-margin-visual-box. Check if:
         - there is a more current specification URL
         - the specification is listed in https://github.com/w3c/browser-specs
         - the specification has a "good" standing
         - the fragment id (#) is valid according to @webref/xref.
  ✖ css.types.random.dashed-ident - Error → Invalid specification URL found: https://drafts.csswg.org/css-values-5/#valdef-random-dashed-ident. Check if:
         - there is a more current specification URL
         - the specification is listed in https://github.com/w3c/browser-specs
         - the specification has a "good" standing
         - the fragment id (#) is valid according to @webref/xref.
  ✖ css.types.random.element-shared - Error → Invalid specification URL found: https://drafts.csswg.org/css-values-5/#valdef-random-element-shared. Check if:
         - there is a more current specification URL
         - the specification is listed in https://github.com/w3c/browser-specs
         - the specification has a "good" standing
         - the fragment id (#) is valid according to @webref/xref.
  ✖ css.types.random.step - Error → Invalid specification URL found: [https://drafts.csswg.org/css-values-5/#ref-for-typedef-calc-sum①⑧](https://drafts.csswg.org/css-values-5/#ref-for-typedef-calc-sum%E2%91%A0%E2%91%A7). Check if:
         - there is a more current specification URL
         - the specification is listed in https://github.com/w3c/browser-specs
         - the specification has a "good" standing
         - the fragment id (#) is valid according to @webref/xref.

overflow-clip-margin: border-box seems to have been removed from the spec
<dashed-ident> is only in css-values-4: https://drafts.csswg.org/css-values-4/#typedef-dashed-ident
<element-shared> and <step> seem to have been replaced by less semantic <calc-sum> values

Elchi3 · 2026-07-07T10:47:21Z

For random(), see #29954. The old sub features are now non-standard. The new sub features aren't shipping anywhere yet.

For overflow-clip-margin: border-box, it should use the same link as content-box, padding-box. It comes down to the same type.

Elchi3 · 2026-07-07T10:48:31Z

This is passing for the first time in almost(!) two years 😄 Marking as ready for review.

Export `processData` and give it an optional `deps` object (`lookup`, `exceptions`) defaulting to the real `xref.lookup` and the loaded `specsExceptions`. This lets unit tests inject fakes instead of depending on the bundled `@webref/xref` data. `check` still calls it with the defaults, so behavior is unchanged.

Add `test-spec-urls.test.js` exercising every branch of `processData` with an injected fake `lookup`/`exceptions`: missing/empty `spec_url`, exception-host skips, fragment-less skips, fragment lookup hits/misses, text-fragment handling, and arrays. Includes one `check` test using the real default dependencies. Reaches 100% coverage of the linter.

caugner

LGTM.

PS: I took the liberty of pushing two commits adding full test coverage for the test-spec-urls linter.

Elchi3 · 2026-07-08T10:43:15Z

I'm merging this. Thanks everyone for your help! See https://github.com/mdn/browser-compat-data/blob/main/docs/data-guidelines/spec_links.md for documentation.

Validate spec_urls based on webref ids

f64e360

github-actions Bot added infra Infrastructure issues (npm, GitHub Actions, releases) of this project linter Issues or pull requests regarding the tests / linter of the JSON files. labels Jul 26, 2024

tidoust reviewed Jul 26, 2024

View reviewed changes

Comment thread lint/linter/test-spec-urls.ts Outdated

Address feedback from François

7116139

decode hashes as suggested

b0c0149

Merge branch 'main' into spec-url-validator

480d503

This was referenced Jul 31, 2024

Update Temporal data #24005

Merged

Fix various CSS spec_urls #24008

Merged

Merge branch 'main' into spec-url-validator

7be9f82

Elchi3 mentioned this pull request Sep 16, 2024

RTCPeerConnection.createDTMFSender is non-standard #24442

Merged

tidoust mentioned this pull request Sep 18, 2024

Consistent guidelines spec links, especially in CSS web-platform-dx/web-features#1785

Open

Merge branch 'main' into spec-url-validator

a512cd7

github-actions Bot added the size:m [PR only] 25-100 LoC changed label Dec 10, 2024

Merge branch 'main' into spec-url-validator

fea6ce6

This was referenced Dec 17, 2024

Update spec urls for Login Status API #25446

Merged

Fix spec urls for DedicatedWorkerGlobalScope message events #25447

Merged

GPUSupportedLimits.maxInterStageShaderComponents no longer standard #25448

Merged

Remove MLContext.compute #25449

Merged

Elchi3 mentioned this pull request Jan 24, 2025

AggregateError serialization is standard track #25744

Merged

Elchi3 mentioned this pull request Feb 19, 2025

Allow standard-track sub-features without implementation / intent to implement #25954

Open

github-actions Bot removed the merge conflicts 🚧 This PR needs to merge latest "main" branch to resolve a merge conflict or other issue. label Jun 11, 2026

Merge branch 'main' into spec-url-validator

9aa29d9

Elchi3 force-pushed the spec-url-validator branch from 1d3fc19 to 9aa29d9 Compare June 11, 2026 14:11

Elchi3 mentioned this pull request Jun 15, 2026

Updates for Safari 27 beta #29823

Merged

github-actions Bot added the merge conflicts 🚧 This PR needs to merge latest "main" branch to resolve a merge conflict or other issue. label Jun 19, 2026

Merge branch 'main' into spec-url-validator

8571e9d

github-actions Bot removed the merge conflicts 🚧 This PR needs to merge latest "main" branch to resolve a merge conflict or other issue. label Jul 2, 2026

Merge branch 'main' into spec-url-validator

4c3603a

Fix remaining four issues

1975478

github-actions Bot added data:css Compat data for CSS features. https://developer.mozilla.org/docs/Web/CSS size:l [PR only] 101-1000 LoC changed and removed size:m [PR only] 25-100 LoC changed labels Jul 7, 2026

Elchi3 marked this pull request as ready for review July 7, 2026 10:48

Elchi3 requested review from a team and mdn-bot as code owners July 7, 2026 10:48

Elchi3 requested a review from caugner July 7, 2026 10:48

caugner added 2 commits July 7, 2026 16:20

caugner approved these changes Jul 7, 2026

View reviewed changes

Elchi3 merged commit c45e643 into mdn:main Jul 8, 2026
12 checks passed

Elchi3 deleted the spec-url-validator branch July 8, 2026 10:43

Navigation Menu

Uh oh!

Validate spec_urls based on webref ids - #23958

Validate spec_urls based on webref ids#23958
Elchi3 merged 62 commits into
mdn:mainfrom
Elchi3:spec-url-validator

Elchi3 commented Jul 26, 2024 •

edited by caugner

Loading

Uh oh!

tidoust commented Jul 26, 2024 •

edited

Loading

Uh oh!

Uh oh!

Elchi3 commented Jul 27, 2024 •

edited

Loading

Uh oh!

tidoust commented Jul 27, 2024

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

Elchi3 commented Jun 11, 2026

Uh oh!

Elchi3 commented Jun 15, 2026

Uh oh!

github-actions Bot commented Jun 19, 2026

Uh oh!

caugner commented Jul 7, 2026

Uh oh!

Elchi3 commented Jul 7, 2026

Uh oh!

Elchi3 commented Jul 7, 2026

Uh oh!

caugner left a comment

Uh oh!

Elchi3 commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

Elchi3 commented Jul 26, 2024 • edited by caugner Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tidoust commented Jul 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Elchi3 commented Jul 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tidoust commented Jul 27, 2024

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

Elchi3 commented Jun 11, 2026

Uh oh!

Elchi3 commented Jun 15, 2026

Uh oh!

github-actions Bot commented Jun 19, 2026

Uh oh!

caugner commented Jul 7, 2026

Uh oh!

Elchi3 commented Jul 7, 2026

Uh oh!

Elchi3 commented Jul 7, 2026

Uh oh!

caugner left a comment

Choose a reason for hiding this comment

Uh oh!

Elchi3 commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Elchi3 commented Jul 26, 2024 •

edited by caugner

Loading

tidoust commented Jul 26, 2024 •

edited

Loading

Elchi3 commented Jul 27, 2024 •

edited

Loading