Skip to content

docs: gate the new feature docs pipeline on worthiness and trigger it from releases - #586

Merged
rachaelrenk merged 15 commits into
mainfrom
rrenk/release-docs-gate
Aug 24, 2026
Merged

docs: gate the new feature docs pipeline on worthiness and trigger it from releases#586
rachaelrenk merged 15 commits into
mainfrom
rrenk/release-docs-gate

Conversation

@rachaelrenk

@rachaelrenk rachaelrenk commented Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

The new feature docs pipeline treated detection as permission to draft. Any merged spec, and later any changelog entry, became a docs PR. warpdotdev/docs accumulated unvetted pages faster than anyone could review them, including pages for features that had not shipped or shouldn't require docs.

Three problems, three fixes:

  1. No worthiness gate. Nothing asked whether a change warranted docs. Now everything runs through an ordered gate with a default of "no docs."
  2. Wrong trigger. Spec merge happens before a feature ships. The trigger is now a new stable release.
  3. No content design step. Drafting began at the template, so pages were shaped by the scaffold rather than the reader's need. A design plan now precedes drafting, sized to the change.

How the new feature docs pipeline works

In plain language. The pipeline checks daily and wakes up when a release ships, then answers two questions: what shipped that needs documenting, and what should that documentation actually say.

Once per release, an agent:

  1. Notices a release shipped. It checks daily but only works when a new stable version appears, so release timing does not matter.
  2. Looks at what went out. The changelog, plus the code surfaces behind it — settings, CLI commands, API routes, slash commands.
  3. Decides what actually deserves docs. This part is new. Most changes do not actually need docs.
  4. Plans the page before writing it. This is also new. For anything beyond a correction, agents must define who it is for, what problem it solves, what it should not cover — reading the product spec where one exists.
  5. Drafts only what passed, and opens one focused PR per feature with a named reviewer.
  6. Writes down every decision, including the "no" ones.

How it decides whether something needs docs. In order, and the default answer is no:

  • Has it shipped? Not GA, not public, or still behind a flag means we wait. This is a hard stop.
  • Can a user configure it? A setting, a flag, an API field. The agent has to name the specific one.
  • Can a user get stuck? Fiddly setup, or an error message whose fix is not obvious.
  • Would a user be surprised? A changed default, permission, billing consequence, or behavior.
  • Could a user never find it on their own? Something genuinely new that has no setting, no menu item, and nothing in the product that hints it exists.

Some things are ruled out regardless: pure UI changes, anything obvious at a glance, internal work with no user-visible effect, and bug fixes that just make the product match what the docs already said.

The agent also has to point at specific evidence — an actual setting name, an actual error string. That single rule is what stops it talking itself into a page.

What it does when something passes. Whether the page should exist is already settled — most candidates died at the gate. The only question left is where the content goes, and updating always wins:

  • Update a page that already exists — the default
  • Create a new page — only when nothing covers the surface, and justified against the existing structure rather than just against the change

New-page sprawl is the main way an automated pipeline degrades a docs set: each page is defensible on its own, and collectively they fragment a topic across places no reader will assemble. Anything that passed only on the last question above is capped at an update — a capability nobody can stumble onto needs a findable sentence, not a page of its own.

What it deliberately does not do:

  • Touch major launches. Those are planned by the docs team; this handles the weekly release stream.
  • Write docs for anything unreleased.
  • Second-guess a person who asks for a page. When a human requests docs, the agent only checks that the feature has shipped.

Changes

.agents/references/docs-worthiness-criteria.md (new)

The gate every docs agent reads before proposing or drafting anything.

  • Gate 0 — shipped and public. A hard prerequisite that defers rather than rejects. Reuses the existing public/private and rollout rules in missing_docs rather than restating them.
  • Gates 1–3 — configurable / can a user get stuck / would a user be surprised. Alternatives; passing one is enough.
  • Gate 4 — a capability the reader cannot discover. Added after review. Gates 1–3 are all friction-based: they assume a reader already trying to do something. None fire for a reader who does not know the capability exists, which is a real gap for a pipeline aimed at new features.
  • Disqualifiers override any pass: pure UI affordance, small and intuitive, no user-observable change, bug fix restoring intended behavior.

The load-bearing rule is the justification requirement: every verdict must name the gate and concrete evidence — a setting key, CLI flag, quoted error string, changed default, or API field. Gates can be argued into; naming a real toml_path key cannot.

Gate 4 is the one most easily talked into, so it takes two checkable facts instead of an impression: the job that was impossible before (not merely easier), and the in-product surfaces checked and found silent. "This is new and users should know about it" is exactly the sentence it replaces, and still fails on its own.

It also forces three outcomes rather than two — new page, update an existing page, or no docs — with updating preferred. A Gate 4-only pass is capped at an update, never a new page: anything that truly warrants a page also passes Gate 1, 2, or 3. That cap is what keeps a discoverability gate from reopening new-page sprawl.

.agents/references/content-design-plan.md (new)

Audience and JTBD, problem, goals, purpose and value, content type, skills and templates, and high-impact scenarios with explicit exclusions. Grounded in GitHub's content design principles.

Produced only after a gate pass, so the agent never designs a doc that should not exist. It lives in the PR body so a reviewer can disagree with who the page is for — the disagreement worth having, and the expensive one to resolve after the prose is written.

Sized to the change, after review feedback that it looked heavy for small updates. The plan promised "four sentences" while presenting seven fields. Routing is now mechanical and comes before the fields:

  • New page → full form, every field
  • Update that adds a concept → a three-line short form, with a filled example in the reference
  • Correction → no plan at all

An update inherits the page's content type, opens no template, and needs no page justification, so Content type, Skills and templates, and Purpose and value drop out. The short form is deliberately not a template file — templating three lines reintroduces exactly the weight being removed. The skip list also now names docs bug fixes, factual corrections, adding a missing entry to an existing reference table, and redirects.

Release trigger

  • scripts/check_new_release.py — compares current stable from app.warp.dev/client_version against a committed state file. A daily schedule therefore does per-release work, absorbing flexible release timing without guessing at a cron.
  • Check and commit are separate steps. A run that crashes mid-triage retries the same release instead of skipping it.
  • A fetch failure exits 1, never 10. It must not be mistaken for "nothing shipped."
  • test_check_new_release.py — 17 stdlib-only tests, network stubbed, covering the exit-code contract and the full check → commit → no-op → next-release cycle.

references/changelog_decisions.md (new)

Records every verdict, including rejections. feature_surface_map.md keys on flags, commands, routes, and settings — a changelog item has no key there, so without this ledger every rejection would be re-proposed next run and re-rejected by the same reviewer. This is what makes "no" stick.

missing_docs

Gate wired into Phase 2 triage, the drift-watch loop, and Phase 3 drafting. "No docs needed" and "deferred (Gate 0)" are now first-class recordable resolutions rather than silent skips. Adds the release gate as drift-watch step 1, and documents the three sources beyond the client changelog (oz_updates, the public Agent API, then warp-server specs behind a hard rollout check — the layer that caused most of the original noise, so the most conservative).

draft_docs and create_pr

The gate only works if every path into the repo runs it. draft_docs gets step 3.5; create_pr requires a ## Content design plan section on any PR that adds a page or substantially updates one. Humans need to answer the same gate questions.

Product specs as a drafting input

Moving the trigger to the release kept the release-driven half of write-feature-docs and dropped the reason it read specs at all. Phase 3 research was code-only, and draft_docs reviewed a spec only when a requester handed one over — which a scheduled run never does.

Code answers what a surface does, never who needed it or what problem it solves. Those are the content design plan's first three fields, so they were being inferred from a one-line changelog bullet. warp-server specs/<id>/PRODUCT.md answers them almost directly: its Problem, Goals, Non-goals, and User experience sections map onto the plan's Problem, Goals, Excludes, and high-impact scenarios.

Bounded so this does not recreate the old spec-triggered pipeline: framing only, never behavior (labels and flags are still verified against code), never evidence a feature shipped (that is Gate 0), and never quoted into a public page (warp-server is private). Only some specs have a PRODUCT.md, so its absence is recorded in the plan rather than papered over.

Content type alignment with established industry content models

Sequenced after the gate. Warp's conceptual, procedural, reference, troubleshooting, quickstart, and combined types already agreed, as did the content-order rule. Four real gaps:

  • FAQ keeps its type but gains admission rules and defaults to "not an FAQ." The same test applies to adding a question to an existing FAQ page, since these grow by accretion.
  • Guide split into quickstart and tutorial. "Guides" stays the section name and holds both; the split is by scope. Quickstart is ~5 minutes / ~600 words for someone who already knows the product. A tutorial walks a full workflow, carries its own troubleshooting and a conclusion, and requires that a quickstart already exist.
  • Combined type guardrail — no quickstart or tutorial content folded in. It is the most common type at ~75+ pages and had no limit.
  • Per-type title conventions — conceptual takes "About [subject]" or a noun, procedural a gerund, reference a noun without stacked modifiers.

Incidental fixes

Found while editing, worth knowing independently of this PR:

  • AGENTS.md and five per-type drafting skills pointed at .warp/templates/ and .warp/skills/, which do not exist. The real location is .agents/. Every agent following those pointers to find a template hit nothing.
  • AGENTS.md referenced src/content/docs/university/, which is now guides/.

Validation

  • 17/17 release-gate tests pass.
  • Live dry-run of the full cycle against current stable: detects → commits → no-ops with exit 10.
  • Every .agents/ path referenced across the new and edited files resolves to a real file (22/22).
  • Gate replayed against the two open recurrence cases, both recorded in the ledger:

No src/content/docs/ pages change, so there is nothing for npm run build to validate here.

Notes for reviewers

The pipeline starts quiet by design. last_release_processed.json is committed recording the current stable, so the first scheduled run no-ops until the next release rather than firing on a 32-entry backlog unattended. Delete the file to force a run. This follows the repo's existing "start with a manual run" convention in skill-authoring-guidelines.md.

This PR is inert until a schedule is created. There is currently no scheduled missing_docs agent — the old gitbook-based docs schedules were deleted, so nothing is running the ungated skill today. Creating the daily schedule against this repo is a separate external change, and it has to happen after this merges: skill_spec resolves from the default branch at runtime.

Companion PR: warpdotdev/common-skills retires scan-new-specs and removes ambient mode from write-feature-docs. Merge this PR first — that one references the two new reference files by path.

Unverified claims

None — no page content changes. Skill and reference behavior was verified by running the scripts and tests, and by resolving every referenced path against the working tree.

Co-Authored-By: Warp agent@warp.dev

rachaelrenk and others added 4 commits August 19, 2026 18:22
@rachaelrenk
docs: gate the new feature docs pipeline on documentation-worthiness
1305f5f 
Not every shipped change warrants documentation, and nothing in the pipeline
asked the question. Spec-merge and changelog detection were both treated as
permission to draft, so the docs repo accumulated unvetted PRs -- including one
that documented an unreleased feature and said so in its own title.

Add two shared references and wire them into missing_docs:

- docs-worthiness-criteria.md: an ordered gate with a default of "no docs".
  Gate 0 (shipped and public) is a hard prerequisite; Gates 1-3 (configurable,
  can a user get stuck, would a user be surprised) are alternatives. Every
  verdict needs concrete evidence -- a setting key, CLI flag, quoted error
  string, changed default, or API field. Restating the changelog is not
  evidence. Three outcomes, with "update an existing page" preferred over a new
  one to counter new-page sprawl.

- content-design-plan.md: audience and JTBD, problem, goals, purpose and value,
  content type, skills and templates, and high-impact scenarios with explicit
  exclusions  exclusions  exclusions  exclusions  exclusions  exclusions  exclusions reasoning is reviewable next to the diff. Grounded in GitHub's content
  design principles.

Also add the release trigger. check_new_release.py compares currentAlso add the release trigger. check_new_release.py compares curren file, so a daily schedule
does per-release work and absorbs fldoes per-release work and absorbs fldoes per-release work and absorbs fldoshes mid-triage retries the same release instead
of skipping it. A fetch failure of skipping it. A fetch failure of skipping it. A fetch failure of skippngelog_decisions.md records every verdict including rejections, which is what
of skipping it. A fetch failure of skipping it. A fetch failure oflined.
feature_surface_map.md has no key for a changelog item, so without this ledger
feature_surface_map.md has no kgatefeature_surface_map.md has no kgatefeature_surface_map.md has no kgatefeature_mit-msg-gate.txt --quiet && git --no-pager log --oneline -1 && git --no-pager diff --stat HEAD~1 HEAD
@rachaelrenk @warp-agent
docs: require the gate and content design plan across drafting skills
a248b61 
The gate only works if every path into the docs repo runs it. Wiring it into
missing_docs alone would leave draft_docs -- the skill a person invokes directly
-- able to produce a page nobody gated.

draft_docs: add step 3.5, which runs the worthiness gate and then the content
design plan before any template is opened. Step 4 now records the content type
the plan chose rather than choosing it, because starting at the template is what
produces pages shaped by the template instead of by the reader's need. Note
explicitly that a human asking for a page is not itself evidence the page should
exist.

create_pr: add a required "Content design plan" section to the PR description
guidelines, alongside the existing "Unverified claims" section, with a filled-in
example. Keeping the plan next to the diff lets a reviewer disagree with who the
page is for -- the disagreement worth having, and the expensive one to resolve
after the prose is written.

missing_docs: document the three sources beyond the client changelog. The
client changelog only covers warpdotdev/warp, so server and platform features
never appear in it -- which is exactly why they used to reach docs through the
sssssssssssssssssssssssssssssssssssssssn the release-gate payload), then the
public Agent API via sync-openapi-spec, and treat warp-server specs as a last
resort behind a hard rollout check. A merged spec is not a shipped feature.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk @warp-agent
docs: align content types with GitHub's content model
f86a35b 
Compared Warp's eight content types against GitHub's content model. Conceptual,
procedural, reference, troubleshooting, quickstart, and combined already agree,
as does the content-order rule. Four gaps were real:

FAQ now has admission rules and defaults to "not an FAQ". GitHub has no FAQ type
because an FAQ page pulls answers away from the page that owns the topic -- the
reader on the owning page does not find the answer, the reader on the FAQ gets
one without context, and the two drift apart. Keeping the type, but all three
admission rules must hold, and the same test applies to adding a question to an
existing FAQ page, since these grow by accretion.

Guide is split into quickstart and tutorial. "Guides" stays as the section name
and holds both; the split is by scope. A quickstart is ~5 minutes and ~600 words
for someone who already knows the product. A tutorial walks a full workflow,
includes its own troubleshooting and a conclusion, and requires that a quickstart
already exist -- otherwise it absorbs setup content that belongs in a shorter
page. The one-word "Guide" gave authors no scope guidance at all.

The combined type gets a guardrail: no quickstart or tutThe combined type gets a guardrail: no quickstart oe budget and a single continuous path, and both lose
their purpose once embedded. Cotheir purpose once embedded. Cotheir purpose once embedded. Cotheir purpose ongetheir purpose once embedded. Cotheir purpose once embedded. Cotheir purposes "About
[subject]" or a noun, procedural a gerund, reference a noun without stack[subject]" or a noun, procedural a gerund, reference a noun without stack[se per[subject]" or a noun, procedural a gerund, refes/ a[subject]" or a noun, procedural a gerund, reference a noun without stack[sery a[subject]" or a noun, procedural a gerund, reference a noun without stack[sntent/docs/university/, which is now guides/.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk @warp-agent
docs: record the gate replay against the two recurrence cases
2f6e719 
Validation, not new policy. Replayed the worthiness gate against the two open
docs PRs that showed the gap was still live, and recorded both verdicts.

docs#581 fails Gate 0: it documents agentDefaults.computerUseModel and says
"(unreleased feature)" in its own title. Deferred with the blocking condition.

docs#582 passes on Gate 3. Factory Dashboard metric counting rules are
non-obvious and live only in hover tooltips -- the By-model view caps at eight
and folds the rest into Other, there is an Unknown bucket, and opened/merged PR
counts can legitimately disagree. A reader interpreting the dashboard without
them draws wrong conclusions, and no existing page carries per-metric detail.

Keeping a pass in the regression set is the point. A gate that rejects
everything is as broken as one that accepts everything, and only having
rejections as worked examples would teach the wrong bias.

Co-Authored-By: Warp <agent@warp.dev>
@cla-bot cla-bot Bot added the cla-signed label Aug 20, 2026
@vercel

vercel Bot commented Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 24, 2026 5:56pm

Request Review

@warp-for-oss

warp-for-oss Bot commented Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

@rachaelrenk

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@rachaelrenk rachaelrenk mentioned this pull request Aug 20, 2026
Open
warp-for-oss[bot]
warp-for-oss Bot reviewed Aug 20, 2026
View reviewed changes

@warp-for-oss warp-for-oss Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR adds documentation-worthiness gates, a content design plan reference, release-triggered missing_docs guidance, and a new release-gate script with tests. The overall direction is coherent, but one newly added scheduled-agent instruction points at a script path that does not exist from the repo root, so the recurrence would fail at the first step.

Concerns

  • The recommended scheduled-agent prompt uses scripts/check_new_release.py, while the PR adds the executable at .agents/skills/missing_docs/scripts/check_new_release.py; update the prompt/reference path before merge.
  • Security: no security issues found in the changed guidance or release-gate script.
  • Spec alignment: spec_context.md contains no approved or repository spec context, so no spec drift was identified.

Verdict

Found: 0 critical, 1 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

Comment thread .agents/skills/missing_docs/SKILL.md Outdated
> and surface_snapshot.json changes; assign each PR's resolved owner as reviewer, and list
> any findings you deferred in the relevant PR body.
> Run the missing_docs skill in drift-watch mode. First run
> scripts/check_new_release.py; if it reports no new stable release, record the no-op

@warp-for-oss warp-for-oss Bot Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] This path does not match the added script under .agents/skills/missing_docs/scripts/check_new_release.py, so a scheduled agent following the prompt will fail before the release gate runs.

Suggested change
> scripts/check_new_release.py; if it reports no new stable release, record the no-op
> .agents/skills/missing_docs/scripts/check_new_release.py; if it reports no new stable release, record the no-op

@rachaelrenk rachaelrenk Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, fixed in 93e8d89.

Confirmed the path does not resolve from the repo root, and this matters specifically because the prompt block is copied verbatim into a scheduled agent config with no surrounding context. The failure would also have been misleading: the agent would die before the release gate ran, so it would look like a broken gate rather than a bad path.

I also fixed scripts/suggest_reviewers.py two lines below, which had the identical problem but sat outside the diff so it was not visible to you. Patching only the flagged line would have left the same bug in the same block.

Left the other relative script paths alone deliberately — inside the skill body the reader already has the directory context, and the References section lists paths relative to the skill directory by convention. Only the copy-paste prompt needs to be self-contained.

@rachaelrenk rachaelrenk self-assigned this Aug 20, 2026
@rachaelrenk
docs: scope the worthiness gate by who is asking
f984c70 
The gate as written applied in full to human-invoked drafting, and told the
agent to decline a person's request and propose an alternative. That went
further than intended and further than the problem justified.

The failure mode this work exists to fix is automation flooding the repo.
Human-invoked drafting was never it. And the evidence bar -- name a toml_path
key, quote an error string -- is calibrated for an agent triaging a changelog
with no context. A docs writer who has been in the planning meeting will
routinely fail that bar while being right, which turns the gate into an agent
arguing with someone who knows more than it does.

Split the gate by who is asking:

- Automated runs apply the full gate. A scheduled agent has no context beyond
  what it can read, and unattended drafting at scale is the thing being
  controlled.
- A person asking directly is subject to Gate 0 only. "Has this shipped, is the
  surface public" is factual and a requester can be wrong about it, so it is
  worth verifying regard  worth verifying regard  worth verifying regard  worth verifying regard  at  worth verifying regard  worth verifying regard  worth verifying regardoe  worth verifying regard  worth verifying regara Gate 1-3 conce  worth verifying regard  worth verifying regard t drafts: prefer updating  worth verifying regard  worth verifying regard  w needs.

Applied in draft_dApplied in draft_dApplied in draft_dApplied in draftd
checklist so an agent rechecklist so an agent rechecklist so an agent rechhored-By: Warp <agent@warp.dev>
@rachaelrenk
Revise content design plan for clarity on purpose
d140ef5 
Clarify the purpose and audience definition in the content design plan.
@rachaelrenk @warp-agent
docs: align content-type templates with GitHub's content model
63cf15b 
Reviewed all eight templates against GitHub's templates page. Three real bugs
and several gaps.

Six templates taught a pattern no page uses. faq, feature-doc, procedural,
quickstart, reference, and troubleshooting showed a body "# [Title]" H1 with no
frontmatter title. All 377 pages under src/content/docs use frontmatter title,
and Starlight renders it as the H1 -- conceptual.md and guide-page.md already
said so explicitly. All eight now set title in frontmatter with no body H1.

quickstart.md contradicted what shipped earlier in this branch. It still said
~10 minutes in two places after AGENTS.md and draft_quickstart moved to ~5
minutes / ~600 words, and it put Next steps before Troubleshooting so the
closing section was not last. Both fixed, and the scope budget is now stated in
the template itself the way GitHub's quickstart does.

Four templates had no closing cross-link section at all -- faq, procedural,
reference, troubleshooting -- while the AGENTS.md checklist required one. The
rule is now explicit and applied everywhere: Next steps for quickstarts and
tutorials, because the reader just finished something and needs forward
momentum; Related pages for every other type, because they want lateral
material. Not "Further reading", which GitHub uses but appears zero times in
our corpus against 54 Related pages and 46 Next steps.

Adopted from GitHub: a "[BEFORE PUBLISHING: delete every bracketed instruction]"
note at the top of every template. GitHub says this in each of theirs and we
said it nowhere, so nothing told an author to strip the guidance before
shipping. Also added their note that content not under a header is not linkable
in the table of contents.

Kept brackets rather than adopting GitHub's comment blocks. Instructions in
comments get deprioritized by agents, which is the whole reason we use brackets.
Corrected AGENTS.md, which claimed the templates use HTML comments -- they do
not, and have not.

Trimmed the duplicated AEO-brief and pre-handoff-review blocks in procedural.md
and guide-page.md to one-line pointers at the owning skills. Net 460 to 390
lines across the set despite adding the removal notes and scope guidance.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk @warp-agent
docs: split the content design plan into reference plus template, and…
0d08520 
… fix its review-timing claim

Two changes, one of them a correction to a claim that was simply wrong.

The reference argued that a plan is cheaper to disagree with than finished
prose, and then told the author to put it in the PR body. Those do not go
together. If the plan lands in the PR body, the draft already exists and the
expensive work is done -- the reviewer is not saved a rewrite. The argument
described a pre-draft checkpoint while the placement described a record.

The pipeline deliberately opens draft PRs rather than proposing first, so the
automated path has no pre-draft human checkpoint by design. Overclaiming here
obscured that tradeoff instead of stating it.

Now framed by mode, because the two cases really are different:

- Interactive drafting is a real checkpoint. draft_docs presents the plan and
  waits before writing prose, the way write-feature-docs already waits on its
  outline. Redirecting costs a conversation instead of a rewrite.
- Automated runs get a record and a consistency check. The prose exists by the
  time anyone reads it, but the reasoning is inspectable next to the diff, so a
  reviewer can catch drift and reject on aim rather than on prose.

The benefit that survives in both cases is that writing the plan disciplines the
agent -- it inverts the template default regardless of when a human reads it.

Separately, the reference was doing two jobs: explaining the fields and carrying
the fill-in artifact. Split them along the same line the repo already uses for
content types, where AGENTS.md defines and .agents/templates/ scaffolds. The
artifact moves to .agents/templates/content-design-plan.md; the reference keeps
the definitions and the reasoning.

It is the one template that is not a page scaffold, so AGENTS.md now scopes the
frontmatter-title rule to page templates and says why this one is exempt.

Co-Authored-By: Warp <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot mentioned this pull request Aug 21, 2026
Closed
warp-agent-staging Bot pushed a commit that referenced this pull request Aug 21, 2026
@warp-agent
docs: address review — per-reviewer requests, first-content check, CI…
aeda3e6 
… wiring

Blocking fix. The reviewer verification was emptiness-only, so the owning
engineer could be dropped silently — the exact bug this PR exists to fix.
`gh pr edit --add-reviewer a,b,c` is one atomic mutation, so a single
unassignable entry rejected the whole list and the `||` then replaced every
resolved owner with the fallback; a non-empty readback still passed. This is
live: `warpdotdev/oss-maintainers` is the root-rule owner in the warp client
repo and appears in most resolutions, but `/repos/warpdotdev/docs/teams` is
empty, so it cannot be requested here. Now each reviewer is requested in its
own call and the readback is compared against the resolved set, with partial
results reported. Also fixed the readback jq: the old
`[.reviewRequests[].login // .reviewRequests[].name]` silently drops teams
from a mixed list (verified).

Also:
- check_lead_section now asserts the summary is the first *content*, not just
  the first heading. A body opening with unheaded spec/workflow/run-ID
  preamble previously exited 0, which is the shape the check exists to stop.
- _iter_non_code_lines skips HTML comments, so a `##` inside a multi-line
  comment no longer displaces the lead section — same class already handled
  for code fences.
- Wired test_check_new_release.py into CI. The earlier deferral was wrong:
  #586 does not touch ci.yml and this PR already edits it, while
  missing_docs/SKILL.md advertises the test as covered.
- suggest_reviewers.py routes resolution diagnostics to stderr under
  --reviewers-only, so a fallback leaves a trace without polluting stdout.
- Removed the duplicated reviewer snippet from missing_docs; create_pr holds
  the canonical copy. The copies had already diverged, and the missing_docs
  one used `[[ -z ... ]] && ...`, which returns 1 and would abort a `set -e`
  scheduled run.
- Backticked the date in the worked example; marked the drafting-only lines
  in the copy-paste heredoc.
- Tests locking in first-content, HTML-comment banners, multi-line comments,
  CRLF bodies, and the stderr diagnostics.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk @warp-agent
missing_docs: use full script paths in the scheduled-agent prompt
93e8d89 
The recommended prompt told the agent to run scripts/check_new_release.py,
which does not resolve from the repo root. A scheduled agent pasting the prompt
verbatim would fail before the release gate ran, and the failure would look like
a broken gate rather than a bad path.

Caught by warp-for-oss[bot] on PR #586.

Fixed the flagged line and the identical pre-existing one two lines below it:
scripts/suggest_reviewers.py had the same problem but was outside the diff, so
the bot could not see it. Patching only the flagged path would have left the
same bug in the same block.

Left the other relative script paths alone. Inside the skill doc the reader
already has the directory context, and the References section lists paths
relative to the skill directory by convention. The prompt block is different
because it is copied verbatim into an agent config with no surrounding context.

Also reflowed the block to a consistent width; the inserted full paths had left
it ragged.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk @warp-agent
docs: read the product spec before drafting, not just the code
a777aa7 
The trigger fold kept the release-driven half of write-feature-docs and
dropped the reason it read specs in the first place. Phase 3 research is
code-only, and draft_docs step 1 reviews a spec only when a requester
hands one over -- which a scheduled run never does.

Code answers what a surface does. It cannot answer who needed it or what
problem it solves, which are the content design plan's first three
fields, so those got inferred from a one-line changelog bullet.

warp-server specs/<id>/PRODUCT.md answers them almost directly: its
Problem, Goals, Non-goals, and User experience sections map onto the
plan's Problem, Goals, Excludes, and high-impact scenarios.

Bounded so this does not become the old spec-triggered pipeline:
framing only and never behavior, never evidence a feature shipped, and
never quoted into a public page. Only some specs have a PRODUCT.md, so
absence is recorded in the plan rather than papered over.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk rachaelrenk mentioned this pull request Aug 22, 2026
Merged
rachaelrenk and others added 2 commits August 24, 2026 11:03
@rachaelrenk @warp-agent
Merge origin/main into rrenk/release-docs-gate
0b4e879 
Resolves conflicts with the tone overhaul (#606), which landed after this
branch was approved and touched the same drafting surface.

Templates (conceptual, faq, guide-page, procedural, reference): keep this
branch's rewritten scaffolds and add #606's [BREVITY] line. #606 added that
line to the pre-rewrite templates, so git could not reconcile the two.
Main's verbose [AEO GUIDANCE] blocks stay dropped -- this branch replaced
them with the one-line [AEO] pointer, which was part of the reviewed
rewrite, not a loss from the merge.

procedural.md also adopts main's corrected API keys path
(Settings > Cloud platform > API keys, from #605). This branch still had
the stale Settings > Platform.

draft_docs: keep both rule sets. Main's stricter callout rule and its two
new bullets (AI-ism buzzwords, user-visible model) plus this branch's
link-text rule, which is the more precise one -- it names the per-type
closing section and bans "Further reading" / "See also".

Verified: no conflict markers, no duplicated checklistVerified: no conflict markers, no duplicated checkliste-check tests, 21/21 factory-noun tests.
No src/content pages differ from main.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk
Merge remote-tracking branch 'origin/rrenk/release-docs-gate' into rr…
4bde968 
…enk/release-docs-gate
@rachaelrenk @warp-agent
docs: add a discoverability gate and right-size the content design plan
4aec2eb 
Addresses HYC's review feedback on #586.

Gate 4 -- is it a capability the reader cannot discover on their own?

Gates 1 through 3 are all friction-based: they assume a reader who is
already trying to do something and hits a wall. None of them fire for a
reader who does not know the capability exists. That is a real gap for a
pipeline aimed at new features.

The risk is that "users should know about this" is exactly the hand-wave
the gate was built to reject, so Gate 4 requires two checkable facts and
refuses the impression:

  1. The change makes a user job possible, not merely easier. "Faster",
     "more reliable", and "now also works in X" are existing jobs.
  2. Nothing in the product surfaces it -- no Command Palette entry, menu
     item, settings toggle, onboarding hint, or error message. Name the
     surfaces you checked.

The second condition is the discipline; without it every changelog line
passes. Reconciled the two places that previously rejected capability
outright so they now distinguish the outright so they now distinguish the ouap:outright so they now distinguish the outright so they now distinguidate, never a new page. Anything that truly warrants a page also
passes Gate 1, 2, or 3. This is what keeps a discoverability gate from
reopening new-page sprawl.

Three worked examples calibrate it, including a case that fails Gate 4 on
discoverabilitdiscoverabilitdiscoverabilitdiscoverabilitdiscoverabilitdiscoverabault

HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH plan
promised "four sentences" promised "four sentences" promised "four sentences" promised "four seshopromised "four sentences" promised "four sentences" promised "foew promised "four sentences" promised "four sentences" promised "four ction -promised "four sentences" promised "four sentences" promised "four s reference with a
    filled exa    filled exa    filled exa    filled exa    filled exa    filled eate,    filled exa    filled exa    filled exa    filled exa    filled emplates, and Purpose and value drop out. It is deliberately not a
    template file -- templating three lines reintroduces the weight.
  - Expanded the skip list: docs bug fixes, factual corrections, adding a
    missing entry to an existing reference table, redirects.

Co-Authored-By: Warp <agent@warp.dev>
@rachaelrenk
rachaelrenk merged commit fee1b6c into main Aug 24, 2026
8 checks passed
@rachaelrenk
rachaelrenk deleted the rrenk/release-docs-gate branch August 24, 2026 18:11
This was referenced Aug 24, 2026
Merged
Merged
rachaelrenk added a commit that referenced this pull request Aug 25, 2026
@rachaelrenk @warp-agent @warp-agent-staging
docs skills: make reviewer requests real, and fix drift-watch sandbox…
4101522 
… footguns (#619)

* docs: request reviewers for real and lead ambient PRs with a feature summary

Two fixes to the ambient new-feature docs pipeline (GROW-6093).

1. Actually request reviewers. The drafted PR only named reviewers in
   prose, which puts nothing in GitHub's review queue: docs #414, #415,
   #416 and #417 all named reviewers in the body and received zero
   reviews, three with an empty requested-reviewers list. Wire a required
   `gh pr edit --add-reviewer` step into missing_docs drift-watch step 7
   and into the create_pr skill, with the `dannyneira` fallback that
   release-docs-update.yml already uses, plus a verification read-back so
   a silently skipped assignment is caught. The prose /cc mention stays.

   suggest_reviewers.py gains `--reviewers-only` so the step can consume
   the resolved set without scraping the human-readable table.

2. Lead the PR body with a feature summary. Drafting PRs must open with
   `## What this feature does`: plain language, what the feature does for
   the user, ending with the shipped-in version and date read from
   check_new_release.py --json. Budget 75 words. check_pr_body.py gains
   `--require-lead-section`, asserting the heading is present once, is the
   first heading, is non-empty, and is within budget.

Co-Authored-By: Warp <agent@warp.dev>

* docs: address review — per-reviewer requests, first-content check, CI wiring

Blocking fix. The reviewer verification was emptiness-only, so the owning
engineer could be dropped silently — the exact bug this PR exists to fix.
`gh pr edit --add-reviewer a,b,c` is one atomic mutation, so a single
unassignable entry rejected the whole list and the `||` then replaced every
resolved owner with the fallback; a non-empty readback still passed. This is
live: `warpdotdev/oss-maintainers` is the root-rule owner in the warp client
repo and appears in most resolutions, but `/repos/warpdotdev/docs/teams` is
empty, so it cannot be requested here. Now each reviewer is requested in its
own call and the readback is compared against the resolved set, with partial
results reported. Also fixed the readback jq: the old
`[.reviewRequests[].login // .reviewRequests[].name]` silently drops teams
from a mixed list (verified).

Also:
- check_lead_section now asserts the summary is the first *content*, not just
  the first heading. A body opening with unheaded spec/workflow/run-ID
  preamble previously exited 0, which is the shape the check exists to stop.
- _iter_non_code_lines skips HTML comments, so a `##` inside a multi-line
  comment no longer displaces the lead section — same class already handled
  for code fences.
- Wired test_check_new_release.py into CI. The earlier deferral was wrong:
  #586 does not touch ci.yml and this PR already edits it, while
  missing_docs/SKILL.md advertises the test as covered.
- suggest_reviewers.py routes resolution diagnostics to stderr under
  --reviewers-only, so a fallback leaves a trace without polluting stdout.
- Removed the duplicated reviewer snippet from missing_docs; create_pr holds
  the canonical copy. The copies had already diverged, and the missing_docs
  one used `[[ -z ... ]] && ...`, which returns 1 and would abort a `set -e`
  scheduled run.
- Backticked the date in the worked example; marked the drafting-only lines
  in the copy-paste heredoc.
- Tests locking in first-content, HTML-comment banners, multi-line comments,
  CRLF bodies, and the stderr diagnostics.

Co-Authored-By: Warp <agent@warp.dev>

* create_pr: stop the fallback reviewer from masking a dropped owner

Review catch on #619. Step 4 appended FALLBACK_REVIEWER to GOT, but GOT
answers "which resolved owners did I actually request". Counting the
fallback there let the verification pass on a run where every real owner
was rejected -- the exact silent failure the section exists to prevent,
two paragraphs after it says "verify against the resolved set, not
against emptiness".

Traced against the documented snippet with a stubbed gh. Resolved owners
alice and bob, both rejected, fallback accepted:

  before:  warning: requested 1/2 resolved reviewers
           Requested reviewers: dannyneira
           exit 0

  after:   ERROR: none of the 2 resolved owners could be requested
           (wanted: alice bob); only the fallback is assigned.
           exit 1

The fallback now stays out of GOT, and the outcomes are reported as four
distinct states rather than one count: all owners requested, a partial
result naming who is missing, owners resolved but none requested (an
error, because the PR has the wrong reviewer), and nothing resolved at
all (a note, because the fallback is the intended pall there). Not even
the fallback landing remains a hard failure.

Verified by extracting the snippet from SKILL.md and executing it against
a stubbed gh across all five cases, so the documented text is what was
tested rather than a paraphrase of it.

Co-Authored-By: Warp <agent@warp.dev>

* create_pr: trust the read-back, and fix comment/fence ordering

Two review catches on #619.

1. Reviewer verification trusted gh's exit status

The section warns that `gh pr edit` can exit 0 while quietly skipping a
reviewer, then verified against GOT -- which is built from those exit
statuses. A silently skipped owner passed.

Verification now compares the read-back against WANT. Step 4's fallback
also keys off the read-back rather than GOT, because when gh exits 0 for
every owner and requests none of them, a GOT-based check skips the
fallback entirely and leaves the PR with no reviewer at all.

Match on the last path segment, lowercased: a team resolves as org/team
but reads back as its bare slug, so a naive compare reported every team
as missing.

Verified by extracting the snippet from SKILL.md and running it against
a stubbed gh across nine cases, including a stub that exits 0 without
recording the reviewer:

  bob silently skipped -> warning names bob   (previously silent)
  all silently skipped -> fall  all silently skipped -> fall  all silently skipped -> fall es its bare slug, no false "missing"

2. Fence detection ran before comment stripping

A ``` line inside an HTML comment opened a phantom code block that ate
the closing --> and every line after it, including the lead heading. A
valid body failed with "missing required lead section", which reads as
an authoring mistake rather than a parser bug.

Precisely: only an *odd* number of fence lines inside a comment breaks
it. A balanced pair opens and closes a phantom block that happens to end
before the -->, so it passed by luck. The tests say which case is the
real regression rather than implying all of them were.

The fix honors fence state first, then strips comments, then looks for a
fence in the visible text. Simply reordering the two would break the
mirror case:mirror case:mirror case:mirror case:mirror case:mirror case:mirror cs a comment would swallow tmirror case:mirror case:mirror cver both directions plus a guard tmirror case:mirror case:mirror case:mirror case:mirror case:mirror case:mirror cs a comment would swallow tmirror caseent@warp.dev>

* create_pr: verify the fallback reviewer by name, not by read-back emptiness

Review catch on #619 (QUALITY-1875 rework). When owner resolution came back
empty and the PR already carried an unrelated reviewer, the prior "is
$REQUESTED non-empty" check treated that unrelated reviewer as proof the
dannyneira fallback had landed, so it skipped verifying/re-requesting the
fallback by name -- and the script still printed "fallback requested" and
exited 0 even when dannyneira was never assigned.

Added a has_reviewer helper that checks the read-back for a specific
reviewer, used it to gate the fallback request/verification when resolution
was empty, and split the final error check so a fallback that truly can't be
assigned is reported as a failure instead of masked by an unrelated
reviewer already on the PR.

Added test_request_reviewers.py, which extracts the documented snippet from
SKILL.md and runs it against a stubbed gh/suggest_reviewers.py across the
normal-resolution, empty-resolution, and pre-existing-unrelated-reviewer
cases. test_unrelated_reviewer_does_not_mask_fallback_failure fails against
the pre-fix snippet and passes after the fix. Wired into ci.yml.

Co-Authored-By: Warp <agent@warp.dev>

* missing_docs: fix the three sandbox footguns validation run #2 hit

The second drift-watch validation run surfaced three ways the skill misleads an
unattended agent. All three are documentation gaps in the skill, not code bugs.

1. Working directory. Every path in the skill is relative to the docs repo root,
   but the skill never says so, and a sandbox commonly starts one level up. The
   failure mode is the problem: python3 exits 2 with "can't open file", which is
   the same exit code audit_docs.py uses to fail loud on a broken environment.
   An agent that reads the code and not the message concludes a sanity guard
   tripped and stops. State the cwd requirement up front, name the collision,
   and repeat it at the release-gate step and in the scheduled-agent prompt --
   the prompt is the only one of the three a cron run is guaranteed to read.

2. npm install. `npm run build` is the only validation this repo has and it
   needs node_modules, which a fresh sandbox does not have. Add `npm ci` as a
   stated precondition in Requirements and at both build sites.

3. Surface-map key edits. A rename sweep run across feature_surface_map.md
   corrupted an    corrupted an    corrupted an    corrupted an    corrupted an    corrupted aon    corrupted an    corrupted an    corrupted an    corrupted an    corrupted an map entry is a literal code identifier that only matches
   because it matches ex   because it matches ex   because it matches ex   beCo-Authored-By: Warp <agent@warp.dev>

---------

Co-authored-by: Warp <agent@warp.dev>
Co-authored-by: warp-agent-staging[bot] <240773466+warp-agent-staging[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@warp-for-oss warp-for-oss[bot] warp-for-oss[bot] left review comments
@hongyi-chen hongyi-chen hongyi-chen approved these changes

Assignees

@rachaelrenk rachaelrenk

Labels

cla-signed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rachaelrenk @hongyi-chen