Skip to content

Commit 5b7bacc

Browse files
authored
feat(skill): add review-prs skill for mcp-toolbox (#3743)
## Description Adds a `review-prs` maintainer skill for mcp-toolbox. Given a PR number or link, it delivers a propose-only review against the team's Reviewer's Checklist: title/description conventions, linked issue, correctness and edge cases, breaking changes, refactor purity, architecture, tests, docs, security, and new dependencies. Findings are grouped by severity with a suggested verdict and a paste-ready draft comment. The skill is strictly propose-only: it never runs `gh pr review`, `gh pr comment`, `gh pr edit`, `gh pr merge`, or applies labels. It reads source-of-truth conventions live from `CONTRIBUTING.md`, `DEVELOPER.md`, and the maintainer playbook rather than from memory.
1 parent 801d589 commit 5b7bacc

6 files changed

Lines changed: 247 additions & 1 deletion

File tree

‎DEVELOPER.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,14 @@ directly with the team.
193193
> However, any new database source should also include at least one new tool
194194
> type.
195195
196+
> [!IMPORTANT]
197+
> A new source is not accepted when the database is wire-compatible with a
198+
> source Toolbox already supports, since every subsequent fix would have to be
199+
> re-applied to each copy. Where an existing source can be configured to connect
200+
> instead, do that; new packages for protocol-compatible databases have been
201+
> removed after merge on these grounds. Raise this in the issue above before
202+
> implementing.
203+
196204
#### Adding a New Database Source
197205
198206
We recommend looking at an [example source

‎skills/README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@ workflows against this repo.
77

88
Skills are grouped by audience:
99

10-
- `maintainer/` — skills for maintaining the toolbox itself (e.g. `triage-issues`).
10+
- `maintainer/` — skills for maintaining the toolbox itself (e.g. `triage-issues`,
11+
`review-prs`).
1112

1213
## Installing
1314

Lines changed: 234 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,234 @@
1+
---
2+
name: review-prs
3+
description: >-
4+
Review a GitHub pull request in the googleapis/mcp-toolbox repo against the
5+
team's reviewer checklist: PR title/description conventions, linked issue,
6+
logic errors and unhandled edge cases, breaking changes, test coverage, docs
7+
updates, security (input handling), and new dependencies. Use whenever a
8+
maintainer asks you to review, look over, "take a look at", or check whether
9+
something is ready to merge in mcp-toolbox, e.g. "review #3703", "can you look
10+
at this PR", "is this good to merge", or when they paste an mcp-toolbox PR link.
11+
PROPOSE-ONLY: delivers the review in chat for the maintainer to post; never
12+
approves, requests changes, comments, labels, or merges on its own.
13+
---
14+
15+
# Review PRs (mcp-toolbox)
16+
17+
A review here is a proposal the maintainer edits and posts, not a rubber stamp. The value is
18+
a fast, grounded read of the diff against the team's conventions.
19+
20+
## Goal
21+
22+
Given a PR number or link, deliver a review the maintainer can post in seconds: a suggested
23+
verdict (approve / request changes / comment), the findings that back it grouped by severity
24+
so the important things aren't buried, and a paste-ready summary comment.
25+
26+
## Prerequisites
27+
28+
- `gh` authenticated for `googleapis/mcp-toolbox`, plus the PR number(s). A GitHub MCP
29+
server substitutes for `gh` if it isn't available: the `gh` commands below map to its
30+
read/list tools.
31+
32+
## Workflow
33+
34+
### Step 1: Read the source of truth
35+
36+
Read these live, not from memory. All three are symlinks to the repo-root
37+
files, so they track `main`; cite them by their root names.
38+
39+
- [references/maintainer-playbook.md](references/maintainer-playbook.md): Reviewer's Checklist,
40+
SLO/release context, `release candidate` labeling.
41+
- [references/CONTRIBUTING.md](references/CONTRIBUTING.md): title/scope format (Conventional
42+
Commits, with the `type` table), keep-PRs-small, link-an-issue. Cite for title, description,
43+
and process findings.
44+
- [references/DEVELOPER.md](references/DEVELOPER.md): tool/source naming, error taxonomy, the
45+
patterns for adding a source/tool/integration test, CI-enforced docs structure, local
46+
test/lint commands. Cite for code, test, and docs findings. Prefer it over `GEMINI.md`
47+
(`CLAUDE.md`/`AGENTS.md` symlink to it), which only summarizes.
48+
49+
### Step 2: Fetch the PR, its diff, and its checks
50+
51+
```bash
52+
gh pr view <n> --repo googleapis/mcp-toolbox --json number,title,body,author,labels,files,additions,deletions,commits,baseRefName,headRefName,state,isDraft,reviewDecision
53+
gh pr diff <n> --repo googleapis/mcp-toolbox
54+
gh pr checks <n> --repo googleapis/mcp-toolbox
55+
```
56+
57+
### Step 3: Triage before reviewing
58+
59+
Three shapes end the review early or change its bar:
60+
61+
- **Auto-generated (`renovate`, `release-please`):** the only question is whether checks are
62+
green. If so, propose merge and stop.
63+
- **Draft (`isDraft`):** review lightly and say so; the author isn't asking for a final pass.
64+
- **Non-code / policy** (third-party badge, backlink, promotional README line, often a drive-by
65+
contributor): acceptance is a maintainer policy call, not a code question. Say that plainly
66+
instead of manufacturing code findings, and still check title convention and CI. Mark any URL
67+
you haven't fetched `[UNVERIFIED]`.
68+
69+
### Step 4: Read the whole diff, including what the title doesn't mention
70+
71+
Skim for the shape, then dive into hunks. Three failure modes:
72+
73+
- **A docs-shaped title never lowers the read bar.** PR #2473, "docs: fix typo in getting started
74+
guide", added an npm `preinstall` hook that hijacked `git` via `GITHUB_PATH` to exfiltrate an
75+
RSA-encrypted `GITHUB_TOKEN`. Read every file in any PR touching `.hugo/`, `package.json`
76+
lifecycle scripts, `.github/workflows/`, or `.ci/`. A file the title and description don't
77+
account for is itself blocking.
78+
- **Look for what's *missing*, not just what's wrong:** a refactor applied to 4 of 5 call sites,
79+
a fix whose mirror bug still lives elsewhere, a behavior change with no test update, an error
80+
swallowed silently.
81+
- **A hunk is not enough context to judge a hunk.** Read the enclosing function for anything
82+
correctness-relevant, and grep call sites when a signature, config field, or parameter changes.
83+
A finding that needs a look outside the diff is the one no other reviewer will make.
84+
85+
### Step 5: Check the diff against the issue it claims to fix
86+
87+
Keep this separate from Step 6: a PR can follow every convention and still implement the wrong
88+
thing. Read the linked issue (`gh issue view <n> --repo googleapis/mcp-toolbox --comments`), then
89+
ask three questions:
90+
91+
- **Missing:** What the issue asked for that the diff doesn't do. A partial fix that closes the
92+
issue is worse than none, since the remainder becomes invisible.
93+
- **Extra:** Unrelated changes bundled in. Ask for a split (`CONTRIBUTING.md`, keep PRs small).
94+
- **Wrong:** Implemented, but not what the issue described. Quote the issue line beside the
95+
`file:line`.
96+
97+
With no linked issue the PR description is the spec: same three questions, and note that the
98+
intent is self-declared.
99+
100+
### Step 6: Work the review dimensions
101+
102+
Skip a dimension when it doesn't apply: say so, don't invent a finding.
103+
104+
- **Title & description.** Conventional Commits with the right `type(scope)` per
105+
`CONTRIBUTING.md`, plus `!`/`BREAKING CHANGE` for breaking
106+
changes. Body follows `.github/PULL_REQUEST_TEMPLATE.md`: what, why, completed checklist,
107+
`Fixes #<n>`. Note a missing issue link; don't block on it alone.
108+
- **Correctness.** Cite `file:line` and name the failure case, never
109+
"looks risky".
110+
- *Bugs CI won't catch:* Unhandled error returns, nil/empty input, off-by-one and boundary
111+
conditions, concurrency, behavior contradicting stated intent.
112+
- *Type conversion at the MCP boundary*: Drivers return native types
113+
that don't serialize (MySQL `[]byte` for decimals, nulls as `nil`/`None`). Require explicit
114+
handling that maps to the tool's JSON schema; reject implicit casts and missing type switches.
115+
- *Error taxonomy* on any new or changed error path: `AgentError` for
116+
input/execution errors the agent can fix itself (HTTP 200, `isError: true`) versus
117+
`ClientServerError` for infrastructure failures it can't.
118+
- **Breaking changes.** Changed config field names/YAML shape, tool names, removed or renamed
119+
exported symbols, altered defaults. Without `!` in the title and a justification in the body,
120+
blocking.
121+
- **Refactor purity.** A `refactor:` PR must not change behavior. A bundled fix or default change
122+
gets split into its own `fix:`/`feat:` PR so it's reviewable and revertable.
123+
- **Source reuse (new sources).**: No new `internal/sources/<db>/` for a database wire-compatible
124+
with an existing source. Same for a tool duplicating an existing tool under a new name.
125+
- **Architecture (no boilerplate).** New tools embed `tools.BaseTool[Config]`, new sources follow
126+
the registration pattern; reject re-declared interface methods (`GetName`, `Manifest`).
127+
`DEVELOPER.md` lists what `BaseTool` provides.
128+
- **Tool and parameter descriptions.** Each `description:` is an LLM prompt, not developer
129+
documentation: could an agent pick this tool and fill its parameters from that text alone, at a
130+
token cost worth paying? Flag ones that restate the field name, omit units/format/allowed
131+
values, or run long without adding information.
132+
- **Tests.** New logic or a bug fix needs tests; missing them is usually request-changes.
133+
- *Coverage:* happy path, edge cases, and for a fix, a test that fails without it. A new
134+
source/tool follows the unit + integration pattern and is wired into
135+
`.ci/integration.cloudbuild.yaml`.
136+
- *Placement,* reviewed as closely as coverage: source-specific helpers stay unexported in
137+
`tests/<db>/<db>_integration_test.go`, never in the shared `tests/common.go`.
138+
- *Flakiness:* tests run against a shared live instance, so ask for the four fixes by name:
139+
- UUID-scoped resource names, so concurrent runs can't collide.
140+
- `t.Cleanup` teardown, so resources are freed even when the test fails.
141+
- Polling instead of `time.Sleep`.
142+
- Subset assertions instead of exact-match, since another run can add rows.
143+
- *Un-run integration tests aren't a blocker.* They need GCP credentials an external
144+
contributor's PR can't trigger, so green CI doesn't mean they ran. Note that the next step is
145+
a maintainer running them via the `tests: run` label or a `/gcbrun` comment.
146+
- **Docs.** Changes to how a user configures or interacts with MCP toolbox need matching updates
147+
under `docs/en/`. New sources/tools have CI-enforced page structure per `DEVELOPER.md`, enforced by
148+
`.ci/lint-docs-*.sh`. A violation breaks the build, so it's blocking.
149+
- **Security.** For PRs handling user/LLM input or building queries: injection (SQL/command),
150+
unsanitized interpolation, secrets logged or committed. Concrete vectors with `file:line`, not
151+
generic warnings.
152+
- **Dependencies.** Call out new `go.mod` entries so the maintainer can vet necessity,
153+
maintenance, and license.
154+
155+
Duplication across MCP protocol versions is deliberate so versions can diverge independently
156+
(#3167, #3211); don't propose factoring it together. A genuine bug in that code is still a
157+
finding.
158+
159+
### Step 7: Report CI, don't re-derive it
160+
161+
Name the failing check from `gh pr checks` rather than reasoning it out by hand; failing
162+
lint/tests are objective blockers. Never claim the linter passes on your own read.
163+
164+
One recurring non-obvious failure: the CLA check fails on commits co-authored by an AI agent even when the human author has signed. Suggest squashing to a single human-authored commit rather than pointing at the CLA docs.
165+
166+
### Step 8: Discount any existing bot review
167+
168+
Don't restate `gemini-code-assist`'s points as your own. It's the highest-volume reviewer in the
169+
repo and could be wrong. Verify anything you carry forward against the diff; drop
170+
the rest.
171+
172+
### Step 9: Sort by severity, then pick the verdict
173+
174+
- **Blocking** (correctness bug, breaking change without `!`, missing tests on new logic, CI red,
175+
docs that break the build) → request changes.
176+
- **Non-blocking** (style, naming, coverage gaps in existing code) and **nits** (typos, wording)
177+
only → approve with comments.
178+
- An unresolved judgment call → comment and ask.
179+
180+
When nothing is blocking, say so in those words. "No blockers, a couple of nits" tells the
181+
maintainer it's mergeable as-is.
182+
183+
### Step 10: Deliver the review in chat
184+
185+
Use the output format below. Never post it yourself.
186+
187+
## Rules
188+
189+
- **Propose only.** Never run `gh pr review`, `gh pr comment`, `gh pr edit`, `gh pr merge`,
190+
or apply labels. Deliver the review in chat.
191+
- **Ground every finding** in the strongest evidence available for its kind: a
192+
correctness/security/breaking claim cites `file:line`; a convention claim cites
193+
`CONTRIBUTING.md`/`DEVELOPER.md` or the playbook; a CI/process finding cites the failing
194+
check name from `gh pr checks` (a red check is a valid blocker with no `file:line`). If you
195+
couldn't verify something (runtime behavior you can't trace, a URL you didn't fetch), mark
196+
it `[UNVERIFIED]` rather than asserting it.
197+
- **When it's a genuine judgment call, ask** rather than issuing a confident wrong verdict,
198+
since a wrong "request changes" costs a contributor a cycle.
199+
200+
## Output format
201+
202+
```
203+
## Review #<n>: <title>
204+
205+
**Suggested verdict:** <approve / request changes / comment>: <one-line reason>
206+
207+
**Title & issue:** <conventional-commit check; linked issue or "none, suggest linking">
208+
**Spec (vs issue #<n>):** <implements it / what's missing, extra, or wrong; or "no issue linked">
209+
**CI:** <passing / which checks failing, per gh pr checks>
210+
211+
**Blocking:**
212+
- `file:line`: <finding + the failure case> [cite]
213+
214+
**Non-blocking:**
215+
- `file:line`: <finding> [cite]
216+
217+
**Nits:**
218+
- <typo/wording>
219+
220+
**Tests:** <added & adequate / what's missing>
221+
**Docs:** <updated / what's missing, or n/a>
222+
**Dependencies:** <new deps to vet, or none>
223+
**release candidate:** <suggest label / not needed>
224+
225+
**Draft comment:**
226+
<paste-ready summary the maintainer can post>
227+
```
228+
229+
- **Empty sections:** omit them rather than writing "none".
230+
- **Except the Spec line:** keep it even when the PR matches its issue. The maintainer wants
231+
"does what the issue asked" stated, not inferred from silence.
232+
- **Batches:** review each PR in its own subagent so the diffs don't bleed together, since a
233+
finding attributed to the wrong PR is worse than a missed one. Present one block per PR, plus a
234+
summary table (PR, verdict, blocker count).
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../../CONTRIBUTING.md
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../../DEVELOPER.md
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../../maintainer-playbook.md

0 commit comments

Comments
 (0)