Skip to content

implement-issue's plan-comment locator never detects a genuine no-plan-comment case #1551

Description

@phmatray

What happened?

implement-issue's legacy plan-comment locator (.claude/skills/implement-issue/SKILL.md §"Step 2 —
Read the plan", and the duplicated recipe in references/github-mechanics.md §2) has a fallback branch
for issues whose plan comment lacks the 🛠️ Implementation plan marker: it re-searches for the latest
comment containing - [ ]/- [x] checkbox lines instead. That fallback is guarded by:

PLAN_COMMENT_ID=$(gh api "repos/{owner}/{repo}/issues/$ISSUE/comments" --paginate --slurp \
  | jq -r '[.[][] | select(.body | contains("🛠️ Implementation plan"))] | last | .id')

# Fallback if no marker (older/hand-written plan): latest comment that has checkbox lines.
if [ -z "$PLAN_COMMENT_ID" ]; then
  PLAN_COMMENT_ID=$(gh api "repos/{owner}/{repo}/issues/$ISSUE/comments" --paginate --slurp \
    | jq -r '[.[][] | select(.body | (contains("- [ ]") or contains("- [x]")))] | last | .id')
fi

When the primary select(...) | last | .id filter matches nothing, last on an empty array is JSON
null, and null | .id is also null. Both jq -r and gh api --jq print a null result as the
literal 4-character string null, not an empty string — confirmed locally: echo '{}' | jq -r '.foo' prints null. So PLAN_COMMENT_ID is assigned the string "null", and [ -z "$PLAN_COMMENT_ID" ] is false (the string is non-empty) — the fallback branch is dead code, and the same problem
recurs one line later at [ -z "$PLAN_COMMENT_ID" ] && { echo "No implementation plan…"; exit 1; },
which also never fires on a genuine no-match. Instead the flow falls through to gh api "repos/{owner}/{repo}/issues/comments/null" --jq .body, a REST call against the literal path segment
null, which 404s with a confusing HTTP 404: Not Found rather than either finding the right comment
or a clear "no plan" message.

Discovered while implementing #1544 (a sibling bug in the same lines — --paginate --jq not
merging pages before filtering — fixed in PR #1550), but left alone there since it's a distinct bug
class (an emptiness check, not a pagination-merge bug) and #1544 explicitly scoped itself to "do not
re-audit unrelated --paginate call sites in this PR."

Minimal .koi model

N/A — this is a bug in a .claude/skills/implement-issue shell recipe (agent tooling), not the Koine
compiler, emitter, or a .koi model.

Steps to reproduce

  1. Pick (or seed) a GitHub issue whose comments contain no 🛠️ Implementation plan marker and
    no - [ ]/- [x] checkbox lines at all (e.g. a plain issue with only prose comments).
  2. Run the primary locator from github-mechanics.md §2 against it:
    gh api "repos/{owner}/{repo}/issues/<ISSUE>/comments" --paginate --slurp \
      | jq -r '[.[][] | select(.body | contains("🛠️ Implementation plan"))] | last | .id'
  3. Observe the output is the literal text null, not an empty string.
  4. Trace that value through [ -z "$PLAN_COMMENT_ID" ] — the test is false, so the fallback search
    (checkbox-comment lookup) and the final "no plan, exit 1" guard are both skipped, and execution
    proceeds to fetch comment id null.

Actual output / error

$ echo '{}' | jq -r '.foo'
null
$ [ -z "null" ] && echo EMPTY || echo NOT-EMPTY
NOT-EMPTY

Expected: the emptiness checks should also treat the literal string null as "not found," so the
fallback search and the final stop-with-a-clear-message branch actually run.

Koine version

N/A — no koine CLI/compiler version is implicated. Repro is against .claude/skills/implement-issue
at main@73ed1568 (post PR #1550).

Environment

N/A — pure gh/jq/bash shell-recipe bug in an agent skill, reproducible on any platform with gh
and jq installed (verified with jq-1.7.1, gh per gh --version in this environment).

Related: #1544 (root cause discovered while implementing its fix), PR #1550 (fixed the sibling
pagination-merge bug in the same lines).

🧠 Brainstorm

Problem / context. implement-issue's legacy (comment-based) plan locator uses [ -z "$PLAN_COMMENT_ID" ] to detect "no match found" and fall through to a secondary search, then to a hard
stop. But a jq/gh api --jq filter that resolves to JSON null prints the 4-character string
null, not an empty string — so the emptiness check silently never fires, and a "no plan comment
found" issue instead attempts a REST call against comment id null, producing a confusing 404 instead
of either the intended fallback search or a clear "No implementation plan on #N" stop message.

Approaches.

  1. Widen the emptiness checks to also match the literal string null (recommended). Minimal,
    surgical: if [ -z "$PLAN_COMMENT_ID" ] || [ "$PLAN_COMMENT_ID" = null ]; then …. Applied at both
    sites in references/github-mechanics.md §2 (the fallback-trigger check and the final stop-guard).
    SKILL.md's Step 2 doesn't duplicate the fallback/stop logic (it just points at §2 as "the full
    locator"), so only github-mechanics.md needs the fix.
  2. Add // empty to each jq filter (... | last | .id // empty), so jq itself normalizes a
    missing match to a true empty string instead of the text null, keeping the existing -z checks
    correct without widening them. Arguably cleaner (fixes it at the source rather than patching every
    call site that reads the variable), but every recipe using the last | .id idiom elsewhere would
    need the same // empty suffix to stay consistent — same shape of fix, different anchor point.
  3. Leave it — the trigger (an issue with truly zero plan-marker comments AND zero checkbox
    comments) is rare for issues this skill is actually invoked on (they exist because they carry a
    plan). Rejected: cheap to fix, and the failure mode (an opaque 404 instead of "No implementation
    plan on #N") wastes real debugging time when it does happen — same reasoning fix(skills): implement-issue's plan-comment lookup can corrupt its id across paginated comments #1544 used to justify
    its own fix.

Recommendation: Approach 2 (// empty at the jq filter) is marginally more robust — it fixes the
value at its source so every consumer of $PLAN_COMMENT_ID (not just the two explicit -z checks) sees
a true empty string on no-match, matching the convention the rest of the recipe already assumes.
Approach 1 is an acceptable fallback if a null-safety review would rather see the check itself widened
without touching the jq filters #1544's PR just fixed.

📋 Spec

Goal. Make the "no plan comment found" fallback and stop-guard in implement-issue's legacy
plan-comment locator actually fire, instead of silently falling through to a null-id REST call.

Scope. .claude/skills/implement-issue/references/github-mechanics.md §2 only — the two
jq/--jq filters that resolve to .id after a select(...) | last, and the two [ -z "$PLAN_COMMENT_ID" ] guards that read their output. SKILL.md Step 2 does not duplicate this fallback
logic (it's the "no marker → fall back to the latest comment with - [ ] lines" case), so no change
needed there beyond what #1544/PR #1550 already made.

Non-goals. Re-auditing other --paginate/jq call sites in this or other skills; changing the
merge-pr skill's analogous check-runs recipe (already correct per #1535/PR #1541, and doesn't hit this
same failure mode since it filters on state, not .id off a select | last).

Design.

# Primary marker search — append `// empty` so "no match" is a true empty string, not the text "null".
PLAN_COMMENT_ID=$(gh api "repos/{owner}/{repo}/issues/$ISSUE/comments" --paginate --slurp \
  | jq -r '[.[][] | select(.body | contains("🛠️ Implementation plan"))] | last | .id // empty')

# Fallback if no marker (older/hand-written plan): latest comment that has checkbox lines.
if [ -z "$PLAN_COMMENT_ID" ]; then
  PLAN_COMMENT_ID=$(gh api "repos/{owner}/{repo}/issues/$ISSUE/comments" --paginate --slurp \
    | jq -r '[.[][] | select(.body | (contains("- [ ]") or contains("- [x]")))] | last | .id // empty')
fi

# Nothing in the body AND nothing in comments? Stop — there is no plan to execute (Autonomy contract).
[ -z "$PLAN_COMMENT_ID" ] && { echo "No implementation plan on #$ISSUE"; exit 1; }

// empty is the standard jq idiom for "null-or-missing becomes nothing" and composes cleanly with
-r: a real id still prints as a bare number, but a null result now prints nothing at all, so
$(...)'s command substitution captures a true empty string and the existing -z checks work as
originally intended — no widening needed at the call sites.

Edge cases.

  • An issue where the primary search matches nothing but the checkbox fallback matches something: the
    first -z check must correctly detect "empty" and run the fallback (this is the exact case that's
    currently broken).
  • An issue with genuinely no plan anywhere: the final stop-guard must fire with the clear message, not
    fall through to a null-id REST call.
  • An issue where the primary search does match: unaffected — // empty is a no-op when .id is a
    real integer.

Verification model. No code path to unit-test (shell recipe in a markdown skill). Verify by hand
the way #1544's Task 1 Step 3 did: run the filter against a seeded empty-match array and a
seeded matching array, confirm the old filter prints null and the // empty-suffixed one prints
nothing (jq -r '... | last | .id // empty' <<< '[]' → empty output).

🛠️ Implementation plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal. Fix implement-issue's legacy plan-comment locator so a genuine "no plan comment" case is
correctly detected (fallback search runs, then the stop-guard fires) instead of silently falling
through to a REST call against the literal comment id null.

Architecture / Tech Stack. Markdown skill definitions under .claude/skills/implement-issue/ plus
gh / gh api / jq shell recipes. No compiler, emitter, or workflow YAML is touched.

Global Constraints.

Task 1: Make null-vs-empty detection correct in the plan-comment fallback

Files: modify .claude/skills/implement-issue/references/github-mechanics.md (§2 locator, both
jq filters and their surrounding prose).

Interfaces: the documented recipe — ... | last | .id // empty piped from gh api --paginate --slurp.

  • Step 1: In references/github-mechanics.md §2, append // empty to the primary marker
    search's jq filter (... | last | .id... | last | .id // empty).
  • Step 2: Append the same // empty to the checkbox-fallback search's jq filter.
  • Step 3: Update the prose immediately below the code block to note why // empty is required
    (jq/gh api --jq prints the literal string null for a JSON null result, which the existing
    -z checks don't treat as empty).
  • Step 4: Verify by hand: run both filters against a seeded empty-match JSON array and confirm
    output is truly empty (jq -r '... | last | .id // empty' <<< '[]' prints nothing), and against a
    seeded matching array to confirm a real id still prints unaffected. Paste the verification into the
    PR description.
  • Step 5: Commit: fix(skills): treat a null jq match as empty in the plan-comment fallback.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ciCI, build, release, test infrastructurebugSomething isn't workingeffort: SSmall — ~hours to ~1 daypriority: lowTier 3 — differentiator / polish

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions