Skip to content

fix(onboard): reconcile registered extra providers#6587

Merged
cv merged 3 commits into
mainfrom
codex/6501-drop-stale-web-search-providers
Jul 9, 2026
Merged

fix(onboard): reconcile registered extra providers#6587
cv merged 3 commits into
mainfrom
codex/6501-drop-stale-web-search-providers

Conversation

@apurvvkumaria

@apurvvkumaria apurvvkumaria commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reconcile registry-recorded extra providers against the authoritative OpenShell gateway list before sandbox creation. This prevents a stale tavily-search record from breaking onboarding while preserving every healthy gateway-wide user provider. The PR replaces #6531, preserves Ho Lim's original issue work, and credits Shawn Xie's provider-reconciliation direction from #6518.

Related Issue

Fixes #6501.

Changes

  • Query provider list -g <gateway> --names once when registry extras exist.
  • Attach only exact provider names returned by a successful authoritative list.
  • Preserve all recorded providers if the gateway query fails or throws, avoiding silent user-state loss during an outage.
  • Keep local registry state unchanged; reconciliation affects only the current sandbox-create plan.
  • Preserve healthy bare brave-search, tavily-search, custom, and bridge providers without treating their names as NemoClaw-owned.
  • Add focused and spawn-level regressions for healthy providers, stale records, exact-name matching, gateway scoping, and fail-open behavior.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: this silently restores the documented provider source-of-truth contract without adding or changing a command, option, output, configuration, or remediation step.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — exact-name gateway reconciliation was independently audited; it avoids diagnostic regexes, preserves healthy user-owned providers, fails open on gateway-list failure, and does not mutate registry state.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
  • Targeted tests pass for changed behavior — 17 focused reconciliation tests and all 65 onboard integration tests passed.
  • Full npm test passes (broad runtime changes only)
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Additional exact-head validation: CLI type-check, file/commit/push hooks, commitlint, test-size/source-shape budgets, secret scan, and npm run check:diff passed. Documentation review found no update required. src/lib/onboard.ts is net-neutral.


Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

  • New Features

    • Sandbox creation now better matches configured providers to what’s actually available, reducing unexpected provider options during onboarding.
  • Bug Fixes

    • Fixed cases where stale or unavailable extra providers could still appear in sandbox setup.
    • Improved behavior when provider lookup fails, helping preserve previously configured choices instead of removing them unexpectedly.

Co-authored-by: Ho Lim <subhoya@gmail.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

A new reconcileRegisteredExtraProviders function is introduced to filter locally recorded extra providers against a live OpenShell gateway provider list, replacing the prior direct use of the registry list. It is wired into sandbox creation, re-exported from an existing module, and covered by new unit and integration tests; an existing onboarding test is updated to match.

Changes

Extra provider reconciliation against gateway

Layer / File(s) Summary
Reconciliation function and dependency types
src/lib/onboard/extra-provider-reconciliation.ts
Adds reconcileRegisteredExtraProviders and ReconcileExtraProvidersDeps, querying OpenShell for gateway-scoped provider names and filtering recorded extras against them, with fallback to the original list on failure or when no override is needed.
Wiring into sandbox creation and re-export
src/lib/onboard.ts, src/lib/onboard/sandbox-provider-cleanup.ts
onboard.ts now sources extraProviders from reconcileRegisteredExtraProviders(GATEWAY_NAME, { runOpenshell }) instead of registry.listExtraProviders(); sandbox-provider-cleanup.ts re-exports the new type and function.
Reconciliation unit and integration tests
src/lib/onboard/extra-provider-reconciliation.test.ts, test/onboard-extra-provider-reconciliation.test.ts, test/onboard.test.ts
New unit tests cover exact-name matching, stale-record omission, and failure fallback; a new integration test simulates sandbox creation excluding a stale provider; an existing onboarding test fixture/assertion is updated to drop the stale tavily-search expectation.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant onboard.ts
  participant reconcileRegisteredExtraProviders
  participant registry
  participant OpenShellGateway

  onboard.ts->>reconcileRegisteredExtraProviders: reconcileRegisteredExtraProviders(GATEWAY_NAME, { runOpenshell })
  reconcileRegisteredExtraProviders->>registry: listExtraProviders()
  reconcileRegisteredExtraProviders->>OpenShellGateway: provider list -g GATEWAY_NAME --names
  OpenShellGateway-->>reconcileRegisteredExtraProviders: gateway names or failure
  reconcileRegisteredExtraProviders-->>onboard.ts: filtered or original extraProviders
Loading

Suggested labels: bug-fix, area: onboarding, area: sandbox

Suggested reviewers: jyaunches, cv

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The fix reconciles recorded extras against the gateway list and regression tests cover the stale tavily-search case in #6501.
Out of Scope Changes check ✅ Passed The new reconciliation utility, re-export, and tests all support the onboard sandbox-provider fix and stay within scope.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: onboarding now reconciles registered extra providers during sandbox creation.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/6501-drop-stale-web-search-providers

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the codex/6501-drop-stal... branch remains at 77%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.
File main 4df9fc2 codex/6501-drop-stal... dd93942 +/-
src/lib/inferen.../vllm-models.ts 81% 66% -15%
src/lib/inference/config.ts 98% 93% -5%
src/lib/messagi.../persistence.ts 95% 92% -3%
src/lib/actions...troy-gateway.ts 10% 7% -3%
src/lib/credentials/store.ts 61% 59% -2%
src/lib/state/config-io.ts 95% 93% -2%
src/lib/inferen...time-context.ts 96% 94% -2%
src/lib/actions...dbox/destroy.ts 71% 72% +1%
src/lib/onboard...conciliation.ts 0% 44% +44%

Updated July 09, 2026 17:48 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/onboard.test.ts (1)

1880-1880: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assertion only checks tavily-search; add brave-search coverage too.

The updated regex only forbids --provider tavily-search, but the fix (and PR objective) is to remove all managed web-search providers when disabled, including brave-search. If a regression only reintroduces a stale brave-search attachment, this assertion won't catch it.

✅ Suggested strengthened assertion
-    assert.match(createCommand.command, /^(?=.*nemoclaw-start)(?!.*--provider tavily-search)/s);
+    assert.match(
+      createCommand.command,
+      /^(?=.*nemoclaw-start)(?!.*--provider (?:tavily|brave)-search)/s,
+    );
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/onboard.test.ts` at line 1880, The onboarding test assertion only
excludes `tavily-search`, so it can miss a regression where `brave-search` is
still attached. Update the check in `test/onboard.test.ts` around the
`createCommand.command` assertion to verify that both managed web-search
providers are absent when disabled, using the existing command-matching test
logic so it covers `nemoclaw-start` without `--provider tavily-search` or
`--provider brave-search`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/onboard.test.ts`:
- Line 1880: The onboarding test assertion only excludes `tavily-search`, so it
can miss a regression where `brave-search` is still attached. Update the check
in `test/onboard.test.ts` around the `createCommand.command` assertion to verify
that both managed web-search providers are absent when disabled, using the
existing command-matching test logic so it covers `nemoclaw-start` without
`--provider tavily-search` or `--provider brave-search`.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 90f1f207-902d-42e5-8f7a-5ccb35e23f75

📥 Commits

Reviewing files that changed from the base of the PR and between 4df9fc2 and cee553b.

📒 Files selected for processing (4)
  • src/lib/onboard.ts
  • src/lib/onboard/sandbox-create-plan.test.ts
  • src/lib/onboard/sandbox-create-plan.ts
  • test/onboard.test.ts

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>

@cjagwani cjagwani left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on exact head cee553b.

The clean replacement fixes the DCO defect from #6531, but it carries forward the substantive provider-ownership bug.

  1. filterManagedWebSearchExtraProviders treats bare brave-search and tavily-search entries from registry.listExtraProviders() as NemoClaw-managed solely by name. That registry also holds user-created, gateway-wide extra providers. The current Deep Agents quickstart explicitly registers bare tavily-search through credentials add and says it attaches to every later build or rebuild; NemoClaw-managed OpenClaw and Hermes search providers are instead sandbox-scoped as -brave-search or -tavily-search and come through messaging preparation. With webSearchConfig null, including Deep Agents, this patch silently drops a healthy user provider. The changed integration assertion now pins that regression.

  2. When Tavily is selected, the filter retains bare tavily-search without checking that the gateway provider exists or that NemoClaw owns it. A stale missing registry entry can therefore still reach openshell sandbox create and reproduce #6501.

Please reconcile extra-provider state using provider existence or explicit provenance instead of reserving these bare names. Add regressions proving that a healthy gateway-wide tavily-search survives for Deep Agents or null managed-web-search config, and that a missing stale bare provider is excluded under null, Brave, and Tavily selections.

  1. The required codebase-growth-guardrails check currently fails because src/lib/onboard.ts grows by one line. Please make that entrypoint net-neutral and rerun exact-head validation after updating from main.

The sole commit is GitHub Verified, the PR body now has a literal Signed-off-by declaration, and DCO passes. Those prior compliance defects are resolved. This is still a sensitive onboarding and host-glue change even though the generic gate script does not classify the nested planner file as risky.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: onboard-repair, onboard-resume, cloud-onboard
Optional E2E: brave-search

Dispatch hint: onboard-repair,onboard-resume,cloud-onboard

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • onboard-repair (medium): Required by the deterministic lifecycle-state risk plan. The changed onboarding/provider cleanup path must still repair persisted metadata, gateway/provider state, and live sandbox state without ghost or stale resources.
  • onboard-resume (medium): Required by the deterministic lifecycle-state risk plan. The changed sandbox create provider reconciliation can affect interrupted onboarding retry/resume convergence and must be validated with the live resume flow.
  • cloud-onboard (high): The PR changes the full onboarding sandbox creation path by reconciling extra providers immediately before prepareSandboxCreatePlan. A hosted live onboarding run is needed to validate real OpenShell gateway/provider interaction and sandbox lifecycle behavior end to end.

Optional E2E

  • brave-search (medium): Useful adjacent confidence because the changed extra-provider reconciliation directly affects search-provider names such as brave-search and tavily-search in sandbox create provider attachment lists.

New E2E recommendations

  • extra-provider-reconciliation (medium): Existing live jobs exercise onboarding and search provider flows, but there does not appear to be a dedicated live E2E that seeds stale and live extra-provider registry entries, verifies only live gateway-scoped provider names are passed to sandbox create, and confirms registry state is preserved for retry.
    • Suggested test: Add a live E2E target for stale extra-provider reconciliation during sandbox create, covering exact-name matching, gateway-list failure fail-open behavior, and preservation of unrelated/user-owned provider registry entries.

Dispatch hint

  • Workflow: .github/workflows/e2e.yaml
  • jobs input: onboard-repair,onboard-resume,cloud-onboard

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Recommendation

Required E2E targets: onboard-repair, onboard-resume
Optional E2E targets: None

Dispatch required E2E targets:

  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-repair
  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-resume

Workflow run

Full E2E target advisor summary

E2E Target Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E targets

  • onboard-repair: Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-repair
  • onboard-resume: Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-resume

Optional E2E targets

  • None.

Relevant changed files

  • src/lib/onboard.ts
  • src/lib/onboard/extra-provider-reconciliation.ts
  • src/lib/onboard/sandbox-provider-cleanup.ts

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Changes requested

Merge posture: Do not merge yet
Primary next action: Resolve or justify PRA-1: Add live resume/repair coverage for stale extra-provider convergence.
Open items: 0 required · 1 warning · 0 suggestions · 8 test follow-ups
Since last review: 2 prior items resolved · 0 still apply · 1 new item found

Action checklist

  • PRA-1 Resolve or justify: Add live resume/repair coverage for stale extra-provider convergence in src/lib/onboard/extra-provider-reconciliation.ts:50
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Add live resume/repair coverage for stale extra-provider convergence
  • PRA-T6 Add or justify test follow-up: Acceptance clause
  • PRA-T7 Add or justify test follow-up: Acceptance clause
  • PRA-T8 Add or justify test follow-up: Acceptance clause

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify tests src/lib/onboard/extra-provider-reconciliation.ts:50 Add or record behavior-specific runtime validation for `onboard-resume` and `onboard-repair` using a reused gateway where registry state contains stale `tavily-search` but the gateway provider list does not; verify the flow converges without attaching `tavily-search`, preserves live custom providers, and real `openshell sandbox list`/gateway provider state agree.
Review findings by urgency: 0 required fixes, 1 item to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Add live resume/repair coverage for stale extra-provider convergence

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:50
  • Category: tests
  • Problem: The changed create path reconciles registry-recorded extra providers against the live gateway list, but available tests are unit or mocked-spawn tests and do not exercise the lifecycle-state invariants against a real/reused gateway during resume or repair after stale provider state.
  • Impact: A stale registry provider could still diverge from real gateway/sandbox state during retry, resume, or repair even though the current mocked create command omits it, leaving lifecycle convergence regressions undetected.
  • Recommended action: Add or record behavior-specific runtime validation for `onboard-resume` and `onboard-repair` using a reused gateway where registry state contains stale `tavily-search` but the gateway provider list does not; verify the flow converges without attaching `tavily-search`, preserves live custom providers, and real `openshell sandbox list`/gateway provider state agree.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read the changed tests and riskPlan evidence; the shortest confirmation is to find runtime evidence or a test that exercises `onboard-resume`/`onboard-repair` against a reused gateway with stale `tavily-search` absent from `provider list -g nemoclaw --names`.
  • Missing regression test: Add an automated lifecycle regression or required E2E scenario that seeds stale `tavily-search` in NemoClaw registry state, leaves it absent from a real/reused gateway provider list, runs resume and repair convergence paths, then asserts sandbox creation/status succeeds, no `--provider tavily-search` is attached, live custom providers remain attached, and observed gateway/sandbox state matches persisted metadata.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read the changed tests and riskPlan evidence; the shortest confirmation is to find runtime evidence or a test that exercises `onboard-resume`/`onboard-repair` against a reused gateway with stale `tavily-search` absent from `provider list -g nemoclaw --names`.
  • Evidence: Risk plan marks changed onboarding files (`src/lib/onboard.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/sandbox-provider-cleanup.ts`) as tier-2 `lifecycle-state` and lists required validation floor jobs `onboard-resume` and `onboard-repair`. `src/lib/onboard/extra-provider-reconciliation.test.ts` covers exact-name filtering and fail-open branches with mocked `runOpenshell`. `test/onboard-extra-provider-reconciliation.test.ts` invokes `createSandbox` through a spawned script, but it mocks `provider list -g nemoclaw --names` and `sandbox list` rather than probing a live OpenShell gateway/sandbox state.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — Run the `onboard-repair` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/sandbox-provider-cleanup.ts`.. Deterministic regression risks require live validation: lifecycle-state. RiskPlan tier 2 marks changed onboarding files under `lifecycle-state`; unit and mocked-spawn tests cover create-command filtering and error branches, but live runtime evidence is missing for resume/repair convergence and independently probed gateway/sandbox status.
  • PRA-T2 Runtime validation — Run or add behavior-specific `onboard-resume` validation with a reused gateway where NemoClaw registry state contains stale `tavily-search` but `provider list -g nemoclaw --names` does not; verify resume converges without attaching `tavily-search`, preserves live custom providers, and observed sandbox/gateway state matches persisted metadata.. Deterministic regression risks require live validation: lifecycle-state. RiskPlan tier 2 marks changed onboarding files under `lifecycle-state`; unit and mocked-spawn tests cover create-command filtering and error branches, but live runtime evidence is missing for resume/repair convergence and independently probed gateway/sandbox status.
  • PRA-T3 Runtime validation — Run the `onboard-resume` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/sandbox-provider-cleanup.ts`.. Deterministic regression risks require live validation: lifecycle-state. RiskPlan tier 2 marks changed onboarding files under `lifecycle-state`; unit and mocked-spawn tests cover create-command filtering and error branches, but live runtime evidence is missing for resume/repair convergence and independently probed gateway/sandbox status.
  • PRA-T4 Runtime validation — Run or add behavior-specific `onboard-repair` validation for the same stale-extra-provider setup; verify repair converges, no stale provider is attached, live custom providers remain attached, and `openshell sandbox list` plus gateway provider state agree with NemoClaw metadata.. Deterministic regression risks require live validation: lifecycle-state. RiskPlan tier 2 marks changed onboarding files under `lifecycle-state`; unit and mocked-spawn tests cover create-command filtering and error branches, but live runtime evidence is missing for resume/repair convergence and independently probed gateway/sandbox status.
  • PRA-T5 Add live resume/repair coverage for stale extra-provider convergence — Add or record behavior-specific runtime validation for `onboard-resume` and `onboard-repair` using a reused gateway where registry state contains stale `tavily-search` but the gateway provider list does not; verify the flow converges without attaching `tavily-search`, preserves live custom providers, and real `openshell sandbox list`/gateway provider state agree.
  • PRA-T6 Acceptance clause — Sandbox <code>test</code> is created successfully with no web-search provider configured. — add test evidence or identify existing coverage. The spawn-level test invokes `createSandbox(null, "gpt-5.4")` with default `webSearchConfig: null` and asserts the sandbox create command omits `--provider tavily-search`; it uses mocked OpenShell behavior and creates `my-assistant`, not a live sandbox named `test`.
  • PRA-T7 Acceptance clause — <code>openshell sandbox list</code> shows <code>test</code> in a healthy/running state — add test evidence or identify existing coverage. The changed spawn test mocks `runner.runCapture` so `sandbox list` returns `my-assistant Ready`; no live `openshell sandbox list` evidence is present in the reviewed diff.
  • PRA-T8 Acceptance clause — no <code>tavily-search</code> provider is referenced anywhere in the gateway configuration (since option 1, "No web search", was selected). — add test evidence or identify existing coverage. Unit and spawn tests prove the current sandbox create command filters stale `tavily-search` when absent from `provider list -g nemoclaw --names`; they do not inspect a live gateway configuration, and the test intentionally preserves local registry state containing `tavily-search`.
Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Add live resume/repair coverage for stale extra-provider convergence

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:50
  • Category: tests
  • Problem: The changed create path reconciles registry-recorded extra providers against the live gateway list, but available tests are unit or mocked-spawn tests and do not exercise the lifecycle-state invariants against a real/reused gateway during resume or repair after stale provider state.
  • Impact: A stale registry provider could still diverge from real gateway/sandbox state during retry, resume, or repair even though the current mocked create command omits it, leaving lifecycle convergence regressions undetected.
  • Recommended action: Add or record behavior-specific runtime validation for `onboard-resume` and `onboard-repair` using a reused gateway where registry state contains stale `tavily-search` but the gateway provider list does not; verify the flow converges without attaching `tavily-search`, preserves live custom providers, and real `openshell sandbox list`/gateway provider state agree.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read the changed tests and riskPlan evidence; the shortest confirmation is to find runtime evidence or a test that exercises `onboard-resume`/`onboard-repair` against a reused gateway with stale `tavily-search` absent from `provider list -g nemoclaw --names`.
  • Missing regression test: Add an automated lifecycle regression or required E2E scenario that seeds stale `tavily-search` in NemoClaw registry state, leaves it absent from a real/reused gateway provider list, runs resume and repair convergence paths, then asserts sandbox creation/status succeeds, no `--provider tavily-search` is attached, live custom providers remain attached, and observed gateway/sandbox state matches persisted metadata.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read the changed tests and riskPlan evidence; the shortest confirmation is to find runtime evidence or a test that exercises `onboard-resume`/`onboard-repair` against a reused gateway with stale `tavily-search` absent from `provider list -g nemoclaw --names`.
  • Evidence: Risk plan marks changed onboarding files (`src/lib/onboard.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/sandbox-provider-cleanup.ts`) as tier-2 `lifecycle-state` and lists required validation floor jobs `onboard-resume` and `onboard-repair`. `src/lib/onboard/extra-provider-reconciliation.test.ts` covers exact-name filtering and fail-open branches with mocked `runOpenshell`. `test/onboard-extra-provider-reconciliation.test.ts` invokes `createSandbox` through a spawned script, but it mocks `provider list -g nemoclaw --names` and `sandbox list` rather than probing a live OpenShell gateway/sandbox state.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor (Nemotron Ultra) — Changes requested

Merge posture: Do not merge yet
Primary next action: Resolve or justify PRA-1: Overlapping PR #6518 addresses same issue with potentially conflicting approach.
Open items: 0 required · 5 warnings · 0 suggestions · 7 test follow-ups
Since last review: 0 prior items resolved · 3 still apply · 2 new items found

Action checklist

  • PRA-1 Resolve or justify: Overlapping PR fix(onboard): prune dangling extra providers before sandbox create #6518 addresses same issue with potentially conflicting approach in src/lib/onboard.ts:1
  • PRA-2 Resolve or justify: Workaround filters at create-time instead of fixing stale registry state at source in src/lib/onboard/extra-provider-reconciliation.ts:31
  • PRA-3 Resolve or justify: Lifecycle-state invariants lack regression test coverage; requiredJobs onboard-repair/onboard-resume not verified in src/lib/onboard/extra-provider-reconciliation.ts:1
  • PRA-4 Resolve or justify: Silent fail-open masks reconciliation skip during gateway outage in src/lib/onboard/extra-provider-reconciliation.ts:68
  • PRA-5 Resolve or justify: Missing mocked retry test for prepareSandboxCreatePlan provider list stability in src/lib/onboard.ts:2770
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Lifecycle-state invariants lack regression test coverage; requiredJobs onboard-repair/onboard-resume not verified
  • PRA-T4 Add or justify test follow-up: Missing mocked retry test for prepareSandboxCreatePlan provider list stability
  • PRA-T5 Add or justify test follow-up: Acceptance clause
  • PRA-T6 Add or justify test follow-up: extra-provider-reconciliation.ts:68 catch { return recorded } — silent fail-open on gateway throw
  • PRA-T7 Add or justify test follow-up: stale extraProviders remain in registry (workaround not fixing source)

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify scope src/lib/onboard.ts:1 Coordinate with PR #6518 author to determine which approach is canonical. One PR should be superseded or they should be merged with a unified design.
PRA-2 Resolve/justify acceptance src/lib/onboard/extra-provider-reconciliation.ts:31 Document the workaround limitation. Consider follow-up to make --fresh purge extraProviders or add gateway-side prune on sandbox create. The current fix is acceptable as a safety net.
PRA-3 Resolve/justify tests src/lib/onboard/extra-provider-reconciliation.ts:1 Run onboard-repair and onboard-resume E2E jobs in PR CI and link results. If not feasible, add mocked retry test verifying prepareSandboxCreatePlan produces stable provider list across retries with unchanged webSearchConfig.
PRA-4 Resolve/justify security src/lib/onboard/extra-provider-reconciliation.ts:68 Add a structured warning log (e.g., console.warn with redacted gateway name) when reconciliation is skipped due to gateway error or throw. Consider emitting a metric/counter for monitoring.
PRA-5 Resolve/justify tests src/lib/onboard.ts:2770 Add a unit/mocked test: call prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers), assert identical createArgs.extraProviders. This validates invariant 1 without full E2E.
Review findings by urgency: 0 required fixes, 5 items to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Overlapping PR #6518 addresses same issue with potentially conflicting approach

PRA-2 Resolve/justify — Workaround filters at create-time instead of fixing stale registry state at source

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:31
  • Category: acceptance
  • Problem: Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501 root cause: reused gateway has stale extraProviders from prior onboarding. This PR filters them out at create-time via reconcileRegisteredExtraProviders. The source (registry.listExtraProviders() returning stale entries) is not fixed. --fresh flag doesn't purge them.
  • Impact: Stale providers remain in registry and could affect other flows that read extraProviders. Workaround is effective for sandbox create but incomplete for full cleanup.
  • Recommended action: Document the workaround limitation. Consider follow-up to make --fresh purge extraProviders or add gateway-side prune on sandbox create. The current fix is acceptable as a safety net.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Missing regression test: Test that --fresh onboarding with prior stale extraProviders succeeds (covered by new unit tests and onboard.test.ts assertion change).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Evidence: Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501 description: 'reused gateway has stale state from a prior onboarding attempt' extra-provider-reconciliation.ts JSDoc notes workaround: 'Local registry state is never mutated: a provider omitted for this create remains available for later retry'

PRA-3 Resolve/justify — Lifecycle-state invariants lack regression test coverage; requiredJobs onboard-repair/onboard-resume not verified

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:1
  • Category: tests
  • Problem: The PR modifies provider list for sandbox create which is part of the lifecycle-state risk family. Three invariants must hold but new unit tests only cover filtering branches, not convergence invariants. Required E2E jobs (onboard-repair, onboard-resume) are not confirmed executed in PR CI.
  • Impact: If invariants regress, onboarding retries/resumes could leave ghost resources, report incorrect status, or delete unrelated sandboxes. No automated evidence in this PR that invariants hold.
  • Recommended action: Run onboard-repair and onboard-resume E2E jobs in PR CI and link results. If not feasible, add mocked retry test verifying prepareSandboxCreatePlan produces stable provider list across retries with unchanged webSearchConfig.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check PR CI logs for onboard-repair and onboard-resume job runs. If absent, these are missing validation gates.
  • Missing regression test: Mocked retry test: call prepareSandboxCreatePlan twice with same webSearchConfig and extraProviders (including stale managed providers), assert identical createArgs. Or E2E job evidence.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check PR CI logs for onboard-repair and onboard-resume job runs. If absent, these are missing validation gates.
  • Evidence: riskPlan from pr_review_scope_risk_context shows lifecycle-state family with 3 invariants and requiredJobs onboard-repair/onboard-resume testDepth verdict: runtime_validation_recommended with suggestedTests listing both E2E jobs New unit tests only cover filterManagedWebSearchExtraProviders branches, not convergence invariants

PRA-4 Resolve/justify — Silent fail-open masks reconciliation skip during gateway outage

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:68
  • Category: security
  • Problem: When runOpenshell throws or returns non-zero status, reconcileRegisteredExtraProviders returns all recorded providers without any logging, warning, or metric emission. Operators cannot detect that gateway reconciliation was bypassed.
  • Impact: Operational blind spot: during gateway degradation, stale providers may be attached to new sandboxes with no audit trail. Security-relevant because it hides a trust-boundary validation gap.
  • Recommended action: Add a structured warning log (e.g., console.warn with redacted gateway name) when reconciliation is skipped due to gateway error or throw. Consider emitting a metric/counter for monitoring.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check extra-provider-reconciliation.ts lines 64-70: catch block and status !== 0 branch both return recorded silently.
  • Missing regression test: Test that a warning is logged when runOpenshell throws or returns non-zero status.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check extra-provider-reconciliation.ts lines 64-70: catch block and status !== 0 branch both return recorded silently.
  • Evidence: Category 5 verdict: WARNING - silent fail-open Category 9 verdict: WARNING - holistic posture includes silent fail-open Code at lines 64-70: catch { return recorded } and if (result.status !== 0) return recorded

PRA-5 Resolve/justify — Missing mocked retry test for prepareSandboxCreatePlan provider list stability

  • Location: src/lib/onboard.ts:2770
  • Category: tests
  • Problem: reconcileRegisteredExtraProviders is called inside prepareSandboxCreatePlan. On retry/resume, the gateway provider list must return the same filtered result for convergence. No test calls prepareSandboxCreatePlan twice with same inputs and asserts identical createArgs.
  • Impact: If reconciliation behaves non-deterministically across retries (e.g., gateway list changes between calls), onboarding resume could produce different sandbox create args, violating lifecycle-state invariant 1.
  • Recommended action: Add a unit/mocked test: call prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers), assert identical createArgs.extraProviders. This validates invariant 1 without full E2E.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check test/onboard-extra-provider-reconciliation.test.ts and src/lib/onboard/extra-provider-reconciliation.test.ts — neither tests prepareSandboxCreatePlan retry stability.
  • Missing regression test: Mocked retry test: prepareSandboxCreatePlan(webSearchConfig, extraProvidersWithStale) called twice, assert createArgs.extraProviders identical both times.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check test/onboard-extra-provider-reconciliation.test.ts and src/lib/onboard/extra-provider-reconciliation.test.ts — neither tests prepareSandboxCreatePlan retry stability.
  • Evidence: riskPlan lifecycle-state invariant 1: 'partial failure and retry converge without ghost resources or stale ports' testDepth verdict: runtime_validation_recommended with suggestedTests listing onboard-resume prepareSandboxCreatePlan at onboard.ts:2770 calls reconcileRegisteredExtraProviders No existing test verifies idempotency of provider list across retries

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — Run the `onboard-repair` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/sandbox-provider-cleanup.ts`.. Deterministic regression risks require live validation: lifecycle-state.
  • PRA-T2 Runtime validation — Run the `onboard-resume` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/sandbox-provider-cleanup.ts`.. Deterministic regression risks require live validation: lifecycle-state.
  • PRA-T3 Lifecycle-state invariants lack regression test coverage; requiredJobs onboard-repair/onboard-resume not verified — Run onboard-repair and onboard-resume E2E jobs in PR CI and link results. If not feasible, add mocked retry test verifying prepareSandboxCreatePlan produces stable provider list across retries with unchanged webSearchConfig.
  • PRA-T4 Missing mocked retry test for prepareSandboxCreatePlan provider list stability — Add a unit/mocked test: call prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers), assert identical createArgs.extraProviders. This validates invariant 1 without full E2E.
  • PRA-T5 Acceptance clause — --fresh should reset gateway-side provider config or sandbox-create should tolerate/prune dangling references — add test evidence or identify existing coverage. Create-time reconciliation works (sandbox-create tolerates), but --fresh does not purge registry extraProviders; stale entries remain for other callers
  • PRA-T6 extra-provider-reconciliation.ts:68 catch { return recorded } — silent fail-open on gateway throw — unit test "preserves every recorded extra when the gateway list throws" (line 71-84 in extra-provider-reconciliation.test.ts). Category 5 verdict: WARNING - silent fail-open Category 9 verdict: WARNING - holistic posture includes silent fail-open Code at lines 64-70: catch { return recorded } and if (result.status !== 0) return recorded
  • PRA-T7 stale extraProviders remain in registry (workaround not fixing source) — test that --fresh onboarding with prior stale extraProviders succeeds (covered by new unit tests and onboard.test.ts assertion change). Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501 description: 'reused gateway has stale state from a prior onboarding attempt' extra-provider-reconciliation.ts JSDoc notes workaround: 'Local registry state is never mutated: a provider omitted for this create remains available for later retry'
Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Overlapping PR #6518 addresses same issue with potentially conflicting approach

PRA-2 Resolve/justify — Workaround filters at create-time instead of fixing stale registry state at source

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:31
  • Category: acceptance
  • Problem: Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501 root cause: reused gateway has stale extraProviders from prior onboarding. This PR filters them out at create-time via reconcileRegisteredExtraProviders. The source (registry.listExtraProviders() returning stale entries) is not fixed. --fresh flag doesn't purge them.
  • Impact: Stale providers remain in registry and could affect other flows that read extraProviders. Workaround is effective for sandbox create but incomplete for full cleanup.
  • Recommended action: Document the workaround limitation. Consider follow-up to make --fresh purge extraProviders or add gateway-side prune on sandbox create. The current fix is acceptable as a safety net.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Missing regression test: Test that --fresh onboarding with prior stale extraProviders succeeds (covered by new unit tests and onboard.test.ts assertion change).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Evidence: Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501 description: 'reused gateway has stale state from a prior onboarding attempt' extra-provider-reconciliation.ts JSDoc notes workaround: 'Local registry state is never mutated: a provider omitted for this create remains available for later retry'

PRA-3 Resolve/justify — Lifecycle-state invariants lack regression test coverage; requiredJobs onboard-repair/onboard-resume not verified

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:1
  • Category: tests
  • Problem: The PR modifies provider list for sandbox create which is part of the lifecycle-state risk family. Three invariants must hold but new unit tests only cover filtering branches, not convergence invariants. Required E2E jobs (onboard-repair, onboard-resume) are not confirmed executed in PR CI.
  • Impact: If invariants regress, onboarding retries/resumes could leave ghost resources, report incorrect status, or delete unrelated sandboxes. No automated evidence in this PR that invariants hold.
  • Recommended action: Run onboard-repair and onboard-resume E2E jobs in PR CI and link results. If not feasible, add mocked retry test verifying prepareSandboxCreatePlan produces stable provider list across retries with unchanged webSearchConfig.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check PR CI logs for onboard-repair and onboard-resume job runs. If absent, these are missing validation gates.
  • Missing regression test: Mocked retry test: call prepareSandboxCreatePlan twice with same webSearchConfig and extraProviders (including stale managed providers), assert identical createArgs. Or E2E job evidence.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check PR CI logs for onboard-repair and onboard-resume job runs. If absent, these are missing validation gates.
  • Evidence: riskPlan from pr_review_scope_risk_context shows lifecycle-state family with 3 invariants and requiredJobs onboard-repair/onboard-resume testDepth verdict: runtime_validation_recommended with suggestedTests listing both E2E jobs New unit tests only cover filterManagedWebSearchExtraProviders branches, not convergence invariants

PRA-4 Resolve/justify — Silent fail-open masks reconciliation skip during gateway outage

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:68
  • Category: security
  • Problem: When runOpenshell throws or returns non-zero status, reconcileRegisteredExtraProviders returns all recorded providers without any logging, warning, or metric emission. Operators cannot detect that gateway reconciliation was bypassed.
  • Impact: Operational blind spot: during gateway degradation, stale providers may be attached to new sandboxes with no audit trail. Security-relevant because it hides a trust-boundary validation gap.
  • Recommended action: Add a structured warning log (e.g., console.warn with redacted gateway name) when reconciliation is skipped due to gateway error or throw. Consider emitting a metric/counter for monitoring.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check extra-provider-reconciliation.ts lines 64-70: catch block and status !== 0 branch both return recorded silently.
  • Missing regression test: Test that a warning is logged when runOpenshell throws or returns non-zero status.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check extra-provider-reconciliation.ts lines 64-70: catch block and status !== 0 branch both return recorded silently.
  • Evidence: Category 5 verdict: WARNING - silent fail-open Category 9 verdict: WARNING - holistic posture includes silent fail-open Code at lines 64-70: catch { return recorded } and if (result.status !== 0) return recorded

PRA-5 Resolve/justify — Missing mocked retry test for prepareSandboxCreatePlan provider list stability

  • Location: src/lib/onboard.ts:2770
  • Category: tests
  • Problem: reconcileRegisteredExtraProviders is called inside prepareSandboxCreatePlan. On retry/resume, the gateway provider list must return the same filtered result for convergence. No test calls prepareSandboxCreatePlan twice with same inputs and asserts identical createArgs.
  • Impact: If reconciliation behaves non-deterministically across retries (e.g., gateway list changes between calls), onboarding resume could produce different sandbox create args, violating lifecycle-state invariant 1.
  • Recommended action: Add a unit/mocked test: call prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers), assert identical createArgs.extraProviders. This validates invariant 1 without full E2E.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check test/onboard-extra-provider-reconciliation.test.ts and src/lib/onboard/extra-provider-reconciliation.test.ts — neither tests prepareSandboxCreatePlan retry stability.
  • Missing regression test: Mocked retry test: prepareSandboxCreatePlan(webSearchConfig, extraProvidersWithStale) called twice, assert createArgs.extraProviders identical both times.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check test/onboard-extra-provider-reconciliation.test.ts and src/lib/onboard/extra-provider-reconciliation.test.ts — neither tests prepareSandboxCreatePlan retry stability.
  • Evidence: riskPlan lifecycle-state invariant 1: 'partial failure and retry converge without ghost resources or stale ports' testDepth verdict: runtime_validation_recommended with suggestedTests listing onboard-resume prepareSandboxCreatePlan at onboard.ts:2770 calls reconcileRegisteredExtraProviders No existing test verifies idempotency of provider list across retries

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>

@cjagwani cjagwani left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewing exact head 6393639.

The net-neutral follow-up resolves the top-level src/lib/onboard.ts growth CI failure, and that required check is now green. The provider-semantics blockers remain unchanged.

A focused comparison against overlapping #6518 confirms that #6518 has the correct core ownership model: probe every registry-recorded extra provider against the selected gateway, retain confirmed providers, prune only an explicit provider-not-found result, and remove that provably stale record from registry state. In contrast, this PR still reserves bare brave-search and tavily-search by name. It drops a healthy gateway-wide Tavily provider when managed search is disabled, yet can retain a missing stale Tavily provider when Tavily is selected.

Please defer this name-based implementation. Do not stack it on #6518: the reconciliation layer could correctly retain a healthy confirmed tavily-search, after which this filter would incorrectly remove it based on webSearchConfig. If #6518 becomes the delivery vehicle, this PR should close as superseded after that fix lands. If this fresher branch must be used, replace the filter with the existence-based reconciliation design and carry forward credit; also incorporate the narrower provider-not-found matching that #6518 still needs.

Both exact-head PR advisors independently say merge_after_fixes. They also require planner and test monolith growth to be extracted or offset, and they require onboard-repair and onboard-resume evidence after the source-level fix; the E2E advisor additionally requires cloud-onboard. Do not spend those live runs on the current incorrect implementation.

No merge performed.

@apurvvkumaria apurvvkumaria changed the title fix(onboard): drop stale web-search providers fix(onboard): reconcile registered extra providers Jul 9, 2026
@apurvvkumaria

Copy link
Copy Markdown
Collaborator Author

Addressed the valid provider-ownership and stale-record blockers on exact head dd93942d98292f926bad8a81a5f2265c300b8e27.

  • Removed all name-based brave-search / tavily-search filtering and the webSearchConfig planner plumbing.
  • Registry extras are treated as user-owned and reconciled with one gateway-scoped provider list --names query.
  • A successful query uses exact full-name membership; a nonzero result or exception fails open and preserves every recorded provider.
  • Reconciliation does not mutate local registry state.
  • Regressions prove healthy bare Brave/Tavily, custom, and bridge names survive; stale missing names are excluded; prefixes do not match; gateway failures preserve state; and the real sandbox-create command attaches only live extras.

Validation: 17/17 focused tests, 65/65 onboard integration tests, CLI type-check, hooks, commitlint, source/test budgets, secret scan, and npm run check:diff pass. src/lib/onboard.ts is net-neutral. The corrective commit is GitHub Verified and credits Shawn Xie as co-author; Ho Lim remains credited on the original implementation commit.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/lib/onboard/extra-provider-reconciliation.ts (1)

51-80: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Missing test coverage for guard/validation paths.

None of the unit tests in extra-provider-reconciliation.test.ts exercise the gatewayName missing throw or the OPENSHELL_GATEWAY_ENDPOINT override guard interaction (Lines 57-58). Given the guard's unusual uncaught-throw behavior flagged above, a regression test would catch this before it reaches production.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/extra-provider-reconciliation.ts` around lines 51 - 80, Add
unit tests for the guard/validation paths in reconcileRegisteredExtraProviders:
cover the missing gatewayName throw and the
assertNoOpenShellGatewayEndpointOverride interaction before runOpenshell is
called. Use the reconcileRegisteredExtraProviders symbol with mocked
listExtraProviders and runOpenshell deps to verify the function throws when
gatewayName is empty, and separately verify the endpoint override guard prevents
execution as expected in extra-provider-reconciliation.test.ts.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/onboard/extra-provider-reconciliation.ts`:
- Around line 51-80: Add unit tests for the guard/validation paths in
reconcileRegisteredExtraProviders: cover the missing gatewayName throw and the
assertNoOpenShellGatewayEndpointOverride interaction before runOpenshell is
called. Use the reconcileRegisteredExtraProviders symbol with mocked
listExtraProviders and runOpenshell deps to verify the function throws when
gatewayName is empty, and separately verify the endpoint override guard prevents
execution as expected in extra-provider-reconciliation.test.ts.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e6ebb17a-15ba-4f27-bc43-d64c56bec346

📥 Commits

Reviewing files that changed from the base of the PR and between 6393639 and dd93942.

📒 Files selected for processing (6)
  • src/lib/onboard.ts
  • src/lib/onboard/extra-provider-reconciliation.test.ts
  • src/lib/onboard/extra-provider-reconciliation.ts
  • src/lib/onboard/sandbox-provider-cleanup.ts
  • test/onboard-extra-provider-reconciliation.test.ts
  • test/onboard.test.ts

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29037459044
Workflow ref: codex/6501-drop-stale-web-search-providers
Requested targets: (default — all supported)
Requested jobs: onboard-repair,onboard-resume,cloud-onboard
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@cv
cv merged commit ac9e1bc into main Jul 9, 2026
123 of 129 checks passed
@cv
cv deleted the codex/6501-drop-stale-web-search-providers branch July 9, 2026 18:21
@cjagwani

cjagwani commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Post-merge follow-up on shipped head dd93942 / verified squash ac9e1bc:

The one-shot provider snapshot is not authoritative for the full gateway. In the pinned OpenShell v0.0.72 source, provider list defaults to --limit 100 and --offset 0. This implementation calls provider list -g --names once, then treats only that first page as the complete name set. On a gateway with more than 100 providers, any healthy registry-recorded extra outside page one is silently omitted from sandbox create, contradicting the stated preservation contract.

This also revives the list-snapshot design that #6518 discarded after cli-test-shards (5) exposed the same unsafe assumption for an exit-0 empty response. The merged tests use only a handful of names and cannot detect truncation or an incomplete success response.

Please track an immediate v0.0.79 follow-up before release: either probe each recorded extra with gateway-scoped provider get and prune only an explicit provider-not-found result, or exhaust pagination without treating a partial page as complete. Add a regression with more than 100 gateway providers where the recorded live extra is on a later page, plus an exit-0 empty/incomplete-response case. Preserve operator-visible diagnostics when reconciliation cannot prove a provider missing.

No merge was performed by this babysitter.

@jyaunches jyaunches mentioned this pull request Jul 9, 2026
21 tasks
cv pushed a commit that referenced this pull request Jul 9, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Adds the pre-tag v0.0.79 release notes entry to
`docs/about/release-notes.mdx` so the release plan can be generated
after docs merge.
The entry summarizes the merged v0.0.79 release train across inference,
diagnostics, runtime hardening, policies, onboarding recovery, and
release validation.

## Changes
- Added the v0.0.79 release notes section with linked follow-up
documentation for OpenRouter onboarding, managed vLLM changes,
completion and logging, Deep Agents runtime limits, policy updates,
onboarding recovery, and release validation.
- Source summary:
- #6461 -> `docs/about/release-notes.mdx`: Documents OpenRouter
onboarding support and links to inference/provider references.
- #6271 and #6272 -> `docs/about/release-notes.mdx`: Documents shell
completion and structured logging highlights.
- #6465, #6539, #6570, and #6528 -> `docs/about/release-notes.mdx`:
Documents status route-drift, orphaned sandbox, gateway cleanup, and DGX
Spark express-install diagnostics.
- #6523, #6551, #6484, #6488, #6324, and #6542 ->
`docs/about/release-notes.mdx`: Documents managed vLLM, Qwen3.6 tool
parser, compaction, and timeout/readiness improvements.
- #6559, #6538, #6560, #6568, #6552, #6567, and #6587 ->
`docs/about/release-notes.mdx`: Documents runtime, credential, proxy,
PID namespace, TOML, and provider-state hardening.
- #6541, #5415, #6246, #6496, and #6573 ->
`docs/about/release-notes.mdx`: Documents GitHub policy, Gmail policy,
MCP allowlist, WhatsApp, and messaging-variant updates.
- #6253, #6572, #6444, #6536, and #5860 ->
`docs/about/release-notes.mdx`: Documents onboarding resume and
create-step recovery improvements.
- #6508, #6527, #5506, #6588, #6446, #6447, #6582, #6296, #6367, #6397,
and #6505 -> `docs/about/release-notes.mdx`: Documents docs,
release-risk, and E2E validation updates.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: Release-note prose only.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: Tests
not applicable, release-note prose only.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

Docs validation note: `npm run docs:check-agent-variants && npm run
docs:check-routes && git diff --check` passed. Full `npm run docs` is
currently blocked before Fern validation because the pinned
`fern-api@5.65.2` package is unavailable from npm (`ETARGET No matching
version found`).

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Added release notes for v0.0.79 with a new summary of recent
improvements, including onboarding and inference options, operator/CLI
diagnostics, sandbox recovery hardening, runtime limits, network policy
behavior, and release validation updates.
  * Added updated references and links for the latest release.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
cv added a commit that referenced this pull request Jul 10, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Fixes the post-merge #6587 gap where one paginated provider-list page
could omit healthy registry extras beyond entry 100. Reconciliation now
uses bounded, gateway-scoped per-provider probes, recognizes the exact
wrapped OpenShell #6501 diagnostic, and preserves every indeterminate
result.

## Related Issue
Follow-up to #6501 and #6587. Supersedes #6518.

## Changes
- Probe every recorded extra with argv-safe `provider get -g <gateway>
<name>` instead of treating one list page as authoritative.
- Omit only exit-1 diagnostics that bind the exact, case-sensitive
quoted provider name to a tightly anchored not-found shape, including
the wrapped not-found-and-unrecognized form reported in #6501.
- Fail open on gateway, transport, authentication, timeout, signal,
ambiguous, name-mismatched, or command-spoofed output.
- Read canonical stderr/stdout without duplicating Node's composite
`output` array, and treat process errors or capture-limit diagnostics as
indeterminate.
- Bound each probe to 5 seconds and 64 KiB, cap total reconciliation
time at 15 seconds, and never mutate the local registry.
- Cover stale providers after index 100, the literal wrapped issue
diagnostic in a realistic `spawnSync` result, capture truncation,
mixed-case names, multi-line diagnostic spoofing, gateway failures,
aggregate timeout, and stable repeated sandbox-create provider
arguments.

The create-time filter is intentionally non-destructive. A provider
omitted from one sandbox create remains in the user-owned registry for a
later retry; `--fresh` does not purge it. Indeterminate probes preserve
the attachment, and the final `sandbox create` remains authoritative and
reports any concrete attachment failure. This avoids deleting healthy
user configuration because of a transient gateway or transport failure.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: internal provider-state
reconciliation only; no CLI, prompt, configuration, policy, or required
user action changes.
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: independent
nine-category security review passed after exact-name, whole-diagnostic,
subprocess-bound, and fail-open checks.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: CLI
unit 14/14 and spawn integration 1/1 passed; the spawn test uses a
realistic composite process result, executes two create attempts, and
proves identical filtered provider arguments.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: not applicable to this
focused reconciliation change; `npm run typecheck:cli`, `npm run
check:diff`, source-shape, test-size, and test-conditional scans passed.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

Exact-head live validation for `onboard-repair`, `onboard-resume`, and
`cloud-onboard` passed at [workflow run
29048610487](https://github.com/NVIDIA/NemoClaw/actions/runs/29048610487)
for head `a0ea6e60f98c6e15a6084dba7a5ba9ff1e89215a`.

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Onboarding and repair now detect and remove stale extra-provider
records.
* Resume operations can explicitly recreate an existing sandbox while
preserving its registry entry.
* Sandbox creation and retries now use a consistent, deduplicated
provider list.

* **Bug Fixes**
* Providers are retained when availability checks are inconclusive,
preventing accidental removal.
* Reconciliation now avoids relying on potentially stale provider-list
snapshots and handles diagnostic variations more reliably.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Reconcile registry-recorded extra providers against the authoritative
OpenShell gateway list before sandbox creation. This prevents a stale
`tavily-search` record from breaking onboarding while preserving every
healthy gateway-wide user provider. The PR replaces NVIDIA#6531, preserves Ho
Lim's original issue work, and credits Shawn Xie's
provider-reconciliation direction from NVIDIA#6518.

## Related Issue

Fixes NVIDIA#6501.

## Changes

- Query `provider list -g <gateway> --names` once when registry extras
exist.
- Attach only exact provider names returned by a successful
authoritative list.
- Preserve all recorded providers if the gateway query fails or throws,
avoiding silent user-state loss during an outage.
- Keep local registry state unchanged; reconciliation affects only the
current sandbox-create plan.
- Preserve healthy bare `brave-search`, `tavily-search`, custom, and
bridge providers without treating their names as NemoClaw-owned.
- Add focused and spawn-level regressions for healthy providers, stale
records, exact-name matching, gateway scoping, and fail-open behavior.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: this silently restores the
documented provider source-of-truth contract without adding or changing
a command, option, output, configuration, or remediation step.
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — exact-name gateway reconciliation was independently audited;
it avoids diagnostic regexes, preserves healthy user-owned providers,
fails open on gateway-list failure, and does not mutate registry state.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Git hooks passed during commit and push, or `npx prek run
--from-ref main --to-ref HEAD` passes
- [x] Targeted tests pass for changed behavior — 17 focused
reconciliation tests and all 65 onboard integration tests passed.
- [ ] Full `npm test` passes (broad runtime changes only)
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

Additional exact-head validation: CLI type-check, file/commit/push
hooks, commitlint, test-size/source-shape budgets, secret scan, and `npm
run check:diff` passed. Documentation review found no update required.
`src/lib/onboard.ts` is net-neutral.

---
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Sandbox creation now better matches configured providers to what’s
actually available, reducing unexpected provider options during
onboarding.

* **Bug Fixes**
* Fixed cases where stale or unavailable extra providers could still
appear in sandbox setup.
* Improved behavior when provider lookup fails, helping preserve
previously configured choices instead of removing them unexpectedly.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Co-authored-by: Ho Lim <subhoya@gmail.com>
Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Adds the pre-tag v0.0.79 release notes entry to
`docs/about/release-notes.mdx` so the release plan can be generated
after docs merge.
The entry summarizes the merged v0.0.79 release train across inference,
diagnostics, runtime hardening, policies, onboarding recovery, and
release validation.

## Changes
- Added the v0.0.79 release notes section with linked follow-up
documentation for OpenRouter onboarding, managed vLLM changes,
completion and logging, Deep Agents runtime limits, policy updates,
onboarding recovery, and release validation.
- Source summary:
- NVIDIA#6461 -> `docs/about/release-notes.mdx`: Documents OpenRouter
onboarding support and links to inference/provider references.
- NVIDIA#6271 and NVIDIA#6272 -> `docs/about/release-notes.mdx`: Documents shell
completion and structured logging highlights.
- NVIDIA#6465, NVIDIA#6539, NVIDIA#6570, and NVIDIA#6528 -> `docs/about/release-notes.mdx`:
Documents status route-drift, orphaned sandbox, gateway cleanup, and DGX
Spark express-install diagnostics.
- NVIDIA#6523, NVIDIA#6551, NVIDIA#6484, NVIDIA#6488, NVIDIA#6324, and NVIDIA#6542 ->
`docs/about/release-notes.mdx`: Documents managed vLLM, Qwen3.6 tool
parser, compaction, and timeout/readiness improvements.
- NVIDIA#6559, NVIDIA#6538, NVIDIA#6560, NVIDIA#6568, NVIDIA#6552, NVIDIA#6567, and NVIDIA#6587 ->
`docs/about/release-notes.mdx`: Documents runtime, credential, proxy,
PID namespace, TOML, and provider-state hardening.
- NVIDIA#6541, NVIDIA#5415, NVIDIA#6246, NVIDIA#6496, and NVIDIA#6573 ->
`docs/about/release-notes.mdx`: Documents GitHub policy, Gmail policy,
MCP allowlist, WhatsApp, and messaging-variant updates.
- NVIDIA#6253, NVIDIA#6572, NVIDIA#6444, NVIDIA#6536, and NVIDIA#5860 ->
`docs/about/release-notes.mdx`: Documents onboarding resume and
create-step recovery improvements.
- NVIDIA#6508, NVIDIA#6527, NVIDIA#5506, NVIDIA#6588, NVIDIA#6446, NVIDIA#6447, NVIDIA#6582, NVIDIA#6296, NVIDIA#6367, NVIDIA#6397,
and NVIDIA#6505 -> `docs/about/release-notes.mdx`: Documents docs,
release-risk, and E2E validation updates.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: Release-note prose only.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: Tests
not applicable, release-note prose only.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

Docs validation note: `npm run docs:check-agent-variants && npm run
docs:check-routes && git diff --check` passed. Full `npm run docs` is
currently blocked before Fern validation because the pinned
`fern-api@5.65.2` package is unavailable from npm (`ETARGET No matching
version found`).

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Added release notes for v0.0.79 with a new summary of recent
improvements, including onboarding and inference options, operator/CLI
diagnostics, sandbox recovery hardening, runtime limits, network policy
behavior, and release validation updates.
  * Added updated references and links for the latest release.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Fixes the post-merge NVIDIA#6587 gap where one paginated provider-list page
could omit healthy registry extras beyond entry 100. Reconciliation now
uses bounded, gateway-scoped per-provider probes, recognizes the exact
wrapped OpenShell NVIDIA#6501 diagnostic, and preserves every indeterminate
result.

## Related Issue
Follow-up to NVIDIA#6501 and NVIDIA#6587. Supersedes NVIDIA#6518.

## Changes
- Probe every recorded extra with argv-safe `provider get -g <gateway>
<name>` instead of treating one list page as authoritative.
- Omit only exit-1 diagnostics that bind the exact, case-sensitive
quoted provider name to a tightly anchored not-found shape, including
the wrapped not-found-and-unrecognized form reported in NVIDIA#6501.
- Fail open on gateway, transport, authentication, timeout, signal,
ambiguous, name-mismatched, or command-spoofed output.
- Read canonical stderr/stdout without duplicating Node's composite
`output` array, and treat process errors or capture-limit diagnostics as
indeterminate.
- Bound each probe to 5 seconds and 64 KiB, cap total reconciliation
time at 15 seconds, and never mutate the local registry.
- Cover stale providers after index 100, the literal wrapped issue
diagnostic in a realistic `spawnSync` result, capture truncation,
mixed-case names, multi-line diagnostic spoofing, gateway failures,
aggregate timeout, and stable repeated sandbox-create provider
arguments.

The create-time filter is intentionally non-destructive. A provider
omitted from one sandbox create remains in the user-owned registry for a
later retry; `--fresh` does not purge it. Indeterminate probes preserve
the attachment, and the final `sandbox create` remains authoritative and
reports any concrete attachment failure. This avoids deleting healthy
user configuration because of a transient gateway or transport failure.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: internal provider-state
reconciliation only; no CLI, prompt, configuration, policy, or required
user action changes.
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: independent
nine-category security review passed after exact-name, whole-diagnostic,
subprocess-bound, and fail-open checks.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: CLI
unit 14/14 and spawn integration 1/1 passed; the spawn test uses a
realistic composite process result, executes two create attempts, and
proves identical filtered provider arguments.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: not applicable to this
focused reconciliation change; `npm run typecheck:cli`, `npm run
check:diff`, source-shape, test-size, and test-conditional scans passed.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

Exact-head live validation for `onboard-repair`, `onboard-resume`, and
`cloud-onboard` passed at [workflow run
29048610487](https://github.com/NVIDIA/NemoClaw/actions/runs/29048610487)
for head `a0ea6e60f98c6e15a6084dba7a5ba9ff1e89215a`.

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Onboarding and repair now detect and remove stale extra-provider
records.
* Resume operations can explicitly recreate an existing sandbox while
preserving its registry entry.
* Sandbox creation and retries now use a consistent, deduplicated
provider list.

* **Bug Fixes**
* Providers are retained when availability checks are inconclusive,
preventing accidental removal.
* Reconciliation now avoids relying on potentially stale provider-list
snapshots and handles diagnostic variations more reliably.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v0.0.79 Release target

Projects

None yet

4 participants