[TypeSpec Authoring] Make it more explicit that versioning evolution should use agentic search#16460
Open
haolingdong-msft wants to merge 6 commits into
Open
[TypeSpec Authoring] Make it more explicit that versioning evolution should use agentic search#16460haolingdong-msft wants to merge 6 commits into
haolingdong-msft wants to merge 6 commits into
Conversation
Resolves the contradiction where the General Fallback tells the agent to use the MCP tool when agentic search fails, while Case 3 forbids the MCP tool -- previously dead-ending the API version evolution workflow when all reference URLs were unreachable. Case 3 now explicitly falls back to the MCP tool result (per the General Fallback) instead of dead-ending, keeps agentic search as the required primary path, and clarifies the concrete example-migration/readme steps are mandatory regardless of grounding path. Fixes #16317 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4912c91-426b-4cb5-8bd2-5ef89f4158fc
haolingdong-msft
requested review from
a team and
catalinaperalta
as code owners
July 23, 2026 03:59
|
Azure Pipelines: 66 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
Resolves a contradiction in the azure-typespec-author skill reference docs where Case 3 (API Version Evolution) could dead-end when agentic search is unavailable by defining an explicit degraded path.
Changes:
- Updates Case 3 guidance to treat agentic search as the primary grounding mechanism.
- Adds a Case 3-specific fallback to use the MCP tool result when agentic search fails, aligning with the General Fallback.
- Clarifies that the concrete Case 3 file-migration steps remain mandatory even in degraded mode.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Member
Author
|
benchmark looks good: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6605975&view=results |
…or version evolution Version-evolution cases (001005-001008) were calling both the MCP generate_authoring_plan tool and agentic search. The MCP plan omits the example-file migration and preview-rebase steps these cases require, so its guidance conflicts with the correct end-state. Restrict Case 3 to agentic search only and forbid the MCP tool; on agentic-search failure, retry rather than fall back to the MCP tool. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4912c91-426b-4cb5-8bd2-5ef89f4158fc
…ch only) Add an explicit precedence note so the general 'use both tools' guidance does not appear to conflict with Case 3's agentic-search-only rule. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4912c91-426b-4cb5-8bd2-5ef89f4158fc
lirenhe
approved these changes
Jul 23, 2026
| ### Case 3 — API Version Evolution (ARM / Data-plane) | ||
|
|
||
| > **Must** use Agentic Search (option 2 above) to build the plan — do not call the MCP tool. | ||
| > **Use Agentic Search (option 2 above) ONLY.** For version-evolution requests you **must not** call the MCP tool `azsdk_typespec_generate_authoring_plan` at all — it does not cover the example-file migration and version-rebase steps this case requires. This exclusive rule overrides the "use both tools" guidance in [3.1 General](#31-general-all-cases). |
Member
There was a problem hiding this comment.
So just for clarity we're not allowing any other fallback in this case?
catalinaperalta
approved these changes
Jul 23, 2026
catalinaperalta
left a comment
Member
There was a problem hiding this comment.
Approving to not block progress here, but had a question
Member
This is in the PR description but I dont see if reflected in the actual guidance for the agent, it looks like this path is still blocked |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In .github/skills/azure-typespec-author/references/authoring-plan.md, the General Fallback conflicted with the Case 3 — API Version Evolution instruction:
When all reference URLs are unreachable, Case 3 forbade the MCP tool while the Fallback pointed to it — leaving no valid path forward (dead-end).
Fix
Case 3 now defines an explicit degraded path instead of dead-ending:
readme.mdupdate) are mandatory regardless of grounding path — these map directly to theexamples/<version>/graders that were failing in the versioning eval suite.Fixes #16317
Ref: Copilot review comment #15840 (comment)