Skip to content

feat(cli): unify transcribe + stream LLM Gateway under --llm#5

Merged
alexkroman merged 1 commit into
mainfrom
llm-default-haiku
Jun 4, 2026
Merged

feat(cli): unify transcribe + stream LLM Gateway under --llm#5
alexkroman merged 1 commit into
mainfrom
llm-default-haiku

Conversation

@alexkroman

Copy link
Copy Markdown
Collaborator

Summary

Unifies the LLM Gateway surface across transcribe and stream under a single --llm flag, makes streaming LLM use a first-class live mode, switches the default model, and adds a security section to the README.

--llm everywhere

  • aai transcribe --llm "…" — renamed from --llm-gateway-prompt. One-shot after the transcript completes; repeatable so prompts chain on each other's output.
  • aai stream --llm "…"new live mode: re-runs the prompt chain over the growing transcript on every finalized turn (Rich panel on a TTY, {"turns","output"} NDJSON when piped). Replaces the old one-shot-at-end behavior. The fused, single-process equivalent of aai stream -o text | aai llm -f.
  • --llm together with -o text is rejected (contradictory).
  • aai stream --llm … --show-code generates the runnable live transcribe→LLM-per-turn Python loop.

Internals

  • New llm.run_chain() helper for the shared prompt-chaining logic.
  • Extracted FollowRenderer into assemblyai_cli/follow.py, now shared by aai llm -f and aai stream --llm.
  • Removed the now-dead StreamRenderer.llm() and its tests.

Other changes bundled in this commit

  • Default LLM Gateway model → claude-haiku-4-5-20251001.
  • README: added an "API key & security" section (env-var precedence, keyring-not-plaintext, secret-manager one-liners); led the live-LLM docs with --llm and kept the stream | llm pipe as the composable alternative; pruned pipe recipes now covered by --llm.

Note: these three concerns share files (llm.py, README.md, test files), so they landed in one commit — clean per-concern splitting would have needed interactive hunk staging.

Test plan

Full check.sh is green:

  • ruff check + format, mypy (77 files), markdownlint
  • 503 passed / 6 skipped, branch coverage 93.54% (gate 90%)
  • build + twine check pass

🤖 Generated with Claude Code

- transcribe: rename --llm-gateway-prompt to --llm (one-shot after the
  transcript, repeatable so prompts chain on each other's output)
- stream: add live --llm that re-runs the prompt chain over the growing
  transcript on every finalized turn (Rich panel on a TTY, {"turns","output"}
  NDJSON when piped), replacing the old one-shot-at-end behavior; reject
  --llm together with -o text
- stream --show-code: generate the runnable live transcribe->LLM-per-turn loop
- add llm.run_chain helper; extract FollowRenderer into follow.py, shared by
  `aai llm -f` and `aai stream --llm`; drop now-dead StreamRenderer.llm
- default LLM Gateway model -> claude-haiku-4-5-20251001
- docs: add README "API key & security" section; lead the live-LLM docs with
  --llm and keep the stream|llm pipe as the composable alternative; prune
  pipe recipes now covered by --llm

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alexkroman alexkroman merged commit 7215a3c into main Jun 4, 2026
5 checks passed
@alexkroman alexkroman deleted the llm-default-haiku branch June 5, 2026 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants