Govern every decision, inside the agent
cascadeflow is the open-source AI agent runtime layer that sees every step your agent takes and steers each model call, tool call, and handoff toward your policies and goals.
$ pip install cascadeflow$ npm install @cascadeflow/coreMIT open source · sub-5ms overhead · full per-step audit trail
$ pip install cascadeflow$ python run_agent.py[cascadeflow] analyzing step⦠predicted cost > threshold, quality > min[cascadeflow] ACTION: switch_model(cost-optimized)[cascadeflow] sub-agent call optimized and routedâ agent run complete · final cost 0.0001 · 92% reduction
Runs in-process
cascadeflow sits inside the AI agent.
An open-source AI agent runtime layer that observes, scores, and enforces every step of agent execution. It is framework-neutral and runs in minutes.
cascadeflow.init(mode="observe")
# every model call now tracked with zero code changes
Agent frameworks
LangChain · OpenAI Agents SDK · CrewAI · PydanticAI · Google ADK · Vercel AI · n8n · OpenClaw · Hermes Agent
cascadeflow runtime
Observe · Score · Enforce
In-process · sub-5ms overhead · full audit trail
Models & tools
OpenAI · Anthropic · Groq · Together · on-prem vLLM · Ollama · 17+ providers
Enforce policy at every agent step
cascadeflow injects your policies and KPIs directly into the agent and steers execution in real time. It scores every step across cost, latency, quality, budget, compliance, and energy, plus any custom dimensions you define in Studio. Eight runtime actions enforce the result.
// 6 built-in dimensions + custom · 8 enforcement actions
allowLet the step proceed when cost and quality are within policy.
switch_modelEscalate or downshift by routing from a flagship model to a cheaper one during the run.
deny_toolBlock expensive or non-compliant tool calls before they execute.
stopTerminate execution on a safety, budget, or compliance rule.
retryrequire_approvalredactserve_from_cache// Domain-aware intelligence
Route every step to the right specialist
cascadeflow detects the domain of each query and routes every agent step to the model cascade built for that work. Small, domain-specialized models routinely outperform large general-purpose models on specialized tasks, so you get better answers and lower cost at the same time.
domain:code"Fix this Python race condition"
â code-tuned drafter, escalate to flagship on failed tests
domain:writing"Draft the launch announcement"
â writing-optimized model, higher quality bar
domain:data"Summarize this CSV of orders"
â fast, low-cost model · speed prioritized
domain:legal"Review this indemnity clause"
â high-accuracy verifier, strict compliance scoring
domain:support"Why was my invoice declined?"
â cheap drafter first, escalate only when unsure
domain:general"Anything else"
â free-first cascade, escalate on quality miss
Inside the agent, not a proxy
Proxies see requests. cascadeflow sees decisions.
Routers and gateways sit at the HTTP boundary. They see the request but miss the decisions inside agent execution. cascadeflow runs in-process where cost, risk, and failure actually occur.
Three lines of code. Works with your stack.
No rip-and-replace. cascadeflow drops into your existing agent framework and model providers, then optimizes every run.
import cascadeflow @cascadeflow.govern() async def run_agent(task): # your agent runs as usual while cascadeflow optimizes each step return await agent.run(task)
Frameworks
OpenAI · Anthropic · Groq · Ollama · vLLM · Together AI · HuggingFace + 100 more via LiteLLM
cascadeflow
Fleet-wide visibility and governance
Business intelligence for AI agents, plus two dedicated visual builders, one for policies and one for domains, so you can govern KPIs, cost routing, and compliance across your entire agent fleet.
Dedicated builder
Policy Builder
Define custom scoring dimensions, KPI weights, and governance policies beyond the six built-in dimensions. Enforce them across every agent with gradual rollouts and automatic rollback.
Dedicated builder
Domain Builder
Map each domain to its specialist model cascade and tune keyword and semantic routing visually without code. Version domains and enforce them fleet-wide.
Self-learning agent intelligence
Every run feeds patterns back to through auto-benchmarks, model fleet suggestions, compounding optimization.
Fleet BI & ROI analytics
Real-time cost breakdowns by provider, model, domain, and user. Spending forecasts and exportable reports.
KPI enforcement & governance
Turn business KPIs into live guardrails for cost routing, compliance rules, and quality bars at every step.
Organization controls
Role-based access, SSO/SAML, audit logs, and multi-org workspaces. Integrates with Slack, Datadog, and your finance stack.
See governance running across your agent fleet.
// FAQ
Frequently asked questions
What is cascadeflow?+
cascadeflow is an agent runtime intelligence layer inside AI agent execution. It sees model calls, tool calls, and sub-agent handoffs, then steers each decision toward your policies and goals. Every step is scored across cost, latency, quality, budget, compliance, and energy in real time.
How is cascadeflow different from a model router or proxy?+
Proxies and model routers see requests going in and out from outside the agent. cascadeflow runs inside agent execution, where it can switch models during a run, deny a tool call, require approval, redact, retry, serve from cache, or stop execution. This gives cascadeflow direct control over the decisions that create cost and risk.
How much can cascadeflow reduce AI costs?+
cascadeflow reduces agent inference cost by up to 90% while retaining roughly 96% of GPT-5 quality, with sub-5ms overhead per decision. Intelligent model cascading and domain-aware routing classify each query and send it through a specialist model cascade rather than always calling a large general model.
Is cascadeflow open source?+
Yes. The cascadeflow core is open source under the MIT license and scores every step across six built-in dimensions with eight runtime enforcement actions. cascadeflow Studio is the full managed, self-optimizing version that adds visual Policy and Domain builders and lets you define custom policies and scoring dimensions beyond the six built-in ones.
What is cascadeflow Studio?+
cascadeflow Studio is the full managed version of cascadeflow. Its UI and CLI help teams run, set up, optimize, and adjust governed agents. The Policy Builder and Domain Builder define, version, and enforce custom policies, KPIs, and domain cascades without hand-written enforcement code.
What can cascadeflow enforce at runtime?+
Every agent step is scored across six built-in dimensions: cost, latency, quality, budget, compliance, and energy. Studio can add custom dimensions. cascadeflow acts on the result with eight enforcement actions: allow, switch_model, deny_tool, stop, retry, require_approval, redact, and serve_from_cache.
Start governing every agent decision
Open source, MIT licensed, three lines of code. See and steer every agent step while cutting inference cost by up to 90%.
$ pip install cascadeflowGet started