A tiny, zero-dependency reverse proxy that injects prompt_cache_retention
into Azure OpenAI request bodies — so clients that don't expose the parameter
(e.g. the Codex CLI) can still opt into
extended (24h) prompt-cache retention.
On Azure OpenAI, gpt-5.x and older models default to in_memory prompt-cache
retention, which is evicted under memory pressure once the cached prefix is
large. In long, tool-heavy agent sessions this shows up as repeated total
cache misses (cached_tokens drops to 0), re-billing the entire context at
full input price call after call.
Azure exposes a per-request fix — prompt_cache_retention: "24h" (extended
retention, which offloads the KV cache to GPU-local storage and survives the
capacity wall) — but some clients have no way to set it (no config field, no
request-body passthrough). This proxy adds it on the way through.
See the upstream Codex feature request: openai/codex#25604.
Buffers each request body; when the path matches (/responses or
/chat/completions by default) and prompt_cache_retention is absent, injects
it; forwards to the upstream origin; and streams the response back unbuffered
(so Server-Sent Events keep working). Non-matching requests pass through
untouched. The proxy never inspects or stores credentials — the api-key /
Authorization header is forwarded as-is.
UPSTREAM=https://<your-resource>.openai.azure.com \
RETENTION=24h \
LISTEN=127.0.0.1:8788 \
node proxy.jsThen point your client's base URL at the proxy. For the Codex CLI
(~/.codex/config.toml):
[model_providers.azure]
base_url = "http://127.0.0.1:8788/openai/v1" # was https://<your-resource>.openai.azure.com/openai/v1| Var | Default | Meaning |
|---|---|---|
UPSTREAM |
(required) | Upstream origin, e.g. https://<resource>.openai.azure.com |
LISTEN |
127.0.0.1:8788 |
host:port to listen on |
RETENTION |
24h |
Value to inject (Azure accepts 24h or in_memory) |
INJECT_PATHS |
/responses,/chat/completions |
Comma-separated path substrings to inject into |
See examples/codex-cache-proxy.service
for a systemd unit (adjust ExecStart and Environment= to your paths/resource).
Send a request through the proxy with no prompt_cache_retention in the body
and set RETENTION to an invalid value — Azure validates the param and returns a
400, which proves the proxy's injected value reached the upstream:
RETENTION=bogus → 400 "Invalid value: 'bogus'. Supported values are: 'in_memory' and '24h'."
RETENTION=24h → 200 OK
- HTTP/SSE only. This handles the HTTP + Server-Sent-Events transport. If your client uses a WebSocket transport, body injection over WS is not handled.
- The proxy is a hard dependency in the request path once your client points at it — run it under a supervisor with restart-on-failure.
MIT — see LICENSE.