Description

Adds a per-client token cache to AgentGatewayClient to avoid redundant IAS token requests during agentic loops. Both the customer (mTLS) and LoB (BTP Destination Service) flows are covered.

Token cache (_token_cache.py):

• _TokenCache - LRU-bounded OrderedDict for system and user tokens with monotonic TTL. Expiry resolves from expires_at → expires_in → exp JWT claim (access then id token) → config fallback TTL. Configurable buffer subtracted from all explicit expiries.
• _GatewayUrlCache - separate LRU-bounded OrderedDict for gateway URLs (stable within a client's lifetime, bounded to prevent memory growth in long-lived multi-tenant clients).
• _parse_jwt_exp - unverified JWT payload parser used only for TTL hints, never for security decisions.

Customer flow (_customer.py):

• _request_token_mtls now returns the full token response dict instead of just access_token, so callers can pass it to compute_expires_at.
• get_system_token_mtls and exchange_user_token accept an optional _TokenCache and short-circuit on cache hit.

LoB flow (_lob.py):

• fetch_system_auth and fetch_user_auth accept optional _TokenCache + _GatewayUrlCache and short-circuit on cache hit.
• Both functions raise ValueError if exactly one of token_cache/gateway_url_cache is provided (previously the cache was silently skipped in that case).

Client (agw_client.py):

• AgentGatewayClient.__init__ creates a _TokenCache and _GatewayUrlCache instance and threads them through all auth calls.

Config (config.py):

• Four new ClientConfig fields: fallback_token_ttl_seconds (300 s), token_expiry_buffer_seconds (30 s), max_system_token_cache_size (32), max_user_token_cache_size (256).

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Code refactoring
• Dependency update

How to Test

1. Run the unit test suite: uv run pytest tests/agentgateway/unit/ -q
2. To confirm cache hit behaviour manually: instantiate AgentGatewayClient, call get_system_auth twice with the same tenant/credentials, observe only one token request is made (mock or inspect the _token_cache directly via client._token_cache).

Checklist

• I have read the Contributing Guidelines
• I have verified that my changes solve the issue
• I have added/updated automated tests to cover my changes
• All tests pass locally
• I have verified that my code follows the Code Guidelines
• I have updated documentation (if applicable)
• I have added type hints for all public APIs
• My code does not contain sensitive information (credentials, tokens, etc.)
• I have followed Conventional Commits for commit messages

Breaking Changes

None. All cache params are optional with None defaults. Existing callers unaffected. _request_token_mtls return type changed from str to dict, but it is a private function.

Additional Notes

• User token cache keys are sha256(user_jwt + "|" + scope_key)[:16].
• Thread safety: CPython GIL makes individual OrderedDict ops atomic, but check-then-set is not. Concurrent coroutines for the same key may both miss and both fetch - produces redundant requests, not corruption. Documented in module docstring.
• _GatewayUrlCache default max size is 64 (independent of ClientConfig). Can be made configurable if needed.