Skip to content

[REFACTOR] DRY Redis client creation in worker cache modules#1857

Merged
chandrasekharan-zipstack merged 1 commit intofix/redis-sentinel-metrics-mixinfrom
refactor/redis-client-creation-dry
Mar 15, 2026
Merged

[REFACTOR] DRY Redis client creation in worker cache modules#1857
chandrasekharan-zipstack merged 1 commit intofix/redis-sentinel-metrics-mixinfrom
refactor/redis-client-creation-dry

Conversation

@chandrasekharan-zipstack
Copy link
Contributor

What

  • Remove duplicated sentinel/standalone Redis connection branching from worker cache modules
  • Add SSL support to create_redis_client() in unstract-core via env vars

Why

  • cache_backends.py and cache_utils.py both had identical if-sentinel-else-standalone branching to create Redis clients
  • create_redis_client() already handles both modes internally — the branching was redundant
  • SSL plumbing existed in cache_utils.py standalone path but not in the factory, so it would be lost when consolidating

How

  • Add {prefix}SSL and {prefix}SSL_CERT_REQS env var reading to _resolve_redis_env() in core's redis_client.py
  • Pass SSL kwargs through _build_connection_kwargs() and _create_standalone_client()
  • Replace both if/else branches in cache_backends.py and cache_utils.py with a single create_redis_client(env_prefix="CACHE_REDIS_", ...) call
  • Remove unused os and redis imports

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

  • No. create_redis_client() reads the same env vars (CACHE_REDIS_HOST, CACHE_REDIS_PORT, etc.) that the removed standalone code was reading. SSL is off by default (CACHE_REDIS_SSL=false), preserving current behavior.

Database Migrations

  • None

Env Config

  • No new env vars required. {prefix}SSL and {prefix}SSL_CERT_REQS are now read by the core factory (previously only read by workers locally). Default behavior unchanged.

Relevant Docs

  • N/A

Related Issues or PRs

Dependencies Versions

  • No changes

Notes on Testing

  • Verify worker cache initialization works in both standalone and Sentinel modes
  • Confirm CACHE_REDIS_SSL=false (default) behavior is unchanged
  • Check that RedisCacheBackend and WorkerCacheManager both connect successfully

Screenshots

N/A

Checklist

I have read and understood the Contribution Guidelines.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 15, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e4c52fa6-865d-4f8c-ac1a-acc94b406c55

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch refactor/redis-client-creation-dry
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 15, 2026

Greptile Summary

This PR consolidates duplicated Redis connection branching in cache_backends.py and cache_utils.py into the shared create_redis_client() factory in unstract-core, while also adding TLS support to that factory so it is not dropped during consolidation.

Key changes:

  • _resolve_redis_env() now reads TLS-related env vars and populates them in the returned env dict.
  • _build_connection_kwargs() forwards TLS fields when TLS is active, centralising that logic.
  • _create_standalone_client() now delegates fully to _build_connection_kwargs() instead of building its own kwargs dict, removing the locally-duplicated TLS handling.
  • RedisClient.__init__() is removed; the class can only be instantiated via RedisClient.from_env() or the __new__() pattern used by cache_backends.py. No callers of the old constructor were found in the codebase.
  • Both worker cache modules (RedisCacheBackend and WorkerCacheManager) drop the if/else Sentinel branches and call create_redis_client() directly.

Issue found:

  • _create_sentinel_client() builds sentinel_kwargs using the include_auth_only branch of _build_connection_kwargs, which skips the newly added TLS fields. Master connections are secured correctly via master_kwargs, but the Sentinel discovery connections carry no TLS parameters. In a cluster where Sentinel servers themselves require TLS, discovery will fail at the handshake step. See the inline comment on redis_client.py line 223.

Confidence Score: 3/5

  • Safe to merge for standard (non-TLS Sentinel) deployments; TLS-protected Sentinel clusters will have broken discovery when SSL is enabled.
  • The DRY refactoring is clean and correct for both standalone mode and standard Sentinel mode. The SSL addition is correct for standalone and for Sentinel master connections. However, the include_auth_only path in _build_connection_kwargs does not propagate SSL to Sentinel discovery connections, meaning any deployment that enables CACHE_REDIS_SSL=true with Sentinel mode will fail to connect to TLS-protected Sentinel servers. This is a logic gap directly introduced by this PR's SSL changes, warranting a score of 3 rather than 4-5.
  • unstract/core/src/unstract/core/cache/redis_client.py — specifically the _build_connection_kwargs include_auth_only branch and _create_sentinel_client

Important Files Changed

Filename Overview
unstract/core/src/unstract/core/cache/redis_client.py SSL reading added to _resolve_redis_env and forwarded via _build_connection_kwargs; standalone path now delegates to _build_connection_kwargs; RedisClient.init removed in favour of new + from_env() pattern. SSL is not propagated to sentinel_kwargs (include_auth_only path), leaving a gap for TLS-protected Sentinel servers.
workers/shared/cache/cache_backends.py if-sentinel-else-standalone branching replaced by a single create_redis_client() call; unused os import removed; log message simplified. No logic issues.
workers/shared/infrastructure/caching/cache_utils.py Identical DRY refactor to cache_backends.py — standalone kwargs and sentinel branching replaced by create_redis_client(); unused os and redis imports removed; log message simplified. No logic issues.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["create_redis_client(env_prefix)"] --> B{Sentinel mode?}
    B -- yes --> C["_create_sentinel_client()"]
    B -- no --> D["_create_standalone_client()"]

    D --> E["_resolve_redis_env()\nreads host, port, auth,\ndb, tls settings"]
    E --> F["_build_connection_kwargs()\nnormal path\ntls params included"]
    F --> G{max_connections?}
    G -- yes --> H["ConnectionPool + redis.Redis"]
    G -- no --> I["redis.Redis(kwargs)"]

    C --> J["_resolve_redis_env()\nreads host, port, auth,\ndb, tls settings"]
    J --> K["_build_connection_kwargs()\nauth-only path\ntls params SKIPPED\nsentinel_kwargs"]
    J --> L["_build_connection_kwargs()\nnormal path\ntls params included\nmaster_kwargs"]
    K --> M["Sentinel(sentinels, sentinel_kwargs)"]
    L --> M
    M --> N["sentinel.master_for\nmaster_kwargs"]
Loading

Comments Outside Diff (1)

  1. unstract/core/src/unstract/core/cache/redis_client.py, line 223-229 (link)

    SSL not forwarded to Sentinel discovery

    The sentinel_kwargs dict is assembled via the include_auth_only code path, which returns before the SSL block added in _build_connection_kwargs (lines 162–164) is reached. Only master_kwargs ends up carrying the SSL fields.

    This means that when the {prefix}SSL env var is enabled and Sentinel servers are themselves TLS-protected, the discovery step will fail at the handshake — even though master connections are correctly secured.

    To close the gap, the include_auth_only branch inside _build_connection_kwargs should also forward the ssl / ssl_cert_reqs fields from the env dict before returning.

Prompt To Fix All With AI
This is a comment left during a code review.
Path: unstract/core/src/unstract/core/cache/redis_client.py
Line: 223-229

Comment:
**SSL not forwarded to Sentinel discovery**

The `sentinel_kwargs` dict is assembled via the `include_auth_only` code path, which returns before the SSL block added in `_build_connection_kwargs` (lines 162–164) is reached. Only `master_kwargs` ends up carrying the SSL fields.

This means that when the `{prefix}SSL` env var is enabled and Sentinel servers are themselves TLS-protected, the discovery step will fail at the handshake — even though master connections are correctly secured.

To close the gap, the `include_auth_only` branch inside `_build_connection_kwargs` should also forward the `ssl` / `ssl_cert_reqs` fields from the `env` dict before returning.

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: 67676a0

@chandrasekharan-zipstack chandrasekharan-zipstack force-pushed the refactor/redis-client-creation-dry branch 2 times, most recently from cdd8d66 to e8cd5c7 Compare March 15, 2026 06:34
@chandrasekharan-zipstack chandrasekharan-zipstack force-pushed the refactor/redis-client-creation-dry branch from e8cd5c7 to 323d05a Compare March 15, 2026 06:37
Worker cache modules (cache_backends.py, cache_utils.py) duplicated
sentinel/standalone branching logic for Redis client creation. Both
now use create_redis_client() from unstract-core as the single entry
point, which already handles mode detection internally.

- Add SSL support (ssl, ssl_cert_reqs) to create_redis_client() via
  env vars ({prefix}SSL, {prefix}SSL_CERT_REQS), disabled by default
- Remove if-sentinel-else-standalone branching in cache_backends.py
  and cache_utils.py — replaced with single create_redis_client() call
- Remove unused os and redis imports from simplified modules

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@chandrasekharan-zipstack chandrasekharan-zipstack force-pushed the refactor/redis-client-creation-dry branch from 323d05a to 67676a0 Compare March 15, 2026 06:43
@sonarqubecloud
Copy link

@chandrasekharan-zipstack chandrasekharan-zipstack merged commit 9486aea into fix/redis-sentinel-metrics-mixin Mar 15, 2026
6 checks passed
@chandrasekharan-zipstack chandrasekharan-zipstack deleted the refactor/redis-client-creation-dry branch March 15, 2026 06:55
hari-kuriakose pushed a commit that referenced this pull request Mar 15, 2026
* [FIX] Fix Redis Sentinel crash in MetricsMixin (prompt-service)

MetricsMixin was creating a raw StrictRedis client using REDIS_HOST/PORT
directly, which in Sentinel HA mode points to the Sentinel process (port
26379). Sentinel doesn't support the SELECT command, causing
ResponseError on every /answer-prompt call.

- Replace StrictRedis with create_redis_client() from unstract-core
  which handles Sentinel discovery via master_for()
- Add unstract-core as sdk1 dependency (all services using sdk1 already
  depend on core, so zero new transitive deps)
- Add error handling around set_start_time() and collect_metrics() so
  Redis failures degrade gracefully instead of crashing requests
- Add missing COPY for unstract/core in tool Dockerfiles (classifier,
  structure, text_extractor) since sdk1 now depends on core

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Commit uv.lock changes

* [FIX] Add unstract-core to tool requirements.txt for pip resolution

Tool Dockerfiles (classifier, text_extractor, structure) use pip, not
uv. Since sdk1 now depends on unstract-core (which isn't on PyPI),
pip can't resolve it via [tool.uv.sources]. Adding explicit editable
install of core before sdk1 ensures pip finds it locally.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [REFACTOR] DRY Redis client creation in worker cache modules (#1857)

Worker cache modules (cache_backends.py, cache_utils.py) duplicated
sentinel/standalone branching logic for Redis client creation. Both
now use create_redis_client() from unstract-core as the single entry
point, which already handles mode detection internally.

- Add SSL support (ssl, ssl_cert_reqs) to create_redis_client() via
  env vars ({prefix}SSL, {prefix}SSL_CERT_REQS), disabled by default
- Remove if-sentinel-else-standalone branching in cache_backends.py
  and cache_utils.py — replaced with single create_redis_client() call
- Remove unused os and redis imports from simplified modules

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant