Skip to content

feat(journaling): add Redis journal storage provider - #9823

Merged
ReubenBond merged 6 commits into
dotnet:mainfrom
jaironalves:feature/redis-durablejobs
Aug 10, 2026
Merged

feat(journaling): add Redis journal storage provider#9823
ReubenBond merged 6 commits into
dotnet:mainfrom
jaironalves:feature/redis-durablejobs

Conversation

@jaironalves

@jaironalves jaironalves commented Nov 27, 2025 •
edited by ReubenBond
Loading

Copy link
Copy Markdown
Contributor

Adds a Redis-backed Orleans.Journaling provider which implements IJournalStorageProvider and IJournalStorageCatalog, so Orleans.DurableJobs can use Redis through the Journaling backing store.

Highlights:

  • Replaces the DurableJobs-specific Redis provider with Microsoft.Orleans.Journaling.Redis.
  • Stores journal data and metadata in Redis with optimistic ETag checks and Redis Cluster-safe per-journal key hash tags.
  • Registers Redis as a GrainJournaling provider and adds AddRedisJournalStorage hosting APIs.
  • Adds Redis journaling integration tests for read/write, catalog listing, metadata CAS, stale writes, and provider-owned metadata validation.

@jaironalves

This comment was marked as resolved.

Sign in to view
@jaironalves
jaironalves marked this pull request as ready for review December 6, 2025 17:48
Copilot AI review requested due to automatic review settings December 6, 2025 17:48
Copilot AI reviewed Dec 6, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a Redis implementation for Orleans Durable Jobs, enabling persistent job storage using Redis Streams. The implementation follows the same patterns as the existing Azure Storage provider, providing distributed job scheduling that survives silo restarts and cluster reconfigurations.

Key changes:

  • Implements RedisJobShardManager and RedisJobShard for Redis-backed job persistence using Redis Streams
  • Adds batching support for job operations with configurable batch sizes and flush intervals
  • Includes comprehensive test coverage mirroring the Azure Storage provider tests

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
Orleans.slnx Adds the new Redis DurableJobs project to the solution
test/Extensions/Tester.Redis/Tester.Redis.csproj Adds project reference for testing the Redis DurableJobs implementation
test/Extensions/Tester.Redis/DurableJobs/RedisJobShardManagerTests.cs Test class delegating common shard manager tests to the shared test runner
test/Extensions/Tester.Redis/DurableJobs/RedisJobShardManagerTestFixture.cs Test fixture providing Redis-specific test infrastructure and cleanup
test/Extensions/Tester.Redis/DurableJobs/RedisDurableJobsTests.cs Integration tests using a test cluster to verify end-to-end functionality
src/Redis/Orleans.DurableJobs.Redis/RedisStreamJsonSerializer.cs Utility for serializing/deserializing job operations to/from Redis Streams using source-generated JSON
src/Redis/Orleans.DurableJobs.Redis/RedisOperationsManager.cs Centralized manager for all Redis operations including Lua scripts for atomic operations
src/Redis/Orleans.DurableJobs.Redis/RedisJobShardManager.cs Core manager implementing shard lifecycle, ownership, and assignment logic
src/Redis/Orleans.DurableJobs.Redis/RedisJobShardManager.Log.cs Logging definitions for RedisJobShardManager using source-generated LoggerMessage attributes
src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.cs Individual shard implementation with batch processing and stream-based persistence
src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.Log.cs Logging definitions for RedisJobShard using source-generated LoggerMessage attributes
src/Redis/Orleans.DurableJobs.Redis/README.md Documentation for the Redis DurableJobs package with configuration examples
src/Redis/Orleans.DurableJobs.Redis/Properties/AssemblyInfo.cs Assembly metadata enabling test visibility
src/Redis/Orleans.DurableJobs.Redis/Orleans.DurableJobs.Redis.csproj Project file defining package metadata and dependencies
src/Redis/Orleans.DurableJobs.Redis/JobOperation.cs Data structure representing job operations with JSON serialization context
src/Redis/Orleans.DurableJobs.Redis/Hosting/RedisJobShardOptionsValidator.cs Configuration validator ensuring valid Redis DurableJobs options
src/Redis/Orleans.DurableJobs.Redis/Hosting/RedisJobShardOptions.cs Configuration options for Redis DurableJobs including connection settings and batching parameters
src/Redis/Orleans.DurableJobs.Redis/Hosting/RedisDurableJobsExtensions.cs Extension methods for configuring Redis DurableJobs in the silo builder

Comment thread test/Extensions/Tester.Redis/DurableJobs/RedisJobShardManagerTestFixture.cs Outdated
Comment thread test/Extensions/Tester.Redis/DurableJobs/RedisJobShardManagerTestFixture.cs Outdated
Comment thread test/Extensions/Tester.Redis/DurableJobs/RedisJobShardManagerTestFixture.cs Outdated
Copilot AI mentioned this pull request Dec 6, 2025
Merged
Copilot AI reviewed Dec 6, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Comment thread src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.cs Outdated
Comment thread src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.cs Outdated
Copilot AI reviewed Dec 6, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Comment thread src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.cs Outdated
Comment thread src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.cs Outdated
Comment thread src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.cs Outdated
Copilot AI mentioned this pull request Dec 6, 2025
Merged
Copilot AI reviewed Dec 6, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated no new comments.

Copilot AI reviewed Dec 6, 2025

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot AI reviewed Dec 6, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Comment thread src/Redis/Orleans.DurableJobs.Redis/Hosting/RedisJobShardOptionsValidator.cs Outdated
Copilot AI reviewed Dec 6, 2025

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@jaironalves
jaironalves requested a review from Copilot December 6, 2025 20:44
Copilot AI reviewed Dec 6, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Comment thread src/Redis/Orleans.DurableJobs.Redis/RedisJobShard.cs Outdated
Comment thread src/Redis/Orleans.DurableJobs.Redis/README.md Outdated
Comment thread test/Extensions/Tester.Redis/DurableJobs/RedisJobShardManagerTestFixture.cs Outdated
@jaironalves
jaironalves requested a review from Copilot December 7, 2025 17:23
Copilot AI reviewed Dec 7, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Comment thread src/Redis/Orleans.DurableJobs.Redis/Hosting/RedisDurableJobsExtensions.cs Outdated
Comment thread src/Redis/Orleans.DurableJobs.Redis/JobOperation.cs Outdated
@jaironalves
jaironalves requested a review from Copilot December 7, 2025 23:09
This was referenced Aug 31, 2026
Merged
Closed
Merged
Open
Closed
Closed
Closed
Closed
Closed
This was referenced Sep 7, 2026
Merged
Open
Merged
Merged
Merged
Merged
Closed
Closed
Closed
Open
Open
Open
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Open
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 10, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

Copilot code review Copilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ReubenBond ReubenBond ReubenBond approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jaironalves @ReubenBond