Skip to content

egarim/SyncFramework

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

185 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SyncFramework

A C# library for synchronizing data between databases using delta encoding.

Current Version: 10.0.0

Target Framework: .NET 10.0

Component Version
EF Core 10.0.0
PostgreSQL 10.0.0
MySQL (Oracle) 10.0.1
SQLite 10.0.0
SQL Server 10.0.0

How It Works

SyncFramework uses delta-based synchronization to efficiently sync data between databases.

graph TB
    subgraph "Client"
        A[Client Application] --> B[DbContext]
        B --> C[Custom BatchExecutor]
        C -->|Intercept SQL| D[Delta Generator]
        D -->|Store Deltas| E[Delta Store]
        E -->|Push Deltas| F[HTTP Client]
    end
    subgraph "Server"
        F -->|HTTP| G[SyncServer]
        G -->|Store Deltas| H[Server Delta Store]
        H -->|Provide Deltas| G
    end
    subgraph "Other Clients"
        G -->|HTTP| I[Other Clients]
        I -->|Push Deltas| G
    end
    style A fill:#d4f1f9,stroke:#333
    style G fill:#ffcccc,stroke:#333
Loading

SyncFramework replaces EF Core's internal BatchExecutor with a custom one that intercepts generated SQL commands (deltas), stores them, and makes them available for replay on remote databases — even across different database engines.


NuGet Packages

Package Description Link
BIT.Data.Sync Core interfaces and base classes NuGet
BIT.Data.Sync.EfCore EF Core implementation NuGet
BIT.Data.Sync.EfCore.Npgsql PostgreSQL support NuGet
BIT.Data.Sync.EfCore.MySql MySQL support (official Oracle driver) NuGet
BIT.Data.Sync.EfCore.Sqlite SQLite support NuGet
BIT.Data.Sync.EfCore.SqlServer SQL Server support NuGet

Getting Started

1. Install the package for your database

dotnet add package BIT.Data.Sync.EfCore.SqlServer   # SQL Server
dotnet add package BIT.Data.Sync.EfCore.Sqlite       # SQLite
dotnet add package BIT.Data.Sync.EfCore.Npgsql       # PostgreSQL
dotnet add package BIT.Data.Sync.EfCore.MySql        # MySQL (Oracle)

2. Register services

var services = new ServiceCollection();
var httpClient = new HttpClient { BaseAddress = new Uri("https://your-sync-server/") };

services.AddSyncFrameworkForSQLite(
    connectionString: "Data Source=deltas.db;",
    httpClient: httpClient,
    serverNodeId: "MemoryDeltaStore1",
    identity: "my-client-id",
    new SqliteDeltaGenerator(),
    new SqlServerDeltaGenerator()
);

var serviceProvider = services.BuildServiceProvider();

3. Use SyncFrameworkDbContext

var options = new DbContextOptionsBuilder().UseSqlite("Data Source=myapp.db;").Options;
var ctx = new MyDbContext(options, serviceProvider);
await ctx.Database.EnsureCreatedAsync();

4. Push, Pull, Fetch

await ctx.PushAsync();   // send local deltas to server
await ctx.PullAsync();   // fetch + apply server deltas locally
await ctx.FetchAsync();  // fetch deltas without applying

5. Implement a SyncServer

// Program.cs / Startup.cs
services.AddSyncServerWithMemoryNode("MemoryDeltaStore1");
// SyncController.cs
[ApiController]
[Route("sync")]
public class SyncController : SyncControllerBase
{
    public SyncController(ILogger<SyncControllerBase> logger, ISyncFrameworkServer server)
        : base(logger, server) { }
}

Dashboard

src/Tools/SyncFramework.Dashboard is a Blazor Server admin UI for managing sync server nodes.

cd src/Tools/SyncFramework.Dashboard
ASPNETCORE_ENVIRONMENT=Development dotnet run
# Open http://localhost:5000
Page Route Description
Nodes / View all active nodes and delta counts
Add Node /nodes/add Spin up a new node (Memory or SQLite)
Replay /nodes/{id}/replay Replay a node's deltas into a SQLite file

Screenshots

Nodes — overview of all active sync nodes Nodes page

Add Node — spin up a new node with store type selection Add Node page

Replay — apply a node's deltas into a local SQLite file Replay page


Documentation

Blog Series

  1. Data Synchronization in a Few Words
  2. Parts of a Synchronization Framework
  3. Let's Write a Synchronization Framework in C#
  4. Synchronization Framework Base Classes
  5. Planning the First Implementation
  6. Testing the First Implementation (Video)
  7. Adding Network Support

Additional Resources


Changelog

All notable changes are documented in CHANGELOG.md, following the Keep a Changelog format.

Contributing to the changelog

When opening a PR or making a commit that changes behavior:

  1. Add an entry under the [Unreleased] section at the top of CHANGELOG.md
  2. Use the appropriate category:
    • Added — new features
    • Changed — changes to existing behavior
    • Deprecated — soon-to-be removed features
    • Removed — removed features
    • Fixed — bug fixes
    • Security — vulnerability fixes
    • Breaking Changes — changes that break backward compatibility
  3. On release, rename [Unreleased] to the version number and date, e.g. [10.1.0] — 2026-05-01

Example entry:

## [Unreleased]

### Added
- `IDeltaStore.PurgeAsync` — bulk delete deltas older than a given date

### Fixed
- Replay page now handles empty delta stores without crashing

Project Templates

dotnet new install BIT.Data.Sync.Templates
dotnet new SyncServer -o MySyncServer

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors