Typhon

It makes the case for Typhon — the problem it solves, what's actually built and measured today, and where it's headed — before you dive into the technical docs below.

Stabilizing fast, not yet production-ready, but ready for POC & early adopters.

Alpha NuGet package (and CLI + Workbench), full reference documentation are now live. Most of the public API is now stable — and you can open an issue or go to the discussions for help, questions, or feedback.

A microsecond-latency ACID data engine combining ECS archetype storage with tick-based parallel execution.

Typhon is an embedded data engine for real-time workloads like game servers, simulations, and stateful dataflow.
It pairs a microsecond-latency ACID store — MVCC snapshot isolation, configurable durability, source-generated ECS archetype accessors — with a tick-based parallel runtime that dispatches fine-grained system chunks across worker threads, each operating directly on the store.
The runtime doesn't sit on top of the database — it runs inside it.

📖 Start here:

• 📚 doc.typhondb.io — the new documentation site: guide, in-depth overview, feature catalog, and full API reference, all in one browsable, searchable place.
• User Guide — learn the key concepts hands-on, backed by a small runnable sample project (doc/guide/example).
• In-Depth Overview — dig into how every subsystem works, and the reasoning behind it.

🗺️ Curious what's next? Current priorities, in-progress work, and the roadmap are tracked live on the GitHub Project.

📚 This is a curated highlight reel — see the full Feature Catalog for every capability, categorized and cross-linked to source and tests.

• Microsecond Operations — Optimized for µs-level latency with pinned memory, SIMD, and lock-free reads
• ACID Transactions — Full transactional semantics with optimistic concurrency control
• MVCC Snapshot Isolation — Readers never block writers; each transaction sees a consistent snapshot
• ECS Archetype System — Entities are typed by archetype; components are blittable structs with source-generated accessors and archetype inheritance
• Larger-Than-RAM Storage — The database is a memory-mapped, paged file on disk, virtualized through a page cache that holds only the hot working set — so resident memory is bounded by the cache, not the database. Manipulate a 100 GiB store on a machine with a fraction of that RAM; component data, indexes, and the entity map all page to/from disk on demand (in-memory ECS frameworks, by contrast, must fit the entire world in RAM)
• Workbench Dev UI — 📣Demo video #1📣 A local, browser-based companion (ASP.NET Core + React/Vite) — DataGrip for Typhon, not just a profiler. Open a recorded .typhon-trace, attach to a live engine over TCP, or open a .typhon database file directly. Performance: per-tick profiler timeline with CPU-sample + off-CPU overlays, Top Spans, Call Tree, Critical Path, System DAG. Data & schema: component/archetype browsers, Schema Inspector, a zoomable Hilbert-curve Database File Map, live Resource Tree. Queries: catalog, plan tree, per-execution inspector — all in a dockable, persisted workspace
• Entity Clusters — SoA batched storage co-locating up to 64 entities per cluster, with cluster-native SIMD predicate evaluation (AVX-512 / AVX2 / scalar dispatch), zone-map pruning, and k-way sorted merge
• Query Engine — Typed queries with Where, With/Without filters, polymorphic iteration, OR logic, navigation joins, and a cluster execution path that routes filtered scans through SIMD gather-compare on cluster SoA columns
• Entity Relationships — Typed EntityLink<T> references for 1:1, 1:N, and N:M relationships, with declarative cascade delete and reactive FK-joined queries
• Views & Change Tracking — Incremental entity-set monitoring (added/removed/modified detection) via ring-buffer delta streaming for indexed queries — opaque/unindexed predicates still work but fall back to a full re-scan per refresh; SingleVersion/Transient view support is still being validated end-to-end
• B+Tree Indexes — Cache-aligned 256-byte nodes with optimistic lock coupling and specialized key-size variants
• Spatial Indexing — Page-backed wide R-Tree for AABB / Radius / Ray / Frustum / kNN queries, plus a per-cell cluster broadphase (SpatialGrid) with BMI2 Morton-encoded cell keys and multi-resolution SimTier dispatch for near/far/coarse simulation budgets
• Write-Ahead Logging — WAL (v2, always on) with configurable durability modes (Deferred, GroupCommit, Immediate) and coalesced tick-boundary snapshots via TickFence / ClusterTickFence chunks for the runtime tick loop
• Page Integrity — CRC32C checksums, suspect-page detection, and crash-recovery rebuild (index + EntityMap); full-page images were retired in favor of a rebuild-based torn-page net
• Schema Versioning — Component/field-level revisions with migration functions, compatible-evolution support, and audit history; automatic archetype layout-drift detection at database open is still landing
• Three Storage Modes — Versioned (full MVCC + WAL), SingleVersion (last-writer-wins; per-tx TickFence or Committed durability discipline — atomic, zero-loss, no revision chain), Transient (heap-only, no persistence)
• Configurable Durability — Choose per-UnitOfWork whether data is deferred, group-committed, or immediately fsynced
• Bulk Load Session — Opt-in exclusive write path for fast dataset ingestion, batching writes through a recycled Transaction and committing the whole load atomically behind a checkpoint barrier
• Game Server Runtime — Tick-based micro-task DagScheduler with any-worker parallel dispatch, per-system change filters, typed MPSC event queues, side-transactions, cluster dormancy with staggered heartbeat wake, checkerboard Red/Black dispatch, and a growing overload response ladder (system throttling, tick-rate modulation up to 6x, a last-resort player-shedding callback) — still evolving, with finer-grained escalation designed but not a current priority
• Subscription Server + Client SDK — TCP delta streaming of published views with MemoryPack serialization, per-client incremental sync, backpressure-driven resync, and a zero-engine-dependency Typhon.Client assembly for game clients
• Observability — Runtime telemetry, metrics, and diagnostics with zero-cost JIT-eliminated toggles
• Deep-Trace Profiler — Per-tick Gantt and flame-graph visualization via a .typhon-trace binary format, live TCP streaming to a React/Vite viewer, and optional dotnet-trace CPU sampling correlation for full managed-stack profiles
• Interactive Shell (typhon) — Database REPL for inspection and debugging
• Public/Internal API split — Two namespaces per assembly (Typhon.Engine / Typhon.Engine.Internals), each subsystem folder split into public/ + internals/, enforced at compile time by the TYPHON008 Roslyn analyzer
• Roslyn Analyzers — Custom analyzers detecting undisposed engine resources at compile time + the TYPHON008 internal-API leak detector

A taste — declare a component and an archetype, spawn an entity, read it back:

// A component is a plain struct; an archetype is the fixed set of components an entity has.
[Component("Game.Health", 1, StorageMode = StorageMode.Versioned)]
public struct Health { public int Current, Max; }

[Archetype(1)]
public sealed partial class Unit : Archetype<Unit>
{
    public static readonly Comp<Health> Health = Register<Health>();
}

// ...build the engine + register the schema (see the Guide), then:
using var tx = dbe.CreateQuickTransaction();
var id = tx.Spawn<Unit>(Unit.Health.Set(new Health { Current = 100, Max = 100 }));
var hp = tx.Open(id).Read(Unit.Health);   // hp.Current == 100
tx.Commit();

Illustrative — engine build + schema registration are elided. The User Guide has the full, runnable version (doc/guide/example).

Install the Typhon.Cli NuGet package and follow the instructions to create a sample app, run it, profile it, open the database and the profiled session in the workbench.

Workbench Database

Workbench Profile viewer

Typhon is in active development targeting an alpha release. Current state:

• Core transaction engine with MVCC
• B+Tree indexes with optimistic lock coupling
• Component-level durability options
• Write-Ahead Logging with configurable durability modes
• Page integrity (CRC32C, suspect-mode rebuild, checkpoints)
• Query engine with filtering, sorting, and navigation
• Views and incremental change tracking
• ECS archetype system with source-generated accessors
• Entity cluster storage (SoA batched, SIMD predicate evaluation, zone-map pruning, k-way sorted merge)
• Dual page store abstraction (PersistentStore + TransientStore)
• Schema versioning and evolution
• Spatial indexing (page-backed wide R-Tree: AABB / Radius / Ray / Frustum / kNN)
• Spatial grid with per-cell cluster broadphase and BMI2 Morton encoding
• Multi-resolution simulation (SimTier dispatch, cluster dormancy, checkerboard Red/Black)
• Game server runtime (DagScheduler, parallel system dispatch, overload management, event queues)
• Subscription server and Typhon.Client SDK (TCP delta streaming, MemoryPack wire protocol)
• Deep-trace runtime profiler (.typhon-trace format, live TCP streaming, React viewer)
• Interactive shell (tsh)
• Observability and monitoring stack
• HashMap collections with key-size specialization
• Workbench dev UI (data browsing, profiler viewer, System DAG view, Data Flow timeline, Access Matrix, internal data API)
• Public/Internal API namespace split + TYPHON008 analyzer
• Crash recovery (WAL v2 replay, scrub + index/EntityMap rebuild, suspect-mode)
• Backup and restore

This project has had quite a journey:

• 2015 — Initial bootstrap with a different design, quickly shelved
• 2020 — COVID resurrection as a POC: "Can we build a µs-latency ACID database for persistent games?" Promising results, then shelved again
• 2025 — Third resurrection with firm intention to reach alpha stage
• 2025-2026 — Rapid progress: WAL & durability, query engine, ECS archetype system with source generators, schema evolution, observability stack, and interactive shell delivered
• Q2 2026 alpha push — Game-server runtime (DagScheduler, parallel system dispatch, overload management), entity clusters with cluster-native SIMD query execution, dual page store abstraction (PersistentStore + TransientStore), spatial indexing (page-backed R-Tree + spatial grid cluster broadphase), multi-resolution SimTier dispatch with cluster dormancy and checkerboard, TCP subscription server with zero-engine-dependency client SDK, and a deep-trace runtime profiler with live-streaming viewer
• Q2 2026 Workbench buildout — Internal Data API + System DAG view (#306, #314, #322), static schema export and Schema Inspector (#326), Data Flow Timeline + Access Matrix panels with cross-panel selection and hover linking (#327)
• Q2 2026 API hardening — Public/Internal namespace migration, accessibility flips, friend-list audit, and leak-tightening pass: ~430 internal types now in Typhon.Engine.Internals enforced by the TYPHON008 analyzer (#329)
• Q2 2026 durability redesign — Minimal WAL (WAL v2 format, RecoveryDriver, checkpoint v2 A/B slot-pairing), the Committed durability discipline (#392), cluster crash durability (#395), BulkLoad write path + storage hardening (#383), and retirement of full-page images in favor of a suspect-mode rebuild recovery net (#399, #401)
• Q2 2026 Workbench redesign — shell rebuild (Stages 0–4), read-only Data Browser, zoomable Database File Map, off-CPU + integrated CPU-sample profiling, Track→DAG / Critical Path rework, the AntHill validation harness, and Query Console Phase 1 — write/execute/browse queries against a live store (#379, #390)
• Q3 2026 Durability hardening, better perfs, less bugs, more user content — in progress...