IngoDB — AI-Native Adaptive Database

Henrik Ingo's announcement of IngoDB, a self-morphing document storage engine in Rust built on lsm-tree architecture. The central idea: the database watches your query patterns and creates indexes automatically, removing the need for manual schema design or DBA tuning. It's positioned as "AI-native" — both built for AI agents to query and partly built by one.

Architecture

IngoDB is an LSM-tree engine at heart. Writes go through the standard path: MemTable in memory, flush to immutable SSTables on disk, background compaction to merge and clean up. The compaction strategy is Cassandra's Unified Compaction Strategy (UCS), controlled by a single parameter W that trades off read amplification against write amplification. IngoDB tunes W adaptively based on workload.

On top of the LSM foundation, three things make it distinct:

Liquid AST queries. No SQL. Queries are Rust data structures — enums and structs that describe filters, projections, and traversals. This is designed for AI agents and application code to construct queries programmatically without SQL parsing ambiguity. It's an interesting bet: SQL exists because humans need a readable language, but if the primary query author is an LLM or agent, a typed AST might be a better interface.

Reactive indexing. Every query records statistics: which fields were filtered, how many documents were scanned versus returned, latency. When the engine detects a repeated pattern with low selectivity (scanning many documents to return few), it creates a secondary SSTable sorted by the filter field. The next query on the same pattern hits a binary search instead of a full scan. The cost is essentially zero — the data was going to be sorted during compaction anyway, so the engine just sorts by a different key. Compaction maintains these secondary indexes over time.

Ad-hoc graph traversal. Any field can become a graph edge at query time. Traverse { from_field: "user_id", to_field: "_id", depth: 2 } walks relationships without predeclared foreign keys or special pointer types. This treats the document store as an implicit graph.

Concurrency

MVCC snapshot isolation using UUIDv7 as version identifiers. Readers see a consistent point-in-time view while writers continue in parallel. UUIDv7's time-ordered nature means versions sort chronologically without extra metadata.

Snapshot isolation is a specific point in a design space that also holds two-phase locking, optimistic control, and serializable-snapshot variants; Graefe's on-transactional-concurrency-control puts all of them under one model and is the compact reference for deciding what a storage engine gives up by picking one.

Benchmarks

On 1 million product documents:

Write amplification measured at 0.72x through adaptive W tuning — meaning the engine wrote less to disk than the raw data size, thanks to effective delta compression and compaction.

Built in a week

The entire codebase (~10K lines, 7 Rust crates, 170 tests) was developed through pair programming between Ingo and Claude over about a week. Ingo made architectural decisions and caught design flaws; Claude generated working implementations. This is a data point in the "building with AI" conversation — a substantial storage engine, not a toy, produced through disciplined human-AI collaboration.

Planned directions

• Neural network-driven decisions for when to create or drop indexes and how to configure compaction
• Semantic shredding: extracting frequently-accessed fields into columnar format during compaction
• Co-location: grouping related documents together based on observed traversal patterns
• WASM custom comparators and gRPC network access

See also

• lsm-tree — the underlying storage architecture
• lsm-trees-nosql — introductory explainer on LSM trees
• building-syntaqlite-ai — another AI-assisted database project, with different lessons about the process

https://github.com/nyrkio/ingodb