# IngoDB — AI-Native Adaptive Database

Henrik Ingo's announcement of IngoDB, a self-morphing document storage engine in Rust built on [[lsm-tree]] architecture. The central idea: the database watches your query patterns and creates indexes automatically, removing the need for manual schema design or DBA tuning. It's positioned as "AI-native" — both built for AI agents to query and partly built by one.

## Architecture

IngoDB is an LSM-tree engine at heart. Writes go through the standard path: MemTable in memory, flush to immutable SSTables on disk, background compaction to merge and clean up. The compaction strategy is Cassandra's Unified Compaction Strategy (UCS), controlled by a single parameter W that trades off read amplification against write amplification. IngoDB tunes W adaptively based on workload.

On top of the LSM foundation, three things make it distinct:

**Liquid AST queries.** No SQL. Queries are Rust data structures — enums and structs that describe filters, projections, and traversals. This is designed for AI agents and application code to construct queries programmatically without SQL parsing ambiguity. It's an interesting bet: SQL exists because humans need a readable language, but if the primary query author is an LLM or agent, a typed AST might be a better interface.

**Reactive indexing.** Every query records statistics: which fields were filtered, how many documents were scanned versus returned, latency. When the engine detects a repeated pattern with low selectivity (scanning many documents to return few), it creates a secondary SSTable sorted by the filter field. The next query on the same pattern hits a binary search instead of a full scan. The cost is essentially zero — the data was going to be sorted during compaction anyway, so the engine just sorts by a different key. Compaction maintains these secondary indexes over time.

**Ad-hoc graph traversal.** Any field can become a graph edge at query time. `Traverse { from_field: "user_id", to_field: "_id", depth: 2 }` walks relationships without predeclared foreign keys or special pointer types. This treats the document store as an implicit graph.

## Concurrency

MVCC snapshot isolation using UUIDv7 as version identifiers. Readers see a consistent point-in-time view while writers continue in parallel. UUIDv7's time-ordered nature means versions sort chronologically without extra metadata.

Snapshot isolation is a specific point in a design space that also holds two-phase locking, optimistic control, and serializable-snapshot variants; Graefe's [[on-transactional-concurrency-control]] puts all of them under one model and is the compact reference for deciding what a storage engine gives up by picking one.

## Benchmarks

On 1 million product documents:

| Operation | Throughput |
|-----------|-----------|
| Ingest (batch=1000) | 210–235K docs/sec |
| Point lookups | 56K ops/sec, p50=14µs |
| First scan (no index) | 3.9s for 100K results |
| Second scan (with reactive index) | 1.8s (2.2x faster) |
| Concurrent 8-thread | 413K ops/sec |
| Mixed read/write | 72K ops/sec |

Write amplification measured at 0.72x through adaptive W tuning — meaning the engine wrote less to disk than the raw data size, thanks to effective delta compression and compaction.

## Built in a week

The entire codebase (~10K lines, 7 Rust crates, 170 tests) was developed through pair programming between Ingo and Claude over about a week. Ingo made architectural decisions and caught design flaws; Claude generated working implementations. This is a data point in the [[building-syntaqlite-ai|"building with AI"]] conversation — a substantial storage engine, not a toy, produced through disciplined human-AI collaboration.

## Planned directions

- Neural network-driven decisions for when to create or drop indexes and how to configure compaction
- Semantic shredding: extracting frequently-accessed fields into columnar format during compaction
- Co-location: grouping related documents together based on observed traversal patterns
- WASM custom comparators and gRPC network access

## See also

- [[lsm-tree]] — the underlying storage architecture
- [[lsm-trees-nosql]] — introductory explainer on LSM trees
- [[building-syntaqlite-ai]] — another AI-assisted database project, with different lessons about the process

https://github.com/nyrkio/ingodb