# How We Built a Virtual Filesystem for Our Assistant

# How We Built a Virtual Filesystem for Our Assistant

**Author:** Dens Sumesh
**Date:** March 24, 2026
**Source URL:** https://www.mintlify.com/blog/how-we-built-a-virtual-filesystem-for-our-assistant
**Category:** Engineering (5-minute read)

## Core Problem

Mintlify's documentation AI assistant faced limitations with traditional RAG (Retrieval-Augmented Generation). The system could only retrieve text chunks matching specific queries, leaving it unable to handle answers spanning multiple pages or exact syntax not appearing in top-K results. The team wanted agents to explore documentation like developers explore codebases.

## The Sandbox Bottleneck

The initial approach used isolated sandboxes that cloned repositories and set up environments. While suitable for asynchronous background work, this created critical problems for frontend-facing assistants where users watch loading spinners:

- P90 session creation time: ~46 seconds (including GitHub clone and setup)
- Annual infrastructure cost: $70,000+ at 850,000 monthly conversations (1 vCPU, 2 GiB RAM, 5-minute sessions)
- Pricing baseline: $0.0504/hour per vCPU, $0.0162/hour per GiB RAM

## ChromaFs Solution

Rather than providing real filesystems, the team built ChromaFs—a virtual filesystem that intercepts UNIX commands and translates them into database queries against their existing Chroma collection.

Performance improvements:
- Session creation: ~100 milliseconds (versus 46 seconds)
- Marginal compute cost: $0 (reuses existing database infrastructure)
- Search mechanism: Database metadata queries instead of linear disk scans

## Architecture

ChromaFs extends Vercel Labs' just-bash (a TypeScript bash reimplementation supporting grep, cat, ls, find, and cd) through its pluggable IFileSystem interface. The system handles command parsing while ChromaFs translates each filesystem call into database operations.

### Directory Tree Bootstrap

The file structure is stored as gzipped JSON containing all paths with metadata:

```json
{
  "auth/oauth": { "isPublic": true, "groups": [] },
  "internal/billing": { "isPublic": false, "groups": ["admin", "billing"] }
}
```

On initialization, this decompresses into two in-memory structures—a path set and a directory-to-children mapping. The ls, cd, and find commands then operate without network calls; the tree is cached for subsequent sessions.

### Access Control

The path tree includes isPublic and groups fields. Before building the file tree, ChromaFs filters paths by user session tokens. Unauthorized files are completely excluded, preventing agents from accessing or even referencing pruned paths. This replaces complex Linux group and chmod management.

### Page Reconstruction

Pages split across chunks are reassembled when agents run cat commands. ChromaFs fetches all chunks matching a page slug, sorts by chunk_index, and joins them into complete files. Results cache to prevent repeated database hits during grep workflows.

Lazy file pointers support large OpenAPI specs stored in customer S3 buckets—files appear in directory listings but only fetch content on access.

All write operations return EROFS (Read-Only File System) errors, keeping the system stateless and preventing agent interference.

### Grep Optimization

Recursive grep operations use a two-stage approach:

1. Coarse filter: Chroma identifies files potentially containing matches (using $contains for fixed strings, $regex for patterns)
2. Fine filter: Matched chunks prefetch into Redis cache, then just-bash performs in-memory regex execution on targeted files

This reduces query latency to milliseconds even for large searches.

## Results

ChromaFs powers documentation assistants across 30,000+ daily conversations for hundreds of thousands of users. The system delivers instant session creation and zero marginal compute cost while building role-based access control into the virtual filesystem layer itself—without additional infrastructure investments.