# The Secret Life of Data in Valkey

Kyle Davis on the Valkey blog (21 July 2026), on the layer between the data model you program against and what actually sits in RAM. Valkey exposes strings, hashes, lists, sets, sorted sets and streams, with commands per type. Underneath, each key carries an *encoding* chosen by the server, and the choice is worth real money.

## One byte

Three hashes, values differing by a single character each:

```
> HSET hash0 field "xxxx…"   # 63-byte value
> HSET hash1 field "xxxx…"   # 64-byte value
> HSET hash2 field "xxxx…"   # 65-byte value
```

`MEMORY USAGE` reports 104, 120 and 212 bytes. The step from 63 to 64 costs 15.38%; the step from 64 to 65 costs 76.67%. `OBJECT ENCODING` explains it: `hash0` and `hash1` are `listpack`, `hash2` is `hashtable`.

A listpack stores elements sequentially in one contiguous chunk of memory, saving the pointer overhead a hash table needs. It is not the hash table you'd assume from the command name, and the abstraction runs the other way too — `LPUSH list1 foo` gives a list encoded as `listpack`, while `SADD` of a 65-byte member gives a set encoded as `hashtable`. Davis's framing is that the classical data structure is the model, not the storage: "Conventional wisdom is that Valkey stores data using the classical data structures like hash tables, linked lists, and sets based on the commands you use. Turns out: no."

The 15.38% jump between 63 and 64 bytes, where the encoding did *not* change, is a separate effect. Listpack overhead is a stair step rather than a line: with the same key pattern and field name, value lengths 49 through 63 all cost 104 bytes, and 64 through 79 all cost 120. So a 64-byte and a 65-byte value both report 120 in the cost tables further down the post even though only one of them is still a listpack.

## The thresholds

Strings use hard-coded logic. Every other type picks its encoding from configuration thresholds — at or below, one encoding; above, another. The Valkey 9.1 defaults for hashes:

```
hash-max-listpack-entries 512
hash-max-listpack-value 64
```

A hash stays a listpack while every value is at most 64 bytes and it holds fewer than 512 field-value pairs. Cross either line and the whole key converts to `hashtable`. The equivalents elsewhere are `list-max-listpack-size`, `set-max-intset-entries` / `set-max-listpack-entries` / `set-max-listpack-value`, and `zset-max-listpack-entries` / `zset-max-listpack-value`. Streams have encodings but no thresholds, because there's only one encoding to pick. Module types are up to the module.

## The cost arithmetic

This is where the post is actually aimed. If your hash values cluster just above 64 bytes, you are paying `hashtable` prices for data that would fit in a listpack under a slightly larger threshold, and the savings scale with the fleet.

Davis's deliberately extreme case: a 100 GB cluster on five 20 GB primaries, with 95 GB of keys sitting one byte over the threshold at 212 bytes each. Raising `hash-max-listpack-value` brings those to 120 bytes, 56.6% of the original, and the cluster drops to 58.8 GB — three primaries instead of five, plus whatever replicas hung off the two that went away.

The milder case is more useful. With 60% of keys above the threshold, the same conversion takes the cluster from 100 GB to 74 GB. That's still five primaries, but each holds 14.8 GB instead of 20, which fits on a 16 GB instance. The point is the pricing breakpoint rather than the percentage: cloud and on-premises hardware both price in tiers, and the win is dropping under one you were barely cresting. Where the instance size can't change, the reclaimed memory goes to caching more, evicting less, or longer TTLs.

The method for finding candidates is unglamorous and doesn't need tooling: sample keys with `OBJECT ENCODING`, find the ones on the expensive encoding, work out which threshold pushed them there, and decide. The caveat Davis attaches is that the thresholds are not a free win: the listpack encoding is compact because it stores elements sequentially, and pushing the numbers up arbitrarily "could result in suboptimal performance or efficiency." His own example is careful to specify low throughput needs alongside the 65-byte values.

## Where it fits

[[redis-cost-of-ambition]] has Charles Leifer arguing that Valkey won by investing in "unglamorous work — multi-threaded performance, memory efficiency, cluster reliability and throughput" instead of chasing feature bullet points. This post is that thesis as a blog entry: no new data type, no new subsystem, just a walkthrough of an existing knob and the instance count it can remove from a bill. The encoding machinery is inherited rather than invented; Valkey is a Redis fork and `src/listpack.c` came with it. That makes it a good illustration of Leifer's actual argument, which is not that [[antirez]]'s original design was wrong but that the handful of tasteful primitives was already enough and the work left to do was on how well they run.

The general shape recurs across storage engines: a compact representation for small objects and a pointer-heavy one past a size cutoff, with a threshold nobody tunes. [[lsm-tree]] and [[lsm-trees-nosql]] cover the on-disk version of the same trade, where write amplification and read amplification sit on opposite ends of a tuning knob that also ships with a default most people never touch.