Engines

The eight embedded Go key/value engines kvbench measures, one page each: what it is, what it is best at with real numbers, and what to watch out for.

Eight embedded key/value engines, all pure Go, all added with go get, all measured through the identical harness. Each page below is a short card: the shape, the best workload with real numbers, the weak spot, and when to reach for it.

At a glance

Random reads are cache-resident (100,000 keys of 1 KB); fresh writes and space are out-of-cache (300,000 keys written to disk); durable writes are the FULL regime with a flush on every commit. All on the Apple M4, 8 concurrent clients. Each column names its own setup because a read that fits in cache and a write that spills to disk are different questions.

Engine	Shape	Random reads	Fresh writes	Ordered scan	Space	Durable writes
tamnd/kv	hash-log	6,955,000	5,711,000	no	0.43x	224
badger	LSM	561,000	320,000	yes	7.41x	4,676
pebble	LSM	713,000	158,000	yes	0.13x	383
bbolt	B+tree	698,000	52	yes	2.28x	52
buntdb	in-memory B-tree	3,236,000	16,000	yes	1.03x	99
pogreb	hash-log	1,748,000	17,000	no	1.05x	158
goleveldb	LSM	815,000	28,000	yes	0.11x	463
sqlite	B-tree	52,000	7,500	yes	4.50x	2,152

No engine wins every column. tamnd/kv owns reads and background-flush writes and stays compact on disk; the LSMs own disk footprint and durable batching; the B-trees own ordered scans; badger and sqlite own per-commit durable throughput through group commit. Pick by the column your workload lives in, then read that engine's page.

What "shape" means

hash-log appends every write to one file with an in-memory index. Fastest point reads, no ordered scan, update churn.
LSM buffers writes and merges them in the background. Cheap writes, small on disk, occasional merge latency.
B+tree keeps keys sorted in pages. Great ordered scans, slower random writes.

The start-here pages explain the shapes in full.

tamnd/kv tamnd/kv is a single-file embedded store with a hash-log core: a sharded resident hash index over a hybrid log with an in-memory hot tier. It leads reads, writes, and the read-update mix on this board. badger badger is a pure-Go LSM key/value store with a separate value log. Fast writes, the lowest tail on mixed workloads, group-commit durability, and a large disk footprint. pebble pebble is the pure-Go LSM that backs CockroachDB. It compresses data to an eighth of its size, stays strong on mixed loads, and is the safest default for a durable server workload. bbolt bbolt is a single-file copy-on-write B+tree, the storage under etcd. The simplest engine here, the fastest ordered scanner, and slow at random and durable writes. buntdb buntdb is an in-memory B-tree with an append-only file for durability. Fast at almost everything, as long as the whole dataset fits in RAM. pogreb pogreb is a pure-Go hash-log built for fast point reads on datasets larger than memory. Like tamnd/kv it has no ordered scan, and it is a simpler bare-bitcask design. goleveldb goleveldb is a pure-Go port of LevelDB. No standout strength, no glaring weakness, and the smallest disk footprint of any engine measured. sqlite sqlite (the pure-Go modernc build) is a full SQL database used as a key/value store. Slowest at raw key/value ops, but it brings real SQL and group-commit durability.