TopolearnThe storage engine explains why some databases ingest writes fast and others read fast, and how to tune each.
Under every database is a storage engine that decides how data is laid out on disk. Two families dominate.
B-tree engines (PostgreSQL, InnoDB, SQLite) keep sorted pages and update them in place (PostgreSQL writes new row versions into heap pages, but indexes are B-trees). Reads are predictable: a few page reads per lookup. Writes may dirty a whole page to change a few bytes, plus a write-ahead log entry for crash safety, so random writes cost several disk writes each.
LSM-tree engines (RocksDB, Cassandra, ScyllaDB, LevelDB) buffer writes in an in-memory sorted table (memtable) plus a log, then flush it as an immutable sorted file (SSTable). Background compaction merges files and discards overwritten and deleted entries (tombstones). Writes are sequential and very fast. Reads may need to check the memtable and several SSTables; Bloom filters let most files be skipped quickly.
Engineers compare them by three amplifications:
- Write amplification: bytes written to disk per byte of user data (compaction rewrites data repeatedly; B-trees rewrite pages).
- Read amplification: disk reads per lookup (higher for LSM).
- Space amplification: extra disk used by stale versions (LSM before compaction, B-tree page fragmentation or MVCC bloat).
Compaction strategy (leveled vs size-tiered) trades these against each other. Choose LSM-based stores for write-heavy, append-like workloads; B-trees for read-heavy, mixed OLTP with range scans.
Remember
- B-trees update pages in place; LSM trees append and compact
- LSM: fast sequential writes, more read work (mitigated by Bloom filters)
- Compare engines by write, read and space amplification