Bitmaps and HyperLogLog

Redis · Practical · 10 min

Daily active users, feature flags and unique counts over millions of ids are cheap with the right structure.

Bitmaps aren't a separate type: they are string values addressed bit by bit. SETBIT key offset 1 sets one bit, GETBIT reads it, BITCOUNT counts set bits, and BITOP AND|OR|XOR|NOT dest k1 k2 combines bitmaps. With integer user ids as offsets, one bitmap per day records who was active: 10 million users take about 1.2 MB. "Active on both days" is a BITOP AND followed by BITCOUNT. BITFIELD packs small integers into a string.

Bitmaps waste space if ids are sparse or huge, because the string grows to the highest offset.

HyperLogLog estimates the number of distinct items. PFADD key item ... adds, PFCOUNT key returns the estimate with a standard error of 0.81%, and PFMERGE combines several, for instance seven daily HLLs into a weekly unique count. Each HLL uses at most 12 KB no matter how many items you add. You can't list or remove members; it only counts.

Choose by question:

  • exact membership and per-id flags with dense integer ids: bitmap,
  • approximate unique count of arbitrary strings: HyperLogLog,
  • exact unique count where memory is fine: set.

Redis 8 also ships Bloom and Cuckoo filters, Count-min sketch and Top-K for other approximate questions.

Example

redis-cli SETBIT active:2026-10-16 42 1      # (integer) 0  (previous bit value)
redis-cli SETBIT active:2026-10-16 7 1       # (integer) 0
redis-cli SETBIT active:2026-10-17 42 1      # (integer) 0
redis-cli BITCOUNT active:2026-10-16         # (integer) 2
redis-cli BITOP AND both active:2026-10-16 active:2026-10-17   # (integer) 6 (bytes)
redis-cli BITCOUNT both                      # (integer) 1   user 42 on both days
redis-cli GETBIT active:2026-10-17 7         # (integer) 0

redis-cli PFADD visitors:mon ana ben cy      # (integer) 1
redis-cli PFADD visitors:tue ben di          # (integer) 1
redis-cli PFMERGE visitors:week visitors:mon visitors:tue   # OK
redis-cli PFCOUNT visitors:week              # (integer) 4

Remember

  • Bitmaps: one bit per integer id, combine with BITOP, count with BITCOUNT
  • HyperLogLog: ~0.81% error unique counts in at most 12 KB
  • PFMERGE turns daily HLLs into weekly or monthly counts
  • Sets for exact counts, HLL when approximate is fine

Watch out for

  • Using a bitmap with very large or sparse ids (like a 64-bit hash) and allocating a gigantic string.

Check yourself

  1. You need the approximate number of distinct search queries per day, possibly hundreds of millions. Which structure?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: redis.io/docs/latest/develop/data-types/probabilistic/hyperloglogs/

Builds on

Unlocks

Nothing builds on this yet.