MULTI/EXEC and WATCH

Redis · Practical · 11 min

Some updates span several keys or depend on a read, and must not interleave with other clients.

MULTI starts a transaction; subsequent commands are queued (the server replies QUEUED), and EXEC runs them all back to back with no other client's commands in between. DISCARD drops the queue.

Redis transactions are different from SQL ones:

  • there is no rollback: if a command fails at runtime (say INCR on a non-number), the others still run,
  • syntax errors detected while queuing make EXEC abort the whole transaction,
  • you can't read a value and branch on it inside MULTI, because replies only arrive at EXEC.

For check-then-act logic use optimistic locking with WATCH. WATCH key before MULTI; read the values you need; queue the writes; call EXEC. If any watched key changed in the meantime, EXEC returns nil and nothing runs, so you retry. This is check-and-set (CAS).

In redis-py, pipeline() gives you MULTI/EXEC, and r.transaction(func, *keys) wraps the WATCH-retry loop.

For complex logic, or when contention makes WATCH retry often, a Lua script or function is usually simpler and faster: it runs atomically on the server and can read and write in one step.

Example

import redis

r = redis.Redis(decode_responses=True)
r.set("stock:pen", 3)

def buy(pipe):                      # called inside WATCH stock:pen
    stock = int(pipe.get("stock:pen"))
    if stock <= 0:
        raise ValueError("sold out")
    pipe.multi()                    # start queuing
    pipe.decr("stock:pen")
    pipe.rpush("orders", "pen")

for _ in range(3):
    r.transaction(buy, "stock:pen")  # retries automatically if stock:pen changes
print(r.get("stock:pen"), r.llen("orders"))   # 0 3

# no rollback: the failing INCR doesn't undo the SET
with r.pipeline() as p:
    p.set("a", "x").incr("a").set("b", "y")
    print(p.execute(raise_on_error=False))
# [True, ResponseError (value is not an integer...), True]

Remember

  • MULTI queues, EXEC runs the queue atomically
  • No rollback: runtime errors don't undo other commands
  • WATCH + EXEC gives optimistic check-and-set; nil means retry
  • Use Lua when logic must read and decide atomically

Watch out for

  • Expecting a failed command inside EXEC to roll back the rest.

Check yourself

  1. Inside MULTI, the second of three commands fails at runtime with WRONGTYPE. What happens?
  2. You WATCH balance, read it, then MULTI ... EXEC, but another client changed balance in between. What does EXEC do?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: redis.io/docs/latest/develop/interact/transactions/