Transactions and concurrency

Django · Advanced · 14 min

Multi-step writes and concurrent requests corrupt data unless you control transactions and locking.

By default Django runs in autocommit mode: each query commits immediately. When several writes must succeed or fail together, wrap them in `transaction.atomic()`, as a decorator or a context manager. If the block raises, everything in it is rolled back.

Nested atomic blocks create savepoints, so an inner block can fail and roll back without aborting the outer one, if you catch the exception outside the inner block. Catching a database error inside an atomic block and continuing is wrong: the transaction is broken, and the next query raises TransactionManagementError.

ATOMIC_REQUESTS = True in a database's settings wraps every view in a transaction. It's simple, but holds transactions open for the whole request; explicit atomic blocks are more precise.

Side effects after commit. Sending emails, enqueuing tasks or calling APIs inside a transaction is risky: the transaction may roll back after the email went out, or a worker may run before the data is committed. Use transaction.on_commit(callback), which runs only after a successful commit and is discarded on rollback.

Concurrency. Two requests reading then writing the same row can overwrite each other. Options:

  • F() expressions for arithmetic updates
  • select_for_update() inside atomic locks the selected rows until commit (nowait=True or skip_locked=True for queues)
  • optimistic locking with a version column checked in filter(...).update(...)
  • unique constraints, and handling IntegrityError from get_or_create races

Keep transactions short: never wait on slow network calls while holding locks.

Example

from django.db import IntegrityError, transaction
from django.db.models import F
from shop.models import Account, Order, Transfer
from shop.tasks import send_receipt
@transaction.atomic
def transfer(from_id, to_id, amount):
    # lock both rows in a consistent order to avoid deadlocks
    accounts = {a.pk: a for a in Account.objects.select_for_update().filter(pk__in=[from_id, to_id]).order_by("pk")}
    if accounts[from_id].balance < amount:
        raise ValueError("insufficient funds")            # rolls everything back
    Account.objects.filter(pk=from_id).update(balance=F("balance") - amount)
    Account.objects.filter(pk=to_id).update(balance=F("balance") + amount)
    t = Transfer.objects.create(source_id=from_id, target_id=to_id, amount=amount)
    transaction.on_commit(lambda: send_receipt.enqueue(t.pk))   # only after COMMIT
    return t
def place_order(user, cart):
    with transaction.atomic():
        order = Order.objects.create(user=user)
        try:
            with transaction.atomic():                  # savepoint
                order.apply_coupon(cart.coupon)
        except IntegrityError:
            order.note = "coupon already used"         # outer transaction still fine
        order.save()
    return order

Remember

  • atomic() makes a block all-or-nothing; nesting creates savepoints
  • Catch database errors outside the atomic block that failed
  • on_commit runs side effects only after a successful commit
  • select_for_update locks rows inside a transaction

Watch out for

  • Sending emails or enqueuing jobs inside a transaction that might roll back.

Check yourself

  1. Inside an atomic block you create an order and enqueue a job that reads it. Why use transaction.on_commit?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.djangoproject.com/en/6.0/topics/db/transactions/