TopolearnWithout confirms a publisher can't know whether the broker actually stored a message; this is where silent data loss happens.
Three separate settings decide whether a message survives problems:
- Durable queue (
durable=True): the queue definition survives a broker restart. Quorum queues and streams are always durable. - Persistent message (
delivery_mode=2,pika.DeliveryMode.Persistent): the message is written to disk. For classic queues, a persistent message in a durable queue survives restarts; a transient message does not. Quorum queues and streams store all messages durably. - Publisher confirms: the broker acknowledges each published message after it has been routed and safely stored (written to disk, and replicated for quorum queues). Without confirms, a message lost in a network failure or broker crash just disappears.
Enable confirms on a channel with confirm_delivery() in pika; each basic_publish then waits for the confirm, and raises UnroutableError (with mandatory=True) or NackError on failure. Waiting per message is slow; high-throughput publishers keep many messages in flight and handle confirms asynchronously, or publish in batches and wait for the batch.
Even with confirms, the publisher needs a strategy for failures: retry (and accept possible duplicates), buffer locally, or fail the request. Combine confirms with consumer acks for end-to-end at-least-once delivery.
Example
import pika
from pika.exceptions import UnroutableError, NackError
ch = pika.BlockingConnection(pika.ConnectionParameters("localhost")).channel()
ch.queue_declare(queue="orders", durable=True) # survives restarts
ch.confirm_delivery() # enable publisher confirms
props = pika.BasicProperties(delivery_mode=pika.DeliveryMode.Persistent,
content_type="application/json")
try:
ch.basic_publish("", "orders", b'{"id": 1}', properties=props, mandatory=True)
print("confirmed by broker") # stored safely
ch.basic_publish("", "no-such-queue", b"x", properties=props, mandatory=True)
except UnroutableError:
print("returned: no queue matched") # returned: no queue matched
except NackError:
print("broker could not store it; retry later")Remember
- Durable queue + persistent message survive a broker restart
- Publisher confirms tell you the broker stored the message
- mandatory=True returns unroutable messages instead of dropping them
Watch out for
- Assuming durable=True on the queue makes messages persistent; classic queue messages also need delivery_mode=2.