TopolearnQueues decouple services, and getting receive/delete semantics wrong causes duplicate processing or lost work.
An SQS queue stores messages until a consumer deletes them. You address it by QueueUrl (get_queue_url(QueueName=...) looks it up).
send_message(QueueUrl=..., MessageBody="...")enqueues a message (up to 256 KB of text; send JSON withjson.dumps).send_message_batchsends up to 10 at once.receive_message(QueueUrl=..., MaxNumberOfMessages=10, WaitTimeSeconds=20)returns up to 10 messages.WaitTimeSecondsenables long polling: the call waits up to 20 seconds for messages instead of returning empty immediately, which cuts cost and latency.delete_message(QueueUrl=..., ReceiptHandle=msg["ReceiptHandle"])acknowledges a message after successful processing.
Receiving does not remove a message. It becomes invisible for the queue's visibility timeout (default 30 seconds); if you don't delete it in time, it reappears for another consumer. So: process, then delete; set the visibility timeout longer than processing takes, or extend it with change_message_visibility.
Standard queues deliver at least once and don't guarantee order: handlers must be idempotent. FIFO queues (names ending in .fifo) preserve order within a MessageGroupId and deduplicate within 5 minutes using a MessageDeduplicationId or content-based deduplication.
Configure a dead-letter queue (a redrive policy with maxReceiveCount) so poison messages stop cycling forever.
Example
import json
import boto3
sqs = boto3.client("sqs", region_name="us-east-1")
url = sqs.create_queue(QueueName="orders",
Attributes={"VisibilityTimeout": "60"})["QueueUrl"]
sqs.send_message_batch(QueueUrl=url, Entries=[
{"Id": str(i), "MessageBody": json.dumps({"order_id": i})} for i in range(3)])
def handle(order):
print("processing", order["order_id"])
while True:
resp = sqs.receive_message(QueueUrl=url, MaxNumberOfMessages=10, WaitTimeSeconds=1)
messages = resp.get("Messages", [])
if not messages:
break # real workers keep polling (WaitTimeSeconds=20)
for msg in messages:
handle(json.loads(msg["Body"]))
sqs.delete_message(QueueUrl=url, ReceiptHandle=msg["ReceiptHandle"])
# processing 0 / processing 1 / processing 2 (order not guaranteed)Remember
- receive_message doesn't remove messages; delete after processing
- Long polling with WaitTimeSeconds=20 reduces empty responses and cost
- Visibility timeout must exceed processing time
- Standard queues are at-least-once and unordered; FIFO queues order per group
Watch out for
- Deleting the message before processing it, losing work when the handler crashes.