Calling tasks: delay and apply_async

Celery · Essentials · 10 min

How you call a task decides when and where it runs, and calling it the wrong way silently runs it inline.

A task object has three ways to run:

  • add(2, 3) calls the function directly in the current process, like any function. No broker involved. Useful in tests, a bug in production code.
  • add.delay(2, 3) sends a message and returns immediately with an AsyncResult. It's a shortcut for apply_async with positional and keyword arguments.
  • add.apply_async(args=(2, 3), kwargs={}, **options) accepts execution options.

The most used options:

  • countdown=60: run no earlier than 60 seconds from now.
  • eta=datetime: run no earlier than an absolute (timezone-aware) time.
  • expires=300 or a datetime: discard the task if it hasn't started by then, good for stale notifications.
  • queue="emails": send to a specific queue.
  • priority, retry and retry_policy (for publishing when the broker is briefly down).

Countdown and ETA are "not before" times, not exact schedules: a busy worker starts it later. For recurring jobs use beat instead.

A common pattern in web apps is to enqueue after the database transaction commits, so the worker doesn't look up a row that isn't visible yet. In Django that's transaction.on_commit(lambda: task.delay(obj.id)); in other frameworks, call delay() after commit().

Example

from datetime import datetime, timedelta, timezone
from celery import Celery

app = Celery("tasks", broker="redis://localhost:6379/0")

@app.task
def notify(user_id, message):
    print(f"notify {user_id}: {message}")

notify(1, "inline")                       # runs right here, no broker
r = notify.delay(1, "queued")             # message sent, returns at once
print(r.id)                               # e.g. 'd9b1c6e2-...'

notify.apply_async(args=(2, "later"), countdown=60)
notify.apply_async(
    args=(3, "flash sale"),
    eta=datetime.now(timezone.utc) + timedelta(hours=1),
    expires=datetime.now(timezone.utc) + timedelta(hours=2),
    queue="notifications",
)

Remember

  • task(...) runs inline; .delay(...) enqueues
  • apply_async adds options: countdown, eta, expires, queue
  • countdown/eta mean 'not before', not 'exactly at'
  • Enqueue after the database commit so workers see the data

Watch out for

  • Calling task(args) instead of task.delay(args) and blocking the request.

Check yourself

  1. What does send_email(42) do when send_email is a Celery task?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.celeryq.dev/en/stable/userguide/calling.html