TopolearnA single hung task can occupy a worker slot forever, and limits keep the system responsive.
Celery offers two limits, per task (@app.task(soft_time_limit=60, time_limit=90)) or globally (task_soft_time_limit, task_time_limit):
- The soft time limit raises
SoftTimeLimitExceededinside the task, giving it a chance to clean up (delete temp files, mark the job failed) and exit. - The hard time limit kills the worker child process and replaces it. No cleanup; the task fails with
TimeLimitExceeded.
Set the hard limit a bit above the soft one. Time limits are enforced by the prefork pool using signals; other pools have limited or no support, and limits don't work on Windows.
Time limits complement, not replace, timeouts in your own code: always pass timeout= to HTTP clients and database calls, which fail faster and more precisely.
Revoking: result.revoke() or app.control.revoke(task_id) tells workers to skip a task that hasn't started. revoke(task_id, terminate=True) also kills a running task's process, a blunt last resort. Revocations are kept in worker memory, so they're lost when all workers restart unless you configure persistent revokes (--statedb).
Use expires when calling tasks that are pointless after a deadline; it's cheaper than revoking.
Example
import time
from celery import Celery
from celery.exceptions import SoftTimeLimitExceeded
app = Celery("proj", broker="redis://localhost:6379/0")
@app.task(soft_time_limit=30, time_limit=40)
def render_video(job_id):
try:
for frame in range(10_000):
time.sleep(0.01) # real work here
return "done"
except SoftTimeLimitExceeded:
print(f"job {job_id}: out of time, cleaning up")
# remove temp files, mark job as failed in the DB
raise
# r = render_video.delay(7)
# r.revoke() # skip it if not started yet
# app.control.revoke(r.id, terminate=True) # kill it if runningRemember
- Soft limit raises SoftTimeLimitExceeded so the task can clean up
- Hard limit kills the child process, no cleanup
- Limits rely on the prefork pool's signals
- revoke() skips pending tasks; terminate=True kills running ones
Watch out for
- Catching SoftTimeLimitExceeded with a bare except: Exception and continuing the work until the hard kill.