Queues and jobs

Laravel · Advanced · 15 min

Sending email, calling APIs and processing files in the request makes apps slow and fragile; queues fix that.

A job is a class representing work to do later: php artisan make:job ProcessPodcast. It implements ShouldQueue and has a handle() method, whose dependencies are injected by the container.

Dispatch it with ProcessPodcast::dispatch($podcast). The job is serialised and stored by the queue driver: database (the default in new apps), redis, sqs, beanstalkd, or sync (runs immediately, handy for tests). A worker process picks jobs up: php artisan queue:work. In production, keep workers alive with Supervisor or systemd, and restart them on deploy with php artisan queue:restart, since they keep old code in memory.

Models passed to a job are stored as identifiers (thanks to SerializesModels) and re-fetched fresh when the job runs.

Reliability controls:

  • public $tries = 5; and public $backoff = [10, 60, 300]; for retries; retryUntil() for a deadline; $timeout for runaway jobs.
  • failed(Throwable $e) runs after the final failure; failed jobs land in the failed_jobs table (queue:failed, queue:retry all).
  • Make jobs idempotent, since a job can run more than once.

Dispatching inside a database transaction is risky: the worker may run before the commit and not find the row. Use ->afterCommit() or the after_commit connection option.

Extras: ->delay(now()->addMinutes(10)), ->onQueue('emails'), Bus::chain([...]) for sequences, Bus::batch([...]) for groups with progress and callbacks, ShouldBeUnique and rate-limiting middleware. Horizon gives Redis queues a dashboard and auto-balancing.

Example

<?php
// app/Jobs/GenerateInvoicePdf.php
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class GenerateInvoicePdf implements ShouldQueue
{
    use Queueable;
    public int $tries = 4;
    public array $backoff = [10, 60, 300];      // seconds between attempts
    public function __construct(public Invoice $invoice) {}   // stored as an id
    public function handle(PdfRenderer $pdf): void            // injected
    {
        if ($this->invoice->pdf_path) return;                  // idempotent
        $path = $pdf->render('invoices.show', ['invoice' => $this->invoice]);
        $this->invoice->update(['pdf_path' => $path]);
    }
    public function failed(Throwable $e): void
    {
        $this->invoice->owner->notify(new InvoiceFailed($this->invoice));
    }
}
// Controller: respond fast, work later (after the transaction commits)
GenerateInvoicePdf::dispatch($invoice)->onQueue('pdfs')->afterCommit();
// $ php artisan queue:work --queue=pdfs,default --tries=4

Remember

  • Jobs implement ShouldQueue and are processed by queue:work workers
  • Configure tries, backoff and failed(); design jobs to be idempotent
  • Dispatch after commit and restart workers on deploy

Watch out for

  • Deploying new code without queue:restart, so long-running workers keep executing the old version.

Check yourself

  1. Why use ->afterCommit() when dispatching a job inside a DB transaction?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: laravel.com/docs/12.x/queues