Background Jobs

Sidekiq13.5k★

Sidekiq handles background jobs in Ruby: sending emails, processing uploads, generating reports, anything that shouldn't block a web request. It uses Redis to queue jobs and processes them in separate worker threads. Sidekiq's killer feature is throughput. It uses threads instead of processes (unlike Resque, the older alternative), so a single Sidekiq process handles 25+ concurrent jobs by default. A $20/mo VPS running one Sidekiq process can chew through millions of jobs per day. The open source version covers most needs: job queuing, retries with exponential backoff, scheduled jobs, a real-time web dashboard, and dead job management. The web UI is excellent: you can see queue depths, retry counts, and job history at a glance. Sidekiq Pro ($99/mo per process) adds reliable fetch (jobs won't be lost if a process crashes), batches (group jobs and run a callback when all complete), and unique jobs. Sidekiq Enterprise ($179/mo per process) adds rate limiting, periodic jobs (cron replacement), and rolling restarts. The catch: Ruby only. If you're not in the Rails ecosystem, this doesn't apply to you. The paid tiers are per-process, which adds up fast: 4 processes on Enterprise is $716/mo. And you need Redis running alongside it, adding another piece of infrastructure to manage.

agenda9.7k★

Agenda is a lightweight job scheduler for Node.js backed by MongoDB. You define jobs as functions, schedule them with human-readable intervals, and Agenda persists them in Mongo so they survive server restarts. backed by MongoDB for persistence. The API is clean: `agenda.define('send email', async job => {.})` then `agenda.schedule('in 20 minutes', 'send email', {to: 'user@example.com'})`. Supports recurring jobs (cron syntax), one-time jobs, job priorities, concurrency limits, and retry logic. Fully free and open source. No paid tier. You need a MongoDB instance, but that can be a free Atlas cluster for small workloads. Solo devs building Node.js apps with background jobs: solid choice. Small teams: works well up to moderate job volumes. The MongoDB dependency means you're either already using Mongo (easy) or adding a database just for job scheduling (less ideal). The catch: Agenda hasn't had a major release in a while; the maintenance pace has slowed. It's tied to MongoDB, so if your stack is Postgres-based, you're adding infrastructure just for scheduling. For high-throughput job processing (thousands per second), BullMQ with Redis is faster and more actively maintained. Agenda is best for low-to-moderate volume scheduling where you already have MongoDB.

BullMQ8.7k★

BullMQ is the go-to job queue for Node.js applications, backed by Redis, with priorities, scheduling, retries, concurrency limits, and rate limiting built in. You push jobs onto a queue, workers pick them up and process them. Retries, delays, rate limiting, priorities, repeatable jobs, and job dependencies are all built in. It's the Node.js equivalent of Celery (Python) or Sidekiq (Ruby). You define a queue, add jobs to it, and BullMQ handles the rest, including the messy parts like failed job retries, stalled job recovery, and concurrency control. MIT. Requires Redis (or any Redis-compatible store like Valkey or KeyDB). The catch: you need Redis running. That's either a managed Redis service ($15-50/mo) or self-hosted Redis (free but you maintain it). BullMQ also has a paid companion: Bull Dashboard Pro for $99/yr gives you a nicer UI for monitoring jobs. The free Bull Board works fine for most needs. The bigger catch: if your stack isn't Node.js, BullMQ doesn't help you. It's Node-only.

bree3.3k★

Bree handles that with worker threads so your main process doesn't block. Basically, it's cron for your Node.js app, but each job runs in its own thread and can't crash your server. MIT license, JavaScript. Each job runs in a Node.js worker thread with its own memory space. If a job crashes, your app keeps running. Supports cron expressions, intervals, one-time runs, and human-readable schedules ("every 5 minutes"). Built-in concurrency control. Prevent the same job from running twice. Fully free. It's a library. No paid tier, no hosted version. Setup is simple: install the package, define your jobs as separate files, configure the schedule. The worker thread approach means jobs are truly isolated. A memory leak in one job doesn't affect your app. Solo projects and small apps: Bree is a clean solution when you need background jobs without adding Redis or a separate queue system. The catch: Bree runs in-process, which means if your server restarts, scheduled jobs reset. There's no persistence layer. Missed jobs don't retry automatically. For durable job processing that survives restarts, you need BullMQ (backed by Redis) or Temporal. Bree is best for periodic tasks where missing one run isn't catastrophic.