A backend API with a background worker and a Redis queue

A backend that sends emails, resizes images or calls slow third-party APIs should not do that work inside the HTTP request. The usual shape is an API that puts a job on a queue and answers at once, and a worker that takes jobs off the queue. On JetDeploy both run from one repository and one image, as two processes of the same App, with a Redis data service as the queue.

This walkthrough uses Node.js with BullMQ. The same layout works for Celery or RQ in Python, Sidekiq in Ruby, or any queue library.

The layout

One image means one build per push, and the API and the worker always run the same version of the code.

1. The shared queue module

// src/queue.js
import { Queue } from "bullmq";
import IORedis from "ioredis";

export function redis() {
  return new IORedis(process.env.REDIS_URL, { maxRetriesPerRequest: null });
}

export const emails = new Queue("emails", {
  connection: redis(),
  defaultJobOptions: {
    attempts: 5,
    backoff: { type: "exponential", delay: 2000 },
    removeOnComplete: { count: 1000 },
    removeOnFail: { age: 7 * 24 * 3600 },
  },
});

BullMQ 6 no longer ships a Redis client, so pass an ioredis connection. maxRetriesPerRequest: null is what BullMQ requires for workers.

The removeOnComplete and removeOnFail options matter more here than usual. See Size Redis for the queue below.

2. The API

// src/web.js
import express from "express";
import { emails } from "./queue.js";

const app = express();
app.use(express.json());

app.post("/signup", async (req, res) => {
  const job = await emails.add("welcome", { to: req.body.email });
  res.status(202).json({ queued: job.id });
});

const server = app.listen(8080, "0.0.0.0", () => console.log("[INFO] web listening on 8080"));

process.on("SIGTERM", () => {
  console.log("[INFO] SIGTERM, closing web");
  server.close(async () => {
    await emails.close();
    process.exit(0);
  });
});

3. The worker

// src/worker.js
import { Worker } from "bullmq";
import { redis } from "./queue.js";

const worker = new Worker(
  "emails",
  async (job) => {
    console.log(`[INFO] sending ${job.name} to ${job.data.to}`);
    // call your email provider here
    return { sent: true };
  },
  { connection: redis(), concurrency: 5 },
);

worker.on("failed", (job, err) => console.log(`[ERROR] job ${job?.id} failed: ${err.message}`));

process.on("SIGTERM", async () => {
  console.log("[INFO] SIGTERM, finishing active jobs");
  await worker.close();
  process.exit(0);
});

worker.close() stops taking new jobs and waits for the active ones. Every deploy sends SIGTERM to the old worker, so without this handler a deploy in the middle of a job interrupts it. BullMQ then retries it later, but the side effect (an email half sent, an API called twice) may already have happened.

4. The Dockerfile

FROM node:24-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY src ./src
USER node
CMD ["node", "src/web.js"]

The CMD starts the API. The worker process overrides it with its own command.

5. Set it up on JetDeploy

1. Create the Redis service orders-redis. Choose its memory now: it cannot be changed later.
2. Set the environment variable on the App, from the service page's password and internal endpoint:

text REDIS_URL=redis://default:<password>@orders-redis:6379

The connection stays on the organization's private network and uses no TLS. 3. Add the processes:

1. Push and deploy. Both processes roll out from the same build.

A process without a port has no port check: it counts as ready once its container runs.

Size Redis for the queue

A JetDeploy Redis refuses writes once its memory is full, instead of evicting keys. For a queue that is the right behaviour: eviction would silently drop jobs. BullMQ in fact asks for exactly this policy, noeviction. It does mean the queue has to fit in memory:

• keep removeOnComplete and removeOnFail bounded, as above, or finished jobs pile up forever;
• keep job payloads small: store an id, not the whole document;
• watch the Redis memory chart on the service page after launch.

Memory is chosen in multiples of 128 MiB, from 128 to 16384, 256 by default. It is also the only size you set: the container limit and the volume follow from it, and Redis writes its data to disk with an append-only file.

Scaling

• More throughput: raise the worker replicas (1 to 5), or its concurrency for I/O-bound jobs. Several workers on the same queue share the jobs without extra setup.
• More API capacity: raise the web replicas. The API holds no state, so it scales independently of the worker.
• Scheduled jobs: BullMQ's job schedulers (queue.upsertJobScheduler) put recurring jobs on the same queue, and the worker you already have runs them. No separate scheduler process is needed.

All processes share the organization's plan of vCPU and memory. The metrics on the App page show how much each one uses.

Next step

Watch the worker with the runtime logs of the worker process, filtered to level=error to see the failed jobs. The worker above logs plain text; if job errors can mention words like "info", log JSON with a level field so the filter reads the level instead of guessing it. When the API gets a public domain, Custom domain and HTTPS goes through the DNS records and the certificate.

Tested locally with BullMQ 6.3, ioredis 6.0, Express 5.2, Node.js 24 and Redis 8 with maxmemory-policy noeviction: three jobs were queued through the API, and the worker finished all of them after receiving SIGTERM mid-job. The JetDeploy steps follow the documentation.