A backend API with a background worker and a Redis queue

A backend that sends emails, resizes images or calls slow third-party APIs should not do that work inside the HTTP request. The usual shape is an API that puts a job on a queue and answers at once, and a worker that takes jobs off the queue. On JetDeploy both run from one repository and one image, as two processes of the same App, with a Redis data service as the queue.

This walkthrough uses Node.js with BullMQ. The same layout works for Celery or RQ in Python, Sidekiq in Ruby, or any queue library.

The layout

One image means one build per push, and the API and the worker always run the same version of the code.

// src/queue.js
import { Queue } from "bullmq";
import IORedis from "ioredis";

export function redis() {
  return new IORedis(process.env.REDIS_URL, { maxRetriesPerRequest: null });
}

export const emails = new Queue("emails", {
  connection: redis(),
  defaultJobOptions: {
    attempts: 5,
    backoff: { type: "exponential", delay: 2000 },
    removeOnComplete: { count: 1000 },
    removeOnFail: { age: 7 * 24 * 3600 },
  },
});

// src/web.js
import express from "express";
import { emails } from "./queue.js";

const app = express();
app.use(express.json());

app.post("/signup", async (req, res) => {
  const job = await emails.add("welcome", { to: req.body.email });
  res.status(202).json({ queued: job.id });
});

const server = app.listen(8080, "0.0.0.0", () => console.log("[INFO] web listening on 8080"));

process.on("SIGTERM", () => {
  console.log("[INFO] SIGTERM, closing web");
  server.close(async () => {
    await emails.close();
    process.exit(0);
  });
});

// src/worker.js
import { Worker } from "bullmq";
import { redis } from "./queue.js";

const worker = new Worker(
  "emails",
  async (job) => {
    console.log(`[INFO] sending ${job.name} to ${job.data.to}`);
    // call your email provider here
    return { sent: true };
  },
  { connection: redis(), concurrency: 5 },
);

worker.on("failed", (job, err) => console.log(`[ERROR] job ${job?.id} failed: ${err.message}`));

process.on("SIGTERM", async () => {
  console.log("[INFO] SIGTERM, finishing active jobs");
  await worker.close();
  process.exit(0);
});

worker.close() stops taking new jobs and waits for the active ones. Every deploy sends SIGTERM to the old worker, so without this handler a deploy in the middle of a job interrupts it. BullMQ then retries it later, but the side effect (an email half sent, an API called twice) may already have happened.

4 The Dockerfile

FROM node:24-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY src ./src
USER node
CMD ["node", "src/web.js"]

5 Set it up on JetDeploy

1. 1. Create the Redis service orders-redis. Choose its memory now: it cannot be changed later.
2. 2.

   Set the environment variable on the App, from the service page's password and internal endpoint:

   textCopy

   REDIS_URL=redis://default:<password>@orders-redis:6379

   The connection stays on the organization's private network and uses no TLS.

3. 3.

   Add the processes:

   Process	Command	Port	External	Grace period
   web	empty (uses CMD)	8080	on	30
   worker	node src/worker.js	empty	off	150

4. 4. Push and deploy. Both processes roll out from the same build.

A process without a port has no port check: it counts as ready once its container runs.

Size Redis for the queue

A JetDeploy Redis refuses writes once its memory is full, instead of evicting keys. For a queue that is the right behaviour: eviction would silently drop jobs. BullMQ in fact asks for exactly this policy, noeviction. It does mean the queue has to fit in memory:

• keep removeOnComplete and removeOnFail bounded, as above, or finished jobs pile up forever;
• keep job payloads small: store an id, not the whole document;
• watch the Redis memory chart on the service page after launch.

Size goes from 256 MiB to 96 GiB, 1 GiB by default. It is also the only size you set: the dataset cap and the volume follow from it, and Redis writes its data to disk with an append-only file.

Scaling

All processes share the organization's plan of vCPU and memory. The metrics on the App page show how much each one uses.

Tested locally with BullMQ 6.3, ioredis 6.0, Express 5.2, Node.js 24 and Redis 8 with maxmemory-policy noeviction: three jobs were queued through the API, and the worker finished all of them after receiving SIGTERM mid-job. The JetDeploy steps follow the documentation.