Graceful shutdown in Go: deploys that finish every request

Every deploy, restart and stop on JetDeploy ends the old process the same way: it receives SIGTERM, and if it is still running when the shutdown grace period ends, it is killed. A Go HTTP server that does nothing about SIGTERM exits at once, and every request still in flight is cut off. With a few lines of net/http, the same server finishes those requests first.

What goes wrong by default

A minimal server:

func main() {
	http.HandleFunc("GET /slow", func(w http.ResponseWriter, r *http.Request) {
		time.Sleep(10 * time.Second)
		w.Write([]byte("finished\n"))
	})
	http.ListenAndServe("0.0.0.0:8080", nil)
}

We started it in a container, opened a request to /slow, and sent SIGTERM two seconds later:

On a deploy, the edge only sends traffic to the new version once it is ready, but requests already running on the old one are not moved anywhere. Checkouts, uploads and slow reports are the ones that fail.

The fix

package main

import (
	"context"
	"errors"
	"log"
	"net/http"
	"os"
	"os/signal"
	"syscall"
	"time"
)

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("GET /", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("ok\n"))
	})

	srv := &http.Server{
		Addr:              "0.0.0.0:8080",
		Handler:           mux,
		ReadHeaderTimeout: 10 * time.Second,
	}

	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt)
	defer stop()

	go func() {
		log.Printf("[INFO] listening on %s", srv.Addr)
		if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
			log.Fatalf("[ERROR] %v", err)
		}
	}()

	<-ctx.Done()
	log.Print("[INFO] SIGTERM received, draining")

	shutdownCtx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
	defer cancel()
	if err := srv.Shutdown(shutdownCtx); err != nil {
		log.Printf("[ERROR] shutdown: %v", err)
		os.Exit(1)
	}
	log.Print("[INFO] stopped cleanly")
}

The Dockerfile

FROM golang:1.27-alpine AS build
WORKDIR /src
COPY go.* ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/api .

FROM alpine:3.24
RUN adduser -D -u 10001 app
COPY --from=build /out/api /usr/local/bin/api
USER app
EXPOSE 8080
CMD ["api"]

The final image is about 22 MB. It could be smaller on scratch or a distroless base, but Alpine brings /bin/sh, which pays off on JetDeploy:

• the browser shell on the App page needs a shell to open;
• a one-off command started through the API is stopped cleanly when its timeout expires only if the image has /bin/sh. Without it, the command keeps running in the container.

Configure the process

The readiness check is a TCP connection to the container port. Bind the port only when the server can actually answer. If it has to load something first (a cache, a model, a large config), do it before ListenAndServe, or set a warmup delay.

What a rolling deploy looks like now

1. newThe new version starts and binds 0.0.0.0:8080.
2. newIt passes the port check and starts receiving new requests.
3. oldThe old version receives SIGTERM, stops accepting connections and finishes what it has.
4. oldIt exits with status 0 and the log line stopped cleanly.

Before the SIGTERM, the old copy of a process with a port also gets 5 seconds without new requests, so connections already on their way are not lost either.

Tested locally with Go 1.27 and Alpine 3.24: the image was built, and both versions of the server were sent SIGTERM during a 10-second request. The JetDeploy steps follow the documentation.