Graceful shutdown in Go: deploys that finish every request

Every deploy, restart and stop on JetDeploy ends the old process the same way: it receives SIGTERM, and if it is still running when the shutdown grace period ends, it is killed. A Go HTTP server that does nothing about SIGTERM exits at once, and every request still in flight is cut off. With a few lines of net/http, the same server finishes those requests first.

What goes wrong by default

A minimal server:

func main() {
    http.HandleFunc("GET /slow", func(w http.ResponseWriter, r *http.Request) {
        time.Sleep(10 * time.Second)
        w.Write([]byte("finished\n"))
    })
    http.ListenAndServe("0.0.0.0:8080", nil)
}

We started it in a container, opened a request to /slow, and sent SIGTERM two seconds later:

On a deploy, the edge only sends traffic to the new version once it is ready, but requests already running on the old one are not moved anywhere. Checkouts, uploads and slow reports are the ones that fail.

The fix

package main

import (
    "context"
    "errors"
    "log"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("ok\n"))
    })

    srv := &http.Server{
        Addr:              "0.0.0.0:8080",
        Handler:           mux,
        ReadHeaderTimeout: 10 * time.Second,
    }

    ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt)
    defer stop()

    go func() {
        log.Printf("[INFO] listening on %s", srv.Addr)
        if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
            log.Fatalf("[ERROR] %v", err)
        }
    }()

    <-ctx.Done()
    log.Print("[INFO] SIGTERM received, draining")

    shutdownCtx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
    defer cancel()
    if err := srv.Shutdown(shutdownCtx); err != nil {
        log.Printf("[ERROR] shutdown: %v", err)
        os.Exit(1)
    }
    log.Print("[INFO] stopped cleanly")
}

What each part does:

• signal.NotifyContext turns SIGTERM (and Ctrl-C locally) into a cancelled context instead of an immediate exit.
• srv.Shutdown closes the listener, waits for active requests to finish, then returns.
• The 25-second timeout is shorter than the default grace period of 30 seconds. The server gives up on stuck requests and exits by itself, logging why, before the platform kills it without a trace.
• [INFO] and [ERROR] prefixes are recognised by the log viewer, so filtering the runtime logs by level works without JSON logging.

The Dockerfile

FROM golang:1.27-alpine AS build
WORKDIR /src
COPY go.* ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/api .

FROM alpine:3.24
RUN adduser -D -u 10001 app
COPY --from=build /out/api /usr/local/bin/api
USER app
EXPOSE 8080
CMD ["api"]

The final image is about 22 MB. It could be smaller on scratch or a distroless base, but Alpine brings /bin/sh, which pays off on JetDeploy:

• the browser shell on the App page needs a shell to open;
• a one-off command started through the API is stopped cleanly when its timeout expires only if the image has /bin/sh. Without it, the command keeps running in the container.

CMD ["api"] makes the binary PID 1, so the signal reaches it directly. If you ever start it through a wrapper script, for example to run a setup step first, end the script with exec api. Otherwise the shell stays PID 1, receives SIGTERM in place of the server, and the Go handler never runs.

Configure the process

The readiness check is a TCP connection to the container port. Bind the port only when the server can actually answer. If it has to load something first (a cache, a model, a large config), do it before ListenAndServe, or set a warmup delay.

What a rolling deploy looks like now

1. The new version starts and binds 0.0.0.0:8080.
2. It passes the port check and starts receiving new requests.
3. The old version receives SIGTERM, stops accepting connections and finishes what it has.
4. It exits with status 0 and the log line stopped cleanly.

Before the SIGTERM, the old copy of a process with a port also gets 5 seconds without new requests, so connections already on their way are not lost either.

Next step

Background goroutines need the same treatment: pass them the context from signal.NotifyContext and let them stop at a safe point. The Start, stop and restart section of the docs describes the signal sequence for every kind of operation.

Tested locally with Go 1.27 and Alpine 3.24: the image was built, and both versions of the server were sent SIGTERM during a 10-second request. The JetDeploy steps follow the documentation.