Graceful SDK Shutdown and Flag Flush

Q: How long should the termination grace period be?

Long enough to cover the propagation delay, drain, flush, and connection close with headroom. Measure the worst-case shutdown wall-clock under load and set the period above it; if the platform caps the period, shorten the flush timeout so the flush still gets a bounded chance.

Q: Should I flush on SIGINT and other signals too, or only SIGTERM?

Handle every signal a graceful stop can arrive as, including SIGINT, with identical drain-flush-close behavior. SIGKILL cannot be trapped and always skips your handler, which is why the buffer needs a periodic durable checkpoint rather than relying on the handler alone.

Q: What happens to exposure events for requests still in flight when the deadline hits?

Those evaluations are cut off and any events they had not yet buffered are lost — the residual the drain deadline trades for a bounded exit. Keep it small by capping per-request timeouts below the drain deadline so almost every request finishes and buffers its event first.

Q: Does closing the provider connection flush automatically?

Do not assume so. Some SDKs flush inside shutdown; many discard the buffer when the connection tears down because that connection was its only delivery path. Read your SDK's shutdown implementation, and always call an explicit flush before shutdown regardless.

This how-to is part of Server-Side SDK Integration Patterns. When a pod receives SIGTERM during a deploy, a naive process exit drops whatever the flag SDK was holding — buffered exposure events, an open streaming connection, in-flight evaluations. This how-to shuts the SDK down cleanly: flush the analytics buffer, close the provider connection, and drain in-flight work so a routine rolling deploy does not silently corrupt your experiment data.

The failure is quiet, which is what makes it dangerous. Nothing errors, no request fails, no alert fires — the pod exits zero and the deploy looks healthy. What you lose is a fraction of a percent of exposure events, biased toward whichever pods happened to be replaced during the rollout. On a high-throughput service doing a rolling deploy across dozens of pods, that adds up to a systematic undercount in every experiment that was live during the deploy, and it undercounts unevenly across variants because traffic is rarely balanced at the instant of termination. An experiment platform cannot distinguish “this variant genuinely saw fewer exposures” from “this variant’s pods flushed less on the way out,” so the bias lands directly in your effect-size estimate. Getting shutdown right is not housekeeping — it is a correctness requirement for the numbers your rollout decisions depend on.

The order matters: stop taking new work, let in-flight work finish, flush the events it produced, then close the connection — only then exit.

Prerequisites

A signal handler for SIGTERM (and the platform’s termination grace period)
An SDK that exposes a flush and a close/shutdown method
Knowledge of your termination grace period so shutdown fits inside it
Analytics or exposure events buffered by the SDK (if you track experiments)
A readiness probe you can flip to draining before shutdown

Knowing the grace period is essential — if flush plus close cannot finish before the platform sends SIGKILL, you lose data regardless.

Step-by-Step Procedure

Step 1 — Flip readiness to draining first

On SIGTERM, immediately fail the readiness probe so the load balancer stops sending new requests, while the process keeps serving in-flight ones. Ordering this first is what makes the rest of the sequence bounded — if you start flushing while the load balancer is still routing fresh traffic, every new request produces new exposure events and the buffer never stops growing, so the flush chases a moving target. Flipping readiness turns the drain into a shrinking set with a clear finish line.

Be aware of the propagation gap: failing a readiness probe does not stop traffic instantly. Kubernetes marks the pod NotReady, the Endpoints controller removes it, and kube-proxy or your ingress reprograms its routing — a chain that typically takes a few hundred milliseconds to a couple of seconds. During that window the pod is still receiving new requests even though it has decided to shut down. This is why production shutdown handlers usually sleep for a fixed preStop delay (commonly 5 seconds) after flipping readiness and before starting the drain, giving the data plane time to catch up. Skip that delay and you will still drop the events belonging to requests that arrived in the gap.

# drain.py
draining = False
def on_sigterm(*_):
    global draining
    draining = True          # readiness now returns 503; no new traffic routed

def readiness():
    return 503 if draining else 200

Step 2 — Drain in-flight evaluations

Wait for outstanding requests to complete (bounded by the grace period) so no evaluation is cut off mid-flight and no exposure event is lost.

# drain.py
async def drain_inflight(deadline_s: float):
    start = time.monotonic()
    while inflight_count() > 0 and time.monotonic() - start < deadline_s:
        await asyncio.sleep(0.05)   # let in-flight requests finish

The deadline here is not the same as the grace period — it is a slice of it. Budget the total grace period across all four phases: the preStop propagation delay, this drain, the flush, and the connection close. If the platform grants 30 seconds and you spend 5 on propagation, you might allocate 20 to the drain and reserve 5 for flush-plus-close. Reserve generously; a flush that gets starved because the drain ran long is exactly the data loss you are trying to prevent. Also watch for a request whose own work outlives the deadline — a slow downstream call or a long poll can keep inflight_count() above zero forever. The loop already handles this by giving up at the deadline, but that means those specific evaluations may still be cut off. Cap request-level timeouts below your drain deadline so the drain reliably reaches zero instead of hitting the wall.

Step 3 — Flush buffered exposure events

Force the SDK to send its buffered analytics before closing, so experiment exposures recorded in the last seconds are not dropped.

# flush.py
async def flush_flags(client, timeout_s: float = 3.0):
    try:
        await asyncio.wait_for(client.flush(), timeout=timeout_s)
    except asyncio.TimeoutError:
        log.warning("flag flush timed out; some exposure events may be lost")

Most SDKs buffer exposure events and ship them on a timer — every few seconds or once a batch fills — precisely because sending one HTTP request per evaluation would dwarf the cost of the evaluation itself. That batching is the whole reason a buffer exists at shutdown: on a busy pod there is almost always a partial batch waiting for the next tick that will never come. flush() forces that partial batch out now rather than on the timer. Log the outcome on both paths, not just the timeout — knowing a flush succeeded but took 2.8 seconds tells you the grace period is close to the edge and one slow network moment will push you over. If the SDK returns a count of events flushed, record it; a flush that ships zero events every time is a sign the buffer is being drained elsewhere, or that this client was never the one accumulating exposures. Emit the timeout as a metric, not just a log line, so you can alert when flushes start failing across a fleet rather than discovering it in a post-mortem.

Step 4 — Close the provider connection

Shut down the SDK so its streaming connection and background threads terminate cleanly, releasing the connection back to the provider.

# close.py
async def shutdown(client):
    await flush_flags(client)     # events out first
    await client.shutdown()       # then close the provider connection + threads

The ordering in these two lines is not stylistic — it is the crux of the whole procedure. Many SDKs discard whatever is in the analytics buffer when the connection tears down, because the buffer’s only delivery path is that connection. Call shutdown() first and the flush has nothing to send over. Some SDKs fold a flush into their own shutdown(), which is convenient but hides the timeout: if the built-in flush is unbounded, shutdown() inherits the hang described in step 3. Reading your SDK’s shutdown implementation once — does it flush, does it bound that flush — is worth more than any amount of defensive wrapping. Closing also matters on the provider’s side: a streaming connection left dangling ties up a slot on the flag delivery service until its own idle timeout reaps it, so on a large fleet doing frequent deploys, sloppy close leaks connections faster than the provider expects. A clean shutdown() returns the slot immediately. Finally, make the whole handler idempotent and guarded against a second SIGTERM — orchestrators sometimes re-send the signal, and a double flush-and-close on an already-closed client should be a no-op, not a crash that masks the exit code.

Flushing (step 3) before closing (step 4) is the ordering that saves your experiment data — close first and the buffered events go with the connection.

Verification

Send SIGTERM during load and confirm no exposure events are lost and the connection closes cleanly, with the whole sequence fitting inside the grace period.

# Under steady load, terminate and check the analytics count matches evaluations
kubectl delete pod flag-service-xyz --grace-period=30
# Compare exposure events received downstream vs evaluations served in the last window:
# they should match — no drop at shutdown.

Test under real load, not an idle pod — an idle pod has an empty buffer and will pass a shutdown test that a busy one fails, because the whole failure mode lives in the partial batch that only accumulates under traffic. Drive steady evaluation traffic, terminate mid-stream, and reconcile counts. An exact match is the ideal, but expect to reason in small margins: if your instrumentation samples or if a genuinely in-flight request was cut at the deadline, a handful of events may legitimately differ. What you are ruling out is a structural gap — hundreds or thousands of events short — that scales with buffer size and throughput. Run the test a few times and watch the shape of the loss, not a single number. It also helps to log a timestamped line at each of the four phases and measure the wall-clock span from SIGTERM to exit; if that span creeps toward the grace period under load, you have no headroom for a bad day and should either widen the grace period or tighten the flush timeout before it bites you in production.

Matching counts across a termination is the proof — a gap means events were buffered when the process exited without flushing.

Gotchas & Edge Cases

The hung-flush case is why step 3 has a timeout — a flush that waits forever on an unreachable provider turns a graceful shutdown into a SIGKILL.

The grace period must fit the flush. If the platform’s termination grace period is shorter than drain-plus-flush, SIGKILL arrives mid-flush and events are lost. Measure the worst-case shutdown time and set the grace period above it.
SIGKILL runs no handler. A hard kill (OOM, forced termination) skips your handler entirely. For durability, periodically persist the exposure buffer to disk or a sidecar so an ungraceful death loses at most the last interval, not everything.
A flush can hang on a dead provider. If the provider is unreachable, an unbounded flush blocks shutdown until SIGKILL. Always bound the flush with a timeout (step 3) so a dead provider does not convert a graceful shutdown into a forced one.
preStop and SIGTERM can race. In Kubernetes a preStop hook runs before SIGTERM is delivered, and both eat into the same grace period. If you put the readiness flip in a preStop sleep and the drain in the SIGTERM handler, verify they compose rather than double-count the clock — the grace period starts when termination begins, not when your handler wakes. Budget them together or you will discover the flush had less time than you allocated on paper.
Serverless and short-lived workers may never get a clean signal. On platforms that freeze or reclaim an instance between requests — some function runtimes, spot instances, scale-to-zero workers — there is no reliable SIGTERM window to flush in. There, prefer flushing synchronously at the end of each request or on a very short timer, so the buffer is near-empty whenever the instance vanishes. A shutdown handler you cannot count on running is not a durability strategy.
Log-and-drain background threads need the same treatment. The exposure buffer is often owned by a background thread or task that also needs to stop. If you flush the buffer but leave that thread running, it can enqueue one more batch after the flush and before the connection closes — a small, maddening residual loss. Stop the producer, then flush, then close.

Troubleshooting & FAQ

We lose experiment exposures on every deploy. Where do they go?

They are buffered in the SDK and dropped when the process exits before flushing. Add the flush step to your SIGTERM handler and confirm the grace period is long enough for it to complete. Verify by comparing downstream exposure counts across a deploy.

Does OpenFeature have a standard shutdown method?

Yes — OpenFeature exposes a shutdown that closes providers and lets them flush. Call it in your termination handler, and if your provider buffers analytics separately, flush that too before invoking shutdown so nothing is left in a buffer that the connection close discards.

Should shutdown block the process exit?

Yes, but bounded. Block on drain-and-flush up to a deadline safely inside the grace period, then exit regardless. Blocking guarantees a clean shutdown in the common case; the deadline guarantees you exit before SIGKILL in the pathological one.

How long should the termination grace period be?

Long enough to cover the sum of the propagation delay, the drain, the flush, and the connection close, with headroom for a slow network moment. Measure the worst-case shutdown wall-clock under load rather than guessing, then set the period comfortably above it. If you cannot lengthen the grace period — some platforms cap it — shorten the flush timeout instead so the flush at least gets a bounded chance rather than being cut mid-send.

Should I flush on SIGINT and other signals too, or only SIGTERM?

Handle every signal a graceful stop can arrive as. SIGTERM is what orchestrators send on a rolling deploy, but local runs and some supervisors send SIGINT, and you want identical drain-flush-close behavior for both. SIGKILL cannot be trapped and always skips your handler, which is exactly why the buffer needs a periodic durable checkpoint rather than relying on the handler alone.

What happens to exposure events for requests still in flight when the deadline hits?

Those specific evaluations get cut off, and any events they had not yet buffered are lost. That is the residual the drain deadline trades away for a bounded exit. Keep it small by capping per-request timeouts below the drain deadline so almost every request finishes and buffers its event before the drain gives up, leaving only genuinely stuck requests exposed.

Does closing the provider connection flush automatically?

Do not assume so. Some SDKs flush inside their own shutdown; many discard the buffer when the connection tears down because that connection was the buffer’s only delivery path. Read your SDK’s shutdown implementation once to learn which it does, and always call an explicit flush before shutdown so the ordering is correct regardless.