Server-Side SDK Integration Patterns

This guide is part of the Backend Evaluation & Server-Side SDKs series. It covers the integration patterns that make a feature flag SDK production-ready: correct initialization sequencing, dependency injection, middleware placement, resilience under control-plane failure, and the observability hooks that let you answer “what variant did this request get?” during an incident.

Server-side evaluation keeps targeting logic and segmentation rules inside your trust boundary. The SDK downloads a compiled rule set once and resolves flags in-process at sub-millisecond cost. This guide does not cover which sync transport to use (polling or streaming) or how to structure a distributed cache across nodes — those decisions are covered in their own deep-dives.

The distinction that drives every pattern below is that the SDK is a stateful component with a lifecycle, not a stateless HTTP client you can call and forget. It holds an open connection to the control plane, a compiled copy of your targeting rules, and an internal state machine that moves between Connecting, Ready, and Stale. Treat it like a database connection pool — bootstrap it once, share it widely, drain it on shutdown — and the failure modes that bite most teams simply never appear. Treat it like fetch, constructing it per request, and you inherit duplicate connection storms, cold rule sets on every call, and evaluations that silently return code defaults because the provider never had time to reach Ready. The five patterns in this guide are the concrete mechanics of getting the lifecycle right in a real service, and each one maps to a specific incident you would otherwise debug at 3 a.m.

Prerequisites

• OpenFeature server SDK ≥ 1.x installed (@openfeature/server-sdk, openfeature Python package, or the Go module)
• flagd or a vendor provider reachable from every service replica
• SDK key injected via environment variable or secret manager — never hardcoded
• Kubernetes liveness/readiness probes wired to the provider health endpoint
• Flag metadata fields owner, created, and expiry populated per your flag taxonomy
• A propagation latency budget agreed on with on-call — needed before you can tune reconnection

Core Concept & Architecture

OpenFeature’s provider abstraction separates the evaluation API from the vendor implementation. Your application code calls one interface (client.BooleanValue, client.StringValue) and the provider behind it handles syncing, caching, and reconnection. Swap providers without touching business logic.

The initialization sequence is strict: the provider must finish its first rule download before the container signals readiness. Any evaluation that runs before the provider is Ready returns the code default with errorCode: PROVIDER_NOT_READY — a footgun that silently serves wrong variants in staging and never shows up in tests. The reason it evades tests is timing: in a test harness the provider is usually pre-seeded or the in-memory provider resolves synchronously, so the race between “listener accepts traffic” and “rules are loaded” simply does not exist. In production, where the control plane is a network hop away and the rule set is larger, that race is wide open for the first few hundred milliseconds of every cold start — precisely the window a rolling deploy hammers hardest.

The Stale state deserves special attention because it is the one that makes server-side evaluation resilient rather than fragile. When the control-plane connection drops, a well-behaved provider does not clear its rule set and start returning defaults — it keeps evaluating against the last-known-good rules it downloaded before the outage. This is almost always what you want: a flag that was on for 20% of users stays on for that same 20% through a control-plane blip, rather than flapping to the code default and back. The trade-off is that flag changes made during the outage do not propagate until the connection recovers, which is why Stale should fail readiness (so orchestrators stop routing new rollout-sensitive traffic to a node whose rules may be minutes old) while still passing liveness (so the node is not killed for a fault it is correctly surviving). Getting that probe split right is the single highest-leverage decision on this page.

Step-by-Step Implementation

The five patterns place SDK concerns at the right layer: bootstrap at startup, a singleton in the container, evaluation in middleware, a circuit breaker around each call, and telemetry on every span. Handlers stay free of flag logic entirely.

Step 1 — Bootstrap the provider before accepting traffic

Initialize the SDK in your startup sequence and block readiness until the provider signals it has loaded its first rule set. An idempotent guard prevents duplicate initialization during hot-reloads.

// startup.ts
import { OpenFeature } from '@openfeature/server-sdk';
import { FlagdProvider } from '@openfeature/flagd-provider';

let initialized = false;

export async function bootstrapFlags(): Promise<void> {
  if (initialized) return; // idempotent — safe to call on hot-reload

  const provider = new FlagdProvider({
    host: process.env.FLAGD_HOST!,
    port: Number(process.env.FLAGD_PORT ?? 8013),
    tls: process.env.NODE_ENV === 'production',
  });

  // Blocks until the first rule set is downloaded; throws on timeout
  await OpenFeature.setProviderAndWait(provider);
  initialized = true;

  process.on('SIGTERM', async () => {
    await OpenFeature.close(); // drain streams, flush telemetry
    process.exit(0);
  });
}

Pitfall: calling setProvider (without AndWait) lets the process start accepting traffic before rules are available. Every evaluation in that window returns the code default and emits PROVIDER_NOT_READY. Wire your readiness probe to the provider state, not just to the HTTP port binding.

Give setProviderAndWait a bounded timeout and decide explicitly what happens when it fires. There are two defensible policies and one dangerous one. The dangerous one is to swallow the timeout and start serving anyway — you have now built the exact PROVIDER_NOT_READY race you were trying to avoid, just with extra steps. The two safe policies are: fail the container start hard (let the orchestrator restart it and try again, appropriate when flags gate correctness-critical behavior), or start in an explicit degraded mode that serves code defaults and loudly emits a metric saying so (appropriate when your defaults are genuinely safe and availability outranks flag freshness). Whichever you pick, make it a deliberate line of code, not an accident of where the await happens to sit. Also note the SIGTERM handler above: OpenFeature.close() drains the streaming connection and flushes any buffered telemetry, so an evaluation that fires during the drain still gets recorded. Skipping it means the last few seconds of every pod’s traces vanish on every deploy — invisible until you go looking for a rollout event and find a gap.

Step 2 — Register the client as a singleton

Register the OpenFeature client as a singleton in your service container. Scoped or transient instances create multiple connection pools, multiplying connection handshakes and bypassing the shared rule-set cache.

// Program.cs (.NET)
builder.Services.AddSingleton<IFeatureClient>(sp => {
    var cfg = sp.GetRequiredService<IConfiguration>();
    OpenFeature.Api.Instance.SetProvider(
        new FlagdProvider(new FlagdConfig {
            Host = cfg["FLAGD_HOST"],
            Port = int.Parse(cfg["FLAGD_PORT"] ?? "8013"),
        })
    );
    return OpenFeature.Api.Instance.GetClient("api");
});

# deps.py (FastAPI)
from openfeature import api
from openfeature.provider.flagd import FlagdProvider
import os

_client = None

def get_flag_client():
    global _client
    if _client is None:
        api.set_provider(FlagdProvider(
            host=os.environ["FLAGD_HOST"],
            port=int(os.environ.get("FLAGD_PORT", "8013")),
        ))
        _client = api.get_client("api")
    return _client

The reason a second client is so costly is that the client is not a lightweight handle — it is the thing that owns the connection, the streaming subscription, and the compiled rule set. A framework that helpfully constructs a fresh instance per request, or a test suite that re-initializes between cases, produces a fleet of providers each doing its own first sync, each opening its own SSE stream to the control plane, and each holding a separate copy of the rules in memory. On a busy service that turns a single expected connection into thousands, and the control plane — which sizes its connection limits assuming one subscriber per replica — starts shedding them. The singleton is not a style preference; it is the contract the sync protocol was designed around.

Pitfall: in languages with async runtimes (asyncio, Tokio), a module-level singleton can initialize on multiple threads simultaneously. The naive if _client is None check above is not thread-safe under concurrent first requests — two coroutines can both see None, both call set_provider, and you are back to duplicate connections. Use a lock or an async-once primitive (asyncio.Lock, sync.Once, Lazy<T>) so the initialization block runs exactly once even under a burst of concurrent cold requests.

Step 3 — Evaluate in middleware, not in handlers

Resolve the flags your handler needs at the request boundary in a middleware or interceptor. This keeps handler logic free of SDK calls, lets you batch evaluations, and ensures evaluation context is assembled once with a consistent snapshot of the request attributes.

# middleware.py (FastAPI)
from fastapi import Request
from contextvars import ContextVar
from openfeature.evaluation_context import EvaluationContext
from .deps import get_flag_client

_flags: ContextVar[dict] = ContextVar("request_flags")

async def flag_middleware(request: Request, call_next):
    client = get_flag_client()
    ctx = EvaluationContext(
        targeting_key=request.headers.get("X-User-ID", "anonymous"),
        attributes={
            "tenantTier": request.state.tenant_tier,
            "region":     request.headers.get("CF-IPCountry", "unknown"),
        }
    )
    flags = {
        "api.search.semantic-rerank":   client.get_boolean_value("api.search.semantic-rerank",   False, ctx),
        "api.checkout.express-pay":      client.get_boolean_value("api.checkout.express-pay",      False, ctx),
    }
    _flags.set(flags)
    return await call_next(request)

The evaluation context assembled here is the input to the rule engine; keep it consistent across a request to avoid split evaluations. See the rule engine performance guide for batching strategies under high concurrency.

The subtle bug this pattern prevents is the split evaluation: the same request evaluates a flag twice, at two different points in its lifecycle, and gets two different answers because the rule set was updated in between or because the two call sites assembled slightly different context. A user who is bucketed into the treatment arm for the “should I show the new checkout?” decision but the control arm for the “should I emit the new-checkout metric?” decision produces analytics that are quietly wrong and impossible to reconcile. Resolving once at the boundary and stashing the results in a request-scoped store — the ContextVar above, an HttpContext.Items bag, a Go context.Value — guarantees every downstream reader sees one consistent snapshot. It also gives you a natural place to record which variants this request saw, which is exactly what the telemetry hook in Step 5 consumes.

Pitfall: evaluating inside a database transaction or after acquiring a lock prolongs the critical section. Resolve flags before entering any lock. Even a sub-millisecond evaluation, multiplied across every row-lock holder under contention, measurably widens your tail latency — and if the provider is momentarily slow, an evaluation stuck inside a transaction pins that transaction open, turning a flag hiccup into lock-wait timeouts on unrelated queries.

Step 4 — Wrap every evaluation in a circuit breaker

Isolate the provider from the rest of your service. A circuit breaker opens after a threshold of evaluation errors and returns a safe default until the provider recovers — so a degraded control plane cannot take down unrelated request paths.

// resilience.go
import (
    "context"
    "time"
    "github.com/sony/gobreaker"
    "go.opentelemetry.io/otel/attribute"
    openfeature "github.com/open-feature/go-sdk/pkg/openfeature"
)

var cb = gobreaker.NewCircuitBreaker(gobreaker.Settings{
    Name:        "flag-provider",
    MaxRequests: 1,
    Interval:    10 * time.Second,
    Timeout:     30 * time.Second,
    ReadyToTrip: func(counts gobreaker.Counts) bool {
        return counts.ConsecutiveFailures >= 5
    },
})

func BoolFlag(ctx context.Context, client openfeature.IClient, key string, defaultVal bool) bool {
    result, err := cb.Execute(func() (interface{}, error) {
        evalCtx, cancel := context.WithTimeout(ctx, 50*time.Millisecond)
        defer cancel()
        val, err := client.BooleanValue(evalCtx, key, defaultVal,
            openfeature.EvaluationContext{})
        return val, err
    })
    if err != nil {
        return defaultVal // safe default on open circuit
    }
    return result.(bool)
}

The counter-intuitive part of wrapping an in-process call in a circuit breaker is that the evaluation itself never touches the network — the rules are already local, so what failure is the breaker guarding against? Two things. First, the provider’s own internal calls: some providers lazily fetch a rule fragment, resolve a large-object flag, or block on a not-yet-ready sync, and those can hang. Second, and more importantly, the breaker converts a slow dependency into a fast failure. Without it, a provider that starts taking 200ms per resolve drags every request path that touches a flag into the same latency, and because flag evaluation is sprinkled everywhere, that is effectively your whole service. The breaker plus the tight per-call timeout means the worst case for any single evaluation is bounded and known: 50ms, then a safe default, then the circuit opens and subsequent calls short-circuit to the default in nanoseconds until the provider proves healthy again.

Choose the breaker thresholds against your actual traffic. ConsecutiveFailures >= 5 is reasonable for a high-QPS service where five failures happen in milliseconds, but on a low-traffic internal API five consecutive failures might span minutes — you may want a failure-ratio trip instead so the breaker reacts to a burst rather than waiting for five serial requests. The Timeout (how long the circuit stays open before probing half-open) is your recovery latency floor: set it to 30s and a control plane that recovers in 5s still serves defaults for the remaining 25.

Pitfall: a timeout that is too generous (>100ms) hides provider latency and can cascade into your P99 budget. Keep evaluation timeouts at 20–50ms; the rule engine should resolve in well under 1ms on warm state.

Pitfall: returning the code default on an open circuit is only safe if your code defaults are genuinely the conservative choice. If a flag defaults to true in code but the safe production value during an incident is false (say it gates an expensive new pipeline), your fail-safe is fail-dangerous. Audit every default-value argument at the call site with the question “is this what I want served when the control plane is on fire?” — the answer decides whether the circuit breaker protects you or amplifies the outage.

Step 5 — Emit structured evaluation telemetry

Attach evaluation metadata to every trace span and emit a structured log entry. This makes it possible to answer “which variant did request X get?” from a trace rather than re-running targeting logic post-incident.

// telemetry.ts
import { OpenFeature, Hook, EvaluationDetails } from '@openfeature/server-sdk';
import { trace } from '@opentelemetry/api';

const telemetryHook: Hook = {
  after(hookContext, evaluationDetails: EvaluationDetails<unknown>) {
    const span = trace.getActiveSpan();
    if (span) {
      span.setAttributes({
        'feature_flag.key':     hookContext.flagKey,
        'feature_flag.variant': String(evaluationDetails.value),
        'feature_flag.reason':  evaluationDetails.reason ?? 'UNKNOWN',
      });
    }
  },
  error(hookContext, err) {
    // flag.stream.error counter lives here
  }
};

OpenFeature.addHooks(telemetryHook);

Align attribute names with the OpenTelemetry semantic conventions for feature flags so your tracing backend can correlate them automatically. Standardizing on feature_flag.key, feature_flag.variant, and feature_flag.provider_name means a query like “show me every span in the last hour where api.checkout.express-pay resolved to on” works across services without per-team conventions, and it lets you overlay a flag flip on your latency graphs to spot a rollout that moved a metric.

The reason field is the piece most teams under-value. It is the difference between “this user got the treatment because they matched a targeting rule” (TARGETING_MATCH), “because they fell into the rollout percentage” (SPLIT), “because nothing matched and this is the flag’s default” (DEFAULT), and “because the provider was unhealthy” (ERROR). During an incident those four look identical if all you logged was the variant, and you will waste twenty minutes re-deriving targeting logic by hand. Log the reason and the answer is already in the trace. Be deliberate about cardinality, though: feature_flag.key and variant are bounded and safe to index, but do not attach the full targeting key (a user ID) as a span attribute on every evaluation — that is unbounded cardinality that will bankrupt a metrics backend. Keep the user identifier on the request span, not on each flag attribute.

Verification & Testing

Confirm the SDK reaches Ready before your container passes its readiness check, and that evaluation fails safe on provider error:

# After startup, confirm the provider state via your debug endpoint
curl -s http://localhost:3000/healthz | jq '.provider_state'
# expect: "READY"

# Simulate provider failure: block egress to the control plane
iptables -A OUTPUT -p tcp --dport 8013 -j DROP

# Confirm evaluation returns safe defaults
curl -s http://localhost:3000/debug/flags/api.search.semantic-rerank | jq .
# expect: { "variant": false, "reason": "DEFAULT", "errorCode": "GENERAL" }

For reconnection behavior after the block lifts, verify a full resync fires rather than relying on a delta. The exponential backoff for SDK reconnection how-to covers testing the exact backoff curve.

Two assertions are worth turning into standing integration tests rather than one-off manual checks. The first is a readiness ordering test: start the container, poll the readiness endpoint, and assert it returns non-200 until the provider debug endpoint reports READY — this catches the day someone reorders the startup sequence and reintroduces the PROVIDER_NOT_READY race. The second is a fail-safe test that runs in CI against a provider you can kill mid-test (a local flagd in a container is ideal): resolve a flag, terminate the provider, resolve again, and assert the second call returns the coded default with a non-error value in bounded time rather than throwing or hanging. Both are cheap to run and they fail loudly the moment a refactor breaks the two properties that matter most — that you never serve on unloaded rules, and that a dead control plane degrades instead of cascading. When you run the fail-safe test, watch the wall-clock time of the second resolve: if it takes anywhere near your circuit-breaker timeout, the breaker is doing its job; if it hangs, your per-call timeout is not actually wired into the evaluation path.

Troubleshooting & FAQ

Why do evaluations return the default variant right after a deploy?

The provider has not reached Ready yet. Your container probably signals readiness before setProviderAndWait resolves. Confirm by checking the evaluation reason field: PROVIDER_NOT_READY with a DEFAULT value is the exact signature. Fix by delaying the HTTP listener start until after provider initialization, or by returning 503 from your readiness probe until the provider state is READY.

How do I test flag behavior without hitting a real control plane?

Use an in-memory provider in tests:

import { InMemoryProvider } from '@openfeature/server-sdk';

await OpenFeature.setProviderAndWait(new InMemoryProvider({
  'api.search.semantic-rerank': { defaultVariant: 'on', variants: { on: true, off: false } },
}));

This keeps CI fast and deterministic without a flagd process.

Can I use the SDK in a serverless function?

Yes, but initialization cost matters. In a cold start, setProviderAndWait adds the full rule-download latency to your first invocation. Consider a short-polling provider with a tight timeout (2–3s), or pre-warm by bootstrapping the SDK in the global scope outside the handler so subsequent warm invocations reuse the state.

How do I confirm the circuit breaker is actually protecting the path?

Expose the circuit breaker state in your metrics: flag.provider.circuit_state (0=closed, 1=open, 2=half-open). Alert on sustained open state — it means evaluation has been falling back to defaults for at least Timeout seconds, which warrants a look at control-plane health.

Should the Stale state pass or fail the readiness probe?

Pass liveness, fail readiness. A node in Stale is correctly surviving a control-plane outage by serving last-known-good rules, so killing it (liveness fail) would only make things worse. But its rules may be minutes old, so you do not want the orchestrator sending it fresh rollout-sensitive traffic while healthy nodes exist — failing readiness quietly rotates it out of the load-balancer pool until it reconnects and returns to Ready. If every node is Stale at once, readiness failing everywhere is a signal, not a self-inflicted outage: your traffic keeps flowing on last-known-good rules while the alert fires.

How long should setProviderAndWait be allowed to block on startup?

Long enough to download and compile your full rule set on a healthy control plane, plus headroom — measure it, do not guess. For most catalogs that is well under two seconds; very large catalogs can take longer. Set an explicit timeout and decide the failure policy: either fail the container start so the orchestrator retries, or enter an explicit degraded mode that serves code defaults and emits a metric. Silently continuing after a timeout recreates the PROVIDER_NOT_READY race you were trying to prevent.

Why not just cache evaluation results in my own application layer?

Because the provider already resolves a flag in under a microsecond against a local read-only rule set, so a result cache saves nothing measurable while adding real risk. Your cache would serve a variant from a rule set that is one or more syncs stale, and you would own a fresh invalidation bug when flags change mid-rollout. Let the provider own the rules and the caching; do not layer your own decision cache above it.

What breaks if I construct a new SDK client per request?

Each client owns its own control-plane connection, streaming subscription, and in-memory copy of the rule set, so per-request construction turns one expected connection into thousands. The control plane sizes its connection limits assuming one subscriber per replica and starts shedding the excess, every new client pays the full first-sync latency before it can evaluate, and memory balloons from duplicate rule sets. Register the client once as a singleton and share it across all requests in the process.

Performance & Scale

At hundreds of concurrent requests, the per-evaluation cost is the rule engine’s in-process lookup — typically under 1ms. The SDK’s internal rule set is read-only after initialization, so no locking is needed on the evaluation path. Connection overhead is bounded by the number of provider instances (one per process with the singleton pattern). Horizontal scaling adds connections linearly; that is expected and cheap compared to per-request network evaluation.

Because the rule set is read-only after each sync, evaluation is embarrassingly parallel — there is no shared mutable state on the hot path, so throughput scales with cores and you never contend on a lock to read a flag. The one moment that mutability appears is at sync time, when the provider atomically swaps in the new rule set; a well-implemented provider does this with a pointer swap so in-flight evaluations either see the old set or the new one, never a torn half-applied state. This is why you should never try to “help” the SDK by caching evaluation results in your own layer: you would be caching a decision that the provider can already make in under a microsecond, and you would risk serving a variant from a rule set two syncs stale while introducing your own invalidation bug. Let the provider own the rules; cache nothing above it.

Memory, not CPU, is the resource that scales with your flag catalog. Each replica holds the full compiled rule set, so a catalog of tens of thousands of flags with large targeting lists is a per-pod memory cost multiplied across the fleet — worth measuring if you run many small replicas. Startup time scales with catalog size too, because the first sync must download and compile everything before Ready; a very large catalog can push cold start past a tight readiness deadline, which is another reason to give setProviderAndWait a timeout you have actually measured rather than guessed.

For cache topology across a fleet, the local in-process rule set is already the first-level cache. See distributed caching for flag evaluations if you need a second-level shared cache to reduce control-plane connection count.