Syed Hasibur Rahman19 min read
The API Key Is the Tenant: Identity Models That Survive Multi-Tenancy
Most cross-tenant data leaks are not exotic exploits; they are a missing WHERE clause, a trusted X-Tenant-ID header, or a query that resolved the tenant from one input and the data from another. This post treats tenancy identity as the security boundary it is: how each resolution model (subdomain, header, key-is-the-tenant) fails, how to store and rotate API keys so a leak is survivable, how Postgres row-level security turns the tenant check into a database invariant, and what blast radius each isolation tier actually buys.
The ticket reads: “Customer Meridian says their dashboard is showing invoices from a company called Atlas Freight. They would like to know who Atlas Freight is.”
You know this ticket. Every engineer who has operated a multi-tenant service for long enough has received some version of it, and the postmortem is almost never interesting cryptography. It is a WHERE tenant_id = ? clause that one query out of four hundred forgot. It is an endpoint that resolved the tenant from the URL path but fetched the record by a client-supplied ID. It is a background job that ran with a connection pool whose session variable still held the previous job’s tenant. The leak is boring, the consequences are not — a single cross-tenant read is an existential trust event for a B2B product — and the root cause, in nearly every case I have seen or read a postmortem for, is the same architectural decision made implicitly and early: tenant identity was resolved in application code, from an input the client influenced, and enforced nowhere else.
This post is about doing it deliberately instead. Where does the tenant come from? The subdomain? A header? The API key itself? And once you know who the tenant is, where is that knowledge enforced — in every query by convention, in the database by policy, or in the infrastructure by physical separation? These are two independent axes — resolution and isolation — and most architectural regret comes from conflating them. We will walk both, with schemas, code, and an honest accounting of the blast radius each choice buys.
Tenancy resolution is authentication’s second half
Authentication answers who is calling. Tenancy resolution answers whose data may this call touch. In a single-tenant system the second question is trivial. In a multi-tenant system it is the security boundary — and it deserves the same rigor we give credentials, because a correct answer to “who” paired with a wrong answer to “whose” is precisely a cross-tenant breach.
OWASP’s API Security Top 10 has ranked Broken Object Level Authorization (BOLA) as API risk number one since 2019, reaffirmed in the 2023 revision: endpoints that authenticate the caller and then fetch whatever object ID the caller names1. Multi-tenant BOLA is the enterprise-grade variant. The caller is legitimate, authenticated, and paying you; the object belongs to someone else who is also paying you. Everything that follows is a strategy for making that class of bug either impossible to write or impossible to execute.
A useful discipline before we start: draw the tenancy resolution point in your request path — the single place where an untrusted request becomes “a request acting on behalf of tenant T.” Everything upstream of that point is hostile input. Everything downstream must be unable to change its mind about T. Systems leak when T can be influenced downstream of the resolution point, or when there are two resolution points that can disagree.
Model one: the subdomain is the tenant
The classic B2B pattern: meridian.yourapp.com, atlas.yourapp.com. The tenant is resolved from the Host header (or the TLS SNI), looked up in a tenants table, and attached to the request context.
It has real virtues. The tenant is visible in every URL, every log line, every browser profile. Cookies scope naturally per subdomain — provided your domain is on the Public Suffix List boundary you think it is2 — so a session minted for meridian.yourapp.com will not ride along to atlas.yourapp.com. Support engineers can reproduce issues by visiting the right hostname. For browser-facing dashboards it remains the right default.
The sharp edges are all at the HTTP layer:
- The
Hostheader is attacker-supplied. James Kettle’s foundational writeup on practical Host-header attacks showed how routing, password-reset poisoning, and cache poisoning flow from servers trustingHost(orX-Forwarded-Host) more than they should3. If your tenant resolution readsHostbehind a proxy layer, the proxy must strictly validate or rewrite it, and the application must reject hostnames that do not match a known tenant — never “default to the first tenant” or “fall through to a marketing site that shares the session store.” - Wildcard ambiguity.
*.yourapp.comcertificates and wildcard DNS mean any label resolves and terminates TLS. The gap between “DNS resolves” and “tenant exists” must return an error page with no session machinery attached. - It does not survive the API. Server-to-server callers do not want per-tenant hostnames; they want one base URL and a credential. Which is why nearly every subdomain-resolved product grows a second resolution model for its API — and now has two resolution points to keep consistent.
Verdict: fine for the browser plane, insufficient alone, and never to be trusted without allowlist validation of the hostname.
Model two: the client-supplied tenant header, and why it is an anti-pattern
Somewhere in many codebases lives this:
GET /v1/invoices HTTP/1.1
Authorization: Bearer eyJhbGciOi...
X-Tenant-ID: 8f14e45f-ceea-4a1e-9a6c-2b1f0d3c7a11
The reasoning is always the same: the token authenticates the user, the user may belong to several tenants, so the client indicates which tenant this request targets. The header is convenient, explicit, and testable with curl. It is also, as deployed in the wild, the single most common cause of the Meridian/Atlas ticket, because the enforcement step — verify this authenticated principal is actually a member of that tenant, with these permissions — is an easy check to write and an easier one to forget on endpoint forty-seven of sixty.
Name the failure precisely: the header is an authorization input sourced from the attacker. It is BOLA with a tenant-sized object1. Change the UUID, replay the request, and every endpoint that skipped the membership check is a cross-tenant read. Unlike a random object ID, tenant IDs are often guessable (sequential integers, leaked in URLs, embedded in webhook payloads), and one miss exposes an entire tenant’s collection, not one record.
There are two legitimate shapes that superficially resemble this pattern, and the difference is who writes the header:
- Gateway-injected tenant context. The edge authenticates the credential, resolves the tenant from the credential, strips any inbound
X-Tenant-ID, and injects its own. Interior services trust the header because the perimeter guarantees no client-supplied value survives. This is fine — the resolution point is the gateway, and the header is transport, not input. The non-negotiable detail is the stripping; a gateway that injects but does not strip has built a spoofing vector for anyone who can reach an interior hop. - Tenant claim in the token. The tenant lives in the signed credential itself — an
org_idclaim in the access token, minted after the user selected a tenant at token time. The client cannot alter it without breaking the signature. Multi-tenant users get one token per tenant context, which is mildly annoying and completely sound.
Both shapes obey the same law: tenancy is derived from the credential, never asserted alongside it. Which points at the cleanest model of all.
Model three: the API key is the tenant
For machine-to-machine APIs — the plane where most data volume and most breach volume lives — the strongest design makes the credential and the tenant the same fact:
POST /v1/charges HTTP/1.1
Authorization: Bearer sk_live_51KxT9dLmR8vQzABC...
No tenant header. No tenant path segment. No tenant query parameter. The server looks up the key, and the key row contains the tenant ID. There is nothing for the client to lie about, because the client supplies exactly one thing and that thing is unforgeable. Stripe’s API is the canonical public example — the secret key alone determines the account and the livemode/testmode axis, and no request parameter can point it at someone else’s data4.
The property this buys is structural, not procedural. In the header model, correctness requires every endpoint to perform a membership check — correctness is the conjunction of four hundred developer decisions. In the key-is-the-tenant model, the tenant is resolved once, at the authentication middleware, from the credential itself; an endpoint that wants to touch another tenant’s data has no vocabulary to express the request. You have not made the bug rarer. You have made it unwritable at the request layer. (You can still write it at the query layer — we get to that in the isolation section.)
The resolution middleware, in full:
# FastAPI-style middleware: credential -> tenant, one resolution point.
import hashlib, hmac, secrets
async def resolve_tenant(request: Request) -> TenantContext:
raw = extract_bearer(request) # "sk_live_51KxT9d..."
prefix, secret = parse_key(raw) # fail closed on format
# Look up by deterministic hash of the full key (see storage section).
digest = hashlib.sha256(raw.encode()).hexdigest()
key = await db.fetchrow(
"""SELECT k.id, k.tenant_id, k.scopes, k.expires_at, k.revoked_at
FROM api_keys k WHERE k.key_hash = $1""", digest)
if key is None or key["revoked_at"] is not None:
raise Unauthorized() # same error, same latency
if key["expires_at"] and key["expires_at"] < now():
raise Unauthorized()
await db.execute( # observability, not vanity
"UPDATE api_keys SET last_used_at = now() WHERE id = $1", key["id"])
# Everything downstream receives tenant identity as *derived fact*.
return TenantContext(tenant_id=key["tenant_id"], scopes=key["scopes"])
Every handler receives TenantContext as its only source of tenant truth. Code review then has a single grep-able rule: no handler mentions tenant_id from any request field, ever.
Engineering the key itself
If the key is the tenant, the key’s design is load-bearing. Four decisions matter.
Entropy and shape. The secret portion should be ≥128 bits from a CSPRNG — secrets.token_urlsafe(32) territory, not a UUID (UUIDv4 has 122 random bits but a recognizable shape that invites misuse as both identifier and secret). Separate the key ID (public, loggable, indexable) from the secret (never stored, never logged). RFC 9562’s UUIDv7 makes a good key ID: time-ordered, index-friendly, safe to expose5.
Prefixes. GitHub’s 2021 token-format redesign is the reference work here: a type prefix (ghp_, gho_, ghs_) makes tokens self-describing, greppable, and — the underrated part — detectable by secret-scanning infrastructure with a near-zero false-positive rate6. Adopt the pattern: yourco_sk_live_<keyid>_<secret>. When a customer pastes one into a public repository (they will; it is a matter of when), scanners can find it and you can auto-revoke it.
Checksums. GitHub also embeds a CRC32 checksum in the token’s tail, Base62-encoded, so offline tooling can distinguish a real token from a random string without a database hit6. For your own keys the same trick lets your edge reject malformed keys before they touch the datastore — cheap DoS insurance and cleaner telemetry.
Display-once semantics. The full key is shown exactly once at creation. Afterward the dashboard shows prefix, key ID, creation date, last-used timestamp, and the final four characters. If you can re-display a key, you are storing it recoverable, which brings us to storage.
Storing keys: hash, but know why fast hashing is fine
Passwords require slow, memory-hard hashing — Argon2id per RFC 9106 — because they are low-entropy and the threat is offline guessing7. API keys are different: a 128-bit random secret cannot be brute-forced regardless of hash speed, so OWASP’s storage guidance explicitly blesses a single round of a fast hash (SHA-256) for high-entropy machine secrets8. That distinction matters operationally, because the fast hash is also your lookup index — you hash the presented key and select by digest, which a per-row-salted Argon2 cannot do without a table scan.
CREATE TABLE api_keys (
id uuid PRIMARY KEY, -- UUIDv7 key ID
tenant_id uuid NOT NULL REFERENCES tenants(id),
name text NOT NULL, -- "prod backend", "CI"
key_prefix text NOT NULL, -- "yourco_sk_live_3ZkQ"
key_hash text NOT NULL UNIQUE, -- sha256(full key), hex
scopes text[] NOT NULL DEFAULT '{}',
created_at timestamptz NOT NULL DEFAULT now(),
expires_at timestamptz, -- NULL = non-expiring (avoid)
last_used_at timestamptz,
revoked_at timestamptz,
rotated_from uuid REFERENCES api_keys(id) -- rotation lineage
);
CREATE INDEX ON api_keys (tenant_id) WHERE revoked_at IS NULL;
Three details worth their bytes. key_prefix lets support match a customer’s “my key starts with…” without anyone possessing the secret. last_used_at is the difference between “we can safely revoke this key, it has been silent for 90 days” and guessing. rotated_from gives you an audit lineage when compliance asks how a credential was retired.
If your threat model includes database-plus-application compromise, upgrade from bare SHA-256 to HMAC-SHA-256 with a pepper held in a KMS or HSM — the digest then cannot be recomputed by anyone holding only the database.
Rotation: multiple active keys or it doesn’t happen
Rotation designs fail socially, not cryptographically. If creating a new key instantly kills the old one, customers must orchestrate a simultaneous deploy across every consumer of the key — so they simply never rotate. The fix is boringly simple: a tenant may hold several active keys at once, and rotation is create → migrate at leisure → observe last_used_at go quiet → revoke.
POST /v1/api_keys -> new key, old keys untouched
GET /v1/api_keys -> list with last_used_at per key
DELETE /v1/api_keys/{id} -> revoke exactly one
The January 2023 CircleCI incident is the argument for having this machinery before you need it: after a compromise of their systems, CircleCI had to ask every customer to rotate every stored secret — OAuth tokens, project keys, environment variables — at once9. Customers with overlap-capable rotation scripted it in an afternoon. Customers whose integrations assumed one immortal key were still finding broken jobs weeks later. Your incident-response plan for “our key material may have leaked” is only as good as the day-two ergonomics of your rotation API. Set default expirations on new keys (a year, even), emit webhook events on key creation and revocation so customers’ security tooling can watch, and rate-limit failed-auth attempts per key prefix so a leaked-then-revoked key becomes a detection signal rather than noise.
The isolation spectrum: where the tenant check is enforced
Resolution tells you the tenant. Isolation determines what happens when code downstream is wrong anyway. The industry-standard vocabulary comes from AWS’s SaaS Tenant Isolation Strategies whitepaper: silo (dedicated resources per tenant), pool (shared resources, logical isolation), and bridge (a mix per component)10. Mapped to the database tier, the pool-to-silo spectrum runs: shared tables with a tenant column, schema-per-tenant, database-per-tenant.
Row-level: shared tables, tenant column, and RLS as the safety net
The default for pooled SaaS: every table carries tenant_id, every query filters on it. The known failure mode is the forgotten filter — which is why the pooled model should never rely on application discipline alone. PostgreSQL’s row-level security turns the tenant predicate into a database invariant11:
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
ALTER TABLE invoices FORCE ROW LEVEL SECURITY; -- applies to table owner too
CREATE POLICY tenant_isolation ON invoices
USING (tenant_id = current_setting('app.current_tenant')::uuid);
# Per-request: bind the resolved tenant into the session, transaction-locally.
async with db.transaction():
# SET LOCAL scopes the setting to this transaction — it cannot leak
# across pooled connections the way SET (session) can.
await db.execute("SET LOCAL app.current_tenant = $1", ctx.tenant_id)
rows = await db.fetch("SELECT * FROM invoices WHERE status = 'open'")
# No tenant_id in the query. The policy appends it. A query that
# forgets the filter returns the right rows; a query that *asserts*
# the wrong tenant returns nothing.
Two operational warnings from production scar tissue. First, use SET LOCAL inside a transaction, never bare SET, with connection pooling — a session-scoped setting that survives connection reuse is itself a cross-tenant bug, just relocated. Second, connect as a non-superuser role without BYPASSRLS, and use FORCE ROW LEVEL SECURITY; RLS silently does not apply to superusers and table owners otherwise11.
With RLS in place, the application’s WHERE tenant_id clauses become a performance optimization and a readability aid, while correctness lives in the policy. This is defense in depth in its most literal form: the resolution point derives the tenant from the credential; the datastore refuses to return rows for any other tenant even if every line of application code between them is wrong.
One more pooled-model discipline, because it is where RLS deployments quietly rot: tenancy must survive leaving the request path. Background jobs, caches, and search indexes all execute outside the middleware that resolved the tenant, and each is a place where tenant context is either carried explicitly or lost. A job enqueued without a tenant_id in its payload will be written, six months later, to “just query all tenants” by someone under deadline. A cache keyed on invoice:{id} instead of tenant:{tid}:invoice:{id} is a cross-tenant read waiting for an ID collision or a guessing attacker. A shared search index without a tenant filter term enforced at the query-builder layer — not in each call site — replays the missing-WHERE-clause bug in a datastore that has no RLS to catch it. The rule generalizes cleanly: every execution context that touches tenant data must receive the tenant as an explicit, non-optional parameter, and the pooled datastores it touches must enforce the same predicate the primary database does. If your RLS story is airtight but your Redis and Elasticsearch stories are vibes, your isolation tier is Redis.
Schema-per-tenant
Each tenant gets a PostgreSQL schema with identical tables; the resolution point sets search_path. Isolation is stronger than a column (a query cannot cross schemas by forgetting a predicate), noisy neighbors are still shared, and per-tenant restore becomes possible (dump one schema) if not pleasant. The costs arrive with scale and migrations: ten thousand tenants means ten thousand copies of every ALTER TABLE, catalog bloat that degrades planning, and connection-pool fragmentation. Salesforce’s classic multitenant-architecture paper is the deep-water counterpoint — they famously rejected per-tenant physical structures entirely, running a metadata-driven pooled model at extreme scale precisely because per-tenant DDL does not scale organizationally12. Schema-per-tenant occupies a real but narrow band: tens to hundreds of tenants, strong contractual isolation language, one engineering team that owns migrations end-to-end.
Database-per-tenant
The silo. One database (or cluster) per tenant: blast radius approaches zero for data-layer bugs, per-tenant encryption keys and per-tenant restore come free, residency requirements (“EU tenant data stays in Frankfurt”) become a placement decision, and compliance reviews get dramatically shorter. The bill: fleet orchestration (provisioning, migrations, backups, monitoring × N), cost floors per tenant, and cross-tenant analytics that now require an ETL pipeline instead of a GROUP BY. This is the right answer for regulated verticals, six-figure contracts, and single-tenant-by-policy enterprise tiers — typically as the premium rung of a bridge architecture whose lower rungs are pooled10.
Blast radius: the honest comparison
The reason to hold resolution and isolation in your head simultaneously is that they cover for each other’s failures. Here is what each combination actually buys when specific things go wrong:
| Failure event | Header-resolved + shared tables | Key-resolved + shared tables | Key-resolved + RLS | Schema-per-tenant | DB-per-tenant |
|---|---|---|---|---|---|
| Missing membership check on one endpoint | Cross-tenant read/write | Not expressible | Not expressible | Not expressible | Not expressible |
Forgotten WHERE tenant_id | All-tenant leak | All-tenant leak | Policy filters rows | Wrong-schema error | Connection scoped |
| SQL injection in one endpoint | Full-database read | Full-database read | Bounded by policy* | Bounded by search_path* | Bounded to one tenant’s DB |
| One tenant’s API key leaks | That tenant + any endpoint bug | That tenant only | That tenant only | That tenant only | That tenant only |
| Bad migration corrupts data | Everyone at once | Everyone at once | Everyone at once | Tenant-by-tenant rollout possible | Tenant-by-tenant rollout possible |
| Noisy-neighbor load | Shared fate | Shared fate | Shared fate | Shared fate | Isolated |
* Injection caveats: RLS bounds an injected query only if the attacker cannot also change app.current_tenant or escalate the role; schema isolation bounds it only if the injected SQL cannot schema-qualify table names. Both are meaningful friction, neither is a substitute for parameterized queries.
Read the first column top to bottom and you see why the client-supplied header keeps producing incidents: it is the only cell in the matrix where a single endpoint-level oversight produces cross-tenant access with no second control in the way. Read the first row left to right and you see what key-is-the-tenant buys: the entire class of membership-check bugs stops being expressible. And read the injection row for the case for RLS: it is the only pooled-model control that stands between one bad string-concatenation and your entire customer base.
Choosing, concretely
A decision procedure that has held up across the systems I have built and reviewed:
- Resolution: browser plane may use subdomains (with strict Host validation); API plane uses key-is-the-tenant or a tenant claim inside the signed token. Client-supplied tenant headers are permitted exactly nowhere. Gateway-injected headers are fine if the gateway strips inbound ones.
- Key mechanics: prefixed, checksummed, ≥128-bit keys; SHA-256 (or peppered HMAC) storage; display-once; multiple active keys;
last_used_at; default expiry; webhooks on lifecycle events. - Isolation: start pooled with
tenant_idcolumns plus RLS from day one — retrofitting RLS onto four hundred queries is a quarter of work; enabling it on table three is an afternoon. Add a database-per-tenant tier when a contract, a regulator, or a residency law asks for it, and run it as a bridge architecture rather than migrating everyone. - Verification: a nightly job that runs known-tenant-A credentials against tenant-B object IDs across every endpoint (an internal BOLA scanner) will catch the regression your unit tests won’t, because the regression is precisely the check someone deleted.
Closing: make the wrong query unwritable
Multi-tenancy discourse loves the isolation debate — schemas versus databases, silo versus pool — because it is an architecture-diagram conversation. But the incidents mostly do not come from choosing the wrong tier of that spectrum. They come from the quieter decision of where tenant identity enters the request and how far your system will follow it when it is wrong. A service that trusts X-Tenant-ID has decided that four hundred endpoints will each independently re-derive its security boundary. A service where the key is the tenant has decided the boundary is derived once, from an unforgeable input, and everything downstream is physically incapable of disagreeing. One of these designs gets safer as the codebase grows; the other accumulates risk with every sprint.
The principle underneath all of it fits in a sentence: tenant identity must be derived from the credential, enforced in the datastore, and never accepted from the client. Derivation kills the lying-client class. Datastore enforcement — RLS, schemas, or a per-tenant database — kills the forgetful-developer class. The client-supplied tenant header survives both classes intact, which is why it earns the “anti-pattern” label without qualification.
And when the Meridian ticket does arrive — because defense in depth is about when, not if, some layer fails — the difference between the pooled shop and the disciplined one is the difference between “we cannot rule out other affected tenants” and “the policy log shows zero rows returned across the blast window.” One of those sentences can be typed in a customer-facing postmortem. Build the system that lets you type it.
References
OWASP Foundation, “OWASP API Security Top 10 — API1:2023 Broken Object Level Authorization”, 2023. https://owasp.org/API-Security/editions/2023/en/0xa1-broken-object-level-authorization/ ↩︎ ↩︎
Mozilla Foundation, “Public Suffix List”. https://publicsuffix.org/ ↩︎
Kettle, J., “Practical HTTP Host Header Attacks”, 2013; and PortSwigger Web Security Academy, “HTTP Host header attacks”. https://www.skeletonscribe.net/2013/05/practical-http-host-header-attacks.html ↩︎
Stripe, Inc., “API keys”, Stripe Documentation. https://docs.stripe.com/keys ↩︎
Davis, K., Peabody, B., and Leach, P., “Universally Unique IDentifiers (UUIDs)”, IETF RFC 9562, May 2024. https://www.rfc-editor.org/rfc/rfc9562 ↩︎
Swanson, H., “Behind GitHub’s new authentication token formats”, The GitHub Blog, April 2021. https://github.blog/engineering/platform-security/behind-githubs-new-authentication-token-formats/ ↩︎ ↩︎
Biryukov, A., Dinu, D., Khovratovich, D., and Josefsson, S., “Argon2 Memory-Hard Function for Password Hashing and Proof-of-Work Applications”, IETF RFC 9106, September 2021. https://www.rfc-editor.org/rfc/rfc9106 ↩︎
OWASP Foundation, “Password Storage Cheat Sheet”, OWASP Cheat Sheet Series. https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html ↩︎
CircleCI, “CircleCI incident report for January 4, 2023 security incident”, January 2023. https://circleci.com/blog/jan-4-2023-incident-report/ ↩︎
Amazon Web Services, “SaaS Tenant Isolation Strategies: Isolating Resources in a Multi-Tenant Environment”, AWS Whitepaper, August 2020. https://docs.aws.amazon.com/whitepapers/latest/saas-tenant-isolation-strategies/saas-tenant-isolation-strategies.html ↩︎ ↩︎
PostgreSQL Global Development Group, “Row Security Policies” and “CREATE POLICY”, PostgreSQL Documentation. https://www.postgresql.org/docs/current/ddl-rowsecurity.html ↩︎ ↩︎
Weissman, C. D., and Bobrowski, S., “The design of the force.com multitenant internet application development platform”, SIGMOD 2009; also published as the Salesforce Multitenant Architecture whitepaper. https://dl.acm.org/doi/10.1145/1559845.1559942 ↩︎