Authenticating with Object Store Backends

The proxy needs credentials to access backend object stores (S3, Azure Blob Storage, GCS). There are two approaches: static credentials stored in the proxy config, and OIDC-based credential resolution where the proxy acts as its own identity provider.

Static Backend Credentials

The simplest approach is to include credentials directly in the bucket's backend_options:

[[buckets]]
name = "my-data"
backend_type = "s3"

[buckets.backend_options]
endpoint = "https://s3.us-east-1.amazonaws.com"
bucket_name = "my-backend-bucket"
region = "us-east-1"
access_key_id = "AKIAIOSFODNN7EXAMPLE"
secret_access_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

This works for any backend type. For anonymous backend access (e.g., public buckets), simply omit the access_key_id and secret_access_key fields — when both are absent, the proxy issues unsigned requests automatically.

NOTE

A skip_signature option appears in some examples, but it is currently not honored by the proxy and has no effect. Anonymous access is determined solely by the absence of credentials.

OIDC Backend Auth

For production deployments, the proxy can act as its own OIDC identity provider. Instead of storing long-lived backend credentials, the proxy mints self-signed JWTs and exchanges them with cloud providers for temporary credentials — the same pattern used by GitHub Actions and Vercel for AWS access.

How It Works

Configuration

OIDC backend auth requires two environment variables:

Variable	Description
OIDC_PROVIDER_KEY	PEM-encoded RSA private key for JWT signing
OIDC_PROVIDER_ISSUER	Publicly reachable URL (e.g., https://s3proxy.example.com)

Generate an RSA key pair:

openssl genrsa -out oidc-key.pem 2048

Set the environment variables:

export OIDC_PROVIDER_KEY=$(cat oidc-key.pem)
export OIDC_PROVIDER_ISSUER="https://s3proxy.example.com"

Then configure buckets to use OIDC:

[[buckets]]
name = "my-data"
backend_type = "s3"

[buckets.backend_options]
endpoint = "https://s3.us-east-1.amazonaws.com"
bucket_name = "my-backend-bucket"
region = "us-east-1"
auth_type = "oidc"
oidc_role_arn = "arn:aws:iam::123456789012:role/DataProxyAccess"

Discovery Endpoints

When OIDC provider keys are configured, the proxy serves two well-known endpoints that cloud providers use to validate JWTs:

GET /.well-known/openid-configuration

{
  "issuer": "https://s3proxy.example.com",
  "jwks_uri": "https://s3proxy.example.com/.well-known/jwks.json",
  "response_types_supported": ["id_token"],
  "subject_types_supported": ["public"],
  "id_token_signing_alg_values_supported": ["RS256"]
}

GET /.well-known/jwks.json

{
  "keys": [{
    "kty": "RSA",
    "alg": "RS256",
    "use": "sig",
    "kid": "proxy-key-1",
    "n": "<base64url-modulus>",
    "e": "<base64url-exponent>"
  }]
}

WARNING

These endpoints must be publicly accessible. Cloud providers fetch them at JWT validation time to verify signatures. If they are behind a firewall or VPN, credential exchange will fail.

The Exchange Flow in Detail

When a request arrives for a bucket with auth_type=oidc:

1. The OIDC backend auth middleware detects auth_type=oidc in the bucket's backend_options
2. It mints a short-lived JWT signed with the proxy's RSA private key:
   • iss: the configured OIDC_PROVIDER_ISSUER
   • sub: a connection identifier (from oidc_subject option, or a default)
   • aud: the cloud provider's STS audience (e.g., sts.amazonaws.com)
   • exp: short expiration (minutes)
3. The proxy sends the JWT to the cloud provider's STS endpoint along with the target IAM role ARN
4. The cloud provider fetches the proxy's JWKS, verifies the JWT signature, evaluates the role's trust policy, and returns temporary credentials
5. The proxy caches the credentials (keyed by role ARN) and injects them into the bucket config
6. The existing create_builder() / build_signer() pipeline consumes the credentials normally

On subsequent requests, cached credentials are reused until they expire.

Cloud Provider Setup

AWS S3

Administrator setup:

1. Register the OIDC provider in your AWS account:

   aws iam create-open-id-connect-provider \
     --url https://s3proxy.example.com \
     --client-id-list sts.amazonaws.com \
     --thumbprint-list <thumbprint>

   TIP

   To get the thumbprint, fetch the TLS certificate chain from your proxy's domain. AWS uses this to verify the HTTPS connection to the JWKS endpoint.

2. Create an IAM Role with a trust policy that allows the proxy to assume it:

   {
     "Version": "2012-10-17",
     "Statement": [{
       "Effect": "Allow",
       "Principal": {
         "Federated": "arn:aws:iam::123456789012:oidc-provider/s3proxy.example.com"
       },
       "Action": "sts:AssumeRoleWithWebIdentity",
       "Condition": {
         "StringEquals": {
           "s3proxy.example.com:aud": "sts.amazonaws.com",
           "s3proxy.example.com:sub": "s3-proxy"
         }
       }
     }]
   }

3. Attach an S3 permission policy to the role:

   {
     "Version": "2012-10-17",
     "Statement": [{
       "Effect": "Allow",
       "Action": [
         "s3:GetObject",
         "s3:PutObject",
         "s3:ListBucket",
         "s3:DeleteObject"
       ],
       "Resource": [
         "arn:aws:s3:::my-backend-bucket",
         "arn:aws:s3:::my-backend-bucket/*"
       ]
     }]
   }

4. Configure the bucket in the proxy:

   [[buckets]]
   name = "my-data"
   backend_type = "s3"
   
   [buckets.backend_options]
   endpoint = "https://s3.us-east-1.amazonaws.com"
   bucket_name = "my-backend-bucket"
   region = "us-east-1"
   auth_type = "oidc"
   oidc_role_arn = "arn:aws:iam::123456789012:role/DataProxyAccess"

At request time, the proxy calls AWS STS AssumeRoleWithWebIdentity with the self-signed JWT. No AWS credentials are stored in the proxy configuration.

Azure Blob Storage

NOTE

Partial — The Azure token-exchange logic exists (behind the azure cargo feature), but the end-to-end backend-auth middleware is currently wired up for AWS/S3 buckets only; an auth_type=oidc Azure bucket will return a configuration error. For now, use Azure with static credentials.

Google Cloud Storage

NOTE

Partial — The GCS token-exchange logic exists (behind the gcp cargo feature), but the end-to-end backend-auth middleware is currently wired up for AWS/S3 buckets only; an auth_type=oidc GCS bucket will return a configuration error. For now, use GCS with static credentials.

Credential Caching

When using OIDC backend auth, the proxy caches temporary credentials to avoid calling the cloud provider's STS on every request. Credentials are:

• Keyed by the IAM role ARN
• Proactively refreshed shortly before they expire, so a credential is never handed out about to expire mid-request
• Single-flighted across concurrent requests to the same bucket — only one token exchange runs while the rest await its result

This means the first request to an OIDC-backed bucket incurs a small latency cost for the credential exchange, but subsequent requests use cached credentials until they expire.

NOTE

How effective this cache is depends on the runtime — an in-memory cache is per-process on the server but per-isolate on Cloudflare Workers. See Caching for the cross-runtime details and best practices (including layering the Cloudflare Cache API).

Choosing Between Static and OIDC

	Static Credentials	OIDC Backend Auth
Setup complexity	Low	Medium (IAM role + OIDC provider registration)
Credential rotation	Manual	Automatic (temporary credentials)
Security	Long-lived secrets in config	No long-lived secrets
Cloud providers	All (S3, Azure, GCS)	AWS S3 (Azure/GCS token exchange exists behind cargo features, but middleware wiring is S3-only)
Latency	None	Small cost on first request (then cached)