s3 subscriber

Batches events per namespace and uploads them as JSONL objects to S3.

Add to the subscribers block in config.yaml:

subscribers:
  s3:
    enabled: true
    bucket: "my-ziti-events"
    region: "us-east-1"
    prefix: "events/"
    flush_size_bytes: 5242880      # 5MB
    flush_interval: 60s
    format: "jsonl"
    workers: 2                     # parallel PutObject uploads
    buffer_size: 1000
    # Per-event filter — events dropped here never enter any per-namespace
    # buffer or get uploaded. include is any-of, exclude is none-of.
    # See ../ "Per-Subscriber Filtering".
    include: []
    exclude: []

Events are written to:

{prefix}{namespace}/{YYYY}/{MM}/{DD}/{HH}/{timestamp}-{uuid}.jsonl

Flushes when the buffer exceeds flush_size_bytes OR the timer fires, whichever comes first. Multiple namespaces can upload in parallel up to workers — a single namespace's flushes are still serial, preserving file order within its prefix.

Available fields and defaults

Field	Default	Description
bucket	—	S3 bucket name
region	—	AWS region
prefix	—	Object key prefix
flush_size_bytes	5242880 (5 MiB)	Per-namespace buffer size that triggers an upload
flush_interval	60s	Max time between uploads regardless of buffer fill
format	jsonl	Object body format
workers	2	Parallel PutObject calls (one per namespace at a time)
buffer_size	1000	Subscriber channel capacity
include	[]	Per-event predicates against the enriched payload; any-of. Empty = pass everything. See Per-subscriber filtering.
exclude	[]	Per-event predicates; none-of — drop on match.

AWS credentials

The S3 subscriber uses the standard AWS SDK credential chain. It picks up, in order:

• AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN env vars
• ~/.aws/credentials and ~/.aws/config files (respecting AWS_PROFILE)
• EC2 instance metadata (IMDS)
• ECS / EKS task role credentials

See Common tuning for workers and buffer_size semantics.