Configuration reference

nf-data-connector is configured via a YAML file (default: config.yaml, override with -config <path>) and environment variables for secrets.

Related docs:

Top-level structure

controller:       # Ziti controller connection
events:           # Event stream subscriptions
enrichment:       # Resource-name resolution cache
subscribers:      # Output targets (enable what you need)
triggers:         # Optional rule-based alerting
logging:          # Log level and format

`controller`: Controller connection

Connects to one or more controller nodes. Supports HA clusters.

controller:
  # Single controller:
  host: "ctrl.example.com:1280"

  # HA cluster (takes precedence over host). When set, the connector tries
  # each host in order and rotates on failure:
  # hosts:
  #   - "ctrl-1.example.com:1280"
  #   - "ctrl-2.example.com:1280"
  #   - "ctrl-3.example.com:1280"

  auth_method: "password"     # "password" or "cert"
  username: ""                # prefer env var ZITI_USERNAME
  password: ""                # prefer env var ZITI_PASSWORD

  # Certificate-based auth (when auth_method: cert):
  cert_file: ""               # path to client cert (PEM)
  key_file: ""                # path to client key (PEM)

  # TLS:
  ca_file: ""                 # path to custom CA bundle (PEM)
  fetch_ca: true              # auto-fetch CA from /edge/client/v1/.well-known/est/cacerts
  skip_verify: false          # disable TLS verification (dev/test only)

Field	Type	Default	Description
`host`	string	:	Single controller `host:port` (no `https://` prefix)
`hosts`	list	:	HA cluster list; takes precedence over `host`
`auth_method`	string	`password`	`password` or `cert`
`username`	string	:	Password auth username (use env var)
`password`	string	:	Password auth password (use env var)
`cert_file`	string	:	Client cert for cert auth
`key_file`	string	:	Client key for cert auth
`ca_file`	string	:	Custom CA bundle (PEM)
`fetch_ca`	bool	`true`	Fetch CA from controller's well-known EST endpoint
`skip_verify`	bool	`false`	Skip TLS verification — dev only

HA behavior: If multiple hosts are configured, authentication and CA fetching try each host in order. On disconnect or connection failure, the connector rotates to the next host. All components (websocket, enrichment API) follow the active host.

`events`: Event subscriptions

events:
  subscriptions:
    - type: usage
      version: 3
    - type: circuit
    - type: link
    - type: router
    - type: terminator
    - type: session
    - type: apiSession
    - type: sdk
    - type: service
    - type: connect
    - type: metrics
      sourceFilter: ".*"
      metricFilter: ".*"
    - type: entityCount
    - type: entityChange
      include:
        - identities
        - services
        - routers

  namespace_filter: []        # optional: only forward events in this list

Each subscription can have the following fields:

Field	Used by	Description
`type`	all	Event namespace. See the Event Types table below.
`version`	`usage`	Schema version (use `3` for current flat format)
`sourceFilter`	`metrics`	Regex matching the event source
`metricFilter`	`metrics`	Regex matching the metric name
`include`	`entityChange`	Array of entity types to report on

Event types

Each type maps to a distinct event namespace on the controller's WebSocket stream.

Type	Description
`apiSession`	API session created/deleted/refreshed
`authentication`	Login success/failure
`circuit`	Fabric circuit lifecycle (created/deleted/failed/pathUpdated)
`cluster`	HA cluster peer events
`connect`	Connections to controllers/routers
`entityChange`	CRUD events for OpenZiti entities
`entityCount`	Periodic entity count snapshots
`link`	Router-link lifecycle
`metrics`	Performance histograms (e.g., `ctrl.latency`, `link.latency`)
`router`	Router online/offline
`sdk`	SDK (identity) online/offline
`service`	Service dial metrics
`session`	Bind/dial session lifecycle
`terminator`	Terminator lifecycle
`usage`	Bandwidth counters (set `version: 3`)

Namespace filter

If set, only events whose namespace matches one of the listed values are forwarded. Empty list = forward everything.

`enrichment`: Resource name resolution

enrichment:
  enabled: true
  cache_ttl: 24h
  cache_max_size: 10000

The enricher resolves OpenZiti resource IDs (services, identities, and edge routers) to human-readable names by querying the management API, caching results in memory.

Field	Type	Default	Description
`enabled`	bool	`true`	Enable name resolution
`cache_ttl`	duration	`24h`	How long to cache lookups
`cache_max_size`	int	`10000`	Maximum cache entries (FIFO eviction on overflow)

Enriched fields added per event type:

Event	Resolved Fields
`usage`	`service_name`, `identity_name`, `host_name`, `edge_router_name`
`circuit`	`service_name`, `identity_name`, `attributes`
`metrics` (`ctrl.latency`)	`edge_router_name` from `source_entity_id`
`metrics` (`link.latency`)	`source_edge_router_name`, `destination_edge_router_name`
`sdk`	`identity_name`, `attributes`
`router`	`edge_router_name`
`apiSession`	`identity_name`
`link`	`source_edge_router_name`, `destination_edge_router_name`
`terminator`	`service_name`, `edge_router_name`
`service.events`	`service_name`

`subscribers`: Output targets

Each subscriber is enabled with enabled: true in its config block. Multiple subscribers can run in parallel — events are fanned out through an internal pub-sub bus, so a slow or failing subscriber never blocks the others.

Subscriber	YAML key	Description
stdout	`stdout`	Write events to stdout as JSON (pretty or single-line)
Raw HTTP	`raw_http`	POST events to any HTTP endpoint (webhooks, Lambda URLs, Collector's `webhookevent`)
Elasticsearch	`elasticsearch`	Bulk-index events into per-namespace datastreams
S3	`s3`	Batched JSONL uploads, partitioned by namespace/date
syslog	`syslog`	Forward each event as a single syslog message (TCP/UDP)
Datadog	`datadog`	Logs intake and/or v2 metrics series
Splunk	`splunk`	HTTP Event Collector with optional indexer acknowledgement
OTel (OTLP)	`otel`	Native OTLP logs and metrics over HTTP or gRPC

See each subscriber's page for the full field reference, defaults, and tuning notes. Subscriber-by-subscriber throughput counters (delivered, dropped, QLEN) live in the TUI Throughput view (-tui, then tab to cycle).

Common tuning

Every subscriber accepts buffer_size, and HTTP-backed subscribers (raw_http, elasticsearch, s3, datadog, and splunk) also accept workers. Defaults are tuned for a small-to-medium network; raise them when you see sustained drops or high QLEN in the TUI Throughput view.

Field	Applies to	Default	Description
`buffer_size`	all	`1000`	In-memory channel capacity. When full, events bound for this subscriber are dropped (drops are counted per subscriber and visible in the TUI).
`workers`	`raw_http`, `elasticsearch`, `s3`, `datadog`, `splunk`	varies	Number of goroutines doing parallel delivery. See each subscriber's page for its default.

Workers parallelize delivery only — events may arrive at the endpoint out of order when workers > 1. Most consumers (Elasticsearch, S3, alert backends) key on timestamp so this is harmless; if your downstream requires strict ordering, keep workers: 1.

Per-subscriber filtering

Every cost-sensitive sink — datadog.logs, splunk, otel.logs, elasticsearch, s3, and raw_http — accepts an optional include and exclude block. The connector evaluates these against the enriched event JSON and drops anything that doesn't pass before the event is batched or compressed, so filtered events never count against your ingest bill.

Use this when an upstream sink charges per event or per indexed field and you want to send only a subset. The two most common cases:

# Only ship circuit events whose event_type is not "created" — drop the
# noisy creates, keep failed / pathUpdated / deleted.
datadog:
  logs:
    enabled: true
    namespace_filter: [fabric.circuits]
    include:
      - { field: event_type, not_equals: created }

# Ship usage events only for production services (name starts with "prod-").
splunk:
  enabled: true
  namespace_filter: [fabric.usage]
  include:
    - { field: service_name, regex: "^prod-" }

Semantics:

include is any-of — at least one condition must match. Empty include accepts every event.
exclude is none-of — if any condition matches, the event is dropped. Empty exclude drops nothing.
Both can be set; the event must pass include AND not match exclude.
namespace_filter (if set) runs first, so you can think of include/exclude as a finer-grained refinement of the namespaces you already opted in to.

Each condition is one field-level predicate. Field paths are gjson paths into the enriched event payload (so service_name, event_type, metrics.p99, tags.hostId all work). See Comparators under the triggers section below for the full set; exactly one comparator must be set per condition.

A condition with an invalid regex or no comparator is rejected at startup — the connector fails fast rather than silently letting everything through.

`triggers`: Rule-based alerting

Optional. Loads a separate rules file and fires webhooks when events match configured conditions. The condition vocabulary is the same as Per-Subscriber Filtering above — field + one comparator — but conditions inside a rule are AND'd (every condition must match) rather than OR'd.

triggers:
  rules_file: "/etc/nf-data-connector/rules.yaml"
  default_webhook:
    url: "https://alerts.example.com/hooks/ziti"
    method: "POST"
    headers:
      Authorization: "Bearer my-token"

Rule file format

A rules file contains a top-level rules array:

# rules.yaml
rules:
  - name: circuit-failure
    description: "Alert on circuit creation failures"
    severity: critical
    cooldown: 5m               # optional: suppress duplicate alerts
    conditions:
      - field: namespace
        equals: circuit
      - field: event_type
        equals: failed

  - name: high-ctrl-latency
    description: "Controller latency p99 over 100ms"
    severity: warning
    cooldown: 10m
    conditions:
      - field: namespace
        equals: metrics
      - field: metric
        equals: ctrl.latency
      - field: metrics.p99
        gt: 100000000            # 100ms in nanoseconds

  - name: prod-identity-offline
    description: "Production identity disconnected"
    severity: critical
    webhook:                     # per-rule override
      url: "https://pagerduty.example.com/hooks/ziti"
    conditions:
      - field: namespace
        equals: sdk
      - field: event_type
        equals: sdk-offline
      - field: attributes
        in_array: "prod-users"

All conditions in a rule are AND'd — every condition must match. For OR logic, create separate rules.

Comparators

Each rule condition uses one of the following comparators. The same vocabulary is used by per-subscriber include/exclude filters above.

Comparator	Description	Example
`equals`	Exact string match	`field: namespace, equals: circuit`
`not_equals`	String negation	`field: event_type, not_equals: created`
`contains`	Substring match	`field: service_name, contains: prod`
`regex`	RE2 regex	`field: identity_name, regex: "^prod-.*"`
`gt` / `gte` / `lt` / `lte`	Numeric comparison	`field: metrics.p99, gt: 50000000`
`exists`	Field presence	`field: tags.hostId, exists: true`
`in_array`	Value in array field	`field: attributes, in_array: "prod-users"`

Fields can be nested (e.g., metrics.p99, tags.hostId) using gjson path syntax.

Alert payload

When a rule fires, the connector POSTs this payload to the configured webhook URL:

{
  "rule": "circuit-failure",
  "description": "Alert on circuit creation failures",
  "severity": "critical",
  "fired_at": "2026-04-13T12:34:56Z",
  "event": { "...": "the full enriched event..." }
}

`logging`

Configure the log level and output format:

logging:
  level: "info"    # debug, info, warn, error
  format: "text"   # text or json

Environment variables

Secrets should come from environment variables rather than being written into the YAML file. Env vars override YAML values.

Variable	Overrides	Notes
`ZITI_USERNAME`	`controller.username`	OpenZiti controller username
`ZITI_PASSWORD`	`controller.password`	OpenZiti controller password
`ES_USERNAME`	`subscribers.elasticsearch.username`	Elasticsearch username
`ES_PASSWORD`	`subscribers.elasticsearch.password`	Elasticsearch password
`DD_API_KEY`	`subscribers.datadog.api_key`	Datadog API key

How those env vars reach the connector depends on how it's installed:

Install method	Where to set env vars
Debian package (systemd)	`/etc/nf-data-connector/env`, loaded via `EnvironmentFile=` in a systemd override (see Install on Debian)
Docker	`-e VAR=value` flags, `--env-file`, or the `environment:` block in compose (see Install with Docker)
Standalone binary	Exported in the shell or in the launchd / Windows-service environment block

AWS credentials for the S3 subscriber: the S3 subscriber uses the standard AWS SDK credential chain — there is no project-specific override. It picks up, in order:

AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN env vars
~/.aws/credentials and ~/.aws/config files (respecting AWS_PROFILE)
EC2 instance metadata (IMDS)
ECS/EKS task role credentials

Command-line flags

Pass these flags when starting nf-data-connector:

Flag	Default	Description
`-config <path>`	`config.yaml`	Path to config file
`-tui`	off	Launch the interactive terminal UI instead of stream mode

Top-level structure​

controller: Controller connection​

events: Event subscriptions​

Event types​

Namespace filter​

enrichment: Resource name resolution​

subscribers: Output targets​

Common tuning​

Per-subscriber filtering​

triggers: Rule-based alerting​

Rule file format​

Comparators​

Alert payload​

logging​

Environment variables​

Command-line flags​