Skip to main content

Install with Docker

This guide covers running nf-data-connector as a Docker container. The image is multi-arch (linux/amd64 and linux/arm64) and published to quay.io/netfoundry/nf-data-connector.

The repository is private — NetFoundry will issue you a username and password that authorize pulls. Authenticate Docker with those credentials before pulling or running the image.

Authenticate to quay.io​

Run docker login with your credentials:

echo "$QUAY_PASSWORD" | docker login quay.io --username "$QUAY_USERNAME" --password-stdin

Use --password-stdin rather than -p so the password isn't captured in shell history or ps output. The credentials are cached in ~/.docker/config.json and reused on subsequent pulls; you only need to log in again when they're rotated.

For unattended hosts (CI runners, systemd services on a server), drop the credentials into a file readable only by the user that will run Docker, then docker login --password-stdin < /path/to/password.

Pull the image​

Pull the image for your platform:

docker pull quay.io/netfoundry/nf-data-connector:latest

Tags:

  • latest: latest main build
  • <git-sha>: specific build (e.g., 048dcbd)

Quick start​

The simplest invocation is to mount a config file and pass credentials via environment variables:

docker run -d \
  --name nf-data-connector \
  --restart unless-stopped \
  -v $(pwd)/config.yaml:/etc/nf-data-connector/config.yaml:ro \
  -e ZITI_USERNAME=my-user \
  -e ZITI_PASSWORD=my-password \
  quay.io/netfoundry/nf-data-connector:latest

View logs:

docker logs -f nf-data-connector

Image layout​

The image ships with these paths pre-configured:

PathPurpose
/usr/bin/nf-data-connectorThe binary (entrypoint)
/etc/nf-data-connector/config.yamlDefault config (override by mounting a volume)
/etc/nf-data-connector/rules.yamlDefault trigger rules
/var/lib/nf-data-connector/Working directory

The container runs as a non-root nf-data-connector user.

Configuration​

Config file​

Override the default config by mounting your own over /etc/nf-data-connector/config.yaml:

-v /path/to/your/config.yaml:/etc/nf-data-connector/config.yaml:ro

Start from config.example.yaml in the repo and edit as needed. See the Configuration reference for all options.

Trigger rules​

If you're using triggers, mount a rules file and point config.yaml at it:

  1. Add to config.yaml:

    triggers:
      rules_file: "/etc/nf-data-connector/rules.yaml"

  2. Add the volume mount to your docker run command:

    -v /path/to/rules.yaml:/etc/nf-data-connector/rules.yaml:ro

Environment variables​

Pass secrets via -e or --env-file (see the full reference):

VariablePurpose
ZITI_USERNAMEOpenZiti controller username
ZITI_PASSWORDOpenZiti controller password
ES_USERNAMEElasticsearch username
ES_PASSWORDElasticsearch password
AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEYAWS credentials for the S3 subscriber (read by the AWS SDK; AWS_PROFILE and ECS/EC2 task roles also work)

Using an env file:

cat > nf-data-connector.env <<'EOF'
ZITI_USERNAME=my-user
ZITI_PASSWORD=my-password
EOF

docker run -d \
  --name nf-data-connector \
  --env-file nf-data-connector.env \
  -v $(pwd)/config.yaml:/etc/nf-data-connector/config.yaml:ro \
  quay.io/netfoundry/nf-data-connector:latest

Send events to stdout​

By default the stdout subscriber is enabled and writes JSON events to stdout. docker logs will show them alongside the log output (which goes to stderr). To separate them:

  • Only events:

    docker logs nf-data-connector 2>/dev/null

  • Only logs:

    docker logs nf-data-connector >/dev/null

Docker Compose​

  1. Create a docker-compose.yaml that mounts a config file and passes credentials via environment variables:

    services:
      nf-data-connector:
        image: quay.io/netfoundry/nf-data-connector:latest
        restart: unless-stopped
        volumes:
          - ./config.yaml:/etc/nf-data-connector/config.yaml:ro
          - ./rules.yaml:/etc/nf-data-connector/rules.yaml:ro   # optional
        environment:
          ZITI_USERNAME: ${ZITI_USERNAME}
          ZITI_PASSWORD: ${ZITI_PASSWORD}
          # ES_USERNAME: ${ES_USERNAME}
          # ES_PASSWORD: ${ES_PASSWORD}

  2. Export your credentials and start the service:

    export ZITI_USERNAME=my-user
    export ZITI_PASSWORD=my-password
    docker compose up -d
    docker compose logs -f

Configure outputs​

By default only stdout is enabled. To send events to S3, Elasticsearch, Datadog, syslog, or a webhook, edit the mounted config.yaml and consult the Configuration reference for each subscriber's schema. Pass any required credentials via -e, --env-file, or compose environment: (see Environment Variables above).

Run the TUI​

The TUI is an interactive terminal UI and needs a TTY:

docker run --rm -it \
  -v $(pwd)/config.yaml:/etc/nf-data-connector/config.yaml:ro \
  -e ZITI_USERNAME=my-user \
  -e ZITI_PASSWORD=my-password \
  quay.io/netfoundry/nf-data-connector:latest \
  -config /etc/nf-data-connector/config.yaml -tui

Troubleshoot​

Container exits immediately​

Check the logs:

docker logs nf-data-connector

Self-signed controller cert​

There are two ways to handle a self-signed controller cert.

Option A: Skip verification (dev only):

Add to config.yaml:

controller:
  skip_verify: true

Option B: Trust a CA bundle:

  1. Add the volume mount to your docker run command:

    -v /path/to/ca.pem:/etc/nf-data-connector/ca.pem:ro

  2. Add to config.yaml:

    controller:
      ca_file: "/etc/nf-data-connector/ca.pem"
      fetch_ca: false

Verify the image manifest​

To confirm multi-arch support for your platform:

docker manifest inspect quay.io/netfoundry/nf-data-connector:latest

Run a one-shot config syntax check​

Start the container without -d to validate your config file:

docker run --rm \
  -v $(pwd)/config.yaml:/etc/nf-data-connector/config.yaml:ro \
  quay.io/netfoundry/nf-data-connector:latest \
  -config /etc/nf-data-connector/config.yaml

The connector exits immediately on config-parse errors. If it reaches the connection phase, the YAML is valid.

More info​