Skip to main content

Expose support tools externally

By default, the support stack tools (Grafana, Kibana, Elasticsearch) are only accessible over Ziti using intercept addresses. If you need to reach them directly — for example from a browser on the same network as the appliance — you can expose them externally.

This guide covers two approaches:

  • Approach 1: LoadBalancer services (recommended) — works out of the box on single-node K3s, requires no internet access, and is the right choice for offline / air-gapped deployments.
  • Approach 2: Traefik Ingress — a connected/online alternative that terminates TLS at the ingress, typically with Let's Encrypt.
warning

Exposing support tools externally opens additional network ports. The recommended approach is to access these tools over Ziti (see Support stack overview). Only expose externally if your environment requires it.

Internal service reference

The support stack creates these internal services:

ToolService nameNamespaceInternal port
Grafanagrafanasupport3000 (HTTP)
Kibanakibana-kb-httpsupport5601 (HTTPS)
Elasticsearchelasticsearch-es-httpsupport9200 (HTTPS)

How the support chart is configured

The support stack is installed by the quickstart as a Helm release named support in the support namespace. During installation the installer generates a support-values.yml file (next to the install scripts) and runs the equivalent of:

helm upgrade -n support --install support ./helm-charts/support/ --values support-values.yml

To expose tools, you add the values shown below to support-values.yml (or a separate override file) and re-run helm upgrade against the same release. Because the installer reuses an existing support-values.yml if one is already present, editing that file before install also works.


Single-node K3s ships with ServiceLB (Klipper) as its default load-balancer implementation. This means a Service of type: LoadBalancer works immediately with no MetalLB and no internet access — Klipper binds the service port directly to the node's IP. This is the recommended way to expose support tools on offline / air-gapped appliances.

tip

The LoadBalancer approach does not use Traefik or Kubernetes Ingress at all, so it cannot collide with the Ziti console (ziti-console-enterprise-ingress) or zLAN console (zlan-console-ingress) Ingress resources in the ziti namespace. For appliance deployments this avoids an entire class of host/path conflicts (see the warning under Approach 2).

Grafana

Grafana is a single Helm value. Set grafana.serviceType to LoadBalancer (default is ClusterIP):

# support-values.yml (or an override file)
grafana:
serviceType: LoadBalancer

Grafana serves plain HTTP on port 3000. If you require TLS for Grafana in an air-gapped environment, front it with your own reverse proxy / TLS terminator — there is no Let's Encrypt path offline.

Kibana

Kibana is managed by the Elastic operator (ECK) via a Kibana custom resource. The chart passes kibana.http straight through to that CR, so set the service type there:

kibana:
http:
service:
spec:
type: LoadBalancer
ports:
- port: 5601
protocol: TCP
targetPort: 5601

Kibana's endpoint stays HTTPS using the ECK self-signed certificate — which is exactly what you want offline, since no public CA is reachable. By default that certificate is not valid for the external hostname or IP you'll use to reach it. Add the external name/IP as a Subject Alternative Name so the cert validates:

kibana:
http:
tls:
selfSignedCertificate:
subjectAltNames:
- dns: kibana.example.com
# - ip: 10.0.0.10
service:
spec:
type: LoadBalancer
ports:
- port: 5601
protocol: TCP
targetPort: 5601

Alternatively, bring your own certificate (see Certificates in air-gapped environments).

Elasticsearch (optional)

Elasticsearch follows the same ECK pattern. Exposing it directly is usually unnecessary (Kibana queries it internally), so treat this as optional:

elasticsearch:
http:
service:
spec:
type: LoadBalancer
ports:
- port: 9200
targetPort: 9200

Like Kibana, Elasticsearch keeps its ECK self-signed HTTPS endpoint; add SANs the same way under elasticsearch.http.tls.selfSignedCertificate.subjectAltNames if you need the cert to validate for the external address.

Apply the overrides

Add the values above to support-values.yml (or a separate override file) and upgrade the existing release:

helm upgrade -n support support ./helm-charts/support/ --values support-values.yml
# or, with a separate override file layered on top:
helm upgrade -n support support ./helm-charts/support/ -f support-values.yml -f support-loadbalancer.yml

Open the host firewall

Exposing these services requires opening the corresponding host firewall / cloud security-group ports:

  • Grafana: 3000/tcp
  • Kibana: 5601/tcp
  • Elasticsearch (only if exposed): 9200/tcp

On hardened appliance boxes only 22, 1280, and 3022 are open by default, so these ports must be opened explicitly.

Verify

Find the assigned external address and port:

kubectl get svc -n support

Look for the EXTERNAL-IP and port on the grafana, kibana-kb-http, and (if exposed) elasticsearch-es-http services. On single-node K3s the external IP is the node's own IP. Then reach the tools at:

  • Grafana: http://<node-ip>:3000
  • Kibana: https://<node-ip>:5601
  • Elasticsearch (if exposed): https://<node-ip>:9200

See Default credentials for login details.


Approach 2: Traefik Ingress (online alternative)

On connected installations you can instead expose the tools through Kubernetes Ingress with TLS termination at Traefik, typically using Let's Encrypt for trusted certificates. This requires internet reachability for ACME validation and is not suitable for air-gapped deployments.

:::danger Do not clobber the console or zLAN Ingress The Ziti console (ziti-console-enterprise-ingress) and the zLAN console (zlan-console-ingress, created when the -z flag is used) both create host-based Traefik Ingress resources in the ziti namespace, each with their own cert-manager issuers. When adding support-tool Ingresses you must:

  • Use distinct hostnames — never reuse a console or zLAN console host.
  • Not define a catch-all path or default backend (no bare / host-less rule) that could capture console traffic.
  • Keep the support-tool Ingress resources in the support namespace, alongside the services they target.

If you don't need TLS-at-ingress or trusted public certs, prefer the LoadBalancer approach above, which sidesteps this entirely. :::

Prerequisites

  • K3s installation with Traefik (included by default with K3s)
  • DNS entries pointing to your K3s node for each tool you want to expose
  • TLS certificates for your domains (this guide uses cert-manager, which is already installed by the quickstart)

Step 1: Create a TLS certificate issuer

If you want to use Let's Encrypt for trusted TLS certificates, create a ClusterIssuer. If you already have one configured, skip to Step 2.

# letsencrypt-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: your-email@example.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: traefik
kubectl apply -f letsencrypt-issuer.yaml
note

The HTTP-01 solver requires that port 80 is accessible from the internet for Let's Encrypt validation. If this is not possible, use a DNS-01 solver or provide your own certificates as a Kubernetes TLS secret. In a fully air-gapped environment Let's Encrypt is unreachable — use Approach 1 instead.

Step 2: Create Ingress resources

Create an Ingress for each tool you want to expose. Replace the host values with your DNS names. These hostnames must be distinct from any console / zLAN console host (see the warning above).

Grafana

Grafana serves plain HTTP on port 3000, so Traefik handles TLS termination directly.

# grafana-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana-ingress
namespace: support
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: traefik
tls:
- hosts:
- grafana.example.com
secretName: grafana-tls
rules:
- host: grafana.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: grafana
port:
number: 3000

Kibana

Kibana's internal service uses HTTPS (managed by the Elastic operator). Traefik needs to pass through TLS traffic to the backend rather than terminating it at the ingress. Use a Traefik ServersTransport to skip backend certificate verification (since the Elastic operator uses self-signed certificates internally), and annotate the Ingress accordingly.

# kibana-transport.yaml
apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
name: kibana-transport
namespace: support
spec:
insecureSkipVerify: true
# kibana-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: kibana-ingress
namespace: support
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
traefik.ingress.kubernetes.io/service.serverstransport: support-kibana-transport@kubernetescrd
spec:
ingressClassName: traefik
tls:
- hosts:
- kibana.example.com
secretName: kibana-tls
rules:
- host: kibana.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kibana-kb-http
port:
number: 5601

Elasticsearch

Elasticsearch also uses internal HTTPS, so it requires the same ServersTransport approach as Kibana.

# elasticsearch-transport.yaml
apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
name: elasticsearch-transport
namespace: support
spec:
insecureSkipVerify: true
# elasticsearch-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: elasticsearch-ingress
namespace: support
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
traefik.ingress.kubernetes.io/service.serverstransport: support-elasticsearch-transport@kubernetescrd
spec:
ingressClassName: traefik
tls:
- hosts:
- elasticsearch.example.com
secretName: elasticsearch-tls
rules:
- host: elasticsearch.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: elasticsearch-es-http
port:
number: 9200

Step 3: Apply the resources

kubectl apply -f letsencrypt-issuer.yaml
kubectl apply -f grafana-ingress.yaml
kubectl apply -f kibana-transport.yaml
kubectl apply -f kibana-ingress.yaml
kubectl apply -f elasticsearch-transport.yaml
kubectl apply -f elasticsearch-ingress.yaml

Step 4: Verify

Check that the Ingress resources have been created and assigned addresses:

kubectl get ingress -n support

Verify that cert-manager has issued TLS certificates:

kubectl get certificates -n support

Once certificates are issued, access the tools at:

  • https://grafana.example.com
  • https://kibana.example.com
  • https://elasticsearch.example.com

Certificates in air-gapped environments

In an air-gapped environment Let's Encrypt is unreachable, so the only viable certificate paths are ECK self-signed certificates (for Kibana and Elasticsearch) or certificates you bring yourself.

  • Kibana / Elasticsearch keep their ECK self-signed HTTPS endpoints. Add the external hostname or IP as a SAN via subjectAltNames (shown under Approach 1) so the certificate validates for the address clients use, or supply your own certificate to ECK.
  • Grafana serves plain HTTP. If you require TLS, front it with your own reverse proxy / TLS terminator.

If you are using the Traefik Ingress approach offline and need to supply your own certificates rather than issue them through cert-manager, create TLS secrets manually from your certificate files:

kubectl create secret tls grafana-tls \
--cert=grafana.crt --key=grafana.key -n support

kubectl create secret tls kibana-tls \
--cert=kibana.crt --key=kibana.key -n support

kubectl create secret tls elasticsearch-tls \
--cert=elasticsearch.crt --key=elasticsearch.key -n support

Then remove the cert-manager.io/cluster-issuer annotation from each Ingress resource, as cert-manager is no longer managing the certificates.

Default credentials

See the support stack overview for Grafana and Elasticsearch/Kibana credentials.