OpenTelemetry Auto-Instrumentation Configuration ¶
When you enable auto-instrumentation in your application the following OpenTelemetry configuration will become available to your application as environment variables:
| Variable | Example Value |
|---|---|
OTEL_SERVICE_NAME | my-application |
OTEL_EXPORTER_OTLP_ENDPOINT | http://opentelemetry-collector.nais-system:4317 |
OTEL_EXPORTER_OTLP_PROTOCOL | grpc |
OTEL_EXPORTER_OTLP_INSECURE | true |
OTEL_PROPAGATORS | tracecontext,baggage |
OTEL_TRACES_SAMPLER | parentbased_always_on |
OTEL_LOGS_EXPORTER | none |
OTEL_RESOURCE_ATTRIBUTES_POD_NAME | my-application-777787df6d-pw9mq |
OTEL_RESOURCE_ATTRIBUTES_NODE_NAME | gke-node-abc123 |
OTEL_RESOURCE_ATTRIBUTES | service.name=my-application,service.namespace=my-team,k8s.container.name=my-application,... |
Tip
These variables are managed by Nais. Do not hardcode them in your application code. The OTel SDK picks them up automatically. You can add extra attributes via OTEL_RESOURCE_ATTRIBUTES in your nais.yaml — see Extra resource attributes below.
Extra resource attributes ¶
OpenTelemetry Resource Attributes are key-value pairs that describe the application and its environment. These attributes are attached to all telemetry data produced by the application and must adhere to the OpenTelemetry Semantic Conventions.
How OTEL_RESOURCE_ATTRIBUTES works on Nais ¶
You can safely set OTEL_RESOURCE_ATTRIBUTES in your nais.yaml — Nais merges your attributes with the platform-managed ones. It does not overwrite them.
Nais manages these attributes and will ignore any attempt to override them:
| Attribute | Set by | Value |
|---|---|---|
service.name | Nais (from app name) | my-application |
service.namespace | Nais (from namespace) | my-team |
nais.backend | Nais (from destinations) | destination1;destination2 |
Any other key=value pairs you provide are appended to the final OTEL_RESOURCE_ATTRIBUTES value.
spec:
env:
- name: OTEL_RESOURCE_ATTRIBUTES
value: "deployment.environment.name=prod"The resulting env var in the pod will be:
service.name=my-app,service.namespace=my-team,nais.backend=...,deployment.environment.name=prodWhen to set extra attributes ¶
| Attribute | When to use | Effect |
|---|---|---|
deployment.environment.name | Apps deployed to multiple environments (dev/prod, q1/q2) | APM and Grafana can filter by environment. The collector also uses this to strip env suffixes (e.g. -q1) from service names. |
Available attributes ¶
| Attribute | Type | Description | Example Value | Stability |
|---|---|---|---|---|
deployment.environment.name | string | Name of the deployment environment (e.g. q1, prod, dev). | q1; prod | experimental |
Sanitizing sensitive data ¶
While the OpenTelemetry SDKs and agents do a best effort to not include any user data, we have added extra protection by redacting some patterns from trace span attributes. The following attributes are scrubbed for personal identification numbers (fødselsnummer):
-
url.path -
url.full -
url.original -
url.query -
http.url -
http.target -
http.route -
db.statement -
messaging.kafka.message.key
We advise you to look over the data collected by your application and ensure that no sensitive data is being stored and contact the nais-team if you have any questions or concerns.
Logs auto-instrumentation ¶
You can enable instrumenting logs using the OpenTelemetry Auto-Instrumentation by setting the OTEL_LOGS_EXPORTER environment variable to otlp. This will intercept all logs produced by the application and send them to the OpenTelemetry Collector.
spec:
env:
- name: OTEL_LOGS_EXPORTER
value: otlpWarning
Enabling logging for the OpenTelemetry Auto-Instrumentation will send all logs to the OpenTelemetry Collector including logs from other libraries and frameworks such as log4j, logback, and slf4j. This should not be enabled if you are using Team Logs.
Agent Versions ¶
The OpenTelemetry Agent is used to automatically instrument your application. The agent is responsible for collecting telemetry data and sending it to the OpenTelemetry Collector.
Java Agent ¶
The Java agent attaches to your JVM automatically at startup using the JAVA_TOOL_OPTIONS environment variable.
Supported libraries, frameworks, application servers, and JVMs ¶
The OpenTelemetry Java Agent supports many popular libraries, frameworks, application servers, and JVMs. A full list of supported libraries and frameworks can be found on the open-telemetry/opentelemetry-java-instrumentation repository.
Advanced Configuration ¶
When using the OpenTelemetry Java SDK and Agent (auto-instrumentation), the following additional environment variables are available:
| Variable | Description | Example Value |
|---|---|---|
OTEL_JAVAAGENT_LOGGING | Controls log output from the Java Agent itself, valid values are none, simple, and application | simple |
OTEL_JAVAAGENT_EXCLUDE_CLASSES | Suppresses all instrumentation for specific classes, format is "my.package.MyClass,my.package2.*" | my.package.MyClass,my.package2.* |
OTEL_INSTRUMENTATION_SPRING_BOOT_ACTUATOR_AUTOCONFIGURE_ENABLED | Enables or disables the Spring Boot Actuator auto-configuration instrumentation | false |
OTEL_INSTRUMENTATION_MICROMETER_ENABLED | Enables or disables the Micrometer instrumentation | false |
OTEL_INSTRUMENTATION_COMMON_EXPERIMENTAL_CONTROLLER_TELEMETRY_ENABLED | Enables or disables controller span instrumentation | false |
OTEL_INSTRUMENTATION_COMMON_EXPERIMENTAL_VIEW_TELEMETRY_ENABLED | Enables or disables view span instrumentation | false |
More OpenTelemetry Configuration ¶
A full list of environment variables that can be used to configure the OpenTelemetry SDK can be found here:
OTLP is the OpenTelemetry Protocol, used to send telemetry data from your application to the OpenTelemetry Collector, which forwards it to backends like Tempo, Loki, and Mimir.
Destinations ¶
If you need to override where telemetry data is stored, you can do so with the following configuration in your nais.yaml:
spec:
observability:
autoInstrumentation:
destinations:
- id: grafana-lgtmThe following destinations are available:
-
grafana-lgtm
Default destination for each environment can be found in the environments overview.