Instrument your app for tracing

The four environment variables

Every OpenTelemetry SDK reads these. Set them once for your service, then pick your language below. Epok's ingest speaks OTLP over HTTP — use http/protobuf or http/json, not gRPC (there is no :4317endpoint; a default gRPC exporter will silently fail).

export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.getepok.dev"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"   # or http/json — NOT grpc (no :4317)
export OTEL_EXPORTER_OTLP_HEADERS="x-api-key=YOUR_API_KEY"
export OTEL_SERVICE_NAME="checkout-service"          # the name your service shows up under

Node.js

Zero code change — the auto-instrumentation package hooks Express, Fastify, HTTP, gRPC, Postgres, Redis, and more.

npm install @opentelemetry/api @opentelemetry/auto-instrumentations-node

# with the env vars above set:
node --require @opentelemetry/auto-instrumentations-node/register app.js

Python

The opentelemetry-instrument wrapper auto-instruments Flask, Django, FastAPI, requests, SQLAlchemy, and more — no import changes.

pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http
opentelemetry-bootstrap -a install

# with the env vars above set:
opentelemetry-instrument python app.py

Java

One JVM flag. The agent instruments Spring, Servlet, JDBC, Kafka, and 100+ libraries with no code change. The agent defaults to gRPC, so the http/protobuf setting above matters here.

curl -L -o opentelemetry-javaagent.jar \
  https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

# with the env vars above set:
java -javaagent:./opentelemetry-javaagent.jar -jar your-app.jar

Go

Go has no runtime agent — set up the SDK once in main, then wrap your handlers (e.g. otelhttp). The exporter uses OTLP/HTTP.

package main

import (
  "context"
  "go.opentelemetry.io/otel"
  "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
  sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

func main() {
  ctx := context.Background()

  exp, err := otlptracehttp.New(ctx) // reads OTEL_EXPORTER_OTLP_* from the env above
  if err != nil {
    panic(err) // in real code: log and continue unintrumented rather than crash
  }
  tp := sdktrace.NewTracerProvider(sdktrace.WithBatcher(exp))
  otel.SetTracerProvider(tp)
  defer tp.Shutdown(ctx) // flushes buffered spans — without this a short-lived
                         // process exits before the first batch is ever sent

  // then wrap handlers: otelhttp.NewHandler(mux, "server")
}

Verify it's working

1. Send a request through your service.
2. Open Traces in the app — the request appears as a span tree within a few seconds.
3. Open Services — your service shows req/s, error %, and p95 automatically (RED is derived from the spans).

Seeing no data? The usual causes: a gRPC exporter pointed at an HTTP endpoint (set OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf); a missing or wrong x-api-key; or no OTEL_SERVICE_NAME (spans arrive but group under unknown_service). For the per-endpoint table, make sure your framework sets http.route on server spans (most do).

Traces visible but the service page empty? If you send spans directly, the page populates from spans — check OTEL_SERVICE_NAME. If you run a collector with spanmetrics, the spanmetrics pipeline must export via prometheusremotewrite to /api/v1/write — sent over the OTLP metrics endpoint it produces the wrong series names and fails silently (see the collector section below).

Once spans flow, trace detectors switch on automatically — new failing operations, latency breaks, throughput collapse — and latency SLOs become available. Nothing to configure.

The 401 that looks like a network problem

An OTLP exporter has no good way to tell you it was rejected. It logs a failed export, retries with backoff, and drops the batch. A wrong key and a right key under the wrong header name look identical from the app side — both are silence — and the second is far more common, because OTEL_EXPORTER_OTLP_HEADERS is free-form text that nothing validates.

Epok's ingest plane reads the key from exactly three carriers: X-API-Key: <key>, Authorization: Bearer <key>, and Authorization: Basic <base64> with the key as the username (any password). Authorization: Token <key> is not accepted on the ingest plane — even though the app plane at app.getepok.dev does accept it, so the exact key that works against a search call will 401 against traces. In OTEL_EXPORTER_OTLP_HEADERS, x-api-key=... is right; api-key=, apikey= and authorization=Token ... all send a header Epok never reads, so the request arrives carrying no credential at all.

Unlike the PaaS drain endpoints, /v1/traces takes the key from headers only — there is no ?api_key= fallback on this endpoint. And because authentication runs before the body is parsed, an empty span batch is enough to test the credential:

curl -i -X POST https://ingest.getepok.dev/v1/traces \
  -H 'Content-Type: application/json' \
  -H 'x-api-key: YOUR_API_KEY' \
  -d '{"resourceSpans":[]}'

# 200 {"partialSuccess":{"rejectedSpans":0}}   key + header are both fine
# 401 {"error":"unauthorized","detail":...}    no key found in ANY accepted header
# 403 {"error":"forbidden","detail":...}       a key WAS found, and rejected
# 429 {"error":"rate_limited",...}             key is FINE — quota or rate limit

401 and 403 are different problems. 401 means no credential was found in any accepted carrier — nearly always the wrong header name. 403 means a key was read and then rejected: rotated or invalid, missing the ingest scope, or the tenant disabled. Read the detail field; it names which. 429 is not an auth failure at all — an expired trial or a blown daily volume cap — and it carries a retry_after, which an exporter will happily back off into forever without ever telling you why. The full carrier matrix is in Authentication and the plan gates are in Limits & error codes.

Optional: the collector accuracy upgrade

You don't need a collector — Epok computes RED from your spans automatically. Run the OpenTelemetry Collector when you operate at a volume where you tail-sample: its spanmetrics connector computes request / error / duration over the full, pre-sampling span stream, so your service-page numbers stay statistically exact while you store only the traces worth keeping (errors, slow requests, a baseline).

The one rule: export the spanmetrics pipeline with the prometheusremotewrite exporter to /api/v1/write — not the OTLP metrics endpoint. Over OTLP the series lose their Prometheus suffixes (_total, _milliseconds_bucket), which are the exact names the service page reads — it fails silently. Spans themselves go via OTLP as usual.

receivers:
  otlp:
    protocols:
      grpc: { endpoint: 0.0.0.0:4317 }   # your apps talk to the COLLECTOR here
      http: { endpoint: 0.0.0.0:4318 }

exporters:
  otlphttp/epok:                      # spans -> trace explorer + stored traces
    endpoint: https://ingest.getepok.dev
    headers: { x-api-key: YOUR_KEY }
  prometheusremotewrite/epok:         # spanmetrics -> service page / SLOs
    endpoint: https://ingest.getepok.dev/api/v1/write
    headers: { X-API-Key: YOUR_KEY }

connectors:
  spanmetrics:
    dimensions: [{ name: http.route }]   # powers the per-endpoint table

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlphttp/epok, spanmetrics]
    metrics/spanmetrics:
      receivers: [spanmetrics]
      exporters: [prometheusremotewrite/epok]   # MUST be remote_write

The full recipe — tail-based sampling, cardinality guards, memory limits — ships with the product; the in-app connect flow (Connect → Traces) links it. A general collector for logs and node metrics is covered in Kubernetes setup.

Your first look: the service page

With logs, the first thing you do is run a search. With traces it isn't — the payoff is the service page, because Epok derives RED from your raw spans server-side. So the honest first destination is Services in the app, then the service named by the OTEL_SERVICE_NAME you set above (checkout-service in these examples).

What should be populated: request rate, error rate, p50 / p95 / p99 latency, and Apdex on the summary strip; a latency distribution; and a per-endpoint table keyed by http.route. Top failing operations fill in as soon as there are error spans to fill them with.

How long to wait. These are two different clocks and conflating them is the usual cause of a false alarm. An individual span shows up in the Traces explorer within a few seconds. The service page is a rate view — at the short time ranges it aggregates spans into 5-minute buckets — so give it a few minutes of real traffic before the numbers mean anything. A blank service page 30 seconds after your first request is expected, not broken.

Two things to check if it stays blank past that. A service whose name starts with unknown_service means OTEL_SERVICE_NAME never reached the SDK — the SDK filled in its own placeholder, so the Java agent shows unknown_service:java and Node unknown_service:node. A service named exactly unknown is a different fault: the spans arrived with no service.nameresource attribute at all, which is the hand-rolled-OTLP case rather than the missing-env-var one. And a healthy summary strip above an empty endpoint table means your framework isn't setting http.route on server spans.