Heroku, PaaS drains & GELF

Heroku

Heroku forwards dyno and router output to any HTTPS endpoint you register as a drain. Point it at /api/v1/drain/herokuand Epok de-frames Heroku's octet-counted wire format and parses the syslog messages inside it. The syslog app-name field becomes service (falling back to the hostname field if Heroku sends no app-name), the syslog severity becomes level, and the hostname is kept alongside as hostname. Which literal values land in serviceis Heroku's choice, not ours — look before you build alerts on it.

The key rides in the URL here, and that is correct. Heroku drains have no field for a custom header — the platform will not send an Authorization or X-API-Key header no matter how you configure them. This is one of the few places where ?api_key= is the intended mechanism rather than a fallback. Every other source on this page should prefer a header; see Authentication for the full table of accepted carriers.

Be honest with yourself about the consequence: a key in a URL lands in proxy logs, shell history, and the output of heroku drains. Use a key that is scoped to ingest only and that you are willing to revoke — not your general-purpose key. You rotate it by removing the drain and adding a new one.

heroku drains:add \
  'https://ingest.getepok.dev/api/v1/drain/heroku?api_key=YOUR_API_KEY' \
  -a your-app

Verify it's working

1. Confirm the drain is registered, then produce a line:
   bashCopy

   heroku drains -a your-app
   # then generate a line to drain — any request to your app produces router
   # output, which is the free way to do it:
   curl -sS https://your-app.herokuapp.com/ >/dev/null
   
   # or emit one from a dyno (note: this starts a one-off dyno, which bills
   # by the second for as long as it runs):
   heroku run echo "hello from epok" -a your-app

2. Open Explore in the app and search ingest_source:heroku. Allow 30–60 seconds — Heroku batches before it POSTs; our own path adds seconds, not minutes.
3. Filter on ingest_source, not on service, for this first check — it is the one field we stamp ourselves, so it proves the route end to end no matter what Heroku puts in the syslog header. Once a line is in front of you, read its service and hostname values and use those from then on.

Nothing arrives, and Heroku reports delivery errors? Read the status code, because the two we return mean different things. 403 means we read a key and rejected it — wrong, rotated, revoked, or missing the ingest scope. 401 means we found no credential at all — the registered drain URL has no api_key on it. Run heroku drains and read the URL Heroku actually stored: that is the whole diagnosis, and it is usually a query string that got dropped on the way in. Quote the URL in single quotes as shown above so your shell hands it over intact. Triage steps are in Authentication.

Heroku reports success but you see nothing?Check the time window before you check anything else. Heroku stamps each line with its own syslog timestamp, and Epok stores a line at the time it claims — a backdated line is written at that historical point, not at the moment it arrived. If a replayed or backlogged batch lands, it will not appear in a “last 15 minutes” view even though ingest accepted it. Widen the range to 24 hours and search again. See Quickstart for the same check in isolation.

Render, Netlify, Fly — the generic drain

/api/v1/drain is the fallback for any platform that can POST its logs to a URL you choose. It has no platform-specific parsing: it reads log objects and stores them, which is why it works for services we have never heard of. Authenticate with a header — there is no reason to put the key in the URL here.

Before you wire this up, check whether your platform has a dedicated guide. Vercel has its own endpoint because it needs a verification handshake, and Railway has no built-in HTTP drain setting at all — the working Railway paths are app-side, so the generic drain will not help you there unless you are the one calling it. Reach for this page for Render, Netlify, Fly, and anything else with a “forward logs to an HTTPS endpoint” box.

curl -X POST 'https://ingest.getepok.dev/api/v1/drain' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '[{"message":"hello from epok","level":"info","service":"my-app"}]'

What shape it accepts

A JSON array, a single JSON object, or NDJSON — the endpoint sniffs the body and does not need to be told which. A message or msg field becomes the log message. level is normalized to a canonical lowercase name, so ERROR and a numeric level both land as error. service is taken from service if you send it, otherwise from the first of app, application, component, container_name, job or source that is present, and unknown if none are — send service explicitly and you never have to think about that cascade. Everything else is kept as a searchable field.

# All three bodies are accepted at the same URL.

# 1. JSON array
[{"message":"a","level":"info","service":"my-app"},
 {"message":"b","level":"error","service":"my-app"}]

# 2. A single JSON object
{"message":"a","level":"info","service":"my-app"}

# 3. NDJSON — one JSON object per line
{"message":"a","level":"info","service":"my-app"}
{"message":"b","level":"error","service":"my-app"}

A successful POST returns the count it took, which is the fastest confirmation you will get that the body parsed the way you expected:

{"status":"ok","accepted":1}

Verify it's working

1. Run the curl above and confirm accepted is not 0. A response of {"status":"ok","accepted":0} means the request authenticated but nothing in the body survived parsing — usually an empty message field.
2. Open Explore and search ingest_source:drain. Allow 30–60 seconds.
3. Then trigger a real log line from your platform and confirm it arrives the same way.

401 Unauthorized? Then the carrier is wrong, not the key — 401 means we found no credential in any accepted header. This endpoint accepts Authorization: Bearer, Basic (key as username), X-API-Key, and ?api_key= — it does not parse Authorization: Token, which the app plane does accept. Same key, different plane, different answer, and the request arrives carrying nothing we can read. A 403 is the other story: the carrier was fine and the key itself was rejected — rotated, revoked, or without the ingest scope. The full matrix is in Authentication.

Got accepted: 1 but see nothing?If your platform sets a timestamp field, Epok honors it. A line stamped with an older time is stored at that older time and will be invisible in a recent window — this is the single most common “ingest worked but the UI is empty” case. Widen the search range to 24 hours. If it appears there, your drain is fine and your clock or your replay window is the story. See Quickstart.

Docker & GELF

Read this before you configure anything. Docker's built-in gelf log driver speaks UDP and TCP only. It cannot POST to an HTTPS endpoint. If you set --log-driver=gelf --log-opt gelf-address=https://..., Docker will reject the address or fail silently at the transport layer, and you will spend an afternoon debugging a path that can never work. This is a property of the Docker driver, not of Epok — no endpoint, ours or anyone's, fixes it.

If you want container logs, use Fluent Bit or Vector instead. Either one reads container stdout and ships it over HTTPS, which is the transport that actually exists. The Kubernetes guide covers the Fluent Bit DaemonSet setup, and the same agent config works on a plain Docker host. The in-app connect gallery carries both as first-class sources.

What /gelf is genuinely for

Applications that emit GELF themselves, over HTTP. That means Log4j2 and logback GELF HTTP appenders, NXLog, and Vector's http sink — the emitter is your logging library or your agent, not the Docker daemon. Both /gelf and /api/v1/gelfserve the same handler; use whichever your tool's config makes easier.

The mapping: short_message becomes the log message, host becomes service, and the numeric level is a syslog severity (6 = info, 3 = error), which Epok converts to a level string. Custom _-prefixed fields are kept with the prefix stripped, except where that would overwrite a field GELF already populated. full_message — where the Java appenders put the stack trace — is kept as its own field whenever it says more than short_message does, so the part you actually need survives. A single object, an array, or NDJSON all parse.

curl -X POST 'https://ingest.getepok.dev/gelf' \
  -H 'X-API-Key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"version":"1.1","host":"web-01","short_message":"hello from epok","level":6}'

Vector's http sink, which is the most common non-Java emitter:

# vector.toml — Vector's http sink emitting GELF-shaped JSON.
[sinks.epok_gelf]
type   = "http"
inputs = ["my_source"]
uri    = "https://ingest.getepok.dev/api/v1/gelf"
encoding.codec = "gelf"
# Turn the healthcheck off: /api/v1/gelf answers POST and nothing else, so
# any probe against it fails and an unhealthy sink never starts sending.
# Same failure class as the Elasticsearch-sink healthcheck trap.
healthcheck.enabled = false

[sinks.epok_gelf.request.headers]
X-API-Key = "YOUR_API_KEY"

Note the header. ?api_key= works on this endpoint too — some GELF HTTP shippers genuinely cannot set headers — but if your emitter can send a header, send a header.

Verify it's working

1. Run the curl above and confirm the accepted count.
2. Open Explore and search ingest_source:gelf. Allow 30–60 seconds.
3. The entry should show service: web-01 — taken from the GELF host field — and level info from level: 6.

401, or 403? 401 means no credential was found at all — the usual cause is a header name we do not read. 403 means a key was read and rejected, and that is the one that catches people here: a header is resolved before ?api_key=, so a Vector or NXLog config carrying a stale X-API-Key alongside a good ?api_key= fails on the stale header and the URL key is never read. It reads like a bad URL key; it is a precedence problem. Remove one of them. Accepted carriers and a step-by-step triage are in Authentication.

Accepted, but nothing in Explore? GELF carries an optional timestamp field as epoch seconds, and Epok honors it. A logging appender with a wrong clock, or one replaying a buffered backlog after a restart, writes those lines at their stated historical time — they are stored, they are searchable, and they are not in your last-15-minutes view. Widen to 24 hours before concluding anything is broken. Omit timestamp entirely and arrival time is used. See Quickstart.

Your first query

Drained logs are ordinary logs — nothing about the delivery path limits what you can ask. Every line from this page carries an ingest_source field (heroku, drain, or gelf), so you can confirm the route end to end. Run this against the logs you just drained, or use Explore in the app, which does the same thing with a text box.

curl -X POST https://app.getepok.dev/api/v1/tenants/YOUR_TENANT_ID/search \
  -H 'X-API-Key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "ingest_source:heroku",
  "start": "-1h",
  "end": "now",
  "limit": 100
}'

You should get back a logs array — each entry with _time, _msg, level, and service, plus whatever fields your platform sent — along with a count and an elapsed_ms. An empty array with count: 0 and a fast elapsed_ms means the query ran fine and matched nothing: go back to the timestamp branch above before you doubt the drain.

Swap the query for something about your own service — level:error, or your app name — and see Search syntax for filters, stats, and grouping.