Skip to the content

DocumentationOpenTelemetry and events

Connect your systems

OpenTelemetry and events

Sending errors, logs and failed traces

A service sends OpsCommand what goes wrong in it — errors, exceptions, failed calls — and OpsCommand shows them on the service, correlates them with checks and alerts, and gathers them into an incident or a developer handover (PLAN 4.5). OpsCommand is not a log store or a tracing store: it keeps what went wrong, redacted, for as long as the organisation's retention says (13 months for checks, 3 for events by default, 4.12).

The key

Each service has its own ingest key: on the service's page, Ingest keys → create. The key (oik_…) is shown once, with the addresses below; OpsCommand keeps only its SHA-256. Revoke it there at any time. Send it as Authorization: Bearer oik_… or X-OpsCommand-Key: oik_….

OpenTelemetry (OTLP/HTTP, JSON)

Point an OpenTelemetry exporter at OpsCommand's API:

SignalAddress
LogsPOST <API>/v1/otlp/v1/logs
TracesPOST <API>/v1/otlp/v1/traces

With the OpenTelemetry SDK for Node.js, for example:

import { OTLPLogExporter } from '@opentelemetry/exporter-logs-otlp-http';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

const headers = { Authorization: `Bearer ${process.env.OPSCOMMAND_INGEST_KEY}` };
const logs = new OTLPLogExporter({ url: 'https://api.opscommand.dev/v1/otlp/v1/logs', headers });
const traces = new OTLPTraceExporter({ url: 'https://api.opscommand.dev/v1/otlp/v1/traces', headers });

Or with the OpenTelemetry Collector, an otlphttp exporter with encoding: json and the header.

  • Logs: each record becomes an event; its severity comes from the severity number; a record with exception.* attributes is an error.
  • Traces: only what went wrong is kept — spans with an error status and exception events. The trace id becomes the event's correlation id, so a failed request is found again across services (a developer handover collects them, 5.2).
  • Only the JSON encoding: protobuf is answered 415.

CloudEvents

POST <API>/v1/ingest/events takes CloudEvents 1.0 in structured mode (application/cloudevents+json), one event or a batch (application/cloudevents-batch+json). Two extensions are read: severity (debug, info, warning, error, critical) and correlationid (or the trace id from traceparent).

Limits and safety

  • At most 1000 events per call; only events from the last day to five minutes ahead.
  • A daily quota per organisation (the plan's; 4.12): over it, the call is refused whole with 429 and Retry-After, and the refusal is counted.
  • Every message and attribute is redacted at the door — secrets, tokens, payment card numbers, identity numbers, contact details — before anything is stored (@opscommand/redaction); attributes are kept within 12 kB.
  • Deployments are reported the same way: POST <API>/v1/ingest/deployments (PLAN 5.6, DEPLOYMENTS.md).

Proven by apps/api/test/ingest.test.ts, with the real OpenTelemetry SDK exporting to the API.