The Sentrinel plugin

@sentrinel/plugin is the agent that runs inside your application. It measures every request, buffers in memory, and flushes batches to the Sentrinel API. It adds no blocking work to the request path: metrics are aggregated in-process and sent on a timer.

Adapters ship for Elysia (first-class), Express, Next.js, and Flutter. Anything that speaks OpenTelemetry can skip the plugin entirely and export OTLP/HTTP straight to /v1/traces.

Not a JavaScript backend? Django and FastAPI have their own packages — DJANGO.md and FASTAPI.md, one middleware each and no runtime dependencies. Flutter, Kotlin and Swift are in MOBILE.md, and anything that speaks OpenTelemetry can export straight to /v1/traces without an SDK at all.

Install

bun add "@sentrinel/plugin@github:Zaga-ltd/sentinel_packages"

Minimum viable setup

import { Elysia } from "elysia";
import { sentrinelPlugin } from "@sentrinel/plugin";

new Elysia()
  .use(sentrinelPlugin({
    serverUrl: "https://api.sentrinel.example.com",
    appName: "checkout-api",
    env: "prod",
    apiKey: process.env.SENTRINEL_API_KEY,
  }))
  .get("/orders", () => listOrders())
  .listen(3000);

That alone gives you request counts, error rates, p50/p95/p99 latency, Apdex, endpoint discovery, host CPU/memory, and deploy markers.

Want everything enabled? See FULL_CONFIG.md for the complete reference covering every option, every signal, and every dashboard page — backend and Flutter side by side.

The API key must match the app name and environment it was issued for.

And it must be the right kind. Every key is bound to one integration, chosen when it is issued (API Keys → Generate → What is this key for?):

The point is the leak. A key inside a phone's bundle, on a database host, or in an agent's config file each leak differently, and a key that could do everything would turn any one of those into all of them. A key used on the wrong surface gets a 403 that names the kind it is and the kind that surface wants. Keys issued before kinds existed are server keys and keep working unchanged. This is the tenant boundary — a leaked key for staging cannot write into prod. A mismatch is a 403, and the plugin says so once in your logs:

[sentrinel] telemetry rejected (403). Check apiKey and env — the key must be
issued for this environment.

It reports failures rather than swallowing them. Silence means it is working.

One app per project, one module per part

An app in Sentrinel is the whole project — the backend, the phone app, the web client, the database, an AI agent. Each part reports into the same app as a module of it, so a request that starts on the phone can be followed into the backend and the query it caused, as one trace.

Give each part its own key (a server key for the backend, a mobile key for the phone), all issued for the one app. The key's name is the module unless you set module:

// backend
sentrinelPlugin({ appName: "fieldops", apiKey: process.env.SENTRINEL_KEY, module: "backend" })

// phone
SentrinelOptions(appName: 'fieldops', apiKey: key, module: 'mobile')

The API Keys page lists the parts of the selected app and when each last reported. Filter requests, logs, errors and traces to one part with ?module=backend.

Options

version is worth setting

sentrinelPlugin({ /* … */ version: process.env.GIT_SHA })

Every chart then gets deploy markers, so "the p95 doubled" becomes "the p95 doubled at 14:20, which is when a3f9c21 went out."

Request logging

Aggregated metrics answer how much and how fast. Request logs answer what exactly happened on this one call.

requestLogging: {
  enabled: true,
  sampleRate: 0.1,              // keep 10% of successful requests
  slowRequestThresholdMs: 2000, // …but always keep slow ones
  logRequestHeaders: true,
  logRequestBody: true,
  logResponseBody: true,
  maxBodySize: 64 * 1024,
  maskHeaders: [/^authorization$/i, /^cookie$/i],
  maskBodyFields: [/^password$/i, /^cardNumber$/i, /^cvv$/i],
  maskQueryParams: [/^token$/i],
}

Three properties are worth understanding, because they are what make sampling safe to turn on:

1. Errors and slow requests are never sampled out. sampleRate applies only to successful, fast requests — the ones you were never going to read.
2. Metrics stay exact. Counters are computed in-process before sampling, so a 1% sample rate still yields correct request counts and error rates. Only the individual log rows are thinned.
3. Every row records the rate it was captured at, so counts derived from raw logs can be extrapolated honestly rather than silently under-reporting.

Where the call came from

Every request row also records who called and which host answered, with no configuration:

This is resolved inside your process, because it has to be. By the time a payload reaches Sentrinel the only address left is your own server's; the caller's address exists only where the request landed.

Two consequences worth knowing before you go looking for the data:

• Country needs an edge in front of you. It is read from a header set by whatever terminated TLS — Cloudflare, Vercel, Fastly, CloudFront. On a bare VPS with no proxy there is no country, and the field stays empty. That is deliberate: shipping a GeoIP database would be a licensing question and tens of megabytes of memory in every one of your processes, and a wrong country is worse than an honest blank.
• Client IP needs the proxy to forward it. Behind a load balancer that does not set any of the headers above, the address recorded is the balancer's.

Both are filterable on the request-logs API:

GET /api/requests?country=TZ
GET /api/requests?host=api.example.com

Masking

Masking runs in your process, before anything leaves it. A masked field never reaches the network, let alone Sentrinel's disk. Patterns match field names, not values, and maskBodyFields descends into nested objects.

Start from this and add your own:

maskHeaders: [/^authorization$/i, /^cookie$/i, /^x-api-key$/i],
maskBodyFields: [/password/i, /secret/i, /token/i, /^cardNumber$/i, /^cvv$/i, /^ssn$/i],

Custom metrics

Tracing tells you how long something took. This tells you what it cost.

import { count, gauge, histogram } from "@sentrinel/plugin";

count("llm.tokens", usage.input_tokens, { model: "deepseek-chat", direction: "input" });
count("revenue.usd", order.total, { plan: order.plan });
gauge("queue.depth", await queue.size());
histogram("llm.cost_per_call_usd", costOf(usage), { model });

They appear on the Metrics page, where you pick a series, choose how to aggregate it, and split it by any label you recorded.

Which one to reach for

count() defaults to 1, so count("signups") is a valid increment.

Safe to call in a loop

An increment mutates a map, not a network buffer. The SDK folds every call into one row per (name, labels) per flush window, so a counter called ten thousand times a second costs one row every thirty seconds, not ten thousand rows a second. Recording a metric per token is a normal thing to do.

Every aggregate travels on every row — sum, last, count, min, max, p50, p95, p99 — regardless of which function you called. That means you can ask a counter "what was the largest single increment" later without re-instrumenting for it.

Labels are dimensions, not identifiers

count("llm.tokens", n, { model, tier });        // good — a handful of values
count("llm.tokens", n, { userId: user.id });    // bad — a series per user

A label per user is a cardinality bomb. The SDK caps a process at 2,000 live series and drops new ones past that rather than growing without bound — a monitoring SDK must not become the outage it was installed to report. It also caps 12 label keys and 128 characters per value.

Label order does not matter: {a, b} and {b, a} are the same series, so two call sites that disagree about ordering still chart as one line.

NaN and Infinity are refused rather than summed — one poisoned value would make every chart drawn from that sum useless.

Requirements

Custom metrics need TELEMETRY_STORE=clickhouse. Ingest answers 501 on the Postgres store rather than accepting and dropping them, so an integration that looks connected is never quietly empty. Retention is its own knob, RETENTION_CUSTOM_METRICS_DAYS, defaulting to 400 days — pre-aggregated rows are small, and a spend series is only useful with months behind it.

Structured logging

Conventional logging optimises for writing — a readable sentence — when what you actually do with logs is query them. console.log(\user ${id} failed checkout`)` is easy to write and impossible to ask questions of: you cannot filter by user, group by failure reason, or tie it to the request that produced it.

import { getLogger, withContext, addRequestContext } from "@sentrinel/plugin";

const log = getLogger(["payment", "checkout"]);

log.info("Checkout completed", { orderId, amount, provider: "stripe" });
log.warn("Checkout rejected", { reason: "insufficient_funds", amount });

Enable it once, in the plugin:

sentrinelPlugin({
  /* … */
  logging: { minLevel: "info", echo: false },
})

Four things make this different from console.log:

1. The message is stable; the values are fields. "Checkout rejected" is one message you can group and count, and reason is a field you can filter. Interpolating the reason into the string would make every occurrence unique and therefore ungroupable. Placeholders ("Fetched {count} orders") are stored verbatim for the same reason — the values are already in the attributes.

2. Every record is correlated automatically. Request id, trace id and span id are attached from the ambient context, so "everything logged during this request" is a lookup, not a grep across timestamps. In the dashboard, a log links to its request and a request lists its logs.

3. Context is inherited, not repeated.

withContext({ userId: user.id, tenantId: org.id, tier: user.tier }, async () => {
  await processOrder();   // every log inside carries those three fields
});

The function three levels down never heard of a user, and its logs still carry one. This is what makes high-cardinality filtering practical — you attach the context once, where you have it.

4. Categories are hierarchical. getLogger(["api", "checkout"]) logs under api.checkout; filtering by api selects the whole subsystem. log.getChild("db") nests further, and log.with({ jobId }) binds attributes to every record from that logger.

Canonical wide events

Scattered log lines make you reassemble a request from fragments. The opposite approach — one rich event per request — answers most questions on its own:

addRequestContext({
  tier: user.tier,
  cartTotal,
  experiment: "new-pricing",
  outcome: "success",
});

Those fields land on the request row itself, next to method, path, status, latency, consumer and trace id. "Slow checkouts for enterprise customers in the new pricing experiment" becomes one query rather than a correlation exercise. Call it as many times as you like during a request; the fields merge.

Everything is linked

One request produces a set of records that name each other:

• the request row carries the trace id and your addRequestContext() fields
• every log record carries the request id, trace id and the span it was written inside
• the trace carries the request id, so the waterfall links back
• an error occurrence carries both, which is what lets an issue open the exact request that produced it

A trace is emitted for every request, not only ones that opened child spans — a request with no traceSpan() calls still has a real trace, the HTTP span itself. Skipping it would leave logs and errors pointing at a trace id that resolves to nothing.

Levels, cost and limits

• minLevel drops records before they are buffered — nothing is sent.
• Attributes are bounded at ingest: 64 fields, 2,000 characters each. Objects and arrays are stored as JSON text. The value is in many small queryable fields, not embedded documents.
• Logging is buffered and flushed on the normal interval. It never blocks a request, and if Sentrinel is unreachable the batch is dropped with one warning rather than backing up your process.
• echo: true also prints each record to stdout. Off by default — your app owns its console output.

Logging outside a request

Background jobs, schedulers and startup all work; those records simply have no request to correlate to.

const log = getLogger("worker");
log.info("Nightly reconciliation started", { batchSize: 5000 });

If several instrumented apps share one process, a request's logs go to the app handling it; logs outside any request go to whichever app initialised first.

Console capture (application logs)

Correlates your console.* output with the request that produced it, so a request-log row can show you the exact lines that ran inside it.

logCapture: {
  enabled: true,
  minLevel: "info",       // "debug" captures everything
  maxPerRequest: 50,      // a hot loop can't flood the buffer
  maxMessageLength: 2000,
}

Two consequences worth knowing:

• A request that logged an error is never sampled out, even if it returned 200 and sampleRate would otherwise have dropped it. The requests where your code complained are the ones you want to keep.
• The Request logs page gains a log-level filter: "show me requests that emitted an error-level line" finds problems that never surfaced as a 5xx.

Distributed tracing

Spans nest automatically. Anything you wrap runs as a child of whatever is already on the stack, and the root span is the HTTP request itself.

import { traceSpan, traced, sentrinelFetch } from "@sentrinel/plugin";

// Wrap a block
const risk = await traceSpan("fraud.check", { kind: "INTERNAL" }, async (span) => {
  span.setAttribute("amount", amount);
  return scoreTransaction(amount);
});

// Or wrap a function once, at definition
const chargeCard = traced(async function chargeCard(amount: number) {
  return stripe.charges.create({ amount });
});

// Cross-service calls propagate the trace (W3C traceparent)
const res = await sentrinelFetch("https://payments.internal/v1/charges", {
  method: "POST",
  body: JSON.stringify({ amount }),
});

traceSpan accepts either shape — traceSpan(name, fn) or traceSpan(name, { kind }, fn).

kind should be one of SERVER, CLIENT, DB, INTERNAL, PRODUCER, CONSUMER; the waterfall colours by it.

This produces a real tree in the dashboard:

POST /api/v1/payment/checkout   SERVER    271ms   ← root
  checkFraudRisk                INTERNAL    0ms
  http.client POST /v1/charges  CLIENT    245ms   ← the actual cost
  payDb.processLedgerEntry      INTERNAL   26ms

Click any span for its duration, self time, status, and attributes. Self time is usually the number that matters: a 271 ms span that spent 245 ms waiting on a child isn't the problem — its child is.

Incoming traceparent headers are honoured, so a trace started upstream continues through your service instead of being split into two.

Other frameworks

Express

import { sentrinelExpressMiddleware } from "@sentrinel/plugin";

app.use(sentrinelExpressMiddleware({
  serverUrl: process.env.SENTRINEL_URL!,
  appName: "legacy-api",
  env: "prod",
  apiKey: process.env.SENTRINEL_API_KEY,
  consumerIdentifier: "x-consumer-id",   // header name
}));

Mount it before your routes so it sees every request.

Next.js

// middleware.ts
import { sentrinelNextMiddleware } from "@sentrinel/plugin";

export const middleware = sentrinelNextMiddleware({
  serverUrl: process.env.SENTRINEL_URL!,
  appName: "storefront",
  env: process.env.VERCEL_ENV ?? "prod",
  apiKey: process.env.SENTRINEL_API_KEY,
});

OpenTelemetry — no plugin at all

Any OTel SDK or Collector can export straight to Sentrinel:

OTEL_EXPORTER_OTLP_ENDPOINT=https://api.sentrinel.example.com
OTEL_EXPORTER_OTLP_PROTOCOL=http/json
OTEL_EXPORTER_OTLP_HEADERS=x-api-key=snt_live_…

GenAI spans (gen_ai.request.model plus gen_ai.usage.*_tokens) are priced automatically and appear on the LLM cost page.

Flutter / mobile

The full SDK ships in the same public repo, at sentrinel_flutter/ — the tree there is flat, so a git: dependency uses that path and not a packages/ prefix. See Flutter and Dart below.

Raw header helper

createFlutterHeaderMap() builds the headers a mobile client should send so its calls are attributed to the right consumer and joined to the server-side trace.

Cron and heartbeat monitoring

Not a plugin feature — a URL. Create a monitor in the dashboard and add its check-in URL to the job:

# at the end of the job
curl -fsS https://api.sentrinel.example.com/api/checkin/<token>

# long jobs: report start, duration, or explicit failure
curl -fsS "https://api.sentrinel.example.com/api/checkin/<token>?state=start"
curl -fsS "https://api.sentrinel.example.com/api/checkin/<token>?duration=1430"
curl -fsS "https://api.sentrinel.example.com/api/checkin/<token>?state=fail"

The token is the credential, so these endpoints need no API key — a cron line stays a one-liner. A sweeper marks the monitor missed when a ping doesn't arrive within its interval plus grace, and notifies the same channels alerts use.

Overhead and failure behaviour

• Request path: an in-memory counter update. No I/O, no awaiting Sentrinel.
• Network: one batched POST per flushInterval per data type.
• Memory: bounded by maxPerRequest for logs; metric buffers are per endpoint, not per request, so they don't grow with traffic.
• When Sentrinel is down: the batch is dropped and your app carries on. The plugin never retries in a way that could back up your process, and it warns once rather than every flush.
• When you're over quota: the API answers 429 and the plugin drops the batch; you'll see one warning in your logs.

Monitoring should never be the reason your service is down.

Flutter and Dart

A real Dart package — not a header helper. It records every request your app makes, the errors and logs around them, and sends traceparent so the backend plugin continues the same trace: a tap and the server work it caused land on one timeline.

This is the pure-Dart core — what a Dart CLI or server depends on:

dependencies:
  sentrinel:
    git:
      url: https://github.com/Zaga-ltd/sentinel_packages.git
      path: sentrinel_flutter

A Flutter app wants sentrinel_flutter instead, which adds what needs the framework — crash handlers that survive a restart, app start and frozen frames, navigation breadcrumbs — and pulls this core in with it. The dependency key and the directory differ, so copy it from MOBILE.md rather than adapting the block above.

import 'package:sentrinel/sentrinel.dart';

void main() {
  Sentrinel.init(
    serverUrl: 'https://api.sentrinel.dev',
    appName: 'mobile-app',
    apiKey: const String.fromEnvironment('SENTRINEL_API_KEY'),
    consumerIdentifier: 'ios_app',
  );
  runApp(const MyApp());
}

// every call through this client is recorded
final client = Sentrinel.httpClient();

Sentrinel.setContext({...}) attaches fields to every later record, Sentrinel.info/warn/error write structured logs, and Sentrinel.captureError takes a caught exception or your runZonedGuarded handler.

It is built not to hurt the host app: nothing blocks on the network, buffers are bounded (500 requests / 500 logs / 200 errors, oldest dropped, with a droppedRecords count), a failed batch is dropped rather than retried forever, and every call before init() is a no-op. Full detail in the package README.

Browsers

@sentrinel/plugin/browser covers the other side of the same trace: uncaught errors, failed and slow fetch calls, breadcrumbs and one session per page load, with traceparent on same-origin requests so a click and the handler it triggered share a waterfall.

It ships no API key — everything in a bundle is public — so the page posts to your own origin and @sentrinel/plugin/tunnel forwards with the key.

// client
initSentrinelBrowser({ endpoint: '/api/sentrinel', release: '2026.8.2' });

// server
export const POST = createSentrinelTunnel({
  serverUrl: 'https://api.sentrinel.dev',
  appName: 'admin', env: 'prod',
  apiKey: process.env.SENTRINEL_API_KEY!,
});

Full detail in BROWSER.md.

See also

• FULL_CONFIG.md — every option, every signal, every dashboard page, backend and Flutter
• BROWSER.md — web apps
• MOBILE.md — Flutter/Dart deep dive
• GUIDE.md — what each dashboard page shows and needs