Full Configuration โ Collect Everything
This document is the single reference for enabling every telemetry signal Sentrinel can collect. It covers the backend plugin (Elysia, Express, Next.js, Bun) and the Flutter SDK side by side, with every option explained and every data type mapped to the dashboard page that displays it.
If you want "just works, show me everything" โ copy the first config block for each side and you are done. The rest of the document explains what each option does and why it exists.
Table of contents
- Backend config: Elysia
- Backend config: Express
- Backend config: Next.js
- Backend config: Bun native
- Flutter config
- Browser config
- OpenTelemetry without a plugin
- Backend environment variables
- What each signal captures
- Data masking
- Sampling safety
- Distributed tracing
- Structured logging deep dive
- What you see in the dashboard
- Troubleshooting
Backend config: Elysia
This enables every signal the backend plugin supports: request metrics, request logs with payloads, console log capture, structured logging, error tracking, distributed tracing, resource monitoring, consumer tracking, and deploy markers.
import { Elysia } from "elysia";
import { sentrinelPlugin } from "@sentrinel/plugin";
const app = new Elysia()
.use(sentrinelPlugin({
// โโ Core โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
serverUrl: "http://localhost:3001", // your Sentrinel API URL
appName: "my-api", // shown in dashboard
env: "prod", // "dev" | "staging" | "prod"
apiKey: process.env.SENTRINEL_API_KEY, // per-app ingest key
version: process.env.GIT_SHA, // deploy markers on charts
debug: false, // verbose plugin logging
// โโ Flush control โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
flushInterval: 30000, // ms between batch sends
// โโ Consumer tracking โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
consumerIdentifier: (ctx) =>
ctx.request.headers.get("x-consumer-id") ?? "unknown",
// โโ Path exclusion โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
excludePaths: ["/health", "/metrics", /^\/internal\//],
// โโ Request logging (raw request/response rows) โโโโโโโโโโโ
requestLogging: {
enabled: true, // turn on request log rows
sampleRate: 1.0, // 100% of fast successes
slowRequestThresholdMs: 500, // always capture slow ones
logRequestHeaders: true, // include request headers
logRequestBody: true, // include request body
logResponseBody: true, // include response body
maxBodySize: 65536, // 64KB max body capture
maskHeaders: [/^authorization$/i, /^cookie$/i, /^x-api-key$/i],
maskQueryParams: ["token", "secret", "key"],
maskBodyFields: [/^password$/i, /^token$/i, /^credit_card$/i, /^cvv$/i],
},
// โโ Console log capture โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
logCapture: {
enabled: true, // capture console.* output
minLevel: "debug", // "debug" = capture everything
maxPerRequest: 100, // cap per request
maxMessageLength: 2000, // truncate long messages
},
// โโ Structured logging โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
logging: {
minLevel: "debug", // drop nothing before buffering
echo: true, // mirror to stdout in dev
},
}))
.get("/", () => "hello")
.listen(3000);
What each option does
| Option | What it controls | Why it matters |
|---|---|---|
serverUrl |
Where telemetry is sent | Must point to the API, not the dashboard |
appName |
Groups all data for this service | Must match the API key's app |
env |
Scopes to an environment | Staging key cannot write to prod |
apiKey |
Authenticates ingestion | Required when SENTRINEL_REQUIRE_INGEST_KEY=true |
version |
Deploy markers | Charts get vertical lines at deploy time |
flushInterval |
How often batches are sent | Lower = more realtime, higher = less network |
consumerIdentifier |
Identifies the API client | Powers the Consumers dashboard page |
excludePaths |
Ignores noisy endpoints | Health checks, metrics scrapes, internal routes |
requestLogging.enabled |
Writes request log rows | Required for Request Logs, Apdex, payloads |
requestLogging.sampleRate |
Fraction of fast successes kept | 0.1 = 10%, errors/slow always kept |
requestLogging.slowRequestThresholdMs |
Always-capture threshold | Requests above this are never sampled out |
requestLogging.logRequestHeaders |
Includes headers in logs | Shows in the Headers tab |
requestLogging.logRequestBody |
Includes request body | Shows in the Payloads tab. Read before your handler, from a clone โ a route that verifies a raw signature still sees its own bytes |
requestLogging.logResponseBody |
Includes response body | Shows in the Payloads tab |
requestLogging.maxBodySize |
Truncates large bodies | Prevents memory bloat |
requestLogging.maskHeaders |
Redacts sensitive headers | Runs in-process before anything leaves |
requestLogging.maskQueryParams |
Redacts sensitive query params | Prevents tokens/keys from leaking |
requestLogging.maskBodyFields |
Redacts sensitive body fields | Recursively masks nested JSON |
logCapture.enabled |
Captures console.* output | Correlates logs to the request that produced them |
logCapture.minLevel |
Minimum level to capture | "debug" = everything, "warn" = warnings+ |
logCapture.maxPerRequest |
Per-request log cap | Prevents hot loops from flooding the buffer |
logCapture.maxMessageLength |
Truncates long messages | Keeps storage bounded |
logging.minLevel |
Drops records before buffering | "debug" = nothing dropped |
logging.echo |
Mirrors to stdout | Useful in dev, off in prod |
When a request shows no payload
An empty Payloads tab on a write request is usually correct. Two cases produce one, and they are worth telling apart before assuming capture is broken:
- The request carried no body.
DELETE /customers/:idand action routes likePOST /workorders/:id/startput everything in the URL. CheckrequestSizeon the row โ0means there was nothing to record. - The body was exactly
{}. An empty object is dropped as noise. Clients that sendJSON.stringify(body ?? {})therefore look identical to clients that send nothing at all.
Anything else with a non-zero requestSize and no recorded body is a bug worth
reporting.
Backend config: Express
import express from "express";
import { sentrinelExpressMiddleware } from "@sentrinel/plugin/express";
const app = express();
app.use(sentrinelExpressMiddleware({
serverUrl: "http://localhost:3001",
appName: "my-express-api",
env: "prod",
apiKey: process.env.SENTRINEL_API_KEY,
version: process.env.GIT_SHA,
consumerIdentifier: "x-consumer-id", // header name (string)
excludePaths: ["/health", "/metrics"],
requestLogging: {
enabled: true,
sampleRate: 1.0,
slowRequestThresholdMs: 500,
logRequestHeaders: true,
logRequestBody: true,
logResponseBody: true,
maxBodySize: 65536,
maskHeaders: [/^authorization$/i, /^cookie$/i],
maskBodyFields: [/^password$/i, /^token$/i],
},
logCapture: {
enabled: true,
minLevel: "debug",
maxPerRequest: 100,
},
logging: {
minLevel: "debug",
echo: false,
},
}));
app.get("/users", (req, res) => {
res.json(getUsers());
});
app.listen(3000);
Important: Mount the middleware before your routes so it sees every request.
Backend config: Next.js
// middleware.ts (App Router)
import { sentrinelNextMiddleware } from "@sentrinel/plugin/next";
const sentrinel = sentrinelNextMiddleware({
serverUrl: process.env.SENTRINEL_URL!,
appName: "my-next-app",
env: process.env.VERCEL_ENV ?? "prod",
apiKey: process.env.SENTRINEL_API_KEY,
version: process.env.VERCEL_GIT_COMMIT_SHA,
requestLogging: {
enabled: true,
sampleRate: 1.0,
slowRequestThresholdMs: 500,
logRequestHeaders: true,
logRequestBody: true,
logResponseBody: true,
maxBodySize: 65536,
maskHeaders: [/^authorization$/i, /^cookie$/i],
maskBodyFields: [/^password$/i, /^token$/i],
},
logCapture: {
enabled: true,
minLevel: "info",
maxPerRequest: 50,
},
logging: {
minLevel: "debug",
echo: false,
},
});
export function middleware(request: Request) {
return sentrinel.middleware(request);
}
export const config = {
matcher: "/api/:path*",
};
For Pages Router API routes, wrap each handler:
import { withSentrinel } from "@sentrinel/plugin/next";
export default withSentrinel({
serverUrl: process.env.SENTRINEL_URL!,
appName: "my-next-app",
env: process.env.VERCEL_ENV ?? "prod",
}, async function handler(req, res) {
res.json({ data: "hello" });
});
Backend config: Bun native
For raw Bun.serve without Elysia:
import { sentrinelBunMiddleware } from "@sentrinel/plugin/bun";
Bun.serve({
port: 3000,
fetch: sentrinelBunMiddleware({
serverUrl: "http://localhost:3001",
appName: "my-bun-app",
env: "prod",
apiKey: process.env.SENTRINEL_API_KEY,
version: process.env.GIT_SHA,
consumerIdentifier: "x-consumer-id",
requestLogging: {
enabled: true,
sampleRate: 1.0,
slowRequestThresholdMs: 500,
logRequestHeaders: true,
logRequestBody: true,
logResponseBody: true,
},
logCapture: {
enabled: true,
minLevel: "debug",
},
logging: {
minLevel: "debug",
echo: false,
},
}, async (req) => {
return new Response("Hello");
}),
});
Flutter config
The Flutter SDK captures HTTP requests, errors, crashes, release health
sessions, performance frames, and structured logs. It sends traceparent
headers so backend traces connect automatically.
Using SentrinelFlutter (recommended)
import 'package:flutter/material.dart';
import 'package:sentrinel_flutter/sentrinel_flutter.dart';
void main() => SentrinelFlutter.run(
options: SentrinelOptions(
serverUrl: 'https://api.sentrinel.dev',
appName: 'mobile-app',
env: 'prod',
apiKey: const String.fromEnvironment('SENTRINEL_API_KEY'),
release: '1.4.2', // critical for release health
flushInterval: Duration(seconds: 30),
),
app: () => runApp(const MyApp()),
);
// In your root widget:
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
// Navigation tracking + breadcrumbs
navigatorObservers: [SentrinelNavigatorObserver()],
home: HomeScreen(),
);
}
}
Key rule: runApp goes inside the app callback. Async errors are only
caught within the zone โ calling runApp outside it silently misses most of
them.
Using the core Dart package (non-Flutter)
import 'package:sentrinel/sentrinel.dart';
void main() {
Sentrinel.init(
serverUrl: 'https://api.sentrinel.dev',
appName: 'my-dart-app',
env: 'prod',
apiKey: const String.fromEnvironment('SENTRINEL_API_KEY'),
release: '1.4.2',
consumerIdentifier: 'ios_app', // or 'android_app', user id, etc.
);
runApp(const MyApp());
}
// HTTP client โ every request is recorded + traceparent is sent
final client = Sentrinel.httpClient();
// Dio adapter
final dio = Dio();
dio.httpClientAdapter = SentrinelHttpClientAdapter();
What gets captured automatically
| Signal | How | What you see |
|---|---|---|
| HTTP requests | Sentrinel.httpClient() |
Method, URL, status, latency, size |
| Crashes | runZonedGuarded + FlutterError.onError |
Stack trace, device info, breadcrumbs |
| Release health | Session per app launch | Crash-free rate per release |
| Performance | Frame timing + app start | Slow/frozen frames, start-to-first-frame |
| Distributed tracing | traceparent header |
Connects Flutter โ backend in one trace |
| Breadcrumbs | HTTP + navigation auto, manual add | Last 25 events before crash/error |
Capturing errors manually
// Uncaught errors (in your zone handler)
runZonedGuarded(() {
// ...
}, (error, stack) {
Sentrinel.captureError(error, stack, path: 'main');
});
// Caught errors
try {
await api.submitPost(post);
} catch (e, stack) {
Sentrinel.captureError(e, stack, path: 'submit_post');
rethrow;
}
Adding context
// After login โ attaches to every later record
Sentrinel.setContext({
'userId': user.id,
'tier': user.subscriptionTier,
'locale': Localizations.localeOf(context).languageCode,
});
// On logout
Sentrinel.clearContext();
Adding breadcrumbs
Sentrinel.addBreadcrumb('tapped pay', category: 'ui', data: {'amount': 42});
Structured logs
Sentrinel.info('User signed in', category: 'auth', attributes: {
'method': 'google',
});
Sentrinel.warn('Cache miss', category: 'cache', attributes: {
'key': 'feed_v2',
});
Sentrinel.error('Payment failed', category: 'billing', attributes: {
'orderId': order.id,
'reason': 'insufficient_funds',
});
Flush on background
When the app goes to the background, flush buffered telemetry so nothing is lost (especially crash reports):
class _MyAppState extends State<MyApp> with WidgetsBindingObserver {
@override
void initState() {
super.initState();
WidgetsBinding.instance.addObserver(this);
}
@override
void didChangeAppLifecycleState(AppLifecycleState state) {
if (state == AppLifecycleState.paused ||
state == AppLifecycleState.detached) {
Sentrinel.flush();
}
}
@override
void dispose() {
WidgetsBinding.instance.removeObserver(this);
super.dispose();
}
}
Full Flutter example
import 'dart:async';
import 'package:flutter/material.dart';
import 'package:sentrinel_flutter/sentrinel_flutter.dart';
void main() => SentrinelFlutter.run(
options: SentrinelOptions(
serverUrl: 'https://api.sentrinel.dev',
appName: 'my-app',
env: 'prod',
apiKey: const String.fromEnvironment('SENTRINEL_API_KEY'),
release: '1.4.2',
),
app: () => runApp(const MyApp()),
);
class MyApp extends StatefulWidget {
const MyApp({super.key});
@override
State<MyApp> createState() => _MyAppState();
}
class _MyAppState extends State<MyApp> with WidgetsBindingObserver {
@override
void initState() {
super.initState();
WidgetsBinding.instance.addObserver(this);
}
@override
void didChangeAppLifecycleState(AppLifecycleState state) {
if (state == AppLifecycleState.paused ||
state == AppLifecycleState.detached) {
Sentrinel.flush();
}
}
@override
void dispose() {
WidgetsBinding.instance.removeObserver(this);
super.dispose();
}
@override
Widget build(BuildContext context) {
return MaterialApp(
navigatorObservers: [SentrinelNavigatorObserver()],
home: Scaffold(
body: Center(
child: ElevatedButton(
onPressed: () async {
final client = Sentrinel.httpClient();
final res = await client.get(
Uri.parse('https://api.example.com/posts'),
);
// Process response...
},
child: const Text('Load Posts'),
),
),
),
);
}
}
Browser config
A web app is the third surface, and the only one where the API key cannot go in the config. Everything in a JavaScript bundle is served to every visitor, and an ingest key cannot be scoped down after the fact. So the setup is two halves: the page holds no secret and posts to your own origin, and a server route forwards with the key.
Client โ everything on
import { initSentrinelBrowser } from '@sentrinel/plugin/browser';
// Before hydration or first render. An error thrown *during* hydration is one
// of the most common real failures, and a handler installed after misses it.
export const sentrinel = initSentrinelBrowser({
// Same origin, so no key ships in the bundle.
endpoint: '/api/sentrinel',
// Crash-free rate is per release. Without this every build reads `unknown`.
release: '2026.8.2',
// โโ What to collect โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
captureErrors: true, // window errors + unhandled rejections
captureRequests: true, // patch fetch: timing, status, trace propagation
captureBreadcrumbs: true, // clicks, SPA navigation, requests
trackSessions: true, // one per page load โ release health's denominator
// โโ Volume โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
sampleRate: 1, // successful requests kept
slowRequestThresholdMs: 2_000, // above this, kept regardless of sampleRate
flushInterval: 10_000,
maxBatch: 100, // flush early once this many records queue
// โโ Tracing โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
// Same-origin only by default: sending trace ids to a third party leaks them
// and turns a simple request into one needing CORS preflight.
tracePropagationTargets: [/^\//, 'https://api.myapp.com'],
// โโ Noise โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
// On top of the built-in list (`Script error.`, ResizeObserver loops,
// browser-extension frames).
ignoreErrors: ['AbortError', 'The user aborted a request'],
// โโ Redaction โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
// Runs before the value is ever on the wire โ the only place that helps for
// data that should not leave the device at all. Return null to drop.
beforeSend: (record) => {
if (record.kind === 'request' && record.queryParams?.token) {
record.queryParams.token = '[redacted]';
}
return record;
},
debug: import.meta.env.DEV,
});
Server โ the tunnel
Any runtime that speaks Request/Response: Bun, Elysia, Hono, Next route
handlers, TanStack Start, Remix, Deno, Workers.
import { createSentrinelTunnel } from '@sentrinel/plugin/tunnel';
export const POST = createSentrinelTunnel({
serverUrl: 'https://api.sentrinel.dev',
// Pinned here, never read from the batch. A browser-supplied app name is a
// claim from an untrusted source โ without pinning, anyone who found the
// endpoint could write into a different app in your account.
appName: 'admin',
env: 'prod',
apiKey: process.env.SENTRINEL_API_KEY!, // never reaches the client
release: process.env.GIT_SHA,
// An open endpoint that will buffer anything is a memory-exhaustion target.
maxBodyBytes: 256 * 1024,
// Read server-side and overrides whatever the browser claimed. An identity a
// page asserts about itself is a claim, not a fact.
consumerIdentifier: (request) => userIdFromSession(request),
beforeForward: (batch, request) => batch, // null drops the batch
debug: process.env.NODE_ENV !== 'production',
});
What gets captured automatically
| Signal | Source |
|---|---|
Uncaught errors โ session crashed |
window error event |
Unhandled rejections โ session crashed |
unhandledrejection |
Handled errors โ session errored |
captureError() |
| Request timing, status, query params | patched fetch |
Network failures (status 0) |
patched fetch |
| Breadcrumbs: clicks, navigation, requests | DOM + history patch |
| One session per page load | pagehide / visibilitychange |
| Language, user agent, cores, connection, screen | navigator, screen |
Manual API
sentrinel.captureError(err, { orderId: 'o_1' });
sentrinel.addBreadcrumb('tapped pay', { category: 'ui', data: { amount: 42 } });
sentrinel.setUser('user_123'); // null on sign-out
sentrinel.setContext({ tier: 'pro' });
await sentrinel.flush();
await sentrinel.close(); // unpatch, end the session, send
Not covered
Source maps (a minified stack stays minified), web vitals, XMLHttpRequest
(only fetch is patched), and session replay.
Full detail: BROWSER.md.
OpenTelemetry without a plugin
Any OpenTelemetry SDK or Collector can export straight to Sentrinel. This works for any language โ Go, Rust, Java, Python, Ruby, .NET โ not just JS/Dart.
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.sentrinel.dev
OTEL_EXPORTER_OTLP_PROTOCOL=http/json
OTEL_EXPORTER_OTLP_HEADERS=x-api-key=snt_live_your_key_here
GenAI spans (gen_ai.request.model + gen_ai.usage.*_tokens) are priced
automatically and appear on the LLM cost dashboard page.
What gets collected
| OTel signal | Dashboard page |
|---|---|
| Traces / Spans | Traces waterfall |
| GenAI span attributes | LLM cost |
| Resource attributes | App auto-provisioning |
Backend environment variables
These control the Sentrinel API server itself. Set them in .env or your
deployment platform (Dokploy, Docker, etc.).
# โโ Storage โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
DATABASE_URL=postgresql://sentrinel:<password>@postgres:5432/sentrinel
TELEMETRY_STORE=clickhouse # "postgres" or "clickhouse"
CLICKHOUSE_DB=sentrinel
CLICKHOUSE_USER=sentrinel
CLICKHOUSE_PASSWORD=<password>
CLICKHOUSE_URL=http://clickhouse:8123
# โโ Network โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
SENTRINEL_API_PORT=3001
PUBLIC_API_URL=https://api.sentrinel.dev # compiled into dashboard at build time
PUBLIC_DASHBOARD_URL=https://app.sentrinel.dev
# โโ Auth & security โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
SENTRINEL_REQUIRE_AUTH=true # enforce session auth on dashboard routes
SENTRINEL_REQUIRE_INGEST_KEY=true # require API keys for ingestion
SENTRINEL_DISABLE_SIGNUP=false # lock signup after first user
SENTRINEL_ALLOWED_ORIGINS=https://app.sentrinel.dev,http://localhost:3000
SENTRINEL_COOKIE_SAMESITE=lax
SENTRINEL_COOKIE_DOMAIN=
# โโ Billing, trials & the operator console โโโโโโโโโโโโโโโโโโ
SENTRINEL_ADMIN_EMAILS= # comma-separated; empty = nobody is an admin
SENTRINEL_TRIAL_DAYS=14 # 0 disables trials (self-hosted wants this)
SENTRINEL_INGEST_GRACE_DAYS=7 # keep accepting telemetry this long after a block
# โโ Retention โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
RETENTION_LOGS_DAYS=30 # raw request/app log retention
RETENTION_METRICS_DAYS=400 # aggregated metrics (~13 months)
RETENTION_REPLAY_DAYS=14 # session replay recordings
# โโ Connection pools โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
DATABASE_POOL_SIZE=10
DATABASE_READ_POOL_SIZE=10
DATABASE_REPLICA_URL= # optional read replica
Key settings explained
| Variable | Why it matters |
|---|---|
TELEMETRY_STORE=clickhouse |
Production recommended. Postgres works for dev/small deployments. |
SENTRINEL_REQUIRE_INGEST_KEY=true |
Don't turn this off. Prevents unauthenticated data writes. |
PUBLIC_API_URL |
Dashboard builds need this at compile time. Changing it requires a rebuild. |
RETENTION_LOGS_DAYS |
Controls disk usage. 30 days is a good default. |
RETENTION_METRICS_DAYS |
400 days gives ~13 months of history. |
RETENTION_REPLAY_DAYS |
Session recordings are the largest thing stored per unit of value, so they get their own, shorter clock. Nobody watches a replay from six weeks ago. |
SENTRINEL_ADMIN_EMAILS |
Who may open Platform admin and change any org's billing. Deliberately an env var, not a database column: a column is one compromised signup away from being set, an env var needs redeploy access. Fails closed โ leave it empty and nobody is an admin, including you. |
SENTRINEL_TRIAL_DAYS |
How long a newly created org may use Sentrinel before paying. Existing orgs are never backfilled onto a trial clock โ they stay active. Set to 0 for self-hosted, where there is nobody to bill. |
SENTRINEL_INGEST_GRACE_DAYS |
How long telemetry keeps being accepted after the dashboard blocks. Cutting ingest instantly means that by the time someone pays, the window they needed is gone. After this, ingest answers 402. |
What each signal captures
Request metrics (always collected)
In-memory aggregation, flushed every 30s. Exact regardless of sampling.
| Field | Description |
|---|---|
| Request count | Total requests per endpoint per minute |
| Success count | Requests with status < 400 |
| Error count | Requests with status >= 400 |
| Response time | avg, min, max, p50, p95, p99 |
| Request/response size | Bytes transferred |
| Status codes | Distribution (200, 404, 500, etc.) |
Dashboard pages: Traffic, Performance, Errors, Consumers
Request logs (when requestLogging.enabled: true)
Individual request/response records with full detail.
| Field | Description |
|---|---|
| Method, path, status | HTTP method, URL path, status code |
| Response time | Latency in milliseconds |
| Request/response size | Bytes |
| Consumer identifier | Which client made the call |
| Request headers | Full headers (masked) |
| Request body | Full body (masked, truncated) |
| Response body | Full body (masked, truncated) |
| Query parameters | URL params (masked) |
| Error message | If status >= 400 |
| Trace ID | Links to distributed trace |
| Business context | Fields from addRequestContext() |
| Sample rate | How this record was sampled |
Dashboard pages: Request Logs, Traffic (endpoint detail), Performance (Apdex)
Application logs (when logging.enabled or logCapture.enabled)
Structured log records correlated to requests.
| Field | Description |
|---|---|
| Level | debug, info, warn, error |
| Message | Stable, groupable string |
| Category | Hierarchical (e.g., "api.checkout") |
| Attributes | Structured fields (user_id, amount, etc.) |
| Request ID | Links to the request |
| Trace ID | Links to the distributed trace |
| Span ID | Links to the specific span |
| Sequence | Order within request |
Dashboard pages: Logs, Request Logs (Logs tab)
Errors (auto-collected for status >= 400)
Fingerprinted and grouped into issues with occurrence counting.
| Field | Description |
|---|---|
| Error type | Exception class or HTTP status |
| Error message | Human-readable description |
| Stack trace | Full call stack (when available) |
| Fingerprint | 16-char hash for grouping |
| Title | Auto-generated from type + message |
| Culprit | Top frame in your code |
| Occurrences | How many times this bug fired |
| First/last seen | Time range |
| Regression | Auto-detected if resolved bug reappears |
Dashboard pages: Issues, Errors
Distributed traces (auto-collected for every request)
| Field | Description |
|---|---|
| Trace ID | Unique per request |
| Root span | The HTTP request itself |
| Child spans | DB queries, external calls, custom spans |
| Span kind | SERVER, CLIENT, DB, INTERNAL, PRODUCER, CONSUMER |
| Duration | Per-span timing |
| Attributes | Custom key-value pairs |
| Status | OK or ERROR |
Dashboard pages: Traces (waterfall), Request Logs (Trace tab)
Resource metrics (auto-collected)
| Field | Description |
|---|---|
| CPU usage | Percentage |
| Memory RSS | Resident Set Size |
| Memory heap total | Total heap allocation |
| Memory heap used | Currently used heap |
Dashboard pages: Resources
LLM cost (auto-detected from GenAI spans)
| Field | Description |
|---|---|
| Provider | openai, anthropic, google, etc. |
| Model | gpt-4o, claude-3, etc. |
| Input/output tokens | Per-call token counts |
| Cost (USD) | Auto-computed from price table |
| Consumer | Which client made the call |
| Endpoint | Which API route triggered it |
Dashboard pages: LLM cost
Release health (Flutter sessions)
| Field | Description |
|---|---|
| Session ID | Unique per app launch |
| Status | ok, crashed, abnormal, errored |
| Release version | Which build |
| Environment | prod, staging, dev |
| Device OS/version | iOS/Android version |
| Duration | Session length |
Dashboard pages: Release Health
Data masking
Masking runs in your process, before anything leaves it. A masked field never reaches the network, let alone Sentrinel's disk.
Headers to mask
maskHeaders: [
/^authorization$/i, // auth tokens
/^cookie$/i, // session cookies
/^x-api-key$/i, // API keys
/^x-auth-token$/i, // custom auth
],
Query parameters to mask
maskQueryParams: [
"token", // exact match
"secret", // exact match
/^key$/i, // regex match
],
Body fields to mask
maskBodyFields: [
/^password$/i, // password fields
/^token$/i, // token fields
/^credit_card$/i, // payment fields
/^cvv$/i, // security codes
/^ssn$/i, // social security
/^secret$/i, // any secret
],
maskBodyFields descends into nested JSON objects. A field named password at
any depth is masked.
Sampling safety
Three properties make sampling safe:
Errors and slow requests are never sampled out.
sampleRateapplies only to successful, fast requests โ the ones you were never going to read.Metrics stay exact. Counters are computed in-process before sampling, so a 1% sample rate still yields correct request counts and error rates. Only the individual log rows are thinned.
Every row records the rate it was captured at, so counts derived from raw logs can be extrapolated honestly rather than silently under-reporting.
Recommended sample rates
| Traffic level | sampleRate |
Why |
|---|---|---|
| Development | 1.0 |
You want every request |
| Low (< 100 req/s) | 0.5 |
Keep half, still plenty of detail |
| Medium (100-1k req/s) | 0.1 |
10% is enough for debugging |
| High (> 1k req/s) | 0.01 |
1% for volume, errors always kept |
Distributed tracing
Traces connect the full request path: Flutter tap โ API โ database โ external
service. The root span is always the HTTP request. Child spans are created with
traceSpan().
Manual spans
import { traceSpan, traced } from "@sentrinel/plugin";
// Wrap a block
const user = await traceSpan("db.findUser", async (span) => {
span.setAttribute("db.system", "postgresql");
span.setAttribute("db.operation", "SELECT");
return db.users.findById(id);
});
// Wrap a function at definition
const getProducts = traced("db.getProducts", () => db.products.find());
Auto-trace an entire service
import { traceObject } from "@sentrinel/plugin";
const db = traceObject(new DatabaseClient(), "db");
// Every method on db is now auto-traced as "db.*"
Cross-service calls
import { sentrinelFetch } from "@sentrinel/plugin";
// Automatically propagates traceparent header
const res = await sentrinelFetch("https://payments.internal/v1/charges", {
method: "POST",
body: JSON.stringify({ amount }),
});
Flutter โ backend correlation
The Flutter SDK automatically sends traceparent on every HTTP request. The
backend plugin picks it up and continues the same trace. No extra configuration
needed โ just make sure both sides use the same Sentrinel API.
Span kinds
Use the right kind for accurate waterfall coloring:
| Kind | Use for |
|---|---|
SERVER |
Incoming HTTP request (auto-created) |
CLIENT |
Outgoing HTTP call |
DB |
Database query |
INTERNAL |
Business logic |
PRODUCER |
Message queue publish |
CONSUMER |
Message queue consume |
Structured logging deep dive
Why structured logging
console.log(\user ${id} failed checkout`)` is easy to write and impossible to
query. You cannot filter by user, group by failure reason, or tie it to the
request that produced it.
Structured logging fixes this by separating the message (stable, groupable) from the attributes (filterable fields).
Context is inherited, not repeated
withContext({ userId: user.id, tenantId: org.id, tier: user.tier }, async () => {
await processOrder();
// every log inside carries those three fields
// even three levels deep, in code that never heard of a user
});
Categories are hierarchical
const log = getLogger(["api", "checkout"]);
// logs under "api.checkout"
const sub = log.getChild("db");
// logs under "api.checkout.db"
// Filter by "api" to see everything in the API subsystem
Canonical wide events
Instead of scattered log lines, attach rich context to the request itself:
addRequestContext({
tier: user.tier,
cartTotal,
experiment: "new-pricing",
outcome: "success",
});
"Slow checkouts for enterprise customers in the new pricing experiment" becomes one query rather than a correlation exercise.
Everything is linked
- The request row carries the trace id and your
addRequestContext()fields - Every log record carries the request id, trace id and span id
- The trace carries the request id, so the waterfall links back
- An error occurrence carries both, linking to the exact request
What you see in the dashboard
| Dashboard page | Requires | Shows |
|---|---|---|
| Apps | Plugin installed | Overview cards per service |
| Traffic | Metrics (auto) | Request volume, endpoints, response times |
| Errors | Errors (auto for >= 400) | Error rates, status breakdown |
| Issues | Errors with stack traces | Fingerprinted error groups, occurrence counts |
| Performance | Metrics (auto) + Request logs | p50/p95/p99, Apdex, slowest endpoints |
| Consumers | Consumer identifier set | Per-client metrics, usage breakdown |
| Request logs | requestLogging.enabled: true |
Individual request entries with full detail |
| Logs | logging.enabled or logCapture.enabled |
Structured logs with filtering |
| Traces | traceSpan() or OTLP |
Distributed trace waterfalls |
| Resources | Metrics (auto) | CPU and memory over time |
| Uptime | Health checks configured | HTTP checks, uptime percentage |
| Alerts | Alert rules configured | Threshold alerts with notifications |
| SLOs | SLOs configured | Error budget, burn-down |
| Cron monitors | Monitors configured | Job check-in status |
| LLM cost | GenAI spans | Token usage, cost by model |
| Release health | Flutter release set |
Crash-free rate per release |
| Dashboards | Custom widgets | Arranged views |
| Status pages | Public config | Customer-facing status |
| SQL console | Any telemetry data | Read-only SQL queries |
Troubleshooting
Nothing shows up
- Check API health:
curl http://localhost:3001/health - Check
serverUrl: Must point to the API, not the dashboard - Check API key: Must match
appName+env - Check
requestLogging.enabled: true: Metrics work without it, request logs don't - Check
excludePaths: Your endpoint might be excluded
Logs not appearing
logging.enabledis off by default. Addlogging: { minLevel: "debug" }.logCapture.enabledis off by default. AddlogCapture: { enabled: true, minLevel: "debug" }.
Traces missing
- No
traceSpan()calls = only the root HTTP span appears. Wrap key operations. - For Flutter:
traceparentpropagation is automatic. No config needed.
Flutter crashes not showing
- Make sure
runAppis inside theappcallback ofSentrinelFlutter.run() - Set
releaseโ without it, release health shows "unknown" - Call
Sentrinel.flush()indidChangeAppLifecycleStatewhen paused
403 errors in logs
The API key doesn't match appName + env. Create a new key in the dashboard
that matches both values.
Reference: all plugin options
| Option | Type | Default | Backend | Flutter |
|---|---|---|---|---|
serverUrl |
string |
โ | required | required |
appName |
string |
โ | required | required |
env |
string |
"dev" |
required | required |
apiKey |
string |
โ | recommended | recommended |
version / release |
string |
โ | deploy markers | crash-free rate |
flushInterval |
number / Duration |
30000 / 30s |
batch interval | batch interval |
consumerIdentifier |
string / (ctx) => string |
โ | client ID | client ID |
excludePaths |
(string|RegExp)[] |
[] |
paths to ignore | โ |
debug |
boolean |
false |
verbose logging | โ |
requestLogging.enabled |
boolean |
false |
request log rows | โ |
requestLogging.sampleRate |
number |
1 |
fraction kept | โ |
requestLogging.slowRequestThresholdMs |
number |
2000 |
always-capture ms | โ |
requestLogging.logRequestHeaders |
boolean |
false |
include headers | โ |
requestLogging.logRequestBody |
boolean |
false |
include req body | โ |
requestLogging.logResponseBody |
boolean |
false |
include res body | โ |
requestLogging.maxBodySize |
number |
65536 |
body truncation | โ |
requestLogging.maskHeaders |
(string|RegExp)[] |
[] |
redact headers | โ |
requestLogging.maskQueryParams |
(string|RegExp)[] |
[] |
redact params | โ |
requestLogging.maskBodyFields |
(string|RegExp)[] |
[] |
redact body fields | โ |
logCapture.enabled |
boolean |
false |
capture console.* | โ |
logCapture.minLevel |
string |
"info" |
min level | โ |
logCapture.maxPerRequest |
number |
50 |
per-request cap | โ |
logCapture.maxMessageLength |
number |
2000 |
truncation | โ |
logging.minLevel |
string |
"debug" |
min buffered level | โ |
logging.echo |
boolean |
false |
mirror to stdout | โ |