Database Monitoring Implementation Guide

Overview

Database Monitoring (DBM) links a user-facing symptom to database work. Start with application database spans and PostgreSQL server metrics. Add pool telemetry, query samples, and workload-level query data when their investigation value justifies the operational and data-handling cost.

This path is PostgreSQL-focused. It links the current internal implementation reference, [[Database Monitoring]], and the product/capability context in [[Tsuga DBM]]. Use those notes for Tsuga-specific fields and current product status. Use this document to agree the rollout order with the application and database owners.

Level 1 depends on the calling application already exporting traces. Without client spans, there is no database inventory to build on. Collect the host, node, or cluster telemetry that owns the database’s compute before using the infrastructure context in level 3.

Application database instrumentation ── client spans ───────────────► Tsuga database page
        │                          \                              ├─ queries and calling services
        └── pool telemetry ──────────► Tsuga metrics               └─ application symptom

PostgreSQL receiver ────────────────► server-health metrics
        │
        ├── query samples + blocking evidence ──► logs
        └── pg_stat_statements top queries ─────► logs

Infrastructure / managed-DB metrics ───────────► capacity and platform context

The receiver reports PostgreSQL state. Client spans report the caller’s experience. Infrastructure telemetry reports host, storage, network, proxy, or managed-service pressure. Use matching database names as an investigative pivot, then verify causality with evidence.

Before starting

Agree the database, environment, application services, database owner, and the user-facing workflow in scope.
Decide which telemetry may leave the database. Query text, role names, table names, and plans can be sensitive even when literal values are masked.
Use a dedicated monitoring role with the minimum privileges. Have the database owner approve every privilege added for query sampling or pg_stat_statements access.
Run the receiver close enough to the database to reach it securely, and send Collector logs and metrics as well as database telemetry. An empty DBM view can mean a network, authentication, permission, configuration, pipeline, or exporter failure.
Establish the identity that links the views: database name, technology, environment, and database instance or cluster. Record whether the PostgreSQL receiver semantic-convention feature gate is enabled. Changing it alters the emitted attribute schema and should be handled as a telemetry migration.

Level 1 - Send database client spans

Instrument every calling application with its OpenTelemetry SDK or supported automatic instrumentation. Database-client spans are the foundation of Tsuga’s database inventory and database page: they provide calling service, request rate, error rate, latency, and query breakdown when statements are available.

Verify that spans carry the current database semantic-convention fields, especially db.system.name and db.namespace. Send db.name as well where legacy compatibility is needed. db.statement enables query-level analysis, but it needs an explicit data-classification decision. Never assume parameterization makes a statement harmless to export.

Validate one known application request end to end:

The trace contains a database client span with the correct database and PostgreSQL technology.
The database appears in Services → Databases for the selected environment/cluster.
The database page identifies the calling service and, where allowed, the query shape.

Exit criteria: a known application request produces a database client span with the correct db.system.name and db.namespace. The database appears in the Databases inventory for the agreed environment. The query-text decision is recorded either way.

References:

Database semantic conventions: current attributes for client database spans.
Part 1: client spans and pool telemetry: why a database span describes the caller’s experience, not necessarily only query execution.

Level 2 - Collect PostgreSQL server health

Deploy the OpenTelemetry Collector Contrib PostgreSQL receiver with a dedicated monitoring role and metrics pipeline. Start with connections, transactions, rollbacks, deadlocks, size, disk runway, applicable replication lag, and locks. Add optional metrics one at a time and measure receiver cost.

receivers:
  postgresql:
    endpoint: "<POSTGRES_HOST>:5432"
    username: ${env:POSTGRES_MONITORING_USER}
    password: ${env:POSTGRES_MONITORING_PASSWORD}
    databases: ["<DATABASE>"]
    collection_interval: 30s

service:
  pipelines:
    metrics/postgresql:
      receivers: [postgresql]
      processors: [memory_limiter, batch]
      exporters: [otlp_http/tsuga]

The exact receiver configuration and emitted attributes vary by Collector version. Pin the Collector version, test its feature gates in a non-production environment, and confirm the database identity used by the Tsuga PostgreSQL panels (context.postgresql.database.name in the current internal reference) before standardizing dashboards or monitors.

These metrics give server-side context during an application incident. They do not prove which request caused the activity, and they do not cover host CPU, storage latency and IOPS, network behaviour, proxy behaviour, or managed-provider limits. Bring those infrastructure signals separately.

Exit criteria: recent connection, transaction, deadlock, size, and lock metrics arrive for the in-scope database with the identity attribute the Tsuga PostgreSQL panels use. The Collector version and feature-gate state are recorded. The monitoring role’s own load has been measured at the chosen collection interval.

References:

Part 1: receiver setup, monitoring role, schema migration, and first metrics.
PostgreSQL receiver: version-specific configuration and permissions.

Level 3 - Add pool and infrastructure context

Add pool telemetry where supported to distinguish query execution from connection wait. Give each pool a stable db.client.connection.pool.name, especially when a framework supplies connection configuration indirectly.

Node.js `pg` example

For Node’s pg, register the instrumentation and metric reader before the application imports pg. This adds to the application’s existing OTLP startup code and retains its trace exporter. The PgInstrumentation currently emits db.client.connection.pending_requests and db.client.connection.count (split by db.client.connection.state). Support for the other pool metrics is driver-version dependent.

// instrumentation.cjs: load before app.cjs imports `pg`
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { PeriodicExportingMetricReader } = require('@opentelemetry/sdk-metrics');
const { OTLPMetricExporter } = require('@opentelemetry/exporter-metrics-otlp-proto');
const { PgInstrumentation } = require('@opentelemetry/instrumentation-pg');

const sdk = new NodeSDK({
  // Keep the service's existing trace exporter and resource configuration here.
  metricReader: new PeriodicExportingMetricReader({
    exporter: new OTLPMetricExporter(),
  }),
  instrumentations: [new PgInstrumentation()],
});

sdk.start();

Set the standard OTLP metric endpoint and authentication variables alongside the application’s existing trace-export configuration. For instrumentation versions that support it, set OTEL_SEMCONV_STABILITY_OPT_IN=database and inspect real telemetry before relying on the emitted field names.

Configure each pool with explicit connection values, then import it only after the instrumentation startup module. This avoids ambiguous pool identity such as unknown_host:unknown_port/unknown_database.

// app.cjs
require('./instrumentation.cjs');
const { Pool } = require('pg');

const pool = new Pool({
  host: process.env.PGHOST,
  port: Number(process.env.PGPORT || 5432),
  database: process.env.PGDATABASE,
  user: process.env.PGUSER,
  password: process.env.PGPASSWORD,
  max: 20,
});

If the application uses ESM, verify query spans as well as pg-pool.connect spans after deployment. The blog’s tested Node path uses createRequire or the OpenTelemetry ESM loader hook so that instrumentation can patch the client library before it loads.

Infrastructure route

Collect infrastructure context through the managed-database provider, Kubernetes/node telemetry, or the relevant host integration. Pool telemetry and server metrics still cannot explain a saturated disk volume, a network path issue, or a provider-level resource limit by themselves.

There is deliberately no universal Collector snippet for this part: a hostmetrics receiver cannot observe a managed PostgreSQL instance, and cloud-provider metrics do not exist for a self-hosted database. Choose the route that owns the database’s compute and storage:

Kubernetes or self-hosted PostgreSQL: deploy the host, node, and cluster collection, then scope the view to the database workload and node.
Managed PostgreSQL: collect the provider’s CPU, storage, IOPS, connection-limit, network, and failover signals through the provider integration or its metrics export path.
VM or bare-metal PostgreSQL: collect host CPU, memory, filesystem, disk-I/O, and network telemetry from the database host in addition to the PostgreSQL receiver.

Use this level to answer the first question in an incident: was the user-visible latency caused by the calling application waiting for a connection, PostgreSQL doing work, or the surrounding platform?

Exit criteria: each application pool reports its own identity rather than an unknown_host:unknown_port placeholder. Connection-wait time is distinguishable from query time on a known slow request. The route that owns the database’s compute and storage telemetry is deployed and identified.

Level 4 - Opt in to query samples and blocking evidence

Enable query samples after the database owner approves the privilege, receiver work, log volume, and query-data policy. Samples are point-in-time pg_stat_activity observations, so short queries and waits can be missed.

The PostgreSQL receiver emits query samples as logs. Its logs support is at development stability as of Collector v0.157.0, so pin the version, test the upgrade, and expect field changes. The baseline pg_monitor grant already covers sample collection, so this level usually needs no extra privilege. Confirm that with the database owner rather than assuming it.

Enable the event explicitly and add the receiver to a logs pipeline as well as the metrics pipeline:

receivers:
  postgresql:
    # Level 2 settings unchanged
    events:
      db.server.query_sample:
        enabled: true
    query_sample_collection:
      max_rows_per_query: 20

service:
  pipelines:
    logs/postgresql-query-evidence:
      receivers: [postgresql]
      processors: [memory_limiter, batch]
      exporters: [otlp_http/tsuga]

PostgreSQL truncates the sampled statement at track_activity_query_size, which defaults to 1024 characters. Raise it only if truncated samples block an investigation, and treat the longer text as a data-handling change.

Filter the monitoring role from dashboards and investigations. In a lightly loaded system, the receiver’s own sessions can dominate the samples. Confirm that literal masking, statement truncation, retention, and access controls meet the customer’s data policy before enabling the feature.

For trace bridging, write the active W3C traceparent to a connection-local PostgreSQL session field before the query and clear it before returning the connection to the pool. A stale value can attach a later query to the wrong request.

Exit criteria: the database owner has approved the added privilege and the query-data policy. Query samples arrive as logs with masking, truncation, retention, and access as agreed. The monitoring role’s own sessions are excluded from investigation views. If the trace bridge is enabled, a test proves the session field is cleared before the connection returns to the pool.

References: Part 2: query samples, blocking evidence, trace correlation, and safeguards.

Level 5 - Add workload-level top-query evidence

Use pg_stat_statements for workload-level query shapes. The database owner must enable the module and settings, install the extension in the receiver’s database, and grant the monitoring role enough access to identify queries.

The receiver reads pg_stat_statements through its default postgres connection, so create the extension there. Without it, the receiver emits no top-query events. One installation covers the monitored databases in the cluster.

Enable the event with explicit limits rather than accepting the receiver defaults. max_explain_each_interval is a database-work budget. Start conservatively and measure it.

receivers:
  postgresql:
    events:
      db.server.top_query:
        enabled: true
    top_query_collection:
      collection_interval: 60s
      top_n_query: 100
      max_rows_per_query: 100
      max_explain_each_interval: 50

Validate the data against a direct read of pg_stat_statements using the monitoring role. Do not reset the shared statistics view to investigate an incident: it destroys the baseline for every other consumer. Compare snapshots instead.

Query plans are a further, separately controlled capability. They can expose schema and query detail and may require a tightly scoped, security-definer helper function. Add them only for a clearly agreed operational use case. They are not a default DBM prerequisite.

Exit criteria: top-query evidence matches a direct read of pg_stat_statements through the monitoring role. The collection and explain limits are set explicitly and their database cost has been measured. No investigation procedure resets the shared statistics view.

References: Part 2: top queries, plans, privileges, and operational cost.

Make it operational

Alert on customer-facing symptoms first: service error rate, latency, and SLO burn. Use DBM to determine whether the database is implicated and which evidence to inspect next. Avoid alerting on every receiver metric without an owner and an agreed response.

For each production database, agree:

The application services and critical journeys that depend on it.
The owner for application instrumentation, database permissions, and infrastructure telemetry.
The approved query-data policy, retention, and who may access the resulting evidence.
A small dashboard or saved investigation view that combines application symptom, pool state, PostgreSQL health, and infrastructure context.
One validation drill: start with a known slow request, find its database span, compare the receiver metrics for the same period, then establish whether a query sample or top-query view adds an actionable next step.

Troubleshooting path

Validate in evidence order: client span, then receiver metrics, then optional query evidence. Stop at the first failing boundary.

Symptom	Check first	Likely corrective action
Database absent from Tsuga	Client span attributes and selected environment/cluster	Correct `db.system.name`, `db.namespace`, and legacy `db.name` where needed. Verify trace export.
Database page has no query breakdown	`db.statement` and data-policy decision	Enable supported client instrumentation or deliberately keep query text unavailable.
Receiver metrics absent	Collector logs, network reachability, credentials, receiver in metrics pipeline	Fix the first failing link and verify with the Collector’s own telemetry.
Query samples absent	Query-sample option, monitoring-role privilege, receiver in logs pipeline	Approve and enable the feature, then validate the logs pipeline.
Wrong trace appears on a sampled query	Connection-pool cleanup around session trace context	Clear the connection-local trace context before every release back to the pool. Otherwise, disable the bridge.
Top queries lack identity	`pg_stat_statements` extension and monitoring-role access in the receiver’s database	Install/verify the extension and test visibility as the monitoring role.

Completion criterion

Starting from a known slow request, an investigator opens its database client span in Tsuga, finds the same database in the Databases inventory, and reads the PostgreSQL health metrics for that period from the same page.

Validate with the Tsuga CLI

# Confirm the calling service emits database client spans.
tsuga traces search \
  --query "context.service.name:<service> db.system.name:postgresql" \
  --from -15m \
  --to now \
  --max-results 10

The command returns spans whose attributes name the PostgreSQL database and technology. If either is wrong, fix the client instrumentation before looking at the receiver. Validate the level 2 metrics by searching Metrics for postgresql., and the level 4 query events in Logs, as described in Collect PostgreSQL metrics and Collect query events. Do not invent a metric or query-event CLI query: the emitted names depend on the Collector version and its feature gates.