Architecture

Native Observability runs entirely inside the Drupal request lifecycle. The base module (native_observability) captures every request through kernel event subscribers and stores the result in a dedicated database table. Eleven sub-modules add spans, metrics, cache and database observation, a dashboard, reporting, and telemetry export, without any of them patching Drupal core. This page documents three mechanisms straight from the source: the capture and persistence pipeline, the module tier graph, and the export fan-out.

Request capture and persistence pipeline

TraceSubscriber (in native_observability) listens to kernel.request, kernel.exception, and kernel.response. It assigns a ULID request ID on kernel.request, captures exception metadata on kernel.exception if one is thrown, and builds the final trace payload on kernel.response according to the privacy modes configured in native_observability.settings. The trace row is inserted synchronously by DatabaseTraceStorage::insert(), which also invalidates the native_observability_trace:list cache tag and dispatches TRACE_RECORDED right after the INSERT, still inside the response phase.

Spans, cache events, and database query records follow a different path. Their subscribers (RequestSummarySpanSubscriber, DatabaseSummarySpanSubscriber, CacheResponseObserverSubscriber, DatabaseQueryObserverSubscriber, and others) run at their own priorities on kernel.response and append rows into an in-memory DeferredPersistenceBuffer instead of issuing a synchronous INSERT. Each buffer is tagged native_observability.deferred_buffer. On kernel.terminate, after the response has already reached the client, DeferredPersistenceFlushSubscriber iterates every tagged buffer and calls flush(), which issues one multi-row INSERT per buffer. Setting deferred_persistence_enabled: false in native_observability.settings switches every buffer back to a synchronous per-row INSERT, which is useful for tests that read rows mid-request without calling flush() explicitly.

flowchart TB
    subgraph REQ["kernel.request"]
        direction TB
        EXCL["RequestLogExclusionSubscriber (priority 1500)<br/>evaluates exclusion rules, flags the request"]
        GEN["TraceSubscriber::onKernelRequest (priority 100)<br/>generates a ULID request_id, records started_at"]
        EXCL --> GEN
    end

    subgraph EXC["kernel.exception"]
        THROW["TraceSubscriber::onKernelException (priority 0)<br/>captures exception class, message, code onto the request"]
    end

    subgraph RESP["kernel.response"]
        direction TB
        OBS["Span, cache and database subscribers<br/>append rows to their own DeferredPersistenceBuffer"]
        TRACE["TraceSubscriber::onKernelResponse (priority -100)<br/>builds the trace payload under the configured privacy modes"]
        OTELSUB["OpenTelemetryTraceSubscriber (priority -110)<br/>builds an independent OTLP payload from request attributes"]
        OBS --> TRACE
        TRACE --> OTELSUB
    end

    STORE[("DatabaseTraceStorage::insert()<br/>native_observability_trace table")]
    EVT{{"TRACE_RECORDED dispatched synchronously,\nstill inside kernel.response"}}

    subgraph TERM["kernel.terminate"]
        FLUSH["DeferredPersistenceFlushSubscriber::onTerminate (priority -100)<br/>calls flush() on every service tagged native_observability.deferred_buffer"]
    end

    BUFTABLES[("spans / cache_event / database_query tables<br/>one multi-row INSERT per buffer")]
    OTLP(["External OTLP collector"])

    REQ --> EXC --> RESP
    TRACE --> STORE --> EVT
    OTELSUB -->|"if enabled and available"| OTLP
    RESP --> TERM
    FLUSH --> BUFTABLES

The trace row itself bypasses the deferred-buffer mechanism on purpose: it is inserted before TRACE_RECORDED fires, and that event is the signal other modules (metrics aggregation, the ECA bridge) react to. Spans, cache events, and database queries can afford to wait until kernel.terminate because nothing downstream needs them before the response is sent.

Module dependencies

The family ships as thirteen modules: native_observability plus twelve extensions. The graph below is the dependency graph declared in the .info.yml files, with the four redundant edges removed (dashboard, export, report and eca_bridge all declare a dependency on native_observability that their other dependencies already imply).

Every module in the graph builds on native_observability, directly or through another module. Arrows point from a module to what it enables, so reading left to right shows the order in which capabilities become available. Which tier installs which module is a separate question, answered by the table in Installation.

flowchart LR
    base(["native_observability"])

    spans["spans"]
    metrics["metrics"]
    cache["cache_observer"]
    dbobs["database_observer"]
    execution["execution"]
    otel["otel"]
    status["status_block"]
    export["export"]
    dashboard["dashboard"]
    report["report"]
    eca["eca_bridge"]
    demo["eca_bridge_demo"]

    base --> spans
    base --> metrics
    base --> cache
    base --> dbobs
    base --> execution
    base --> otel
    base --> status

    spans --> export
    cache --> export

    metrics --> dashboard
    execution --> dashboard
    spans --> dashboard
    cache --> dashboard
    dbobs --> dashboard

    execution --> eca
    spans --> eca

    dashboard --> report
    eca --> demo

    style demo stroke-dasharray: 5 5

native_observability_dashboard is the convergence point: it needs five other extensions (metrics, execution, spans, cache_observer, database_observer), which is why it cannot be installed before them. native_observability_report sits one step further out, on top of the dashboard. native_observability_eca_bridge also requires the external eca module, which is outside this family and omitted from the graph.

native_observability_eca_bridge_demo, drawn with a dashed border, ships five example ECA rules and belongs to no preset. Enable it by hand with drush en native_observability_eca_bridge_demo -y.

Export fan-out

Three export surfaces read from the same storage tables but reach them through different mechanisms: one push, two pull. native_observability_otel registers three exporter plugins (opentelemetry, prometheus, null_exporter) through its ExporterManager, but only opentelemetry performs a network call from export(); prometheus and null_exporter exist to advertise their availability in the plugin registry and return TRUE without doing anything.

flowchart LR
    subgraph SRC["Storage tables"]
        TT[("native_observability_trace")]
        ST[("native_observability_spans")]
        CT[("native_observability_cache_event")]
    end

    REQATTR["Request attributes set during\nkernel.request / kernel.response"]

    subgraph PUSH["Push: one call per request, kernel.response priority -110"]
        OTS["OpenTelemetryTraceSubscriber"]
        REG["ExporterRegistry::get('opentelemetry')"]
        OTE["OpenTelemetryExporter::export()"]
    end
    OTLP(["External OTLP collector"])
    REQATTR --> OTS --> REG --> OTE --> OTLP

    subgraph PULLP["Pull: Prometheus scrape"]
        PC["PrometheusMetricsController<br/>GET /native-observability/prometheus"]
        PMB["PrometheusMetricsBuilder"]
        LPR["LivePrometheusMetricsReader"]
    end
    PC --> PMB --> LPR
    LPR --> TT
    LPR --> ST
    LPR --> CT

    subgraph PULLE["Pull: Elastic export"]
        EC["ElasticTelemetryExportController<br/>GET /admin/reports/native-observability/elastic"]
        ETB["ElasticTelemetryBuilder"]
    end
    EC --> TT
    EC --> ETB

The OTLP path never reads the row that DatabaseTraceStorage just wrote: OpenTelemetryTraceSubscriber builds its own payload straight from the request attributes set earlier in the same response cycle, and calls export() only if isAvailable() returns true, which requires both enabled and a non-empty endpoint in native_observability_otel.settings. The Prometheus and Elastic paths are the opposite: they never run on their own, they respond to an external GET and read the current state of the trace, spans, and cache event tables at scrape time. LivePrometheusMetricsReader aggregates a rolling window (prometheus_window_seconds) across all three tables in a single call; ElasticTelemetryExportController reads only the trace table through TraceStorageInterface::search() and reshapes it into NDJSON.