> ## Documentation Index > Fetch the complete documentation index at: https://docs.lerian.studio/llms.txt > Use this file to discover all available pages before exploring further. # How Streaming Hub works > The path an event takes through Streaming Hub: ingest, classification, consume-once deduplication, subscription matching, dispatch, retries, and dead-letters. An event moves through Streaming Hub in three stages. The hub **ingests** it from the internal stream, **matches** it against your subscriptions, and **dispatches** it to each matched destination. This page walks through each stage and the delivery guarantees that fall out of it. ## Ingesting events from the stream *** Streaming Hub consumes the platform stream as **CloudEvents 1.0** messages in **binary content mode**. The CloudEvents context attributes travel in Kafka record headers (each prefixed `ce-`). The event body is the record value. The hub reads routing and identity from the headers without deserializing the payload. The hub subscribes only to v3 **application fact topics**: ``` lerian.streaming. ``` `ce-source` is one lowercase, dot-free segment. It uses letters, digits, underscores, or hyphens. The topic carries no resource, event, or schema-major suffix. The hub dispatches by CloudEvents headers. The follow set excludes command topics (`.commands`) and dead-letter topics (`.dlq`). Midaz, Tracer, Matcher, Lender, and Consignado application fact topics now match this follow-set contract. A second consumer reads the `.dlq` topics that producers write. That consumer serves **observability only**. The follow-set grammar excludes a trailing `.dlq`, so a dead-letter record never becomes a delivery to one of your subscriptions. Operators read those records through `GET /admin/dlq`. See [DLQ forensics](/en/platform/streaming-hub/operating-streaming-hub). Every record the hub accepts must carry this context set: | Header | Requirement | | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ce-specversion` | The CloudEvents spec version. | | `ce-id` | A unique event id. **Must be a UUIDv7** — it is the deduplication and correlation key. This tightens the [shared envelope contract](/en/reference/events/overview), where `ce-id` is opaque: the hub quarantines a record whose id is not a UUIDv7, and it deduplicates ids within a tenant. | | `ce-source` | The producing service. | | `ce-type` | The event type, exactly `studio.lerian...`; `` must equal `ce-source`. | | `ce-time` | The emission timestamp. | | `ce-schemaversion` | The payload schema version. | | `ce-resourcetype` | The resource segment (for example, `account`). | | `ce-eventtype` | The event segment (for example, `created`). | | `ce-tenantid` | The owning tenant, and the only source of tenant identity at ingest. A record missing it is quarantined. | For each record the hub returns one of three **verdicts**: | Verdict | When | Outcome | | -------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------- | | **Accept** | A well-formed event with a non-empty tenant id. | Persisted to the event inbox and matched against subscriptions for that same tenant. | | **Drop** | A system event. | Counted and skipped; the stream position still advances. No event is stored. | | **Quarantine** | A malformed event, or one missing its tenant. | Recorded as a poison event for forensics. Not retried — a malformed record never becomes valid on replay. | The tenant admitted at ingest comes only from the validated `ce-tenantid` attribute the parser extracted. Ingest has no tenant-roster drop gate. Stored events match only subscriptions with the same tenant id. Three related strings look alike but are distinct. Never conflate them: * **`ce-type`** is the source-qualified CloudEvents type: `studio.lerian...` (for example, `studio.lerian.lender.loan_application.approved`). * **Kafka topic** is one v3 fact topic per application: `lerian.streaming.` (for example, `lerian.streaming.lender`). Resource, event, and schema major do not appear in the topic. * **`X-Lerian-Event-Type`** is the bare `.` key on a delivered webhook (`loan_application.approved`). It intentionally omits the producer. Use a subscription `origin` pin when the same key can come from more than one application. See the [event streaming overview](/en/reference/events/overview) for the full wire contract. ## Consume-once deduplication *** The internal stream is at-least-once, so the same record can arrive more than once. On the way into the event inbox the hub deduplicates on `ce-id`. A duplicate id writes nothing and counts as a dedup drop. The hub persists records per Kafka partition in one transaction. It commits the stream position for a partition only after that partition's write commits. A failure on one partition therefore never loses or double-commits events on another. ## Subscription matching *** After the hub stores an event, it resolves which of your subscriptions should receive it. Subscription matching is internal and **catalog-free**: it evaluates each event directly on `(tenant, origin, event type, schema major)`. The `origin` dimension is optional on the subscription. Without it, the same event key from any producing application can match. An event no subscription wants produces zero deliveries. That is a correct outcome, not an error. Matching admits a subscription only when it is both **`enabled`** and in **`verification_state = active`**. These are two independent conditions, and matching requires both. See [the subscription model](/en/platform/streaming-hub/managing-subscriptions). ## Dispatching to your destinations *** For every match, the hub creates a delivery job and hands it to a worker pool (eight workers by default). Workers claim due jobs atomically with a short lease and interleave across tenants, so no single tenant's backlog starves the others. Each worker loads the stored payload, decrypts the destination's signing secret or credential in memory, and makes exactly one delivery attempt to the sink. Delivery is **at-least-once**: the hub may deliver the same event more than once (through retries or redeliveries after a worker restart). Every delivery carries a stable `X-Lerian-Event-Id` (the `ce-id`) for you to deduplicate on, and a per-attempt `X-Lerian-Delivery-Id` that changes on every retry. Treat a repeated `X-Lerian-Event-Id` as a duplicate and acknowledge it without reprocessing. See [Consuming events](/en/platform/streaming-hub/consuming-events) for the consumer side of this contract. ## Retries and back-off *** When an attempt fails, the hub schedules the next one on a fixed back-off curve with full jitter: ``` 0 · 5s · 5m · 30m · 2h · 5h · 10h · 10h ``` The curve spans roughly 28 hours across its eight steps. Each subscription can override it with its own schedule (up to 12 steps, each capped at 10 hours). A malformed override falls back to the default curve rather than failing delivery. ## Dead-lettering *** The hub **dead-letters** an event that exhausts its retries. The hub stops attempting it and records the terminal outcome. The hub reclaims a job that instead keeps crashing a worker mid-attempt and never records an outcome. The reclaim limit is five by default. The hub then dead-letters it as poison, so a single toxic job can never occupy a worker forever. ## Circuit breaker *** Each `(tenant, destination)` pair has a circuit breaker. When a destination fails repeatedly, its breaker opens. Further jobs for that destination are **shed** (rescheduled without an attempt). A single broken endpoint does not burn worker capacity or hammer a struggling target. The breaker recovers on its own once the destination starts succeeding again. ## Auto-disabling a broken destination *** The hub automatically disables a destination that stays broken for a long time. Auto-disable trips only when the open failure span is **both**: * **sustained**: it lasts at least the failure window (120 hours by default), and * **spread**: at least 12 hours separate the first and last failure in the span. Both gates must hold, and a **single successful delivery clears the span**, so a brief blip never accumulates toward the verdict. When auto-disable trips it flips the subscription's `enabled` flag to `false`. It never touches `verification_state`. Delivery stops (matching requires `enabled`), and the hub records why. You recover an auto-disabled subscription with `POST /v1/subscriptions/:id/verify`, which re-probes the destination and, on success, re-enables it in place. A kill switch (`STREAMING_HUB_AUTODISABLE_ENABLED`) governs auto-disable, so an operator can ship it dark. See [recovering an auto-disabled subscription](/en/platform/streaming-hub/managing-subscriptions). ## Ordering guarantees *** The internal stream is partitioned by tenant. Events for one tenant normally ride a single partition, and the hub preserves their **first-in, first-out** order end to end. The hub attributes and deduplicates on the CloudEvents headers, not the Kafka record key. Because of that, a producer that spreads (salts) a hot tenant across several partitions changes only physical placement. The hub still attributes and dedups correctly. Ordering is the one trade-off. A salted tenant's events span multiple partitions with no cross-partition order guarantee, so that tenant forfeits strict FIFO. The arrival sequence the hub assigns reflects the order events *arrived* at the hub, not the order they were *produced*. A tenant that is not salted keeps single-partition FIFO throughout. ## Next steps *** The subscription model, onboarding flows, and secret rotation. Verify signatures, deduplicate, and pull events.