> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lerian.studio/llms.txt
> Use this file to discover all available pages before exploring further.

# Fetcher Engine ports

> Reference for the eight capability ports of the Fetcher Engine: what each contract requires, whether it is mandatory, and exactly what the Engine does without it.

The [Fetcher Engine](/en/fetcher/fetcher-engine-overview) owns no infrastructure. It reaches the outside world only through **ports** — Go interfaces your host application implements and passes to `engine.New`.

This page is the reference for all eight. The **Without it** column is the point of the page. Graceful degradation is the contract you plan against, so read it before you skip a port.

## The ports at a glance

***

| Port                     | Required                   | Without it                                                                                              |
| ------------------------ | -------------------------- | ------------------------------------------------------------------------------------------------------- |
| `ConnectorRegistry`      | **Always**                 | `engine.New` fails. No Engine exists.                                                                   |
| `CredentialProtector`    | With encrypted persistence | `engine.New` fails when encrypted persistence is on. Otherwise credentials reach the store unprotected. |
| `ConnectionStore`        | Optional                   | Every operation except `Limits()` fails.                                                                |
| `ExecutionStore`         | Optional                   | No durable execution-state tracking.                                                                    |
| `ResultSink`             | Optional                   | Store mode is unavailable. Extraction runs in direct mode.                                              |
| `SchemaCache`            | Optional                   | Schema discovery always hits the live datasource.                                                       |
| `ActiveExecutionChecker` | Optional                   | No conflict gating. Connection updates and deletes always proceed.                                      |
| `Observability`          | Optional                   | Tracing hooks become no-ops.                                                                            |

<Note>
  `engine.New` rejects a port passed as a **typed nil**, not only a literal nil. An interface value that wraps a nil pointer fails at construction with a clear validation error, instead of panicking at first use.
</Note>

## ConnectorRegistry

***

**Required: always.** This is the only port the Engine validates unconditionally.

The registry resolves a connector factory by datasource type. It performs no I/O, it resolves deterministically by type, and it reports `ok=false` for a type nobody registered. Building and connecting a connector happens later, through the factory it returned.

**Without it:** `engine.New` returns a validation error with the message `connector registry is required`. You get no Engine value at all, so extraction is impossible.

## CredentialProtector

***

**Required: only with `WithEncryptedPersistence(true)`.**

The protector encrypts and decrypts credential material for a tenant. The `Protect` call returns the protected bytes plus the key version that protected them. The `Reveal` call decrypts with a given key version, so your host can resolve a rotated key.

Your host owns key derivation, rotation, and storage. The Engine only calls the seam and records the returned key version as secret-free metadata.

**Without it:**

* With encrypted persistence **on**, `engine.New` fails with `credential protector is required when encrypted persistence is enabled`. The Engine refuses to construct rather than persist plaintext credentials.
* With encrypted persistence **off**, the port is genuinely optional, and credentials reach the `ConnectionStore` without protection.

## ConnectionStore

***

**Required: optional at construction, load-bearing at runtime.**

The store persists and resolves connection descriptors owned by a tenant. It is the only persistence seam the connection operations use — the Engine embeds no MongoDB, no SQL, and no host repository. It exposes nine operations: create, find, find-by-id, update, update-by-id, delete, delete-by-id, list, and list-paged.

Two obligations fall on your implementation. It **must** scope every record by the tenant ID, so one tenant never sees another tenant's connections. It **must not** return secret material — the connection descriptor carries none.

**Without it:** `engine.New` succeeds, and then nearly everything fails. Every operation that touches a connection returns the validation error `connection store is not configured`. That covers all nine connection operations plus plan, execute, discover-schema, fresh-schema discovery, validate-schema, and test-connection. An Engine without a connection store can report only its limits.

<Warning>
  Do not read `ConnectionStore` as a CRUD-only convenience. Skipping it disables extraction and schema discovery too.
</Warning>

## ExecutionStore

***

**Required: optional.**

The store upserts execution lifecycle state for a tenant. The Engine writes the transitions synchronously and inline: `running`, then `completed`, `failed`, or `canceled`.

Those writes are **best-effort by design**. The Engine discards a save error, so optional persistence can never corrupt an extraction result. A result-sink write failure behaves differently and does fail the execution. The two are deliberately distinct.

**Without it:** the Engine runs with no durable execution tracking, and your host owns execution state externally.

## ResultSink

***

**Required: optional. It selects the result mode.**

The sink persists result payloads to host-managed storage. Store-mode extraction calls `OpenResultStream`, so the Engine writes the result incrementally in constant memory. `PersistResult` remains for whole-payload writes.

The stream shape is contractual NDJSON — one JSON object per line, newline-terminated, with no enclosing array:

```json theme={null}
{"config":"<configName>","table":"<qualifiedTable>","row":{"<col>":"<val>"}}
```

Lines come out in deterministic order: steps by ascending plan-step ordinal, rows within a step in cursor order. The same input therefore produces byte-identical NDJSON and the same SHA-256 digest on every run.

On an abort — a write error, an exceeded size limit, or a canceled context — the Engine abandons the writer and never calls `Close`. Treat an unclosed writer as a discarded write, because a partial result must never become a returned reference.

**Without it:** store mode is unavailable. The default `auto` mode resolves to direct, so extraction returns inline bytes and persists nothing. An explicit store-mode request fails up front with `store mode requires a configured result sink`, before the Engine builds any connector.

## SchemaCache

***

**Required: optional.**

The cache stores and returns schema snapshots per tenant and config name.

The Engine treats it as an accelerator, never as a source of truth. A failed cache read degrades to fresh discovery. A failed cache write still returns the discovered schema to the caller. The always-fresh discovery call ignores the cache on every invocation, even when you wired one, so the live-datasource contract of the Manager schema endpoint holds.

**Without it:** the Engine discovers schema live from the datasource on every call.

## ActiveExecutionChecker

***

**Required: optional.**

The checker reports whether a connection currently has active executions. The Engine consults it **before** it mutates a connection. A host can therefore keep the Manager behavior, which blocks a change while jobs run against that connection.

The port is deliberately **logical**, not a durable job store. Your host decides how to answer: a job repository, an in-memory tracker, a distributed lock, or always false. The Engine never imports a job repository to ask the question. The connection identity it passes is the config name inside the tenant scope. Your answer **must** be tenant-scoped, so one tenant's running work never blocks another tenant's mutation.

**Without it:** the Engine performs no conflict gating, and connection updates and deletes proceed unconditionally.

## Observability

***

**Required: optional.**

The contract has one method. `StartSpan` takes a context and an operation name, and returns a derived context plus an end function the Engine defers. One method is the whole point: the Engine core never imports a tracing library, and your host adapts its own tracer behind the seam.

**Without it:** span creation returns the incoming context and a no-op end function. Tracing hooks disappear with no other change in behavior.

## Next steps

***

<CardGroup cols={2}>
  <Card title="Embed the Engine" icon="code" href="/en/fetcher/fetcher-embedding-the-engine">
    Import, provide the ports, and construct with a runnable example.
  </Card>

  <Card title="Engine overview" icon="cube" href="/en/fetcher/fetcher-engine-overview">
    The three-layer model, the import boundary, and the two result modes.
  </Card>
</CardGroup>