> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lerian.studio/llms.txt
> Use this file to discover all available pages before exploring further.

# Extraction jobs

> How a Fetcher extraction job describes datasources, tables, schemas, and fields — and what happens to the job from creation to a stored result.

An extraction job is an asynchronous request to read data out of one or more registered databases. The Manager accepts the job and queues it. A Worker runs the extraction and stores the result. This page explains the request you send and the path the job takes.

## The request

***

A job carries two parts: `dataRequest` and `metadata`.

```json theme={null}
{
  "dataRequest": {
    "mappedFields": {
      "my_postgres": {
        "public.accounts": ["id", "email", "created_at"],
        "accounting.invoices": ["*"]
      },
      "my_mongo": {
        "transactions": ["*"]
      }
    }
  },
  "metadata": {
    "source": "payments",
    "correlationId": "settlement-2026-07-13"
  }
}
```

`mappedFields` is the heart of the request. It maps each datasource config name to the tables you want, and each table to the fields you want.

`metadata.source` is **required**. It names the product that owns the job. The Manager rejects a request without it. The Manager also rejects the request when a referenced connection belongs to a different product, or belongs to no product at all. Internal datasources carry no product, so this check skips them.

## Field projection

***

Name the fields you want, or ask for all of them:

* A list of field names extracts exactly those fields.
* The single entry `["*"]` extracts every column of the table. The wildcard must be the only entry in the list.

Fetcher validates every named field against the discovered schema before it queries. An unknown field name fails the job and names the offending fields in the error. A table with no valid field also fails the job.

## Multiple datasources, tables, and schemas

***

One job can read from several datasources at the same time. Each datasource can contribute several tables. Each table can sit in a different schema.

Write a table name in one of two forms:

* **Unqualified** — `accounts`. Fetcher reads it from the engine default schema: `public` on PostgreSQL, `dbo` on SQL Server.
* **Schema-qualified** — `accounting.invoices`. The prefix before the dot is the schema. On Oracle it is the owner.

Fetcher collects the distinct schema prefixes in `mappedFields` and discovers only those namespaces. If any table name is unqualified, discovery adds the engine default schema as well: `public` on PostgreSQL, `dbo` on SQL Server.

Two engines do not take a schema list. MySQL treats the connected database as the namespace. MongoDB has collections instead of schemas, so a collection name is always unqualified.

## Limits

***

The extraction engine bounds every job. These are the default values:

| Bound                             | Default   |
| --------------------------------- | --------- |
| Datasources per job               | 10        |
| Tables per datasource             | 20        |
| Fields per table                  | 50        |
| Datasources extracted in parallel | 4         |
| Extraction timeout                | 5 minutes |
| Serialized result size            | 256 MiB   |

The Manager rejects a job with more than 10 datasources before it reaches the queue. A result over the size limit fails the job. Fetcher never returns or stores a truncated result. `ENGINE_MAX_RESULT_BYTES` lowers the size ceiling on the Worker. A request can lower a limit, but it can never raise one above the engine default.

## Duplicate jobs

***

The Manager computes a SHA-256 hash over the whole request — `dataRequest` and `metadata` together. It then looks for a job with the same hash created in the last **5 minutes**.

* A match returns the existing job with HTTP 200. No second extraction runs.
* No match creates a new job and returns HTTP 202.
* A match that already **failed** does not block a retry. The Manager creates a new job.

Metadata is part of the hash. Two requests that read the same data under different correlation IDs are different jobs.

## Job lifecycle

***

| Status       | Meaning                                                     |
| ------------ | ----------------------------------------------------------- |
| `pending`    | The Manager accepted the job and published it to the queue. |
| `processing` | A Worker claimed the job and is reading the datasources.    |
| `completed`  | The result is stored, and `job.completed` is published.     |
| `failed`     | The extraction stopped, and `job.failed` is published.      |

Before it creates the job, the Manager resolves every datasource name to a connection and opens a real connection to each one. A datasource that fails this test rejects the whole request. Internal datasources configured through environment variables skip the test.

A Worker claims a job only while the job is still `pending`. It then moves the job to `processing` and starts the extraction. Datasources run in parallel up to the concurrency bound. The run is fail-fast: the first datasource that fails ends the job, and Fetcher stores no partial result.

On success the Worker writes the result to object storage and records two values on the job: the result path and the HMAC signature of the result. It then publishes the terminal event.

## Next steps

***

<CardGroup cols={2}>
  <Card title="Filters" icon="filter" href="/en/fetcher/fetcher-filters">
    The ten filter operators and the value shape each one takes.
  </Card>

  <Card title="Datasources" icon="database" href="/en/fetcher/fetcher-datasources">
    What behaves differently on each of the five database engines.
  </Card>
</CardGroup>