> ## Documentation Index > Fetch the complete documentation index at: https://docs.lerian.studio/llms.txt > Use this file to discover all available pages before exploring further. # Fetcher configuration > The environment variables that shape a Fetcher deployment — encryption, infrastructure, event streaming, multi-tenancy, and readiness. Fetcher's Manager and Worker are configured entirely through environment variables — each component reads its own `.env` file (`components/manager/.env`, `components/worker/.env`). This page covers the variables that matter for an MVP deployment; the `.env.example` files in the [repository](https://github.com/LerianStudio/fetcher) are the exhaustive reference. ## Encryption keys *** | Variable | Component | Description | | --------------------- | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | `APP_ENC_KEY` | Manager + Worker | **Required.** Base64-encoded 32-byte master key. Generate with `make generate-master-key`. Both services must share the same value. | | `APP_ENC_KEY_VERSION` | Manager + Worker | Key rotation counter (default `1`). Increment when rotating; Fetcher tracks which version encrypted each credential. | From this single master key, Fetcher derives three independent keys via HKDF: credential encryption (AES-256-GCM), internal message signing between Manager and Worker (HMAC-SHA256), and external document signing so consumers can verify extracted data. Consumers derive the verification key with `make derive-key KEY=""`. ## Manager API *** | Variable | Description | Default | | -------------------------- | ------------------------------------------------------- | --------------- | | `SERVER_PORT` | HTTP port for the Manager API | `4006` | | `SCHEMA_CACHE_TTL_SECONDS` | How long discovered schemas stay cached | `300` | | `PLUGIN_AUTH_ENABLED` | Delegate authentication/authorization to Access Manager | `false` | | `PLUGIN_AUTH_ADDRESS` | Access Manager address when auth is enabled | — | | `LOG_LEVEL` | Log verbosity | `debug` (local) | ## Infrastructure *** Both services connect to MongoDB (metadata) and RabbitMQ (job queue); the Manager also uses Valkey/Redis for caching, and the Worker writes results to object storage. | Variable | Component | Description | | ---------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | --------------------------------------------------------------------------------------- | | `MONGO_HOST`, `MONGO_PORT`, `MONGO_NAME`, `MONGO_USER`, `MONGO_PASSWORD` | Manager + Worker | MongoDB metadata store | | `RABBITMQ_HOST`, `RABBITMQ_PORT_AMQP`, `RABBITMQ_DEFAULT_USER`, `RABBITMQ_DEFAULT_PASS` | Manager + Worker | RabbitMQ broker | | `RABBITMQ_FETCHER_WORK_QUEUE` | Manager + Worker | Extraction work queue (default `fetcher.extract-external-data.queue`) | | `RABBITMQ_NUMBERS_OF_WORKERS` | Worker | Concurrent extraction workers (default `5`) | | `REDIS_HOST`, `REDIS_PORT`, `REDIS_PASSWORD` | Manager | Valkey/Redis cache | | `OBJECT_STORAGE_ENDPOINT`, `OBJECT_STORAGE_REGION`, `OBJECT_STORAGE_BUCKET`, `OBJECT_STORAGE_ACCESS_KEY_ID`, `OBJECT_STORAGE_SECRET_KEY` | Worker | S3-compatible result storage (SeaweedFS by default; AWS S3 and MinIO work the same way) | | `ENGINE_MAX_RESULT_BYTES` | Worker | Cap on the serialized result artifact per job (default 256 MiB) | TLS-related options (`MONGO_TLS_CA_CERT`, `REDIS_TLS`, `RABBITMQ_TLS`) are available per dependency. ## Job event streaming *** The Worker publishes mandatory `job.completed` and `job.failed` notifications and **fails closed at startup** if streaming is not enabled: | Variable | Description | Default | | ------------------------------ | -------------------------------------- | -------------------- | | `STREAMING_ENABLED` | Must be `true` for the Worker to start | `false` | | `STREAMING_BROKERS` | Kafka/Redpanda bootstrap servers | `localhost:9092` | | `STREAMING_CLOUDEVENTS_SOURCE` | CloudEvents source for Worker events | — | | `RABBITMQ_JOB_EVENTS_EXCHANGE` | RabbitMQ exchange for job events | `fetcher.job.events` | Single-tenant deployments emit events with the stable tenant ID `single-tenant`. ## Multi-tenancy *** | Variable | Description | Default | | ------------------------------------------ | -------------------------------------- | ------- | | `MULTI_TENANT_ENABLED` | Enable multi-tenant mode | `false` | | `MULTI_TENANT_URL` | Tenant Manager service URL | — | | `MULTI_TENANT_MAX_TENANT_POOLS` | Max concurrent tenant connection pools | `100` | | `MULTI_TENANT_IDLE_TIMEOUT_SEC` | Idle tenant connection timeout | `300` | | `MULTI_TENANT_CIRCUIT_BREAKER_THRESHOLD` | Circuit breaker failure threshold | `5` | | `MULTI_TENANT_CIRCUIT_BREAKER_TIMEOUT_SEC` | Circuit breaker reset timeout | `30` | Enabling multi-tenant mode also activates the SSRF host-safety guard: tenant-supplied connection hosts are checked against loopback, private (RFC 1918), and cloud-metadata address ranges before Fetcher connects. With `MULTI_TENANT_ENABLED=false` (the default), all multi-tenant code paths are bypassed with zero performance impact. ## Readiness and TLS posture *** | Variable | Description | Default | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | | `DEPLOYMENT_MODE` | `saas`, `byoc`, or `local` — tags `/readyz` and drives SaaS TLS enforcement | `local` | | `ALLOW_INSECURE_TLS` | Permit plaintext connections to MongoDB, Redis, and RabbitMQ. Fetcher fails **closed** on insecure connection URIs by default — set this only for local development. | unset (fail closed) | | `READYZ_DRAIN_DELAY_SEC` | Graceful-drain window after SIGTERM, during which `/readyz` reports unhealthy before connections tear down | `12` | Both services expose `GET /health` (liveness), `GET /readyz` (readiness with parallel dependency probes), and `GET /version`. ## Internal datasources *** Fixed, operator-owned databases can be declared as env vars instead of API-registered connections, using the `DATASOURCE_{NAME}_*` family: | Variable | Required | Example | | ---------------------------- | -------- | ------------------------------------------------------------ | | `DATASOURCE_{N}_CONFIG_NAME` | yes | `billing_db` | | `DATASOURCE_{N}_TYPE` | yes | `postgresql` / `mysql` / `oracle` / `mongodb` / `sql_server` | | `DATASOURCE_{N}_HOST` | yes | `db.internal.example.com` | | `DATASOURCE_{N}_PORT` | yes | `5432` | | `DATASOURCE_{N}_DATABASE` | yes | `billing` | | `DATASOURCE_{N}_USER` | yes | `fetcher_ro` | | `DATASOURCE_{N}_PASSWORD` | yes | from your secret manager | | `DATASOURCE_{N}_OPTIONS` | no | `authSource=admin` (MongoDB only) | | `DATASOURCE_{N}_SSLMODE` | no | `require` (PostgreSQL) | For TLS to a managed database on this path, `_SSLMODE` is the validated option; custom CA or client-certificate plumbing for internal datasources is not available. ## Observability *** Fetcher emits OpenTelemetry traces and metrics when telemetry is enabled: | Variable | Description | Default | | ----------------------------- | -------------------------------------------------------- | ------------- | | `ENABLE_TELEMETRY` | Turn OTel emission on/off | `false` | | `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP collector endpoint | — | | `OTEL_RESOURCE_SERVICE_NAME` | Service name in telemetry (`fetcher` / `fetcher-worker`) | per component |