Discovery

Discovery automates data source detection and extraction through Matcher’s embedded extraction engine. Instead of manually uploading files, Discovery connects to external systems, identifies available data, and extracts transactions directly into Matcher.

What Discovery solves

Manual file uploads create friction at every step. Teams export files, transfer them, monitor for failures, and re-upload when something goes wrong. This process is time-consuming, error-prone, and breaks when data volume grows. Discovery replaces the manual pipeline. It connects to external systems through the extraction engine, reads their schemas, and pulls transactions into Matcher on demand.

How Discovery works

Discovery runs inside Matcher. The embedded engine manages connections to external databases and runs extractions locally. Discovery exposes those connections and coordinates the extraction process, handing results directly to Ingestion. The workflow has eight steps:

Create a connection: Register a database with its address and user, or a REST connection to a provider API.
Check status: See how many connections the tenant has and when they last synced.
Browse connections: See the tenant’s database connections.
Inspect a connection: Review the schema to understand what fields are available.
Test a connection: Validate the connection before committing to an extraction.
Create an extraction: Request that Matcher pull data from a specific source.
Monitor progress: Track extraction status as data flows in.
Refresh connections: Re-sync connections from the embedded engine.

Discovery workflow

Create a connection

curl -X POST "https://api.matcher.example.com/v1/discovery/connections" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "configName": "prod-ledger-db",
    "databaseType": "POSTGRESQL",
    "host": "db.internal.example.com",
    "port": 5432,
    "databaseName": "ledger",
    "userName": "matcher_ro",
    "password": "..."
  }'

configName, databaseType, host, port, databaseName, and userName are required. schema and password are optional, and Matcher never returns password. A successful create returns 201. Update a connection with PATCH and delete it with DELETE on /v1/discovery/connections/{connectionId}.

Create a REST connection

Create a REST connection to extract data from a provider API instead of a database.

curl -X POST "https://api.matcher.example.com/v1/discovery/rest-connections" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "configName": "stripe-main",
    "connectorType": "STRIPE",
    "apiKey": "..."
  }'

configName and connectorType are required. connectorType is PAGBANK, STRIPE, or PIX_BCB. baseURL is optional and sets the provider API address for this connection. Each connector type also requires its own credential fields:

`connectorType`	Required credential fields
`PAGBANK`	`establishmentID`, `ediToken`
`STRIPE`	`apiKey`
`PIX_BCB`	`clientID`, `clientSecret`, `clientCert`, `clientKey`

Matcher stores the credentials encrypted. List connector types returns the same field names as requiredFields. A successful create returns 201, and the connection appears in the connection list like any other connection. On a REST connection, PATCH accepts only configName. To change anything else, delete the connection and create it again.

Check Discovery status

Read the tenant’s connection count and the time of the last sync.

curl -X GET "https://api.matcher.example.com/v1/discovery/status" \
  -H "Authorization: Bearer $TOKEN"

API Reference: Get Discovery status

Browse connections

List all data sources available through the embedded extraction engine.

curl -X GET "https://api.matcher.example.com/v1/discovery/connections" \
  -H "Authorization: Bearer $TOKEN"

The response lists each database connection with its configName, databaseType, host, database, status, and whether its schema was discovered.

API Reference: List connections

Get a connection

Retrieve a single discovered connection by its internal identifier:

curl -X GET "https://api.matcher.example.com/v1/discovery/connections/{connectionId}" \
  -H "Authorization: Bearer $TOKEN"

GET /v1/discovery/connections/{connectionId} returns the same fields for one connection. Use it when you already hold a connectionId, for example from a source binding’s query rail. It gives the current details without a list of every connection.

API Reference: Get discovery connection

Inspect a connection

Before you extract, review the schema of a specific connection to understand what data fields are available.

curl -X GET "https://api.matcher.example.com/v1/discovery/connections/{connectionId}/schema" \
  -H "Authorization: Bearer $TOKEN"

Use schema inspection to confirm that required fields (transaction IDs, amounts, dates, references) exist before building field mappings.

API Reference: Get connection schema

Get schema history

Read the earlier generations of a connection’s schema, oldest first.

curl -X GET "https://api.matcher.example.com/v1/discovery/connections/{connectionId}/schema/history" \
  -H "Authorization: Bearer $TOKEN"

The response carries connectionId and a generations list. Each generation carries archivedAt, the time it stopped being the live schema, and its tables. When all tables of a generation share one cause, the generation also carries retirementCause:

`retirementCause`	Meaning
`ABSENT_FROM_CATALOG`	The catalog of the source stopped listing the table.
`COLUMN_SET_SUPERSEDED`	The catalog still listed the table, but with a different set of columns.
`CONNECTION_RETIRED`	Someone retired the connection.
`UNKNOWN_LEGACY`	An earlier release archived the generation and recorded no cause.

An unknown connection returns 404.

Test a connection

Validate that Matcher can reach and read from a connection before creating an extraction.

curl -X POST "https://api.matcher.example.com/v1/discovery/connections/{connectionId}/test" \
  -H "Authorization: Bearer $TOKEN"

The test connects to the database and pings it. It does not check read access to tables. A failed test returns 200 with "healthy": false and an errorMessage. Always test before creating an extraction, especially for new or recently modified connections.

API Reference: Test connection

Create an extraction

Request that Matcher pull data from a connection. The request names no context. Matcher ingests the result into the FETCHER source linked to the connection.

curl -X POST "https://api.matcher.example.com/v1/discovery/connections/{connectionId}/extractions" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "tables": {
      "public.transactions": {}
    }
  }'

Each key in tables must be a table name exactly as the connection’s schema lists it. The response returns an extraction ID. Use it to monitor progress.

API Reference: Create extraction

Monitor extraction progress

Track the status of an active extraction by polling its status with GET.

curl -X GET "https://api.matcher.example.com/v1/discovery/extractions/{extractionId}" \
  -H "Authorization: Bearer $TOKEN"

Extraction status transitions from PENDING → SUBMITTED → EXTRACTING → COMPLETE (or FAILED). An extraction fails when its result exceeds 2 GiB. The response carries the extraction status, an errorMessage when it failed, and the linked ingestionJobId once the extraction bridges into ingestion.

API Reference: Get extraction

Refresh available connections

Refresh re-syncs the tenant’s connections from the embedded engine. A connection that the engine no longer reports becomes UNREACHABLE.

curl -X POST "https://api.matcher.example.com/v1/discovery/refresh" \
  -H "Authorization: Bearer $TOKEN"

API Reference: Refresh connections

List connector types

List the connector (datasource) types the engine registry has registered for your environment. Each entry carries a backend-derived category (database or rest). The registry is live. Only connectors registered at boot appear. This list excludes aggregator vendors (Pluggy/Belvo). Provision those through the aggregator-connections surface below.

curl -X GET "https://api.matcher.example.com/v1/discovery/connector-types" \
  -H "Authorization: Bearer $TOKEN"

Response

{
  "types": [
    { "type": "POSTGRESQL", "category": "database" },
    { "type": "MYSQL", "category": "database" }
  ]
}

Aggregator connections (Open Finance)

Open-Finance data-aggregator connections (Pluggy or Belvo) let Matcher pull transactions from bank aggregators. Credential material (clientId/secret) is sealed on write and never returned. Every read is secret-free by construction.

Create an aggregator connection

curl -X POST "https://api.matcher.example.com/v1/discovery/aggregator-connections" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "vendor": "pluggy",
    "configName": "pluggy-main",
    "baseUrl": "https://api.pluggy.ai",
    "accountRef": "a1b2c3d4-5678-90ab-cdef-1234567890ab",
    "clientId": "...",
    "secret": "..."
  }'

Send five required fields: vendor, configName, baseUrl, clientId, and secret. The accountRef field (the Pluggy item id or Belvo link id) is optional. Without it, Matcher creates a connection that waits for consent. See Connect an account through hosted consent. The vendor field is one of pluggy or belvo. The configName field is the tenant-scoped name the webhook token-mint endpoint binds to. A successful create returns 201 with a secret-free connection.

Response

{
  "vendor": "pluggy",
  "configName": "pluggy-main",
  "baseUrl": "https://api.pluggy.ai",
  "accountRef": "a1b2c3d4-5678-90ab-cdef-1234567890ab",
  "awaitingConsent": false
}

List, get, update, and delete

# List (cursor-paginated, secret-free)
curl -X GET "https://api.matcher.example.com/v1/discovery/aggregator-connections" \
  -H "Authorization: Bearer $TOKEN"

# Get one by id
curl -X GET "https://api.matcher.example.com/v1/discovery/aggregator-connections/{id}" \
  -H "Authorization: Bearer $TOKEN"

# Update (PUT). vendor is immutable. Supply clientId+secret together to rotate
# the sealed credential, or omit both to keep the stored secret intact.
curl -X PUT "https://api.matcher.example.com/v1/discovery/aggregator-connections/{id}" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "configName": "pluggy-main",
    "baseUrl": "https://api.pluggy.ai",
    "accountRef": "a1b2c3d4-5678-90ab-cdef-1234567890ab"
  }'

# Delete (soft-delete; frees the config name for reuse). Returns 204.
curl -X DELETE "https://api.matcher.example.com/v1/discovery/aggregator-connections/{id}" \
  -H "Authorization: Bearer $TOKEN"

Use this flow when you do not have the vendor’s item or link id yet. The end customer authorizes access in the vendor’s own consent widget.

Create the connection without accountRef. The response carries "awaitingConsent": true.
Get the connection id from the list response.
Request a consent token for the connection. The request has no body.
Open the vendor’s consent widget with the token from the response.
Send the item or link id that the widget returns as accountRef in PUT /v1/discovery/aggregator-connections/{id}. You do not need to send other fields.

curl -X POST "https://api.matcher.example.com/v1/discovery/aggregator-connections/{id}/connect-token" \
  -H "Authorization: Bearer $TOKEN"

The 201 response carries vendor, configName, token, expiresAt, reconsent, and accountRef. For a connection that already has an accountRef, the token renews consent for that same item or link. Then reconsent is true, and accountRef holds the bound id. Give that accountRef to the update mode of the widget. The answer shows the consent token once, and Matcher does not store it. Use it before expiresAt. After that time, the vendor refuses it. A connection id that is not an aggregator connection returns 404. If the vendor does not issue a token, the call returns 422 (MTCH-0212). If Matcher has no consent flow for the vendor, the call returns 422 (MTCH-0213).

Test an aggregator connection

Run a live connectivity check against an existing connection’s already-sealed credential. Send only the configName: Matcher reads the vendor from the stored connection. This call takes no credential and returns none. Invalid stored credentials give an expected test result: 200 with "healthy": false, not an error. A missing connection returns 404. Matcher cannot test Belvo connections, so a Belvo connection returns 422 (MTCH-0210). A connection that waits for consent returns 422 (MTCH-0211). In these cases, no test runs. Each row of the list response carries testable, which tells you if a test can run.

curl -X POST "https://api.matcher.example.com/v1/discovery/aggregator-connections/test" \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "configName": "pluggy-main"
  }'

Response

{
  "vendor": "pluggy",
  "configName": "pluggy-main",
  "healthy": true
}

Aggregator webhook tokens

Aggregators push data change signals to Matcher via webhooks. Mint an opaque token bound to an aggregator connection, then configure the returned URL in the vendor dashboard.

Mint a webhook token

Matcher returns the raw token and its provider-facing URL exactly once. Matcher stores only the token’s SHA-256 hash.

curl -X POST "https://api.matcher.example.com/v1/discovery/webhooks/tokens" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "vendor": "pluggy",
    "connection_config_name": "pluggy-main"
  }'

Response

{
  "vendor": "pluggy",
  "token": "<raw-token-shown-once>",
  "webhook_url": "https://api.matcher.example.com/v1/discovery/webhooks/pluggy/<raw-token>"
}

Receiving webhooks

The vendor calls POST /v1/discovery/webhooks/{provider}/{webhookToken} (no operator JWT). Two layers authenticate it: the opaque path token plus a per-provider source check. When the provider has an HMAC secret, that check is a valid hex HMAC-SHA256 of the raw body in the X-Webhook-Signature header. Otherwise it is membership in the provider’s source-IP allowlist. Both layers fail closed. A valid first delivery returns 202 Accepted. Matcher then pulls the signaled data asynchronously into the ingestion pipeline. A redelivery of an event that Matcher already received returns 200 OK.

Best practices

Always test connections before extracting

A failed extraction mid-run is harder to recover from than a failed test. Test every connection before creating an extraction, especially when connecting to a new source or after a credential rotation.

Inspect schemas before mapping fields

Field names vary across systems. A bank might call the transaction date value_date while your ledger uses posting_date. Check the schema before configuring field mappings to avoid silent mismatches.

Monitor extractions actively for large datasets

Large extractions take time. Don’t assume completion. Poll the extraction status, then check the linked ingestion job before starting a match run. Starting a run on incomplete data generates incorrect exceptions.

Next steps

External sources

Configure the external data sources that Discovery connects to.

Field mapping

Map fields from extracted data to Matcher’s transaction model.

Discovery API reference

Full API reference for Discovery endpoints.

Overview

Guides

Console

MCP Server

Operations

What Discovery solves

How Discovery works

Discovery workflow

Create a connection

Create a REST connection

Check Discovery status

Browse connections

Get a connection

Inspect a connection

Get schema history

Test a connection

Create an extraction

Monitor extraction progress

Refresh available connections

List connector types

Response

Aggregator connections (Open Finance)

Create an aggregator connection

Response

List, get, update, and delete

Test an aggregator connection

Response

Aggregator webhook tokens

Mint a webhook token

Response

Receiving webhooks

Best practices

Next steps

External sources

Field mapping

Discovery API reference

​What Discovery solves

​How Discovery works

​Discovery workflow

​Create a connection

​Create a REST connection

​Check Discovery status

​Browse connections

​Get a connection

​Inspect a connection

​Get schema history

​Test a connection

​Create an extraction

​Monitor extraction progress

​Refresh available connections

​List connector types

​Response

​Aggregator connections (Open Finance)

​Create an aggregator connection

​Response

​List, get, update, and delete

​Connect an account through hosted consent

​Test an aggregator connection

​Response

​Aggregator webhook tokens

​Mint a webhook token

​Response

​Receiving webhooks

​Best practices

​Next steps

External sources

Field mapping

Discovery API reference

What Discovery solves

How Discovery works

Discovery workflow

Create a connection

Create a REST connection

Check Discovery status

Browse connections

Get a connection

Inspect a connection

Get schema history

Test a connection

Create an extraction

Monitor extraction progress

Refresh available connections

List connector types

Response

Aggregator connections (Open Finance)

Create an aggregator connection

Response

List, get, update, and delete

Connect an account through hosted consent

Test an aggregator connection

Response

Aggregator webhook tokens

Mint a webhook token

Response

Receiving webhooks

Best practices

Next steps