Using Reporter

Use this guide for the recurring Reporter workflow: manage a template, generate a report, verify the result, and download the finished file.

Prerequisites

Before you begin, make sure that:

Reporter runs and you can authenticate with its API.
An operator has configured at least one data source for the data your template queries.
You have a .tpl file that matches the intended output format. For PDF output, author the template as HTML. See Template examples and the template reference.

Manage templates

Reporter uses uploaded .tpl files to define report content and layout.

Upload a template

Call Upload a template as a multipart request with all four required fields:

template: the .tpl file.
outputFormat: the generated file format, such as HTML, PDF, XML, CSV, or TXT.
description: a human-readable description of the template.
category: the report family, such as cadoc. Use lowercase letters, digits, underscores, and hyphens, up to 100 characters.

Reporter returns the template identifier that you use when generating reports.

Maintain existing templates

Use the template endpoints to:

Deleting a template is a soft delete. Reporter excludes it from standard queries but preserves reports already created from it.

Generate a report with filters

Call Create a report with both required fields:

templateId: the identifier returned when you uploaded the template.
filters: the conditions grouped by data source, table, and field.

The following request limits the report to one transaction:

{
  "templateId": "0196159b-4f26-7300-b3d9-f4f68a7c85f3",
  "filters": {
    "midaz_transaction": {
      "transaction": {
        "id": {
          "eq": ["0196d983-a2c2-7d5a-a5b7-029fe0dcb710"]
        }
      }
    }
  }
}

To generate a report without filtering rows, send an empty object. Don’t omit the field:

{
  "templateId": "0196159b-4f26-7300-b3d9-f4f68a7c85f3",
  "filters": {}
}

Reporter returns the report identifier in the id field. Store this value as REPORT_ID to check the generation status and retrieve the output. See Advanced filtering for the supported operators and filter structure.

Discover data source schemas

Inspect the configured data sources before building templates or dynamic filter interfaces:

List data sources returns a page of registered connections without credentials.
Retrieve a data source returns one connection configuration by dataSourceId. Credentials remain hidden.
Get a data source schema introspects the live tables or collections and their typed fields.

The API owns the registry lifecycle: create a data source, partially update it, test the connection, inspect its schema, or soft-delete it. Reporter refuses deletion while a live Template still references the source. A self-hosted deployment can also seed entries from DATASOURCE_* variables at startup.

Interpret report statuses and errors

Call Check report status with REPORT_ID.

Status	Meaning	What to do
`Processing`	Reporter is generating the file.	Keep polling with a reasonable interval.
`Finished`	Generation completed successfully.	Download the report.
`Partial`	Reporter generated only part of the requested output.	Inspect the response details and correct the failed data sections before generating again.
`Error`	Generation failed.	Inspect the error details, template, filters, and data source availability.

Treat Finished and Final reports, and a Draft with no required field blank, as downloadable. A Partial result requires investigation even when Reporter produced some data.

Verify and download the report

When the status is Finished:

Call Download a report with REPORT_ID.
Confirm that the response has the expected content type and Content-Disposition header.
Open the file and verify that its data and layout match the template and filters.

Troubleshooting

Symptom	Check
Template upload is rejected	Send `template`, `outputFormat`, `description`, and `category`, and confirm that the file uses the `.tpl` extension.
Report creation is rejected	Send both `templateId` and `filters`. Use `"filters": {}` when you don’t need row filters.
Report remains in `Processing`	Check Worker health, RabbitMQ connectivity, and the referenced data sources.
Report ends as `Partial`	Inspect which data sections failed and verify their source, table, field, and filter names.
Report ends as `Error`	Check the returned error, template syntax, filter values, data source connectivity, and object storage.
Download is rejected	Check the latest status. Downloads are available for `Finished` and `Final` reports, and for a `Draft` with no required field blank.

Operator configuration

The following deployment settings are for operators. Application users don’t need them for the report generation workflow.

Configure object storage

Reporter stores templates and generated reports in one S3-compatible bucket. It uses the templates/ and reports/ prefixes. Reporter supports AWS S3, MinIO, and SeaweedFS.

Variable	Description	Default
`OBJECT_STORAGE_ENDPOINT`	S3-compatible endpoint. Leave empty for AWS S3.	—
`OBJECT_STORAGE_REGION`	AWS region.	`us-east-1`
`OBJECT_STORAGE_ACCESS_KEY_ID`	Access key.	—
`OBJECT_STORAGE_SECRET_KEY`	Secret key.	—
`OBJECT_STORAGE_USE_PATH_STYLE`	Uses path-style URLs. Commonly required by MinIO and SeaweedFS.	`false`
`OBJECT_STORAGE_DISABLE_SSL`	Uses HTTP instead of HTTPS when `OBJECT_STORAGE_ENDPOINT` has no scheme. An explicit `http://` or `https://` scheme takes precedence.	`false`
`OBJECT_STORAGE_BUCKET`	Bucket name.	`reporter-storage`

AWS S3

OBJECT_STORAGE_ENDPOINT=
OBJECT_STORAGE_REGION=us-west-2
OBJECT_STORAGE_ACCESS_KEY_ID=AKIA...
OBJECT_STORAGE_SECRET_KEY=your-secret-key
OBJECT_STORAGE_USE_PATH_STYLE=false
OBJECT_STORAGE_DISABLE_SSL=false
OBJECT_STORAGE_BUCKET=reporter-prod-bucket

The MinIO and SeaweedFS examples below use HTTP only for local development. Production deployments require HTTPS and TLS.

MinIO (local development)

OBJECT_STORAGE_ENDPOINT=http://minio:9000
OBJECT_STORAGE_REGION=us-east-1
OBJECT_STORAGE_ACCESS_KEY_ID=minioadmin
OBJECT_STORAGE_SECRET_KEY=minioadmin
OBJECT_STORAGE_USE_PATH_STYLE=true
OBJECT_STORAGE_DISABLE_SSL=true
OBJECT_STORAGE_BUCKET=reporter-storage

SeaweedFS (local development)

OBJECT_STORAGE_ENDPOINT=http://reporter-seaweedfs:8333
OBJECT_STORAGE_REGION=us-east-1
OBJECT_STORAGE_ACCESS_KEY_ID=any
OBJECT_STORAGE_SECRET_KEY=any
OBJECT_STORAGE_USE_PATH_STYLE=true
OBJECT_STORAGE_DISABLE_SSL=true
OBJECT_STORAGE_BUCKET=reporter-storage

S3 doesn’t support per-object TTL. Configure S3 bucket lifecycle policies if generated reports must expire automatically.

Configure external data sources

Set DATASOURCE_CRED_ENC_KEY to a persistent hexadecimal AES key before Reporter starts. Generate a 32-byte key with openssl rand -hex 32. Manager and worker fail startup when the key is absent or malformed. Keep the same key available to every Reporter runtime that reads the registry, because stored passwords are encrypted with it. Use the API for the normal data-source lifecycle. In a self-hosted deployment, you may also seed PostgreSQL or MongoDB entries at startup with DATASOURCE_<NAME>_* variables:

Variable	Description	Required
`DATASOURCE_<NAME>_CONFIG_NAME`	Identifier used in templates, such as `midaz_onboarding`.	Yes
`DATASOURCE_<NAME>_HOST`	Database host.	Yes
`DATASOURCE_<NAME>_PORT`	Database port.	Yes
`DATASOURCE_<NAME>_USER`	Database user.	Only when the database requires user authentication
`DATASOURCE_<NAME>_PASSWORD`	Database password.	Only when the database requires password authentication
`DATASOURCE_<NAME>_DATABASE`	Database name.	Yes
`DATASOURCE_<NAME>_TYPE`	`postgresql` or `mongodb`, in lowercase.	Yes
`DATASOURCE_<NAME>_SSLMODE`	PostgreSQL SSL mode, such as `disable` or `require`.	PostgreSQL only
`DATASOURCE_<NAME>_SSLROOTCERT`	PostgreSQL root certificate path.	PostgreSQL only
`DATASOURCE_<NAME>_SSL`	Enables MongoDB SSL.	MongoDB only
`DATASOURCE_<NAME>_SSLCA`	MongoDB CA certificate path.	MongoDB only
`DATASOURCE_<NAME>_SSLCA_BASE64`	Standard Base64 of a PEM CA bundle, as an alternative to a certificate file path. For PostgreSQL, set `SSLMODE` to `verify-ca` or `verify-full`, or leave it empty for `verify-full`.	No
`DATASOURCE_<NAME>_OPTIONS`	Additional MongoDB URI options.	MongoDB only
`DATASOURCE_<NAME>_ROW_CEILING`	Rows one table can return. Defaults to `DATASOURCE_DEFAULT_ROW_CEILING`.	No
`DATASOURCE_<NAME>_QUERY_TIMEOUT_SEC`	Query timeout in seconds. Defaults to `DATASOURCE_DEFAULT_QUERY_TIMEOUT_SEC`.	No
`DATASOURCE_<CONFIG_NAME>_SCHEMAS`	Comma-separated PostgreSQL schemas to expose. The variable prefix derives from `CONFIG_NAME`.	PostgreSQL only

For a source whose CONFIG_NAME is midaz_onboarding:

DATASOURCE_ONBOARDING_CONFIG_NAME=midaz_onboarding
DATASOURCE_ONBOARDING_HOST=midaz-postgres-replica
DATASOURCE_ONBOARDING_PORT=5702
DATASOURCE_ONBOARDING_USER=midaz
DATASOURCE_ONBOARDING_PASSWORD=CHANGE_ME
DATASOURCE_ONBOARDING_DATABASE=onboarding
DATASOURCE_ONBOARDING_TYPE=postgresql
DATASOURCE_ONBOARDING_SSLMODE=require

Reference it in a template by its CONFIG_NAME:

{% for account in midaz_onboarding.account %}
  {{ account.id }} - {{ account.name }}
{% endfor %}

For multiple PostgreSQL schemas, derive the schema variable from CONFIG_NAME. For example, external_db maps to DATASOURCE_EXTERNAL_DB_SCHEMAS:

DATASOURCE_EXTERNAL_CONFIG_NAME=external_db
DATASOURCE_EXTERNAL_HOST=external-postgres
DATASOURCE_EXTERNAL_PORT=5432
DATASOURCE_EXTERNAL_USER=db_user
DATASOURCE_EXTERNAL_PASSWORD=CHANGE_ME
DATASOURCE_EXTERNAL_DATABASE=external_database
DATASOURCE_EXTERNAL_TYPE=postgresql
DATASOURCE_EXTERNAL_SSLMODE=require
DATASOURCE_EXTERNAL_DB_SCHEMAS=sales,inventory,reporting

Use database:schema.table in templates and schema.table as the filter table key:

{% for order in external_db:sales.orders %}
  {{ order.id }} - {{ order.total }}
{% endfor %}

{
  "templateId": "00000000-0000-0000-0000-000000000000",
  "filters": {
    "external_db": {
      "sales.orders": {
        "created_at": { "gte": ["2025-01-01"] }
      }
    }
  }
}

When the schema variable isn’t set, Reporter uses the public schema. Manager and worker connect to a data source on demand, when a request needs it.

Overview

Concepts

Guides

Integration

Console

Operations

Using Reporter

Prerequisites

Manage templates

Upload a template

Maintain existing templates

Generate a report with filters

Discover data source schemas

Interpret report statuses and errors

Verify and download the report

Troubleshooting

Operator configuration

Configure object storage

Configure external data sources

​Prerequisites

​Manage templates

​Upload a template

​Maintain existing templates

​Generate a report with filters

​Discover data source schemas

​Interpret report statuses and errors

​Verify and download the report

​Troubleshooting

​Operator configuration

​Configure object storage

​Configure external data sources

​Related tasks

Prerequisites

Manage templates

Upload a template

Maintain existing templates

Generate a report with filters

Discover data source schemas

Interpret report statuses and errors

Verify and download the report

Troubleshooting

Operator configuration

Configure object storage

Configure external data sources

Related tasks