Skip to main content

​
Why this matters

When a client sends a transaction, two things need to happen: validate it and persist the results. In synchronous mode, both happen in the same request — the client waits until everything is written to the database before getting a response. That’s simple and predictable, but it has a ceiling. At high volumes, database writes become the bottleneck. Every transaction holds a connection, waits for locks, and competes for I/O. Async mode breaks that dependency. The transaction is validated, the response is returned immediately, and the persistence happens in the background through RabbitMQ. The client gets faster responses. The database gets writes in controlled, optimized batches. The system handles more with less. For broader scaling guidance, see Scalability strategies.

​
How it works

​
Synchronous mode (default)

The transaction is validated and written directly to PostgreSQL in the same request cycle. The API response is only sent after all database operations complete.
Sequence diagram showing the client sending a POST /transaction to the Midaz API, which validates it, writes the transaction, operations, and balances to PostgreSQL, waits for confirmation, and only then returns 200 OK to the client.
Here’s the full flow, step by step:
  1. The client sends a POST /transaction to the Midaz API.
  2. The API validates the request — DSL parsing, balance checks, and limit enforcement all happen here.
  3. The API writes to PostgreSQL — transaction, operations, and balance updates are persisted in the same request cycle.
  4. PostgreSQL confirms the write, acknowledging that all records are committed.
  5. The API returns 200 OK to the client with the created transaction. The response only leaves the server after the database has confirmed everything.
Characteristics:
  • Response time includes database write latency.
  • Each transaction is an independent database operation.
  • Simpler to reason about — what you see in the response is what’s persisted.

​
Asynchronous mode

The transaction is validated the same way, but instead of writing to the database, Midaz publishes a message to RabbitMQ. A background consumer picks up the message and handles persistence separately.
Sequence diagram showing the client sending a POST /transaction to the Midaz API, which validates it, publishes the payload to RabbitMQ, and immediately returns 200 OK to the client. In parallel, RabbitMQ delivers the message to a background consumer, which writes the transaction, operations, and balances to PostgreSQL.
Here’s the full flow, step by step:
  1. The client sends a POST /transaction to the Midaz API.
  2. The API validates the request — DSL parsing, balance checks, and limit enforcement happen exactly as in synchronous mode.
  3. The API publishes the transaction payload to RabbitMQ instead of writing to the database directly.
  4. The API returns 200 OK to the client immediately after the message is accepted by the queue — the client doesn’t wait for database persistence.
  5. RabbitMQ delivers the message to a background consumer, decoupled from the API request.
  6. The consumer writes to PostgreSQL — transaction, operations, and balance updates are persisted separately from the original request.
Characteristics:
  • Response time excludes database write latency — the client only waits for validation and queue publish.
  • Messages are serialized with MessagePack for compact, efficient transport.
  • Background consumers write to the database at their own pace, with batching and retry capabilities.
The validation step is identical in both modes. Balance checks, DSL parsing, limit enforcement — all of that happens before the API responds, regardless of processing mode. The difference is only in when the data hits the database.

​
Built-in resilience

If RabbitMQ is unavailable when async mode tries to publish a message, Midaz doesn’t fail the transaction. Instead, it falls back automatically to a direct database write — the same path as synchronous mode. This means:
  • No transaction is lost because of a queue outage.
  • The client still gets a successful response.
  • The fallback is logged so your operations team can investigate the queue issue.
The automatic fallback ensures data safety, but it also means latency will spike during a queue outage (since writes go directly to the database). Monitor your RabbitMQ health to keep async mode operating as intended.

​
Enabling async mode

Set one environment variable in the ledger application: 
RABBITMQ_TRANSACTION_ASYNC=true
When set to false (the default), all transactions use synchronous processing. No RabbitMQ consumer is needed. When set to true, the ledger publishes transaction payloads to the configured RabbitMQ exchange and a background consumer handles persistence.

​
RabbitMQ configuration

Async mode uses the following RabbitMQ settings (all in the ledger .env):
VariableDescriptionDefault
RABBITMQ_TRANSACTION_ASYNCEnable async processing.false
RABBITMQ_HOSTRabbitMQ server hostname.midaz-rabbitmq
RABBITMQ_PORT_HOSTManagement API port.3003
RABBITMQ_PORT_AMQPAMQP protocol port.3004
RABBITMQ_DEFAULT_USERProducer credentials (user).transaction
RABBITMQ_DEFAULT_PASSProducer credentials (password).—
RABBITMQ_CONSUMER_USERConsumer credentials (user).consumer
RABBITMQ_CONSUMER_PASSConsumer credentials (password).—
RABBITMQ_NUMBERS_OF_WORKERSNumber of consumer worker goroutines.5
RABBITMQ_NUMBERS_OF_PREFETCHMessages prefetched per worker.10
RABBITMQ_TRANSACTION_BALANCE_OPERATION_EXCHANGEExchange name for transaction messages.transaction.transaction_balance_operation.exchange
RABBITMQ_TRANSACTION_BALANCE_OPERATION_KEYRouting key.transaction.transaction_balance_operation.key
RABBITMQ_TRANSACTION_BALANCE_OPERATION_QUEUEQueue name.transaction.transaction_balance_operation.queue
The consumer uses separate credentials (RABBITMQ_CONSUMER_USER / RABBITMQ_CONSUMER_PASS) from the producer. This follows the principle of least privilege — the consumer only needs read access to the queue.

​
Balance synchronization

When running in async mode, balance updates are coordinated through a dedicated synchronization worker that uses Redis as a coordination layer. This ensures balances remain consistent even when multiple consumers process messages concurrently.
VariableDescriptionDefault
BALANCE_SYNC_BATCH_SIZENumber of balance updates to batch before flushing.50
BALANCE_SYNC_FLUSH_TIMEOUT_MSMaximum wait time (ms) before flushing an incomplete batch.500
BALANCE_SYNC_POLL_INTERVAL_MSHow often (ms) the worker checks for pending updates.50
The balance sync worker runs automatically when async mode is enabled. No additional setup is required beyond having Redis available.

​
RabbitMQ circuit breaker

When async mode is enabled, Midaz depends on RabbitMQ for transaction persistence. To protect against broker outages, Midaz includes a built-in circuit breaker that monitors the health of the RabbitMQ connection and fails fast when the broker is unavailable — preventing request pileups and cascading failures. The circuit breaker is always active when RabbitMQ is in use. It follows the standard three-state model:
  • Closed (normal): requests flow through to RabbitMQ. Failures are counted.
  • Open (tripped): requests are rejected immediately without contacting RabbitMQ. A background health checker monitors the broker and attempts recovery.
  • Half-open (probing): a limited number of requests are allowed through to test if RabbitMQ has recovered. If they succeed, the circuit closes. If they fail, it reopens.
The circuit opens when either condition is met:
  • The number of consecutive failures reaches the threshold, OR
  • The failure ratio exceeds the configured percentage within the counting window

​
Circuit breaker configuration

VariableDescriptionDefault
RABBITMQ_CIRCUIT_BREAKER_CONSECUTIVE_FAILURESConsecutive failures before the circuit opens.15
RABBITMQ_CIRCUIT_BREAKER_FAILURE_RATIOFailure percentage (0–100) that triggers open state.50
RABBITMQ_CIRCUIT_BREAKER_MIN_REQUESTSMinimum requests before evaluating the failure ratio.10
RABBITMQ_CIRCUIT_BREAKER_INTERVALTime window (seconds) for counting failures. Counters reset after each interval.120
RABBITMQ_CIRCUIT_BREAKER_TIMEOUTHow long (seconds) the circuit stays open before transitioning to half-open.30
RABBITMQ_CIRCUIT_BREAKER_MAX_REQUESTSRequests allowed through in half-open state to probe recovery.3
RABBITMQ_CIRCUIT_BREAKER_HEALTH_CHECK_INTERVALHow often (seconds) the background health checker pings RabbitMQ.30
RABBITMQ_CIRCUIT_BREAKER_HEALTH_CHECK_TIMEOUTTimeout (seconds) for each health check ping.10
When the circuit is open, async transactions fall back to direct synchronous database writes — the transaction is not lost. This fallback ensures data integrity even during broker outages.
For most production deployments, the defaults work well. Tune CONSECUTIVE_FAILURES and TIMEOUT if your RabbitMQ cluster has known recovery patterns — for example, lower the timeout if your broker typically recovers within seconds, or increase consecutive failures if you experience transient network blips.

​
How async mode connects to Bulk Recorder

Async mode and the Bulk Recorder are complementary features that work together:
  1. Async mode decouples the API response from persistence — transactions go to RabbitMQ instead of directly to PostgreSQL.
  2. Bulk Recorder optimizes how the consumer writes those messages to the database — batching multiple messages into single bulk inserts.
Bulk Recorder activates only when async mode is enabled (RABBITMQ_TRANSACTION_ASYNC=true AND BULK_RECORDER_ENABLED=true). Without async mode, there’s no message queue — and without a queue, there’s nothing to batch.
ConfigurationProcessing behavior
Async falseDirect database write per transaction (synchronous)
Async true, Bulk Recorder falseQueue-based, one message processed at a time
Async true, Bulk Recorder trueQueue-based, messages batched for bulk inserts (10×+ throughput)

​
When to use async mode

Use async mode when:
  • You need lower API response times for transaction creation.
  • Your workload involves high transaction volumes (hundreds+ per second).
  • You’re running batch operations like mass payouts or settlements.
  • You want to decouple your API tier from database performance.
Keep synchronous mode when:
  • You need the simplest possible setup (no RabbitMQ dependency).
  • Transaction volume is low to moderate.
  • You want the guarantee that a successful API response means the data is already persisted.
  • You’re in a development or testing environment where simplicity matters more than throughput.
You can switch between modes at any time by changing RABBITMQ_TRANSACTION_ASYNC and restarting the ledger application. No data migration is needed — the transaction format is the same in both paths.