> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lerian.studio/llms.txt
> Use this file to discover all available pages before exploring further.
# Async transaction processing
> Async transaction processing validates the call fast, then persists the write through RabbitMQ, for lower API latency and higher throughput.
## Why this matters
***
A client sends a transaction. Midaz must do two things: validate it and persist the result. In synchronous mode, both steps run in the same request. The client waits for every write to reach the database before it gets a response.
This model is simple and predictable, but it has a ceiling. At high volume, database writes become the bottleneck. Each transaction holds a connection, waits for locks, and competes for I/O.
Async mode breaks that dependency. Midaz validates the transaction, returns the response at once, and persists the data in the background through RabbitMQ. The client gets faster responses. With async processing enabled, the Bulk Recorder batches inserts by default; set `BULK_RECORDER_ENABLED=false` to persist queued messages individually.
For broader scaling guidance, see [Scalability strategies](/en/midaz/scalability-strategies).
## How it works
***
### Synchronous mode (default)
Midaz validates the transaction and writes it directly to PostgreSQL in the same request cycle. The API sends its response only after every database operation completes.
Here's the full flow, step by step:
1. **The client sends** a `POST /transaction` to the Midaz API.
2. **The API validates the request** — it runs request-shape validation, balance checks, and limit enforcement here.
3. **The API writes to PostgreSQL** — it persists the transaction and its operations in the same request cycle.
4. **PostgreSQL confirms the write** — it commits all records.
5. **The API returns `201 Created`** to the client with the created transaction. The response leaves the server only after the database confirms everything. The response carries the transient `CREATED` status; Midaz promotes the transaction to `APPROVED` asynchronously after balance processing. Do not treat the `201` as final approval; wait for the status to reach `APPROVED` before you treat the transaction as settled.
**Characteristics:**
* Response time includes database write latency.
* Each transaction is an independent database operation.
* Simpler to reason about. The response shows exactly what Midaz persists.
Even in synchronous mode, Midaz updates balances atomically in Redis during the request. Redis is authoritative for balances. The write above persists the transaction and its operations, not the Postgres balance rows. The always-on balance-sync worker reconciles those rows (see [Balance synchronization](#balance-synchronization)).
### Asynchronous mode
Midaz validates the transaction the same way. Instead of a direct database write, Midaz publishes a message to RabbitMQ. A background consumer picks up the message and handles persistence separately.
Here's the full flow, step by step:
1. **The client sends** a `POST /transaction` to the Midaz API.
2. **The API validates the request** — it runs request-shape validation, balance checks, and limit enforcement exactly as in synchronous mode.
3. **The API publishes the transaction payload to RabbitMQ** instead of a direct database write.
4. **The API returns `201 Created`** to the client as soon as the queue accepts the message — the client does not wait for database persistence. The response carries the transient `CREATED` status; Midaz promotes the transaction to `APPROVED` asynchronously after balance processing. Do not treat the `201` as final approval; wait for the status to reach `APPROVED` before you treat the transaction as settled.
5. **RabbitMQ delivers the message** to a background consumer, decoupled from the API request.
6. **The consumer writes to PostgreSQL** — it persists the transaction and its operations from the queued message. The balance-sync worker coordinates balance updates and keeps balances consistent in both modes (see the **Balance synchronization** section).
**Characteristics:**
* Response time excludes database write latency — the client waits only for validation and queue publish.
* Midaz serializes messages with MessagePack for compact, efficient transport.
* Background consumers write to the database at their own pace, with retries; batch inserts require an enabled Bulk Recorder.
The validation step is identical in both modes. Balance checks, request-shape validation, and limit enforcement — all of that happens before the API responds, regardless of processing mode. The difference is only in *when* the data hits the database.
## Built-in resilience
***
If RabbitMQ is unavailable when async mode tries to publish a message, Midaz attempts a direct database write. If that write fails, Midaz returns the database error.
This means:
* During a queue outage, Midaz attempts to write directly to the database.
* The client can receive an error if the fallback database write fails.
* Midaz logs the queue failure and, if the direct write also fails, the fallback-write failure so your operations team can investigate them.
During a queue outage, latency can increase because writes go straight to the database. Monitor your RabbitMQ health to keep async mode active.
## Enabling async mode
***
Set one environment variable in the ledger application:
```bash Environment variable theme={null}
RABBITMQ_TRANSACTION_ASYNC=true
```
With `false` (the default), all transactions use synchronous processing and persist directly to PostgreSQL. The current Ledger bootstrap still initializes RabbitMQ and wires its consumer.
With `true`, the ledger publishes transaction payloads to the configured RabbitMQ exchange. A background consumer then handles persistence.
## RabbitMQ configuration
***
Async mode uses the following RabbitMQ settings (all in the ledger `.env`):
| Variable | Description | Default |
| :------------------------------------------------ | :-------------------------------------- | :--------------------------------------------------- |
| `RABBITMQ_TRANSACTION_ASYNC` | Enable async processing. | `false` |
| `RABBITMQ_HOST` | RabbitMQ server hostname. | `midaz-rabbitmq` |
| `RABBITMQ_PORT_HOST` | AMQP protocol port. | `3003` |
| `RABBITMQ_PORT_AMQP` | Management API port. | `3004` |
| `RABBITMQ_DEFAULT_USER` | Producer credentials (user). | `transaction` |
| `RABBITMQ_DEFAULT_PASS` | Producer credentials (password). | — |
| `RABBITMQ_CONSUMER_USER` | Consumer credentials (user). | `consumer` |
| `RABBITMQ_CONSUMER_PASS` | Consumer credentials (password). | — |
| `RABBITMQ_NUMBERS_OF_WORKERS` | Number of consumer worker goroutines. | `5` |
| `RABBITMQ_NUMBERS_OF_PREFETCH` | Messages prefetched per worker. | `10` |
| `RABBITMQ_TRANSACTION_BALANCE_OPERATION_EXCHANGE` | Exchange name for transaction messages. | `transaction.transaction_balance_operation.exchange` |
| `RABBITMQ_TRANSACTION_BALANCE_OPERATION_KEY` | Routing key. | `transaction.transaction_balance_operation.key` |
| `RABBITMQ_TRANSACTION_BALANCE_OPERATION_QUEUE` | Queue name. | `transaction.transaction_balance_operation.queue` |
The consumer uses separate credentials (`RABBITMQ_CONSUMER_USER` / `RABBITMQ_CONSUMER_PASS`) from the producer. This follows the principle of least privilege — the consumer only needs read access to the queue.
## Balance synchronization
***
A dedicated balance-sync worker coordinates balance updates. It uses Redis as a coordination layer. This worker runs in both synchronous and asynchronous modes. It keeps balances consistent even when multiple consumers process messages at the same time.
| Variable | Description | Default |
| :------------------------------ | :---------------------------------------------------------- | :------ |
| `BALANCE_SYNC_BATCH_SIZE` | Number of balance updates to batch before flushing. | `50` |
| `BALANCE_SYNC_FLUSH_TIMEOUT_MS` | Maximum wait time (ms) before flushing an incomplete batch. | `500` |
| `BALANCE_SYNC_POLL_INTERVAL_MS` | How often (ms) the worker checks for pending updates. | `50` |
The balance-sync worker runs automatically in both modes. You need no extra setup beyond an available Redis instance.
## RabbitMQ circuit breaker
***
When you enable async mode, Midaz depends on RabbitMQ for transaction persistence. A built-in circuit breaker protects against broker outages. It monitors the health of the RabbitMQ connection and fails fast when the broker goes down. This prevents request pileups and cascading failures.
The circuit breaker is active on the single-tenant RabbitMQ path. Multi-tenant RabbitMQ uses tenant connection management instead. The circuit breaker follows the standard three-state model:
* **Closed** (normal): requests flow through to RabbitMQ. The breaker counts failures.
* **Open** (tripped): the breaker does not contact RabbitMQ. Midaz bypasses the broker and attempts a direct database write for each async transaction; if that write fails, Midaz returns the database error. A background health checker monitors the broker and attempts recovery.
* **Half-open** (probing): the breaker lets a limited number of requests through to test RabbitMQ recovery. If they succeed, the circuit closes. If they fail, it reopens.
The circuit opens when either condition is true:
* The number of consecutive failures reaches the threshold, OR
* The failure ratio exceeds the configured percentage within the counting window
### Circuit breaker configuration
| Variable | Description | Default |
| :----------------------------------------------- | :------------------------------------------------------------------------------- | :------ |
| `RABBITMQ_CIRCUIT_BREAKER_CONSECUTIVE_FAILURES` | Consecutive failures before the circuit opens. | `15` |
| `RABBITMQ_CIRCUIT_BREAKER_FAILURE_RATIO` | Failure percentage (0–100) that triggers open state. | `50` |
| `RABBITMQ_CIRCUIT_BREAKER_MIN_REQUESTS` | Minimum requests before evaluating the failure ratio. | `10` |
| `RABBITMQ_CIRCUIT_BREAKER_INTERVAL` | Time window (seconds) for counting failures. Counters reset after each interval. | `120` |
| `RABBITMQ_CIRCUIT_BREAKER_TIMEOUT` | How long (seconds) the circuit stays open before transitioning to half-open. | `30` |
| `RABBITMQ_CIRCUIT_BREAKER_MAX_REQUESTS` | Requests allowed through in half-open state to probe recovery. | `3` |
| `RABBITMQ_CIRCUIT_BREAKER_HEALTH_CHECK_INTERVAL` | How often (seconds) the background health checker pings RabbitMQ. | `30` |
| `RABBITMQ_CIRCUIT_BREAKER_HEALTH_CHECK_TIMEOUT` | Timeout (seconds) for each health check ping. | `10` |
When the circuit is open, Midaz attempts direct database writes for async transactions. If a direct write fails, Midaz returns the database error; the fallback does not guarantee transaction delivery during broker outages.
For most production deployments, the defaults work well. Tune `CONSECUTIVE_FAILURES` and `TIMEOUT` if your RabbitMQ cluster has known recovery patterns. For example, lower the timeout if your broker recovers within seconds. Increase consecutive failures if you see transient network blips.
## How async mode connects to Bulk Recorder
***
Async mode and the [Bulk Recorder](/en/midaz/bulk-recorder) are complementary features that work together:
1. **Async mode** decouples the API response from persistence — transactions go to RabbitMQ instead of directly to PostgreSQL.
2. **Bulk Recorder** optimizes how the consumer writes those messages to the database — it batches multiple messages into single bulk inserts.
Bulk Recorder is active with async mode unless you explicitly set `BULK_RECORDER_ENABLED=false`. Without async mode, transaction persistence uses direct database writes and there is no transaction queue to batch.
| Configuration | Processing behavior |
| :-------------------------------------------- | :-------------------------------------------------- |
| Async `false` | Direct database write per transaction (synchronous) |
| Async `true`, Bulk Recorder `false` | Queue-based, one message processed at a time |
| Async `true`, Bulk Recorder enabled (default) | Queue-based, messages batched for bulk inserts |
## When to use async mode
***
**Use async mode when:**
* You need lower API response times for transaction creation.
* Your workload involves high transaction volumes (hundreds+ per second).
* You run batch operations like mass payouts or settlements.
* You want to decouple your API tier from database performance.
**Keep synchronous mode when:**
* You need direct, request-bound transaction persistence.
* Transaction volume is low to moderate.
* You want a successful API response to mean that Midaz already persisted the data.
* You work in a development or testing environment where simplicity matters more than throughput.
You can switch between modes at any time. Change `RABBITMQ_TRANSACTION_ASYNC` and restart the ledger application. You need no data migration, because the transaction format is the same in both paths.