> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lerian.studio/llms.txt
> Use this file to discover all available pages before exploring further.

# Midaz production best practices

> Set up Midaz for production with multi-AZ deployment, autoscaling, managed services, and the Kubernetes resilience patterns Lerian recommends.

Midaz runs in production at high volume. This guide gives you the deployment, high-availability, and observability patterns Lerian recommends. Follow them to keep downtime low and protect your data.

## Best-fit setup

***

Start a production deployment with these choices:

* Deploy across multiple availability zones.
* Run at least 3 worker nodes with autoscaling.
* Separate application and database workloads.
* Use managed services such as RDS, ElastiCache, and MongoDB Atlas.
* Apply Kubernetes patterns for resilience, security, and observability.
* Automate backups and alerting from day one.

## Infrastructure planning

***

### Cluster architecture

Plan the cluster for resilience and performance:

* Deploy across multiple availability zones.
* Run at least 3 worker nodes for high availability.
* Enable node autoscaling to absorb workload spikes.
* Separate application and database workloads when possible.

### Resource sizing

* Match node sizes to the expected workload.
* Give critical services enough resources first.
* Apply resource quotas to prevent contention.
* Monitor usage and tune the sizing over time.

### Storage

* Use SSD-backed storage for all database components.
* Define a storage class for each cloud provider.
* Provision volumes with headroom for growth.
* Use replicated or durable storage for critical data.

## Database architecture and high availability

***

Midaz uses CQRS (Command Query Responsibility Segregation) to separate reads from writes. This design lets you scale each path on its own.

### PostgreSQL

* Use a dedicated primary for writes and replicas for reads.
* Enable synchronous replication for critical data.
* Configure automatic failover with Patroni or AWS RDS.
* Monitor replication lag and consistency.
* Prefer managed services such as AWS RDS or GCP Cloud SQL.

### Redis / Valkey

* Deploy in cluster mode across multiple zones.
* Enable automatic failover with native clustering, or a Sentinel topology via `REDIS_MASTER_NAME`. Verify your client configuration — a managed service (ElastiCache, Memorystore) is the safer default.
* Use managed services such as AWS ElastiCache or GCP Memorystore.

### MongoDB

* Use replica sets with members across zones.
* Monitor role transitions and lag.
* Schedule regular backups.
* Do not write to secondaries unless you intend to.
* Use managed services such as MongoDB Atlas or AWS DocumentDB.

## Messaging infrastructure

***

Midaz runs two messaging surfaces, both off by default:

* **RabbitMQ** carries the internal async transaction balance-operation pipeline (`RABBITMQ_TRANSACTION_BALANCE_OPERATION_*`, enabled with `RABBITMQ_TRANSACTION_ASYNC=true`) plus the legacy outbound transaction, overdraft, and audit event exchanges. Use a managed RabbitMQ service such as AWS MQ or CloudAMQP in production.
* **RedPanda** (via lib-streaming) is the forward-looking event backbone. Set `STREAMING_ENABLED=true` and `STREAMING_BROKERS`; events publish to `lerian.streaming.<resource>.<event>` topics. Transaction-lifecycle and overdraft events currently publish to both transports during the migration window.

CQRS read/write separation is served by PostgreSQL replicas (`DB_*_REPLICA_*` DSNs), not by broker consumers rebuilding read models.

## High availability strategies

***

### Service redundancy

* Deploy multiple replicas for every service.
* Use anti-affinity rules to spread services across zones.
* Apply Pod Disruption Budgets to limit downtime during updates.

### Load balancing

* Use ingress controllers with health checks.
* Avoid session affinity unless a service requires it.
* Enable connection draining for smooth rollouts.

## Security considerations

***

### Network security

* Apply Kubernetes network policies to control traffic.
* Give each service account minimal permissions.
* Secure external access with TLS.
* Restrict admin interfaces with IP allowlists.

### Secret management

* Use Kubernetes Secrets for credentials and tokens.
* Rotate secrets on a regular schedule.
* Never hardcode secrets in containers or config files.
* Use an external secret manager for a stronger posture.

## Monitoring and observability

***

### Metrics

* Monitor the key application and infrastructure KPIs.
* Set alert thresholds that lead to action.
* Use dashboards for real-time visibility.

### Logging

* Centralize logs across all services.
* Use a structured format for easier filtering.
* Apply log retention and rotation policies.
* Define log-based alerts for critical events.

### Tracing

* Enable distributed tracing across services.
* Sample traces to balance performance and cost.
* Correlate traces with logs and metrics for full visibility.

### Alerting

* Create clear, reliable alerts.
* Tune thresholds to reduce noise.
* Route each alert through the right channel.
* Keep runbooks for recurring issues.

## Backup strategy

***

* Automate regular backups for critical systems.
* Store backups in more than one location or region.
* Test the restore procedure on a regular schedule.
* Keep the backup documentation current and accessible.

## Idempotency

***

Protect critical operations against duplicate processing in production:

* Send an idempotency key on every transaction create request with the `X-Idempotency` header.
* Use explicit, deterministic keys tied to your business IDs (order IDs, payment references), not auto-generated keys.
* Read the `X-Idempotency-Replayed` response header to tell a new transaction from a cached replay.
* Set the `X-TTL` header in seconds to match your retry window. The default is 300. Use a shorter value for synchronous flows and a longer one for asynchronous flows.

<Note>
  All Lerian products support idempotency through their own header conventions. For implementation details and a comparison across products, see [Retries and idempotency](/en/reference/retries-idempotency).
</Note>

## Final notes

***

Align your infrastructure to the Midaz architecture and you gain:

* Clean read/write separation with CQRS.
* Compatibility with managed cloud services.
* A clear path to observability, failover, and secure operations.

Review your setup on a regular schedule to keep this foundation solid as you grow.

## What’s next?

***

Ready to scale, migrate, or harden your production environment?

* Read the [Midaz deployment guide](/en/midaz/deployment).
* [Contact our team](https://lerian.studio/contact) for tailored support.
