Observability is the ability to understand a system’s internal state and behavior by analyzing its external outputs, such as logs, metrics, and traces. It is essential for monitoring, diagnosing, and improving the performance of complex, distributed systems like Midaz.

By implementing observability, teams gain:

• Performance Insights: Ensure systems operate efficiently.
• Faster Issue Resolution: Quickly identify and fix problems.
• Predictive Capabilities: Detect patterns to prevent failures.
• System Transparency: Understand interactions between components.

The Three Pillars of Observability

Observability is built on three key data types:

• Logs: Time-stamped records of events in a system. Logs provide essential context for diagnosing errors and understanding specific incidents.
• Metrics: Numerical data representing the system's performance and health over time, such as CPU usage, request latency, and throughput. Metrics highlight trends and performance bottlenecks.
• Traces: End-to-end tracking of requests as they flow through different components of a system. Traces reveal dependencies and pinpoint latency in distributed architectures.

Together, these pillars offer a comprehensive view of system behavior, enabling teams to address issues proactively.

Observability in Midaz Infrastructure

In Midaz, observability is powered by OpenTelemetry, an industry-standard project maintained by the Cloud Native Computing Foundation (CNCF). OpenTelemetry enables the collection and export of telemetry data (logs, metrics, traces) in a unified and extensible way.

Benefits of Observability in Midaz

Midaz’s observability features provide:

• Unified Monitoring: Logs, metrics, and traces in one platform.
• Flexibility: Export telemetry data to preferred backends using OTLP.
• Transparency: A clear understanding of system performance and behavior.
• Scalability: Designed for modern, cloud-based, distributed architectures.

Midaz Observability Stack

Midaz uses otel-lgtm, an observability solution by Grafana Labs that combines:

• OpenTelemetry Collector: Collects, processes, and exports telemetry data.
• Grafana Loki: Stores and analyzes logs.
• Grafana Tempo: Handles distributed tracing.
• Prometheus: Captures system metrics.

These tools integrate seamlessly, providing a unified platform for monitoring, troubleshooting, and optimizing the Midaz system.

Telemetry Data Export

Midaz exports telemetry data using the OpenTelemetry Protocol (OTLP). This standardized format ensures compatibility with various observability backends, including:

• Datadog
• Jaeger
• Zipkin

By using OTLP, you can easily redirect Midaz telemetry data to match your organization’s preferred observability tools.

Accessing Midaz Observability Tools

After completing the Midaz Setup, the observability tools are available through Grafana at http://localhost:3100.

Expand the following topic to access the login credentials.

Next Steps

• Start Monitoring: Access the Observability tool and explore the default dashboards.
• Redirect Data: If needed, configure the telemetry exporter to send data to external backends like Jaeger or Datadog.
• Deep Dive into OpenTelemetry: Learn more about OpenTelemetry by visiting the official documentation.
• Contribute to Midaz: Join the community through our Discord channel and share feedback or ideas.