In my career journey, I started in chemical engineering before moving into software development. In chemical engineering, we often work with chemical reactors — containers where chemical reactions take place under controlled temperature and pressure conditions.
For reactions to succeed, instrumentation is essential: engineers must measure, monitor, and adjust conditions without stopping the process. Without those instruments, we would be flying blind.
This concept of instrumentation is not unique to chemical processes. It directly parallels the world of modern software development.
In monolith or complex distributed systems, we can think of our applications as “reactors,” continuously handling inputs, processing data, and producing outputs. We cannot pause the process each time we have a question.
Instead, we rely on observability: the ability to ask questions about a system’s behavior without needing to alter or stop it. This talk will explore how observability enables reliable software, why it matters for success, and how teams can adopt it practically.
Defining Observability
Observability is often confused with monitoring, but the two are not the same. Monitoring answers the question: “Is the system up?” Observability goes further and lets us ask: “Why is this happening?”
The formal definition is: observability is the ability to understand the internal state of a system from the outputs it produces. In practice, this means instrumenting applications so developers can query logs, metrics, and traces to uncover insights.
A system has good observability when:
1) Developers can troubleshoot problems without adding additional code or instrumentation.
2) Teams can detect and resolve issues quickly.
3)Developers can understand system behavior even if they were not the original author of the code.
The Importance of Observability for Code Success
Reliability is a cornerstone of software success. But reliability is not simply uptime. An application may run 100% of the time yet still fail if it is not doing what users expect it to do. True reliability is about meeting user expectations consistently.
This is where observability becomes critical. A well-instrumented system allows teams to:
1) Respond to incidents faster (reducing Mean Time to Resolution).
2) Detect hidden failures before they impact users’ experience.
3) Improve user trust and satisfaction.
4) Support continuous delivery and innovation by reducing the risks of change.
Just as chemical engineers rely on reactor sensors to ensure chemical processes succeed, developers rely on observability to ensure their applications meet user expectations in real-world conditions.
The Three Pillars of Observability
Observability relies on three key signals — often called the three pillars:
1) Logs – Detailed records of discrete events. Logs help trace what happened, step by step.
2) Metrics – Numeric measurements aggregated over time, such as request latency, error rates, or throughput. Metrics show trends and patterns.
3) Traces – End-to-end records of requests as they travel across distributed systems. Traces show where delays, errors, or bottlenecks occur.
When combined, these three emissions give developers a full picture of system health, much like sensors in a chemical reactor provide data on temperature, pressure, and flow rates.
Observability in Practice
Imagine deploying a new feature in a distributed system. A customer reports that a request intermittently fails. Without observability, the team may spend hours adding logs, redeploying, and reproducing the issue. With observability, the team can:
1) View traces to see exactly where the failure occurred.
2) Use metrics to identify whether the problem correlates with load, memory usage, or external dependencies.
3) Examine logs for detailed error information.
Instead of guesswork, the team diagnoses the issue quickly and restores functionality — reducing downtime and improving customer trust.
Common Pitfalls to Avoid
Adopting observability is not without challenges. Common pitfalls include:
1) Alert fatigue: Too many unfiltered alerts overwhelm teams, especially when alerts are connected to Slack or Teams
2) Collecting everything: Raw data without a strategy creates noise and costs.
3) Siloed dashboards: Data exists but is not shared or acted upon.
The solution is to adopt a culture where observability is part of the definition of done. Just as safety checks are non-negotiable in chemical engineering, observability must be non-negotiable in software development.
Observability in existing cloud infrastructure
Cloud platforms recognize that observability is fundamental to reliability. Both Azure and AWS provide integrated services to help developers and operations teams instrument, collect, and analyze system data.
(1) Azure Monitor and (2) AWS CloudWatch
Observability Tools
Beyond cloud-native offerings, the ecosystem of observability tools is broad and continues to grow. Common categories include:
1) OpenTelemetry (OTel):
2) Prometheus and Datadog.
3) Prometheus and Datadog
Choosing the right tools depends on context: small teams may prefer managed cloud-native tools, while larger organizations benefit from OpenTelemetry-based pipelines that decouple data collection from backend providers.
Building a Culture of Observability
Ultimately, observability is not just about tools — it is about culture. Success requires:
1) Shared responsibility across developers, testers, and operations.
2) Blameless postmortems that use observability data to learn, not assign blame.
This cultural shift transforms observability from a reactive practice into a proactive enabler of success.
Conclusion
Success in software development comes not from simply delivering features but from delivering reliable experiences that meet user expectations.
Observability provides the instrumentation developers need to understand systems, diagnose problems, and continuously improve.
Just as chemical engineers cannot run reactors without sensors, software teams cannot succeed without observability.
