Rethinking Distributed Systems: Why Events Fall Short
Changing how you think about a system can have big implications. Consider the example of Copernicus, where a “simple” change of perspective holds a valuable lesson for all of us.
In the early sixteenth century, Copernicus realized that existing models of the solar system were overly complicated. They looked beautiful, but when it came to figuring out where things might be relative to others, it was a rather difficult working model:
He believed that those models could be dramatically simplified by changing the frame of reference. Rather than modeling everything with Earth at the center of the universe, by placing the Sun at the center you could produce a simpler system that was far easier to understand:
This new model drastically simplified the understanding of planetary movements and enabled a whole new series of discoveries and developments that we now call the Copernican Revolution. This simple change of perspective holds a valuable lesson for all of us; changing how you think about a system can have big implications.
Simpler Diagrams are Better Diagrams
Event-Driven Architecture (EDA) is one of the most common patterns in modern applications built with microservices. Like the model with Earth at the center, some architecture diagrams look rather beautiful. Once you are forced to work with them at scale, however, many developers discover they’re extraordinarily complex, fragile, and frustrating. Consider this relatively simple Payment Accumulation architectural diagram:
Copernicus’ Insight: Try Shifting the Center
If you shift the center of your world view away from events, you might help be able to reduce the intricacy of your software systems. This, in turn, could unlock and unblock projects that are hopelessly complicated by an overly convoluted EDA system.
Here’s how that same Payment Accumulation system looks with Temporal:
Common Problems of Event-Driven Architecture
Event-Driven Architecture is a common approach for building distributed applications. However, EDA has major pitfalls, especially when scaled:
- No APIs: Implementing EDA usually leads to the loss of clear, well-defined APIs. Events lack the documentation and structure that APIs provide, leading to ambiguity about which events each service consumes and produces.
- Tightly Coupled: Despite the perception of loose coupling, EDA systems are often effectively highly coupled, as changes in one service can cause unintended consequences throughout the system. Events are effectively global variables, and changes to those events and event formats can break services throughout the system. For more on this, see Event-Driven Systems and the Truth About "Loosely Coupled" Architectures.
- Scattered Logic: Business logic becomes fragmented and spread across many different services which makes it difficult to understand and maintain.
- Poor Debuggability: If an event handler doesn’t happen as expected, tracing through the prior events to figure out where the chain got broken is tedious and requires a lot of careful investigation. And, often, understanding what that chain of events might include can be its own non-trivial research project.
- Proliferating Ad-hoc State Machines: Each service becomes an ad-hoc state machine, complicating state management. This state is dispersed across different systems and events, making it hard to get a clear picture of what is happening and why.
- Increased Latency: The queuing and infrastructure overhead necessary to manage event flows can introduce considerable latency.
- Race Conditions: Unless you use the outbox pattern to guarantee data consistency, you will have race conditions between queues and databases that can be extremely difficult to diagnose and troubleshoot.
- System Upgradability: Upgrading EDA systems is extremely difficult because of poorly defined and understood dependencies and scattered state management.
Durable Execution: A Better Abstraction
Durable Execution offers a compelling alternative to event-driven architecture, addressing many of its shortcomings. Platforms like Temporal use this approach to provide more resilient and maintainable distributed systems.
Collect Your Business Logic and Simplify Your Life With Durable Execution, developers define business logic with code that lives in one central place, significantly improving understanding and debuggability.
Manage State Consistently Durable Execution handles state management, retries and more so that network outages, flaky endpoints, long-running processes and other transient failures don’t overwhelm your system with the complexity of managing unreliable processes.
Survive Crashes with Grace Durable Execution inherently supports resilience. The execution state is preserved even in the face of process crashes or system outages, allowing the app to resume from where it left off. This capability ensures consistency and reliability, making operations more dependable and resilient.
Design Processes Flexibly Durable Execution handles long-running and intricate processes. This flexibility supports synchronous and asynchronous processes, accommodating a wide range of application needs. Developers can build systems that span hours, days, or even longer periods without sacrificing reliability.
Embracing a New Approach
Shifting from Event-Driven Architecture to Durable Execution is more than just a different way to model systems; it's a fundamental change in how you can build reliable and maintainable software. By addressing the shortcomings of event-driven designs and offering a more intuitive and effective framework, Durable Execution empowers you to create distributed systems that are easier to understand, maintain, and expand.
As we seek to build better, more reliable systems, having the right frame of reference can make a big difference. Yes, you can model the whole universe as revolving around Earth, but if you take a page from Copernicus, you might find that changing your frame of reference will make your system much, much easier.
Read next: Event-Driven Architectures - A better way forward ›
Ready to Rethink EDA?
Whether you're just starting with events or already have built an event-driven system and feel the pains, we encourage you to check out Temporal and see how it can simplify your architecture and code and make building distributed systems durable, flexible, and clear:
- Get Started with Temporal and try it out yourself
- Ask an Expert for advice and assistance on your project
- Join our Community Slack and dive into how you can simplify your architecture with Temporal