Event Sourcing

What is Event Sourcing

Event sourcing is an architectural pattern where you store application state in a sequence of events and in a microservices architecture this can be also seen as a way to replace synchronous blocking calls to other collaborating services with flow that instead broadcasts events to which other services subscribe.

Why event sourcing?

In traditional world you would generally have used a RDBMS to maintain state of your application. For example, in a commerce site you’d have a Order table and which would have the state of each order Placed, Shipped, Cancelled etc. but fundamental problem here is that you cannot recreate a Orders history? If there was a question like, ‘When exactly was this order placed?’ you wouldn’t be able to provide an answer as state is mutated through out an orders lifecycle until it was fulfilled. However, complete rebuild of order life cycle is possible using event store. In the following picture on left application stores the state of each Order as a row in a table and on the right each activity on Order repository is stored as a sequence of events. Picture is taken from here.

In a microservices architecture, if you have all communication on synchronous calls? A slow service can bring entire/partial system down. At times we need things to happen on the same thread, but if the stuff can wait make it asynchronous. When you buy an item on Amazon.com, complete checkout flow, from the time you add item to cart, till place order page is just broadcasting events based on what user is doing. Since entire check out flow is events based, occasionally you see that if it happens that the item you ordered is out of stock then Amazon sends you a email later on as they cannot fulfill the order.

If you are working on microservices that have their own databases, there are times where you will have to access the data from other services, and if you decide to read from the database of other service (for whatever reasons:-)), you are going to be tightly coupled with that schema thus resulting in that other service losing its autonomy. Here, instead we can broadcast events to update its state.

We all have heard the phrase -> “Data is the new oil” using event sourcing we can retain almost all the transactions of business value. This can help business with lot of cool analytics and possibilities of turning them into eventual business opportunities.

Introducing CQRS

Problem at hand

We have basic understanding of event sourcing is? Now, how do we handle the case, where we need access to a particular business entity/data, for various reasons: may be a particular part of business wants it or a user group, or may be we need render it on the UI. The problem at hand is, I want faster reads! Okay, so we have all the events and we can solve this problem? Yes! we can replay the events and create the current state but imagine an event store which has billions of events already and keeps growing, and yes not all the events are going to be replayed always but it is still lot of data to go through. This is where CQRS (Command and Query Responsibility Segregation) helps us.

CQRS to help

Design principle that CQRS proposes is to keep you database writes and reads separate. The entities useful for business or views that are often read will fall into the Query part and the things that have potential to generate events fall into the Command part. For example: If you placed on order on a commerce site, there is value in generating OrderPlaced event and from this event you can construct the views that business is interested in? In this special case once an order is placed you will update following domain models, -> Order, Customer, Payment etc these domain models are read only and they can be materialized from events. Following picture is from Martin Fowlers blog, picture describes read and write models with various interactions.

Only the most valuable sections of the business/system are considered to be part of Query part as, too many materialized views complicate the event processing pipeline and also the over all system. In short don’t have crazy number of read views. CQRS helps avoids read/write conflicts in the system. Your materialized views can be stored in MongoDB, ElasticSearch etc. I have personally used mongoDB. Each event broadcasted is consumed by number of services and these services use it in deriving their current state or materialize the views in their bounded context. This is so much fun:-)