Skip to main content

Kafka + Cadence + WebSockets + Angular: managing event-driven microservices state with Uber Cadence

In the the previous post of the Event-driven microservices with Kafka series (see here), I showed how to extend the asynchronous event-driven communication all the way from Kafka to the Web frontend passing through the Java backend.

In that proof of concept application which processes money transfers, one Kafka topic is used to receive incoming transfer messages and a second topic is used to store the account balances state and broadcast changes (i.e., to push notifications from the backend to the frontend).
The first topic (called "transfers") and the second topic (called "account-balances") are connected through Kafka Streams.

Uber Cadence

In this post we are bringing Uber Cadence into the mix to manage the state of the application (i.e., to keep the balance of the accounts updated), thus, Cadence replaces Kafka Streams.

Cadence is an orchestration/workflow engine which unlike most of the other workflow engines out there (e.g., Zeebe, Camunda and many others), does not rely on BPMN (nor on any other type of XML) to define a workflow, it uses regular programming languages such as Go and Java instead, which is a huge advantage as eloquently explained by Cadence's lead developer in this comment.

It is out of the scope of this post to provide a detailed description or an introduction to Cadence, you are encouraged to perform your own (even if superficial) research but I will mention its main advantages:
- It makes your application easily escalable
- It hugely simplifies the error handling
- It comes with an admin Web UI and CLI
In short, it takes care of most of the heavy lifting inherent to a distributed application

Enhanced PoC application

Let's go trough the data flow to get an overview of the application functionality.
A new service called Cadence transfers recording service listens for incoming transfer messages on a Kafka topic.
If the message represents the act of opening a new account, the service creates a new workflow by using the corresponding Cadence Java client API method.
The Cadence server will assign the execution of the new workflow to one of the previously started Cadence workers. One important detail to note is that the id of the workflow matches the account id, i.e., there is one workflow per account.

If the incoming Kafka message represents a money transfer which changes the balance of an existing account, the Cadence transfers recording service calls the required method (which must be annotated with @SignalMethod) on the workflow whose id matches the account id.
That triggers the remote execution of the code on one of the Cadence workers, which after updating the value of the account balance as required, will send a Kafka message to the "account-balances" topic to announce the event.

Next, the "transfers websockets" service will pull that message from the "account-balances" Kafka topic and broadcast it to the interested Angular Web clients through websockets.

In addition to this asynchronous/push communication, an Angular client may query the "transfers websockets" service synchronously to get the current balance or the history of an account (i.e., the list of transfers for that account).
The "transfers websockets" service in order to fulfill those requests, will query the Cadence server by calling the corresponding method which must be annotated with @QueryMethod.

Diagram

This is a high-level diagram of the PoC application, I am sorry if it looks a little messy, graphic design is not my strongest skill but it should help you to get an overview of the components and their interactions.



See it in action



Source code repositories

https://github.com/VictorGil/cadence_transfers_recording_service
https://github.com/VictorGil/account_balance_workflow_api
https://github.com/VictorGil/transfers_websockets_service
https://github.com/VictorGil/transfers_frontend
https://github.com/VictorGil/transfers_recording_service
https://github.com/VictorGil/transfers_api
https://github.com/VictorGil/kafka_util

Comments

Popular posts from this blog

Using Apache Kafka to implement event-driven microservices

When talking about microservices architecture, most people think of a network of stateless services which communicate through HTTP (one may call it RESTful or not, depending on how much of a nitpicker one is).

But there is another way, which may be more suitable depending on the use case at hand.
I am talking about event-driven microservices, where in addition to the classic request-response pattern, services publish messages which represent events (facts) and subscribe to topics (or queues depending on the terminology used) to receive events/messages.

To fully understand and embrace this new software design paradigm is not straight-forward but it is totally worth it (at least looking into it).
There are several interconnected concepts which need to be explored in order to discover the advantages of event-driven design and the evolutionary path which led to it, for example:
Log (including log-structured storage engine and write-ahead log)Materialized ViewEvent SourcingCommand Query Respons…

Vert.x microservices: an (opinionated) application

First of all, sorry for the tautology in the title, a library can be either opinionated or un-opinionated (such as Vert.x is), but an application can only be opinionated.
However, I decided to include "opinionated" to help getting my point across even though is redundant.
The Motivation  I am a big fan of Vert.x and the official documentation is quite good, yet it is not straight-forward to understand how it works and how to use it.
There are a lot of blogs and articles describing Vert.x terminology and its concurrency model.
There are also tons of "Hello World" Vert.x applications on Github, and the rest seem to be just variations of the already typical "Web chat application using Vert.x".
On top of that, many of them are outdated (using AngularJS instead of Angular2+, for example).

The only two exceptions which I found are:
vertx-microservices-workshop: a demo application by the Vert.x development team.
ngrx-realtime-app: proof of concept application focu…

Kafka + WebSockets + Angular: event-driven microservices all the way to the frontend

In the the initial post of the Event-driven microservices with Kafka series (see here or here), I talked about the advantages of using event-driven communication and Kafka to implement stateful microservices instead of the standard stateless RESTful ones.
I also presented the architecture and the source code of a related proof of concept application.

In this post, I would like to show how to extend the asynchronous event-driven communication all the way from Kafka to the Web frontend passing through the Java backend.
Hence, in the first post of this series, we got rid of HTTP as the communication protocol among microservices in the backend, and now we are also replacing it (with WebSockets) as the communication protocol between the frontend and the backend.
Ok, but why would you do that? Because it provides a better experience to the end user!.
Using WebSockets you can build legit real-time user interfaces, the updates are pushed immediately from the server to the client, no need for …