Skip to main content

Kafka + Cadence + WebSockets + Angular: managing event-driven microservices state with Uber Cadence

In the the previous post of the Event-driven microservices with Kafka series (see here), I showed how to extend the asynchronous event-driven communication all the way from Kafka to the Web frontend passing through the Java backend.

In that proof of concept application which processes money transfers, one Kafka topic is used to receive incoming transfer messages and a second topic is used to store the account balances state and broadcast changes (i.e., to push notifications from the backend to the frontend).
The first topic (called "transfers") and the second topic (called "account-balances") are connected through Kafka Streams.

Uber Cadence

In this post we are bringing Uber Cadence into the mix to manage the state of the application (i.e., to keep the balance of the accounts updated), thus, Cadence replaces Kafka Streams.

Cadence is an orchestration/workflow engine which unlike most of the other workflow engines out there (e.g., Zeebe, Camunda and many others), does not rely on BPMN (nor on any other type of XML) to define a workflow, it uses regular programming languages such as Go and Java instead, which is a huge advantage as eloquently explained by Cadence's lead developer in this comment.

It is out of the scope of this post to provide a detailed description or an introduction to Cadence, you are encouraged to perform your own (even if superficial) research but I will mention its main advantages:
- It makes your application easily escalable
- It hugely simplifies the error handling
- It comes with an admin Web UI and CLI
In short, it takes care of most of the heavy lifting inherent to a distributed application

Enhanced PoC application

Let's go trough the data flow to get an overview of the application functionality.
A new service called Cadence transfers recording service listens for incoming transfer messages on a Kafka topic.
If the message represents the act of opening a new account, the service creates a new workflow by using the corresponding Cadence Java client API method.
The Cadence server will assign the execution of the new workflow to one of the previously started Cadence workers. One important detail to note is that the id of the workflow matches the account id, i.e., there is one workflow per account.

If the incoming Kafka message represents a money transfer which changes the balance of an existing account, the Cadence transfers recording service calls the required method (which must be annotated with @SignalMethod) on the workflow whose id matches the account id.
That triggers the remote execution of the code on one of the Cadence workers, which after updating the value of the account balance as required, will send a Kafka message to the "account-balances" topic to announce the event.

Next, the "transfers websockets" service will pull that message from the "account-balances" Kafka topic and broadcast it to the interested Angular Web clients through websockets.

In addition to this asynchronous/push communication, an Angular client may query the "transfers websockets" service synchronously to get the current balance or the history of an account (i.e., the list of transfers for that account).
The "transfers websockets" service in order to fulfill those requests, will query the Cadence server by calling the corresponding method which must be annotated with @QueryMethod.

Diagram

This is a high-level diagram of the PoC application, I am sorry if it looks a little messy, graphic design is not my strongest skill but it should help you to get an overview of the components and their interactions.



See it in action



Source code repositories

https://github.com/VictorGil/cadence_transfers_recording_service
https://github.com/VictorGil/account_balance_workflow_api
https://github.com/VictorGil/transfers_websockets_service
https://github.com/VictorGil/transfers_frontend
https://github.com/VictorGil/transfers_recording_service
https://github.com/VictorGil/transfers_api
https://github.com/VictorGil/kafka_util

Comments

Popular posts from this blog

Kafka + WebSockets + Angular: event-driven microservices all the way to the frontend

In the the initial post of the  Event-driven microservices with Kafka series (see here  or  here ), I talked about the advantages of using event-driven communication and Kafka to implement stateful microservices instead of the standard stateless RESTful ones. I also presented the architecture and the source code of a related proof of concept application. In this post, I would like to show how to extend the asynchronous event-driven communication all the way from Kafka to the Web frontend passing through the Java backend. Hence, in the first post of this series, we got rid of HTTP as the communication protocol among microservices in the backend, and now we are also replacing it (with WebSockets) as the communication protocol between the frontend and the backend. Ok, but why would you do that? Because it provides a better experience to the end user!. Using WebSockets you can build legit  real-time user interfaces, the updates are pushed immediately from the server to the client

Cómo pedir cita previa en la DGT y desesperarse en el intento

El uso de la tecnología para la agilización de la administración española es simplemente bochornoso (y me quedo corto). Además todo el mundo lo sabe, sin ir más lejos, la última gran chapuza  informática en el Ministerio de Justicia va a costar al menos  60 millones  extra.  Pero como el dinero sale del bolsillo de los ciudadanos, pues aquí no pasa absolutamente nada y nadie asumirá ninguna responsabilidad. Esta es la triste realidad de la España actual. Pero es que hasta un trámite tan sencillo como pedir cita previa en una de las Jefaturas Provinciales de Tráfico desespera hasta al más templado. Y como quiero ser muy específico y una imagen vale más que mil palabras, a continuación voy a documentar todo el proceso que es de todo menos intuitivo. Todo empieza abriendo este enlace Y como podéis ver en la imagen previa, con la primera pantalla (vamos a obviar que la interfaz parece que ya haya cumplido la mayoría de edad, tiene pinta de ser de finales de los 90) llega la pri

Proof of Social Network Identity: a proof of concept

One of the main criticisms to blockchain technology is based on the huge consumption of electricity required by the biggest networks (e.g., Bitcoin and Ethereum). That huge electric power consumption  comes from using Proof of Work (PoW) algorithms to prevent " double spending " which is closely related to the " 51% attack ". Some other types of algorithm have been developed as an alternative to PoW, the more popular one being Proof of Stake (PoS), which basically implies that individuals with higher stake (i.e., more crypto money) have more power (i.e., are more trusted) than the rest. Hence, PoS, creates an entry barrier for those who want to become members of the blockchain network, since one needs to own a stake in order to join the network. The key to prevent " double spending / 51% attacks"  is to use something  in the block validation mechanism which is complicated or expensive enough for some malicious actor to replicate or to clone or