Skip to main content

Kafka + WebSockets + Angular: event-driven microservices all the way to the frontend

In the the initial post of the Event-driven microservices with Kafka series (see here or here), I talked about the advantages of using event-driven communication and Kafka to implement stateful microservices instead of the standard stateless RESTful ones.
I also presented the architecture and the source code of a related proof of concept application.

In this post, I would like to show how to extend the asynchronous event-driven communication all the way from Kafka to the Web frontend passing through the Java backend.
Hence, in the first post of this series, we got rid of HTTP as the communication protocol among microservices in the backend, and now we are also replacing it (with WebSockets) as the communication protocol between the frontend and the backend.

Ok, but why would you do that?

Because it provides a better experience to the end user!.
Using WebSockets you can build legit real-time user interfaces, the updates are pushed immediately from the server to the client, no need for the user to click any button nor to refresh the browser.

Wait, but we can do the same using good old HTTP, can't we?

Well... HTTP was not designed to provide real-time communication, but since people really wanted it, a number of workaround techniques and technologies have been create on top of it, such as:
  • Long polling
  • Short polling
  • Server-Sent Events
Those are little more than hacks, IMHO.
WebSockets, on the other hand, provide full-duplex bidirectional communication, which means that information can flow from the client to the server and also in the opposite direction simultaneously, it cannot get more real-time than that (that's what I meant when I said 'legit real-time applications' before).

HTTP vs WebSocket (source)

The example

I added a new service to the same PoC application which I used in the previous post and also an Angular web frontend.

Ok, but what does this new service do?

This service allows the frontend user to receive information about the current balance of any account of her/his choice as well as live updates when the balance changes.
Also, information about any transactions related to that currently selected account (i.e., any amount of money credited or debited) past or live, is displayed to the users.

Architecture

The new Transfers WebSockets service main components are:
  • Two Kafka consumers (one for each topic) to retrieve messages from the Kafka cluster
  • Two Kafka Streams local stores to retrieve the latest data associated with a given key (id)
  • A custom local store implemented using a simple Map to store the list of transactions for a given account.
  • Two dispatchers which keep track of what WebSockets sessions (i.e., frontend clients) are registered to receive live balance updates or live transfers data related to which account and then dispatch the updates accordingly.
And these are the main components of the Angular frontend:
  • A WebSockets service which handles the connection lifecycle and dispatches the received messages to the corresponding RxJS Subject. Please note that different types of JSON messages (requests, responses and updates) flow through the only WebSockets connection.
  • A form which passes the user input (i.e., the id of the chosen account) to the WebSockets service so it can be sent to the remote server.
  • An Angular component to display the account balance which gets asynchronously notified of balance changes (updates) through an RxJS Observer which is subscribed to the corresponding RxJS Subject in the previously mentioned WebSockets service, and then re-renders the value accordingly.
  • Another Angular component which similarly as the previous one, gets asynchronously notified of any new transaction related to the selected account and adds it to the list of displayed transaction data.
This is a diagram depicting the PoC application components and several frontend instances:

See it in action



Check out this Youtube video for a demo, where a test process which continuously generates Kafka messages representing random money transfers is used to feed the application.
And on the Web UI the user can see how the transfers pops up as they happen and the balance of the related account changes accordingly, all in real time.

Final Thoughts

HTTP has had a huge success, it made possible the WWW as we know it today.
It has been so successful that it has spread to the backend (RESTful services and open APIs) where it has been the king for more than a decade already.
However, technology evolves, paradigms come and go and for some use cases other tools and designs may be the right choice.
I do not know how software applications will look like in the future, all I can do is guess.
Event-driven microservices look very powerful and implementing them with Kafka and adding WebSockets to the mix seems promising.
In fact, some companies are able to see beyond REST APIs and are starting to offer WebSockets APIs to their clients for faster streaming access to data (example).
And there are other alternative ways to implement event-driven microservices, for example, recently I learnt about Uber Cadence but that's for a future post 😃

Comments

Popular posts from this blog

Using Apache Kafka to implement event-driven microservices

When talking about microservices architecture, most people think of a network of stateless services which communicate through HTTP (one may call it RESTful or not, depending on how much of a nitpicker one is).

But there is another way, which may be more suitable depending on the use case at hand.
I am talking about event-driven microservices, where in addition to the classic request-response pattern, services publish messages which represent events (facts) and subscribe to topics (or queues depending on the terminology used) to receive events/messages.

To fully understand and embrace this new software design paradigm is not straight-forward but it is totally worth it (at least looking into it).
There are several interconnected concepts which need to be explored in order to discover the advantages of event-driven design and the evolutionary path which led to it, for example:
Log (including log-structured storage engine and write-ahead log)Materialized ViewEvent SourcingCommand Query Respons…

Vert.x microservices: an (opinionated) application

First of all, sorry for the tautology in the title, a library can be either opinionated or un-opinionated (such as Vert.x is), but an application can only be opinionated.
However, I decided to include "opinionated" to help getting my point across even though is redundant.
The Motivation  I am a big fan of Vert.x and the official documentation is quite good, yet it is not straight-forward to understand how it works and how to use it.
There are a lot of blogs and articles describing Vert.x terminology and its concurrency model.
There are also tons of "Hello World" Vert.x applications on Github, and the rest seem to be just variations of the already typical "Web chat application using Vert.x".
On top of that, many of them are outdated (using AngularJS instead of Angular2+, for example).

The only two exceptions which I found are:
vertx-microservices-workshop: a demo application by the Vert.x development team.
ngrx-realtime-app: proof of concept application focu…

Shared Ledger Simulator

I have been interested in the shared/distributed ledger technology (a.k.a. block chain, a.k.a. the magic behind cryptocurrencies) for more than a year already but recently I had finally put real time and effort into it.

And since I believe that the best way to understand something is to get your hands dirty, that is what I have done, after I got a grasp of the core principles (or that is what I thought back then), I decided to code my own shared ledger simulator.

Initially, I also considered to look into the main existing code bases (e.g., the source code of the main/official Bitcoin client/miner or Ethereum's) since they are open source, but seeing code like thisput me off... That file is 3229 lines big!!! Plus it is written in C++.
Do not get me wrong, I truly believe Satoshi Nakamoto (i.e., whoever hides behind that name) is a genius and also a great software developer, but he/she/they for sure did not have readability as their main priority.

I also noticed that some other people h…