Skip to main content

Kafka + WebSockets + Angular: event-driven microservices all the way to the frontend

In the the initial post of the Event-driven microservices with Kafka series (see here or here), I talked about the advantages of using event-driven communication and Kafka to implement stateful microservices instead of the standard stateless RESTful ones.
I also presented the architecture and the source code of a related proof of concept application.

In this post, I would like to show how to extend the asynchronous event-driven communication all the way from Kafka to the Web frontend passing through the Java backend.
Hence, in the first post of this series, we got rid of HTTP as the communication protocol among microservices in the backend, and now we are also replacing it (with WebSockets) as the communication protocol between the frontend and the backend.

Ok, but why would you do that?

Because it provides a better experience to the end user!.
Using WebSockets you can build legit real-time user interfaces, the updates are pushed immediately from the server to the client, no need for the user to click any button nor to refresh the browser.

Wait, but we can do the same using good old HTTP, can't we?

Well... HTTP was not designed to provide real-time communication, but since people really wanted it, a number of workaround techniques and technologies have been create on top of it, such as:
  • Long polling
  • Short polling
  • Server-Sent Events
Those are little more than hacks, IMHO.
WebSockets, on the other hand, provide full-duplex bidirectional communication, which means that information can flow from the client to the server and also in the opposite direction simultaneously, it cannot get more real-time than that (that's what I meant when I said 'legit real-time applications' before).

HTTP vs WebSocket (source)

The example

I added a new service to the same PoC application which I used in the previous post and also an Angular web frontend.

Ok, but what does this new service do?

This service allows the frontend user to receive information about the current balance of any account of her/his choice as well as live updates when the balance changes.
Also, information about any transactions related to that currently selected account (i.e., any amount of money credited or debited) past or live, is displayed to the users.

Architecture

The new Transfers WebSockets service main components are:
  • Two Kafka consumers (one for each topic) to retrieve messages from the Kafka cluster
  • Two Kafka Streams local stores to retrieve the latest data associated with a given key (id)
  • A custom local store implemented using a simple Map to store the list of transactions for a given account.
  • Two dispatchers which keep track of what WebSockets sessions (i.e., frontend clients) are registered to receive live balance updates or live transfers data related to which account and then dispatch the updates accordingly.
And these are the main components of the Angular frontend:
  • A WebSockets service which handles the connection lifecycle and dispatches the received messages to the corresponding RxJS Subject. Please note that different types of JSON messages (requests, responses and updates) flow through the only WebSockets connection.
  • A form which passes the user input (i.e., the id of the chosen account) to the WebSockets service so it can be sent to the remote server.
  • An Angular component to display the account balance which gets asynchronously notified of balance changes (updates) through an RxJS Observer which is subscribed to the corresponding RxJS Subject in the previously mentioned WebSockets service, and then re-renders the value accordingly.
  • Another Angular component which similarly as the previous one, gets asynchronously notified of any new transaction related to the selected account and adds it to the list of displayed transaction data.
This is a diagram depicting the PoC application components and several frontend instances:

See it in action



Check out this Youtube video for a demo, where a test process which continuously generates Kafka messages representing random money transfers is used to feed the application.
And on the Web UI the user can see how the transfers pops up as they happen and the balance of the related account changes accordingly, all in real time.

Final Thoughts

HTTP has had a huge success, it made possible the WWW as we know it today.
It has been so successful that it has spread to the backend (RESTful services and open APIs) where it has been the king for more than a decade already.
However, technology evolves, paradigms come and go and for some use cases other tools and designs may be the right choice.
I do not know how software applications will look like in the future, all I can do is guess.
Event-driven microservices look very powerful and implementing them with Kafka and adding WebSockets to the mix seems promising.
In fact, some companies are able to see beyond REST APIs and are starting to offer WebSockets APIs to their clients for faster streaming access to data (example).
And there are other alternative ways to implement event-driven microservices, for example, recently I learnt about Uber Cadence but that's for a future post 😃

Comments

  1. I have read all the comments and suggestions posted by the visitors for this article are very fine,We will wait for your next article so only.Thanks! Combo List Worldwide All Account Fresh RDP

    ReplyDelete
  2. You are a blogger and now my inspiration. I just love the blog post. Its very informative, interactive and quality content. I wish you all good luck for your coming blogs and posts. Keep sharing!
    ios development company

    ReplyDelete

Post a Comment

Popular posts from this blog

A Java dev journey to full-stack: first chapter

The Motivation I am an experienced Java developer and (surprise!) I like Java. I know it is not perfect but it works just fine for me (I enjoy type-safety and I do not consider verbosity a disadvantage, quite the opposite). I also know that some people dislike Java, which is also fine. But recently I decided to step out of my confort zone as developer, my goal isn't to be one of the "cool kids" neither trying to monetize a new skill in the job market. I have a quite practical motivation: I want to be able to build more (different) stuff. That's exactly the same reason why I learnt Android development by myself a couple of years ago. Web applications are ubiquitous, even more than native mobile apps, and thanks to cloud computing, one can easily and inexpensively release their idea/app to the World Wide Web. I already did some Web development in the past, in the bad old days of JSP and JSF, but the process was slow and painful. Nowadays the Web landscape h

Using Apache Kafka to implement event-driven microservices

When talking about microservices architecture, most people think of a network of stateless services which communicate through HTTP (one may call it RESTful or not, depending on how much of a nitpicker one is). But there is another way, which may be more suitable depending on the use case at hand. I am talking about event-driven microservices, where in addition to the classic request-response pattern, services publish messages which represent events (facts) and subscribe to topics (or queues depending on the terminology used) to receive events/messages. To fully understand and embrace this new software design paradigm is not straight-forward but it is totally worth it (at least looking into it). There are several interconnected concepts which need to be explored in order to discover the advantages of event-driven design and the evolutionary path which led to it, for example: Log (including log-structured storage engine and write-ahead log) Materialized View Event Sourcing C o