Microservice + RabbitMQ application - 1

I have written and analyzed a microservice application with spring cloud in my first series of posts. You need to understand that series in order to start with this one. Because i will integrate message queue structure with RabbitMQ on top of that existing project. Because the most effective and clear way to use a message queue tool is usually using it as part of a microservice architecture. We won't write a whole new project from scratch with RabbitMQ. We will see how it is integrated on top of an existing application and we will analyze our gains by using this tool. This series of posts will be divided into these 5 parts.

• Introduction (This post)
  • What is message queue and message broker
  • Which tool did we use and why
  • How is the existing microservice application designed

• RabbitMQ
  • What is RabbitMQ, what are the pros and cons
  • What are the important features
  • How can load balancing and error handling work
  • What are the exchange types
  • What are the alternatives

• Message sender methods
  • How to send messages with Spring AMQP
  • How to configure settings
  • How to use RabbitMQ management console

• Message receiver methods
  • Adding the mail service to the system
  • How to fetch Payload data with RabbitListener
  • How to configure a deadletter queue

• The analysis of the message queue
  • How did we use it, how could it be used
  • What are the possible handicaps
  • How can we take cautions for possible errors

What is a message queue and message broker

The project that i have built before was working with SAGAs and orchestration. Therefore there was an MVC application making calls to small services and managing some transactions according to the responses. All these operations were synchronous requests and responses. You can also develop operations that don't require synchronous communication. For example, sending an email or a report could be working lots of time in the system without the need of any response. Long story short, you can establish asynchronous interactions among the parts of a system with messaging infrastructure and you can queue these messages to deliver them sequentially.

The small parts of the system can send messages to each other or the central coordinator can communicate with small services over these messages. Also, some of these messages might have to reach different endpoints depending on some criterias. Connecting all these parts with each other would create spaghetti relationships and won't be maintainable. Therefore you need a message broker to deliver these messages to relevant correspondents. These brokers constantly receive and add these messages to queues to deliver them. They usually also carry out extra operations during this process. This is what we call a message queue. All of the message broker tools today has this same approach at the basic level and they are introducing complex features on top of them.

Which tool did we use and why

We will integrate RabbitMQ on top of the existing microservice project. Here is the decision making process, which is crucial for the system because it will be almost impossible to change this tool later on. First i have searched for the message broker tools alternatives. I have only had some acquaintance of these tools and it would have been a mistake to decide on one without knowing any details. I have found blog posts and other articles on the internet that compares these message brokers' features, up-to-date states and use cases. And RabbitMQ turned out to be the most logical choice as an open source tool.

Because it has been developed since 2010 and it has plenty of features. Let me mention something here. Since it has lots of capabilities by itself, i have seen weird usages of rabbitmq in older posts while i research. This was confusing at the beginning. This confusion is disappeared when i look at the updated documentation a little bit. I have realized that rabbitmq got new capabilities (like load balancing, resiliency and fault tolerance) over time with the developments in spring cloud and microservices. At this point I have realized that it is mandatory to use the message broker tool with conscious and avoid randomly using it without knowing its capabilities. Otherwise it could turn into a soup with bunch of features in the middle of the system, working as it wishes. This message broker will affect your systems capabilities since it can't play every role that is given. That is why you have to determine the role of the message broker thoroughly in order to not to complicate the system.

Let me get back to the comparison. There are three suitable alternatives to use in this kind of blog post. RabbitMQ, ActiveMQ and Kafka. There is a comparison of these three over the internet. You can find it here. In the end, the reason i chose rabbitmq is that it is a bit easier to use. Kafka looked more professional. Activemq looks like it has less documentation and community support. Rabbitmq sits between these two as it teaches you the message broker logic and it is easy to use. It has support for lots of programming languages. You can find solutions for your problems on the internet. Of you are curious about message brokers, there are products of big players like Azure, Amazon, IBM and Google.

How is the existing microservice application designed

If you have intermediate or advanced knowledge about microservices and spring cloud, you can find the previous project on github and try to run it on your local machine. You can get help to run it from a previous post. This project was an imaginary ticketing application where users were logging in and buying tickets for certain events. This project has 3 small services and an MVC application to coordinate the operations. I will show how to use the rabbitmq on top of this application by using it for emailing to users for reservations or cancelations. It would be nonsense to send dozens of emails so it will be a prototype code :) These emails will wait in a queue and they will be sent one by one. If you are asking where the RabbitMQ will be placed in our system, here is the architecture that i will show in the rest of the series.

As you can see in the drawing, some requests are sent from the MVC application to RabbitMQ while the system is working. Message broker is directing the messages according to the routing logic inside it to the relevant mail service. We will develop this mail service. The undeliverable messages will redirect to deadletter queue. Even though rabbitmq is used for its basic functionalities, it will be a vital part of our system. The decisions we will take is going to affect and communication between the MVC app and the mail service. We will analyze the details of this system in this series.

You can also find the latest working application of this new application on github and try to run it yourself. But you will need rabbitmq installed and working. You will also need the config repository cloned to your own github. I recommend you to follow this series so that you could understand it deeper. Let me summarize the situation: We will use a message broker to handle async operations on our microservice architecture. We will use it for emailing operations. We have investigated various tools and decided to use RabbitMQ. We have positioned the broker next to the existing system and gave it a specific role, which is delivering messages to mail service. We have taken the first step, which is the most important one, in this post. Now it's time to dig into RabbitMQ. See you at the next post :)