A SAGA microservice application - 1

Throughout this 7 part series of blog posts, i will explain how i have implemented an imaginary ticketing application with spring cloud microservices, starting from zero experience and some theoretical knowledge. I will also mention the experience i have gained while learning about this popular subject. Instead of "you can do this with this code", i will connect the dots and explain the spesific need, the troubles we have and the benefits we gained with code or structure. I also want to be able to clearify the "why" questions by relating these desicions with a domain. I haven't come across with a proper blog post which explains the transition from the theory to code :) Our example will be a "SAGA orchestration" based running project and will contain mostly up-to-date technologies, while sticking to mostly relevant tools.

When it comes to microservices, there are couple of questions that must be addressed beforehand: Why are you implementing microservices solutions, upon what requirements? How are you going to implement it, do you have a pattern? Which technologies are you going to use, why? Are there any alternative tools or approaches for the solutions you implement? Microservices is an abstract subject. Your end-product is going to be shaped by 3 main factors: Domain - technology - requirements / expectations. This is why this 7 part blog post is separated like the list below:

• Introduction to the project and the architectural desicions (This post)
  • What is our domain (aldimbilet.com), what is our scope
  • How can this project be divided into microservices
  • What is the resulting microservice arhchitecture and the reasons of our desicions

• Spring cloud and the tools we will use
  • What is Spring cloud and how does it help us
  • What tools and libraries are we going to use

• The backbone system
  • Eureka (Load balance), config server (github) and gateway

• Authentication and authorization with JWT and the user services
  • Development of a sample service
  • The guide to add a new service to the system

• Consumption of the services and the basic SAGA management
  • Developing a prototype MVC app
  • Management of the basic business rules and errors (SAGA)

• Running the microservices project locally
  • Necessary steps to run the services
  • Technical and physical requirements

• Analysis of the system
  • The advantages, disadvantages and the earnings
  • Possible future improvements and alternative designs

Our ticketing app

We took the most important step and decided to develop a project with microservices. I will mention my own thoughts about the microservices architecture while developing the project. Therefore you will be able to grasp the concept and follow along even if you don't have much prior knowledge. As i have mentioned in the beginning, it won't be enough to understand the microservices if i just talk about the abstract concepts and write down code with no explanation. This is why we decided to develop an app and imagined a website called "aldimbilet.com".

Ok but can we turn every project that we think of, into microservices? Technically the answer is yes. What is a microservice architecture: It is carrying out the big operations with the help of small participants. And the small participants are the restful web services, who works independently and prevents the system from getting cumbersome. This is a lot like a country being governed by small cities or states, with a common constitution. So technically you can divide any country into states or cities, but will it always make sense? Of course not. When the management and the volume of the project starts becoming an issue, thet project becomes a microservice transition candidate.

In a programming language, usually, there are helper codes or libraries to support the codes and classes that carries out the main flow. You can build a microservice architecture by applying the same logic on a structure that uses web services to communicate and operate. We don't have an app developed as a monolithic project so we must begin with analyzing the domain. Our sample project will be a ticketing app with users and events, like movies or concerts or etc. We aim to get users to register and make them able to search for events and buy tickets. Nothing major. This app will not be a fully fledged ticketing app because our purpose is to ponder on microservices architecture and learn the technical details. Therefore we will have prototype screens and simple databases. Don't worry, you will be able to run the whole architecture on your pc as well. Then you can imrpove it further and use it for your own initiatives :)

Requirements? Done. What about the method?

At this point, we know what we want. We want to have 3 main functions (users, events and payments) to be implemented. Note that we are mainly focusing on the functions and designing the systems according to them. There can be other concerns while you are developing a microservice architecture. Our system will consider these 3 functions as subdomains and will start a service for each of them. It is like a "ministery of users or events" :) Also, we don't expect high volume of traffic or concurrent operations in our ticketing app, since it is not an e-commerce app or does not have a vital role to play. Our system can tolerate 1 hour of downtime if something goes wrong. Users can wait 1 day to buy a ticket. This is our presumption. In the 7th post of this series, there will be discussions about different scenarios, if there were heavy traffic or lots of concurrent users or other strategic concerns.

Even though the system is not going to work under pressure, our tiny services will keep the system up and running reliably and consistently. What is reliability and consistency: Let's suppose our ticketing app won't allow reservations, in order to prevent spamming. You have to buy a ticket directly or go away. And the system will check the number of seats available before issuing the ticket. This made-up rule will force us to take measures for consistency. We make up a rule and try to find a proper solution for it. The solution for this rule would be a SAGA, which is basically a process of small and connected events. If the process fails at some point, the chain will be broken and the system must be able to compansate it. If you pay for the ticket and all the seats are taken, the payment must be returned. Let's note this here.

Now we have made 2 desicions. We will have 3 tiny services and there will be a center coordinator or orchestrator to manage the interactions inbetween. This was the first step of the development process. We have analyzed the domain and determined our requirements, constraints and goals. We have basically gone through the software engineering process. If you don't have much hands-on microservices experience, all of this will just float around in your head at this point. You might ask are we implementing a microservice just because there will be users and events and payments? This is the point where you realize the trade-offs. Normally, it is not always the best thing to apply microservices to all projects. But we have these reasons:

• If we develop a monolithic application, the ticketing app will grow bigger and become hard to manage
• The huge codebase will result in bigger deployments and it will also be hard to start it up
• Version control will get harder and result in more conflicts as time goes by
• The expenses will grow faster over time because we will need new servers to be able to run the monolithic system
• There could be more effective solutions in different programming languages for specific operations in the system but we will have programming language dependency

If you are having these questions or troubles, you might want to start thinking about migrating to a microservice architecture. We have determined our requirements, realized our risks and decided that microservices could be a proper solution for our system. You must think twice before jumping in if you don't have qualified software developers or if you don't have a corporate culture for software development. Since we are the developers who set sail for microservices development, we don't have these concerns :)

At this point, i want to ask a question to ponder on about the microservices. Imagine the country divided into cities or states again. These states or cities are managed by a central government and they are handling their specific operations within. Would it be ok to not have a central government? Is microservices always structured like a network hub where you have a central router and every small service connect to it? Technically no. As far as i understand, there are 2 main approaches here. First one is the coordinator and it is basically managing the services with a higher level of mechanism. The second one is every service passing a message to another service after it has completed it's responsibility. You can imagine this as cities being connected with roads and handling the management among themselves. We have chosen the first approach since our business requirements are not that much complicated and our web app does not seem to have a lot of load. Of course we may regret these desicions later if our goals are not well thought out or concrete. Since we won't change our idea throughout this series of posts, there is no problem for us :) But always remember to ask these questions to yourself. Is it necessary? If so, how so?

Let's summarize again. I know this post is very abstract but don't worry, the technical side will start with the next post. We have now asked "why would we implement a microservice" and responded to ourselves. We have mastered self conversations as software engineers. And we wondered "How should we implement it" and decided to put an orchestration infrastructure. We know what we want, it is building a reliable and flexible structure within a certain scope. Let me remind you again that these are the most important steps of the process. Now it is time to decide the technologies. We must start coding with certain technologies while taking our requirements, limitations and goals into account.

Method? Done. What are the tools?

Ok but i am a developer. I will eventually write some DB statement codes somewhere. I will write if and else statements at the end of the day. How can i know where to put these codes? Am i just going to start a project and write code? What should i do to create a service? How will a central system know the tiny services running around? Is it about network programming? Is it enough to create 3 small web service projects? If you don't have experience in microservices, these must be similar questions in your mind right now.

This is the part where theoretical concepts turn into practical applications. This was hardest part for me. Because i had techincal knowledge about the microservices but i have never actually implemented them. I didn't know the state of the art and couldn't ask anyone. That's why i have made architectural mistakes at the beginining and changed it along the way. For example, i thought the tiny services would talk to each other via dynamic routing without a messaging tools inbetween. I also didn't know that there were crucial differences between the message queue based systems and central management systems. Right now, you are proceeding on a preset architecture but i will also mention some mistakes i have made in the other posts. You might be subjected to make the same mistakes sooner or later.

I have schematized the architecture with a simple drawing below. This image is designed to set the big picture of the system. You can find tons of different designs about the microservices over the internet. I too found them but they didn't help me code. I found code but they didn't fit my system. So i had to write my own system from scratch. Here is the schema:

It's ok if you don't understand what this schema means. I will explain the logic of this schema and talk about the spring cloud technologies which will help us build this architecture. This image will be placed on top of every post in this series. Because at the end of the day, our requirements, constraints and goals will be materialized if we successfully turn this schema into code. See you at the next technical post :)