A SAGA microservice application - 3

In this post, we are starting to run the projects of our SAGA based microservice architecture with spring cloud. The backbone of our system will run with 3 main components, Eureka, Config Server and Gateway. I would recommend you to read the first post of the series to understand the architectural desicions, requirements and goals, if you haven't. And we have intorduced this 3 components in the last post. Let's start these 3 spring boot projects and set the backbone of our microservice. Here is the architecture to remember.

Ingredients

Let's recap these definitions first. We have Eureka to collect information about the running services and redirect with load balancing among them. We use the gateway for routing and utilizing the Eureka. And the Config server helps us centralizing the configurations of these services. Config server was not mandatory in our design but i have added it thinking it would be helpful if the project evolves into a big microservice. All these 3 projects will be spring boot projects and i have written another post about creating a spring boot project with spring initializr.

Eureka

We need to create a spring boot project with 2 depdendencies at the spring initializr project. Eureka Server and Spring Security. You can set group and artifact names however you want. My example used "com.aldimbilet" as group and "Eureka" as artifact. The selected spring boot version must be higher than 2.4.0. This is necessary to ensure the spring cloud libraries to work compatibly. The project framework is done. You can download the zip and open it up with and IDE.

After importing the project, there should be spring-cloud.version in the pom file. The version name here must be 2020... versions. The old spring cloud versions (like Hoxton or Greenwich) are not compatible with spring boot 2.4.0. For more information, you can check out the "Release train Spring Boot compatibility" at Spring cloud website.

It will be enough to add @EnableEurekaServer in the main class of this application to run it as Eureka server and make the Eureka interface available. It would look like this:

			
@EnableEurekaServer
@SpringBootApplication
public class AldimbiletEurekaApplication
{
	public static void main(String[] args)
	{
		SpringApplication.run(AldimbiletEurekaApplication.class, args);
	}
}
			
		

Also, we need to secure the access to Eureka web console from outside with a username and password. We have already added Spring Security dependency, so we can set the username and password too. In the application.properties you can write these values:

			
spring.security.user.name=aldimbilet
spring.security.user.password=eureka
			
		

We also need to set a port number for this server. Normally, 8761 is the default port for eureka but i wanted to change it to see it working. I also added the name for the application to be able to identify the file easily, when the project creates more and more properties files.

			
server.port=4442
spring.application.name=ab-eureka
			
		

At this stage, we are done with the spring boot project to run. But i said Eureka is a registery service. If you try to run this app as spring boot app, the server will try to register itself and throw some kind of "connection refused" error. We have 2 more properties in this file to prevent this.

			
eureka.client.register-with-eureka = false
eureka.client.fetchRegistry = false
			
		

Now we are ready to run the project properly. In eclipse, you can chose Run as -> Spring boot application. You can access the Eureka console via http://localhost:4442. I recommend you to get familiar with this screen a little bit.

We will have to add a security config too. We will use JWT in the services and those tokens will be passed with headers. Therefore we need to disable CSRF (cross site request forgery) protection with spring security. It is not mandatory if you don't use JWT. You can disable CSRF check and authorize all the other endpoints with a @Component class like this.

			
@Configuration
public class SeConfig extends WebSecurityConfigurerAdapter
{
	@Override
	protected void configure(HttpSecurity http) throws Exception
	{
		http.csrf().disable().authorizeRequests().anyRequest().authenticated().and().httpBasic();
	}
}
			
		

Config server

Now we can prepare the config vault if you are done with Eureka. We will keep all the relevant configuration (.properties files) with cloud config. This vault will be a private git repository. So the config server will go fetch the property files for the related projects from github. This is a lot like a git clone process. Config server can also store the properties in local files but it is always wise to use github, since you can easily access it and track the changes. Let's create gonfig server project like this:

If you take a look at the dependencies, there are both "config server" and "config client". This project acts as the property server and can also fetch its properties from github too. We use spring security to protect it with a username and password. We will have "go register yourself at Eureka upon launch" function with Eureka Client (not server). But the most important dependecy here is the Bootstrap dependency. Spring boot can't use bootstrap.properties after spring boot 2.4.0 version without this dependency.

What is bootstrap? It is basically the properties to set and use while the spring application is booting. Application.properties is the properties to set and use after the application is booted. We will use bootstrap.properties in this project. Because the github connection should be established and settings must be fetched while the app is starting. We will also use bootstrap in other services too. Lastly, make sure the spring cloud version is some 2020... version.

Let's start with application.properties. We will have a port number this file. The default port for gateway is 8888 and for some reason, i couldn't change it. The other services insisted on connecting to 8888 port and didn't listen to me. So i left it as 8888. You might also guess it is almost mandatory to set a username and password to access this server.

			
server.port=8888
spring.security.user.name=aldimbilet
spring.security.user.password=config
			
		

And then the bootstrap.properties (which you should manually create right next to application.properties) for the github settings. First we have a profile name here. It might be useful to have prod, dev or test profiles in your projects. This profile name will make it easy to fetch, set, find and change the necessary DB configurations. Config server will look for the file with the profile name appendix in the property files. We also must provide a name to be able to register to Eureka.

			
spring.profiles.active=local
spring.application.name=ab-config-server
			
		

You can set the github repository settings to connect to github like below. The "clone on start" property tells the server to fetch the last repo from the github on start up. The default-label is important here. Because github recently changed the main branch names from master to main and some apps started crashing. Spring boot hasn't adapted to this yet. Up-to-date info :)

			
spring.cloud.config.server.git.uri=https://github.com/<username>/<reponame>
spring.cloud.config.server.git.clone-on-start=true
spring.cloud.config.server.git.default-label=main
spring.cloud.config.server.git.username=<gituser>
spring.cloud.config.server.git.password=<gitpass>
			
		

With the properties below, you can also let this app fetch its own configurations from github. Even though it is the config server, it would be meaningless to request the configs from itself. "Bootstrap = true" means this fetching and serving must be done at boot process. Fail-fast means "if you can't fetch the properties, it is a fatal error. Stop it and throw an exception".

			
spring.cloud.config.server.bootstrap=true
spring.cloud.config.discovery.service-id=ab-config-server
spring.cloud.config.fail-fast=true
			
		

Now you would ask "where is the git repository?". That repository is private and cannot be accessed from outside. After all, it has DB properties. Let me put the screenshot of the repository here. You can download it here and push it to your own github. I will also mention the contents of these files when it is required.

Now you should create a repository like this and update the github connection information in the properties file. There must be the Eureka connection properties in the ab-config-server-local.properties file. Which is named as "application name - profile.properties". Attention to the username and password was set in the Eureka project's properties file.

			
eureka.client.service-url.defaultZone=http://aldimbilet:eureka@localhost:4442/eureka
			
		

I have saved the most important to last. Just because there is a config server dependency in the pom file does not mean it will start serving the properties automatically. It will register itself to eureka automatically with Eureka client dependency. But you need @EnableConfigServer annotation in the main class for config server to start. This is important to remember.

			
@EnableConfigServer
@SpringBootApplication
public class AldimbiletConfigApplication
{
	public static void main(String[] args)
	{
		SpringApplication.run(AldimbiletConfigApplication.class, args);
	}
}
			
		

The config server must be ready to run at this stage. There must be "Located property source" expression in the console while starting up, meaning it is fetching the properties from github. There will also be "Registering application AB-CONFIG-SERVER with eureka" kind of infoırmation in the console. Upon launching, this server must be visible in the Eureka console. If this is all done, lets jump to the last component.

Gateway

Our architecture requires a gateway for the whole microservice system. You may see load balancing on gateway in some examples. Because it is possible to access to small services without Eureka. As far as i know, there are 2 libraries to achieve this. Zuul and Gatekeeper. I have decided these are not that much flexible alternatives. Because they have routing information in the property files and you can't differentiate or customize routing according to headers or request types or bodies or etc. So i have decided to use spring cloud gateway like the image below:

The first dependency is the spring cloud gateway. We will be able to create router classes and set their properties to route the incoming http requests with this dependency. Resiliance4J is helping us to redirect the routing if the services are not available for some reason. This is also called Circut Breaker. If you think of the request journey like a circuit, it will redirect to an alternative path instead of throwing an exception when there is a cut in the circuit. Hystrix was the old alternative for this functionality. It has been used with Eureka to inform about the errors and it was working when Eureka fails to route the request. But Resiliance4J is used with gateway and it works without the registery server. We can also customize flexible routing logics with it. We have used "eureka client" to register to Eureka and "config client" to fetch the properties from the config server. Of course we need the bootstrap dependency too. Remember to check the spring-cloud.version being 2020... something in the pom file.

We will have only 1 property inside Application.properties and it is the port number. Gateway cannot have a random port number you know.

			
server.port=4441
			
		

We have familiar properties inside the bootstrap.properties. Profile name indicates the profile property to fetch from the config server (ultimately github). We have named the application so that we can identify it in Eureka console.

			
spring.profiles.active=local
spring.application.name=ab-gateway
			
		

Then we set the config service id that we will connect as config client. I think it will automatically go to port 8888 if you don't set this. Fail-fast means "fetch the config or you are screwed". We also set the username and password that has been set in the config server security options. Otherwise you will get security error while fetching the configurations.

			
spring.cloud.config.discovery.service-id=ab-config-server
spring.cloud.config.fail-fast=true
spring.cloud.config.username=aldimbilet
spring.cloud.config.password=config
			
		

If you haven't created the file in the github, you must have ab-gateway-local.properties in your repository. There will be Eureka connection information in there. Otherwise the app will look for a Eureka server at 8761 port and can't find it and throws exceptions and makes you angry.

			
eureka.client.service-url.defaultZone=http://aldimbilet:eureka@localhost:4442/eureka
			
		

Now you can actually run the gateway as spring boot app but it won't redirect any requests. You need to create a RouteLocator for routing. I have shortened these codes to avoid junk code here. You can find the rest of it on github. RouteCreator is my custom class, you can't find it in spring framework :)

			
@Autowired
RouteCreator creator;

@Bean
public RouteLocator loadBalancedRoutes(RouteLocatorBuilder builder)
{
	Function<PredicateSpec, Buildable<Route>> user_path;
	Function<PredicateSpec, Buildable<Route>> user_fail_path;
	user_path = creator.createRouterFunctionWithFallbackFilter("/user/**", "lb://ab-userservice", "user-cb", "forward:/user-failover", "user-failover");
	user_fail_path = creator.createRouterFunction("/user-failover", "lb://ab-userservice-failover");
	Builder routes = builder.routes();
	routes.route("user route", user_path);
	routes.route("user failover route", user_fail_path);
	return routes.build();
}
			
		

There are 2 route functions defined here. First one will go to userservice and the second will go to userservice-failover. Let's take a look at createRouterFunctionWithFallbackFilter parameters. "/user/**" means we are routing /user endpoints. We will use this endpoint when we are coding the userservice. "lb://ab-userservice" means send this to Eureka server and call the service with the name "ab-userservice" with load balancing if more than 1 available. This name must match the name in the userservice properties file that we will implement !

And then we have "user-cb" to indicate the circuit breaking path if the userservice is not available. Resiliance4J helps us here with the definitions. We redirect the requests to the "/user-failover" path (not service name) with the "forward:/user-failover" expression. And the "user-failover" is the route name for this route.

We have the second function for userservice-failover. Let's see the createRouterFunction parameters. We are roting the requests coming to "/user-failover" with "lb://ab-userservice-failover" to Eureka. We will define the paths and endpoints in the ab-userservice-failover with "/user-failover" path. We will also match the service name as ab-userservice-failover there.

There will also be activityservice ve paymentservice route definitions here. You can literally copy and paste it from github. You have to add the necessary routing for all the services that you add to the system. Also if there are failovers, they must be routed as fallbacks too. We have defined the most basic routings here. You can create more detailed routings according to headers or request bodies or other request properties. Keep this in mind too.

This class creates the routelocator bean with @Configuration annotation on start up. This way, the gateway must be able to run as spring boot application and fetch the config from the config server and register itself to Eureka. You must be able to see "Fetching config from server at : http://localhost:8888" and "Registering application AB-GATEWAY with eureka" expressions in the console while starting. If you run these 3 projects with Eureka -> Config Server -> Gateway order, you must be able to see gateway and config server in the Eureka web console.

If you ask why would you register gateway to Eureka, i would say i wanted to know it's situation too. I have used Eureka for snitching purposes. Let me the show result screen in the Eureka web console. If you click on the names of the services here. It will try to access the Actuator endpoints. Actuator is a library that provides information about the spring boot applications. You can try to add it to gateway and config server projects as dependency if you want to.

It's aliveee :)

With couple of thousands of volts or something :) This post was very long but i didn't want to separate these 3 main components. This was the end of the backbone of our microservice. Now we will add small services and the MVC coordinator app to call these services. There were some requirements i have mentioned in the first post of the series. We now have used Eureka and Gateway to create flexible services to mitigate the outages. We have used config server to centralize the configurations and avoided a cumbersome structure. Eureka is ready to load balance the requests among the service instances.

You can add and connect your own new services on top of this system if you want. As long as you set the necessary eureka and config server properties and create the Routelocators, the infrastructure is ready to scale. This is also why i have called this "backbone" and kept it together in one post. It is time to develop userservice, activityservice and paymentservice. I will explain the userservice deeply and leave the other 2 to you, since they will have mostly similar code. See you at the next post :)