Understand Code Refactoring Techniques to Improve your Code Quality

Photo by Danial Igdery

Now a days, agile teams are under tremendous pressure to write code faster with enhanced functionality in short time. There would be some or the other functionality added at the last moment or just before the release. As engineers are under pressure, the functionality gets implemented in a sloppy manner, which may technically work but may lead to dirty code.

Dirty code usually results from a developer’s inexperience, shortcuts taken to meet increasingly tight deadlines, poor management of the project, several different developers working on a project over time, or some combination of all of the above.

Bad code comes at a price, and writing good code isn’t that complicated. Let's understand what's code refactoring.

Code Refactoring

In computer programming and software design, code refactoring is the process of restructuring existing computer code—changing the factoring—without changing its external behavior. Refactoring is intended to improve the design, structure, and/or implementation of the software (its non-functional attributes), while preserving its functionality. 

Potential advantages of refactoring may include improved code readability and reduced complexity; these can improve the source code's maintainability and create a simpler, cleaner, or more expressive internal architecture or object model to improve extensibility. 

Another potential goal for refactoring is improved performance; software engineers face an ongoing challenge to write programs that perform faster or use less memory.

Code refactoring helps to change this dirty code into clean code and it helps to make it easier to extend the code and add new features easily in the future. Also, helps to improve the more objective attributes of code such as code length, code duplication, and coupling and cohesion, all of which correlate with ease of code maintenance and your code will use less memory and perform better and faster.

How to Perform Code Refactoring?

Now that you know the answer to the question of what is code refactoring, and you know many of its potential benefits, how exactly do you refactor code?

There are many approaches and techniques to refactor the code. Let’s discuss some popular ones.

Red-Green Refactor

Red-Green is the most popular and widely used code refactoring technique in the Agile software development process. This technique follows the test-first approach to design and implementation, this lays the foundation for all forms of refactoring.

Red - The first step starts with writing the failing red-test. You stop and check what needs to be developed.

Green - In the second step, you write the simplest enough code and get the development pass green testing.

Refactor - Find ways to improve the code and implement those improvements, without adding new functionality.

Refactoring by Abstraction

This technique is mostly used by developers when there is a need to do a large amount of refactoring. Mainly we use this technique to reduce the redundancy (duplication) in our code. This involves class inheritances, hierarchy, creating new classes and interfaces, extraction, replacing inheritance with the delegation, and vice versa.

Pull-Up Method - It pulls code parts into a superclass and helps in the elimination of code duplication.

Push-Down Method - It takes the code part from a superclass and moves it down into the subclasses.

Refactoring by abstraction allows you to make big changes to large chunks of code gradually. In this way, you can still release the system regularly, even with the change still in progress.

Composing Method

Code that is too long is difficult to understand and difficult to implement. The composing method is a code refactoring approach that helps to streamline code and remove any code duplications. This is done through extraction and inline techniques.

Extraction - We break the code into smaller chunks to find and extract fragmentation. After that, we create separate methods for these chunks, and then it is replaced with a call to this new method. Extraction involves class, interface, and local variables.

Inline - Refactoring also helps to create simpler, more streamlined code. It helps to remove unnecessary methods within the code and replaces them with the content of the method. After that, we delete the method from our program.

Simplifying Methods

As legacy code gets older and older, it tends to become more polluted and complex. In this sense, simplifying methods help to simplify the logic. These methods include adjusting the interaction between different classes, along with adding a new parameter or removing and replacing certain parameters with explicit methods.

Simplifying Conditional Expressions - Conditional statement in programming becomes more logical and complicated over time. You need to simplify the logic in your code to understand the whole program. There are so many ways to refactor the code and simplify the logic. Some of them are: consolidate conditional expression and duplicate conditional fragments, decompose conditional, replace conditional with polymorphism, remove control flag, replace nested conditional with guard clauses, etc.

Simplifying Method Calls - In this approach, we make method calls simpler and easier to understand. We work on the interaction between classes, and we simplify the interfaces for them. Examples are: adding, removing, and introducing new parameters, replacing the parameter with the explicit method and method call, parameterize method, making a separate query from modifier, preserve the whole object, remove setting method, etc.

Extract Method

The extract method is one of the techniques for code refactoring that helps to decrease complexity, while also increasing the overall readability of the code.

When you find that a class has so many responsibilities and too much thing is going on or when you find that a class is unnecessary and doing nothing in an application, you can move the code from this class to another class and remove it completely from the existing class. It involves the moving of a fragment or block of code from its existing method and into a newly created method, which is clearly named in order to explain its function.

Conclusion

Engineers are the only ones responsible for writing good and quality code. We should all make it a habit to write good code from the very beginning. Writing clean code isn’t complicated and doing so will help both you and your colleagues. A clean and well-organized code is always easy to change, easy to understand, and easy to maintain.

Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin

Even bad code can function. Every year, countless hours and significant resources are lost because of poorly written code but it doesn't have to be that way.

Refactoring: Improving the Design of Existing Code by Martin Fowler

Any fool can write code that a computer can understand. Good programmers write code that humans can understand.

Quick Tips

• Always perform code refactoring in small chunks by making your code slightly better and leaves the application in a working state. Run jUnit tests after making small changes in the refactoring process. Without running these tests, you create a risk of introducing new bugs.

• Do not create any new features or functionality during the refactoring process. You should refactor the code before adding any updates or new features into your existing code.

• Refactoring process always results in complete regression, don't forget to involve your QA team in the process.

Read More

Understand and Analyze Java Thread Dump

Photo by Mel Poole

Microservices

Also known as the microservices architecture, is an architectural style that structures an application as a collection of services that are:

• Easily Maintainable and Testable
• Loosely Coupled
• Independently Deployable
• Organized around Business Capabilities
• Owned by a Small Team

The microservices architecture enables the rapid, frequent and reliable delivery of large, complex applications. It also enables an organization to evolve its technology stack.

The decentralization of business logic increases the flexibility and most importantly decouples the dependencies between two or more components, this being one of the major reasons as to why many companies are moving from monolithic architecture to a microservices architecture.

What is a Thread?

All of us have probably written a program that displays "Hello World!!" or given word is a palindrome or not etc. These are sequential programs that have a beginning, an execution sequence and an end, at any given point of time during the execution of a program, there is a single point of execution.

A single thread is also similar, as it has a beginning, an execution sequence and an end. However, a thread itself is not a program, a thread cannot run on its own, it runs within a program.

A program can consist of many lightweight processes called threads. The real excitement surrounding threads is not about a single sequence. It helps to achieve parallelism wherein, a program is divided into multiple threads and results in better performance. All threads within a process share the same memory space and might have dependency on each other in some cases.

Lifecycle of a Thread

For understanding a thread dump in detail, it is essential to know all the states a thread passes through during its lifecycle. A thread can assume one of these following states at any given point of time:

NEW

Initial state of a thread when we create an instance of Thread or Runnable. It remains in this state until the program starts the thread.

RUNNABLE

The thread becomes runnable after a new thread is started. A thread in this state is considered to be executing its task.

BLOCKED

A thread is in the blocked state when it tries to access an object that is currently locked by some other thread. When the locked object is unlocked and hence available for the thread, the thread moves back to the runnable state.

WAITING

A thread transitions to the waiting state while waiting for another thread to perform a task and transitions back to the runnable state only when another thread signals the waiting thread to resume execution.

TIMED_WAITING

A timed waiting state is a thread waiting for a specified interval of time and transitioning back to the runnable state when that time interval expires. The thread is waiting for another thread to do some work for up to a specified waiting time.

TERMINATED

A runnable thread enters the terminated state after it finishes its task.

Thread Dumps

A thread dump contains a snapshot of all the threads active at a particular point during the execution of a program. It contains all relevant information about the thread and its current state.

A new age application development involves multiple numbers of threads. Each thread requires certain resources, performs certain task related to the program. This can boost the performance of an application as threads can utilize available CPU cores. But we do have some trade-offs, for example, sometimes multiple threads may not co-ordinate well with each other and a deadlock situation may arise depending on the program. So, if something goes wrong, we can use thread dumps to identify the state of our threads.

As Java has been most popular language among application development, let's consider our application is built using spring-boot. If you want to take a snapshot of application threads, then we can go ahead with taking thread dump. A JVM thread dump is a listing of the state of all threads that are part of the process at that particular point of time. It contains information about the thread’s stack with other important information. The dump will be in a plain text format, the contents can be saved and analysis can be done either manually or using some UI that are available.

Analysis of thread dumps can help in following areas:

• Tweak JVM performance
• Tweak application performance
• Identify threads related problems within application.

Now we know the basics of thread and it's life cycle. Let's get into the next stage where we will explore how to take a thread dump from any running Java application. 

There are multiple ways to take a thread dumps. Am going to discuss about some JVM based tools and can be executed from the CLI or GUI tools.

Java Stack Trace

One of the easy way to generate a thread dump is by using jStack. jStack is a utility that ships with JVM, it can be used from the CLI and it expects the PID of the process for which we want to generate the thread dump.

jstack -l 1129 > thread_dump.txt

Java Command

JCMD is a command-line utility that ships with the JDK and are used to send diagnostic command requests to the JVM, where these requests are useful for controlling Java Flight Recordings, troubleshoot, and diagnose JVM and Java Applications. It must be used on the same machine where the JVM is running, and have the same effective user and group identifiers that were used to launch the JVM.

We can use the Thread.print command of jcmd to get a list of thread dumps for a particular process specified by the PID.

jcmd 1129 Thread.print > thread_dump.txt

Java Console

The jconsole GUI is a monitoring tool that complies to the Java Management Extensions (JMX) specification. It ships with the JDK and uses the extensive instrumentation of the Java VM to provide information about the performance and resource consumption of applications running on the Java platform.

Using the jconsole tool, we can inspect each thread’s stack trace when we connect it to a running java process. Then, in the Thread tab, we can see the name of all running threads. To detect a deadlock, we can click on the Detect Deadlock in the bottom right of the window. If a deadlock is detected, it will appear in a new tab otherwise a No Deadlock Detected will be displayed.

To launch the GUI tool, just type the below command on CLI.

jconsole

VisualVM

VisualVM is a GUI tool that helps us troubleshoot, monitor and profile Java applications. It perfectly fits all requirements of application developers, system administrators, quality engineers and end users.

As it's an external program, you need to download and install it on your machine. The GUI tool is very easy to use and lot of things you can monitor and troubleshoot things related to Java Applications.

Understanding Thread Dump Contents

Now, Let’s see what are the things we can explore using thread dumps. If we observe the thread dump, we can see a lot of information. However, if we take one step at a time, it can be fairly simple to understand.

1129:

2021-10-13 12:57:15

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode):

"Attach Listener" #142 daemon prio=9 os_prio=31 tid=0x00007f8dc7146000 nid=0x440b waiting on condition [0x0000000000000000]

   java.lang.Thread.State: RUNNABLE

"http-nio-8080-Acceptor" #138 daemon prio=5 os_prio=31 tid=0x00007f8dc7fab800 nid=0x9c03 runnable [0x000070000c59f000]

   java.lang.Thread.State: RUNNABLE

        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method

• The thread dump entry shown above, starts with the name of the thread Attach Listener whose ID is 142 thread (indicated by#142) created by the JVM after the application has started.
• The daemon keyword after the thread number indicates that it's a daemon thread, which means that it will not prevent the JVM from shutting down if it is the last running thread.
• After that we have are less important pieces of metadata about the thread like a priority, os priority, thread identifier, and native identifier.
• The last piece of information is the most important, the state of the thread and its address in the JVM. The thread can be in one of the states as explained earlier in thread life cycle.

I am sure that most of us may not want to analyze the thread dump in plain text file. One can use GUI tools to analyse thread dumps. 

• Fast Thread
• Java Stack Review
• Spotify Analyzer
• Site 24x7 Analyzer

Conclusion

Now you know, what's thread dump and how it can be generated. Also it's useful in understanding and diagnosing problems in multithreaded applications. With proper knowledge, regarding the thread dumps and it's structure, the information contained in dump etc,  can be utilized to identify the root cause of the problems quickly.

Read More

Redis Overview and Benchmark

Image Courtesy Morioh

ReDiS which stands for Remote Directory Server, is an open source in-memory data store, used as a database and as a cache. Redis provides data structures such as strings, hashes, lists, sets and sorted sets. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.

Redis is an open source, advanced key-value store and an apt solution for building highperformance, scalable web applications.

Redis has three main features that sets it apart from others:

• Redis holds its database entirely in the memory, using the disk only for persistence.
• Redis has a relatively rich set of data types when compared to many key-value data stores.
• Redis can replicate data to any number of slaves.
  

Following are certain advantages of Redis:

• Exceptionally fast − Redis is very fast and can perform about 110000 SETs per second, about 81000 GETs per second.
• Supports rich data types − Redis natively supports most of the datatypes that developers already know such as list, set, sorted set, and hashes. This makes it easy to solve a variety of problems as we know which problem can be handled better by which data type.
• Operations are atomic − All Redis operations are atomic, which ensures that if two clients concurrently access, Redis server will receive the updated value.
• Multi-utility tool − Redis is a multi-utility tool and can be used in a number of use cases such as caching, messaging-queues (Redis natively supports Publish/Subscribe), any short-lived data in your application, such as web application sessions, web page hit counts, etc.
  
  

Redis Monitoring

Availability, the redis server will respond to the PING command when it's running smoothly.

$ redis-cli -h 127.0.0.1 ping
PONG

Cache Hit Rate

This information can be calculated with the help of INFO command.

$ redis-cli -h 127.0.0.1 info stats | grep keyspace
keyspace_hits:1069963628
keyspace_misses:2243422165

Workload Statistics

The first two stats talks about connections and commands processed where last two stats talk about bytes received and sent from the redis server.

$ redis-cli -h 127.0.0.1 info stats | grep "^total"
total_connections_received:1687889
total_commands_processed:5602955422
total_net_input_bytes:198210899161
total_net_output_bytes:309040592973

Key Space

Anytime to know number of keys in the database, use this command. The size of the keyspace with a quick drop or spike in the number of keys is a good indicator of issues.

$ redis-cli -h 127.0.0.1 info keyspace
# Keyspace
db0:keys=3857884,expires=277,avg_ttl=259237

Clear Keys

We can clear all the keys from the Redis, using the below command.

$ redis-cli -h 127.0.0.1
127.0.0.1:6379> flushall

How to Perform Redis Benchmark?

Redis benchmark is the utility to check the performance of Redis by running n commands simultaneously.

redis-benchmark [option] [option value]

Option Description
-h Specifies server host name 127.0.0.1
-p Specifies server port 6379
-c Specifies number of parallel connections, default is 50
-n Specifies total number of requests, default is 100000
-d Specifies data size of SET/GET value in bytes, default is 3
-r Use random keys for SET/GET/INCR
-q Forces Quiet to Redis. Just shows query/sec values
-l Generates loop, Run the tests forever
-t Only runs the comma-separated list of tests
--csv Output in CSV format

$ redis-benchmark -h 127.0.0.1 -n 100000 -q
PING_INLINE: 57306.59 requests per second
PING_BULK: 57273.77 requests per second
SET: 56657.22 requests per second
GET: 57012.54 requests per second
INCR: 57240.98 requests per second
LPUSH: 57045.07 requests per second
RPUSH: 56657.22 requests per second
LPOP: 57142.86 requests per second
RPOP: 57175.53 requests per second
SADD: 56369.79 requests per second
HSET: 55679.29 requests per second
SPOP: 54704.60 requests per second
LPUSH (needed to benchmark LRANGE): 52798.31 requests per second
LRANGE_100 (first 100 elements): 35448.42 requests per second
LRANGE_300 (first 300 elements): 17618.04 requests per second
LRANGE_500 (first 450 elements): 12812.30 requests per second
LRANGE_600 (first 600 elements): 10036.13 requests per second
MSET (10 keys): 47281.32 requests per second

References

• Documentation
• Command Reference
• Benchmark

Read More

Materialized Views in RDBMS - Is it a View or Table?

 

Google uses structured data to understand the content on the page and use that data to display in richer features in the search results.

Below you can see the difference in the search result data provided by Google for the OnePlus 8T product from official site vs amazon site. The amazon one has additional data rendered in the same search results.

Image Courtesy - Shushma

As we started to build the feature to feed the product details to Google, the data has to be retrieved from various tables from the catalog schema based on multiple  conditions.

Once we had the API ready, we did load testing, having 25 threads with a ramp-up time of 5 seconds and test duration of 30 seconds. Though we know the search engine bots won't put so much load on the site but the performance of the API was a concern to check.

If you look at the 95 percentile of time, it took 10 seconds to respond and the number of requests reached 430 only. Team members came up with the idea of caching the data on redis, so that next call will be faster. Caching at application layer for this use case will not be of that great help as we have a TTL of 600 seconds on Redis.

I recalled in one of my previous assignments, where we had a similar use case and we had utilised the caching of the data on the database (materialized view) side instead on the application layer which had given better results.

Understand Terminologies

Let's start with TABLE, it's basically an organized storage for your data in the form of rows and columns. You can easily query the table using predicates on the columns. 

To simplify the queries or maybe to apply different security mechanisms on data being accessed you can use VIEWs. Think of a view as glasses through which you can look at your data without knowing the actual tables and realtionship details etc.

If the table is a storage, a view is just a way of looking at it or a projection of the data as view doesn't store the data physically. If you query a table, you fetch its data directly. On the other hand, when you query a view, you are basically querying another query that is stored in the view's definition. But the query planner is aware of that and build a plan to merge the two together and give the results.

Between the table and view, we have the MATERIALIZED VIEW. It's a view that has a query in its definition and uses this query to fetch the data directly from the storage, but it also has it's own storage that basically acts as a cache in between the underlying table(s). 

Now you might have a question, if any data is updated in the underlying table(s) whether they will reflect in materialized view. The simple answer is NO as materialized view acts like a cache, it wont reflect the changes. We need to refresh the materialized view, through a process that would cause it's definition's query to be executed again on actual data and rebuilds the cache.

Materialized View Approach

We created a materialized view to cache the data on the database side, as we were retrieving the data from multiple tables. The required data is available in it, so we can avoid lot of computation at the real-time and also the fetched data won't change very frequently.

With the code changes, we did same load testing having 25 threads with a ramp-up time of 5 seconds and test duration of 30 seconds.

If you look at the 95 percentile of time it took 75 milli seconds to respond and the number of requests reached to 8376. The performance gain with respect to throughput is 19x and 95 percentile of time is 134x time faster. The results looks wonderful and it's without caching on the application layer side.

Query Plan

• The original query which used to join multiple tables with various conditions had the query planning time of 0.852 ms and execution time of 5.511 ms.
• The query which we fire on materialized view from the application layer had the query planning time of 0.087 ms and execution time of 0.052 ms.

Conclusion

• The Materialized View is supported by all relational databases like Oracle, MySQL, Postgres etc.
• The  Materialized View is a powerful tool enabling many performance improvements while providing another way of ensuring data consistency.

References

• Materialized Views by Oracle
• Materialized Views by PostgreSQL

Read More

Latency and Fault Tolerance for Distributed System

Microservices

Also known as the microservices architecture, is an architectural style that structures an application as a collection of services that are

• Highly Maintainable and Testable
• Loosely Coupled
• Independently Deployable
• Organized around Business Capabilities
• Owned by a Small Team

The microservices architecture enables the rapid, frequent and reliable delivery of large, complex applications. It also enables an organization to evolve its technology stack.

The decentralization of business logic increases the flexibility and most importantly decouples the dependencies between two or more components, this being one of the major reasons as to why many companies are moving from monolithic architecture to a microservices architecture.

What's Hystrix?

• Hystrix is designed to do the following:
• Give protection from and control over latency and failure from dependencies accessed (typically over the network) via third-party client libraries.
• Stop cascading failures in a complex distributed system.
• Fail fast and rapidly recover.
• Fallback and gracefully degrade when possible.
• Enable near real-time monitoring, alerting, and operational control.

In simple terms, it isolates external dependencies so that one may not affect the other.

Hystrix provides 2 isolation strategies, thread and semaphore based isolation, but it’s left to one to decide which one works based on the requirements.

Thread Based Isolation

In this isolation method; the external call gets executed on a different thread (non-application thread), so that the application thread is not affected by anything that goes wrong with the external call.

Hystrix uses the bulkhead pattern. In general, the goal of the bulkhead pattern is to avoid faults in one part of a system to take the entire system down. The term comes from ships where a ship is divided in separate watertight compartments to avoid a single hull breach to flood the entire ship; it will only flood one bulkhead.

Hystrix uses per dependency thread-pool for isolation. So, for every call only the threads from the pool of corresponding dependency are utilized. This ensures that only the pool of the dependency which fails get exhausted and leaving the others unaffected. The number of maximum concurrent requests allowed are based on the thread-pool size defined.

It’s very important to set the right thread-pool size for each of the dependency. A very small pool size may result in requests going into fallback although the downstream may be responding timely. On the other hand a very large one may cause all the threads to be blocked in case the downstream is running with high latencies, thereby degrading the application performance.

How to find the thread-pool size?

As mentioned in hystrix documentation, the correct thread-pool size can be computed with the given formula:

RPS = Requests per second at which we are calling the downstream.
P99 = The 99th percentile latency of downstream.
Pool Size = (RPS * P99) + some breathing room to cater spikes in RPS.

Let’s take an example, the cart service is dependent on product service to get the product details. Assume cart is calling product service at 30 RPS. The product service latencies P99 is 200 ms, P99.5 is 300 ms and Median is 50 ms.

30 rps x 0.2 seconds = 6 + breathing room = 10

In above case thread-pool size is defines as 10. It is sized at 10 to handle a burst of 99th percentile requests, but when everything is healthy this thread-pool will typically only have 1 or 2 active threads at any given time to serve mostly 50 ms median calls. Also it is ensured that only 10 requests are processed concurrently at any point of time, any concurrent requests after the 10th will directly go into the fallback mechanism.

Hystrix also provides an option to configure the queue size, where requests are queued up-to a certain number after the thread-pool is filled. The queued requests will be processed as soon as the threads become free. If the requests cross the queue size then it will directly go into the fallback mechanism.

Thread based isolation also gives an added protection from timing out. If, a thread in the hystrix thread-pool is waiting for response for more than a specified time(read timeout), the request corresponding to that thread is served by the fallback mechanism. So thread based isolation provides 2 layers of protection — concurrency and timeout.

To summarise, the concurrency limit is defined such that, if the incoming requests exceed the limit, it implies that the downstream is not in a healthy state and it is suffering high latencies. Also, if the requests are taking more time than the read timeout to respond, again it’s an indication that the downstream is not in a healthy state as the latency is high.

Semaphore Based Isolation

In this method, the external call runs on the application thread and the number of concurrent calls are limited by the semaphore count defined in the configuration.

The “time out” functionality is not allowed in this method, since the external call is running on application thread and hystrix will not interfere with the thread-pool that doesn’t belong to it. Hence, there is only one level of protection, i.e., the concurrency limit.

You might wonder, what is the advantage of using semaphore based isolation?

The limiting of concurrency is nothing but a way of throttling the load in case a downstream is running with high latencies and semaphore based isolation allows us that. Please note, the thread based isolation adds an inherent computational overhead. Each command execution involves the queueing, scheduling and context switching is involved in running a command on a separate thread. All of these are not required in case of semaphores, making them computationally much lighter as compared to threads.

Generally, semaphore isolation should be used when the external call rate is so high that the overhead of thread creation comes out to be very costly, or when the downstream is trusted to respond or fail quickly, so it will ensure our application threads are not stuck. The logic for setting up the semaphore size is same as the one which use for setting thread-pool size, but the overhead when using semaphores is much less and will result in faster execution.

Catch with Thread Isolation

As you know by now, if a downstream application call takes more than the defined timeout to return the result, the caller is served using the fallback defined. Please note this fallback is not served by the hystrix thread that was created for the external call, infact it will keep waiting until it gets a response or an exception from the downstream application. We can’t force the latent thread to stop the work; the best hystrix can do is to throw an InterruptedException. If the implementation wrapped by hystrix doesn’t respect InterruptedException, then thread will continue it’s work though the caller would have received the response from the fallback.

We can handle this situation by defining the read timeout as close as possible with hystrix thread timeout. Once the thread timeout is pass, the external call receives an exception which will mark the task of hystrix thread as completed and is returned to the thread-pool.

To validate the above scenarios, I have created two microservices one is named as rest-producer and other one is rest-consumer.

rest-producer microservice

This service exposes a simple API which takes username as a path variable, prefix Welcome to it and returns back to the caller.

@RestController
@RequestMapping("/api/users")
public class UserController {
    @GetMapping("/{username}")
    public String getWelcomeMessage(@PathVariable("username") String userName) {
        try {
            Thread.sleep(4000);
        } catch(Exception ex) {
            ex.printStackTrace();
        }
        StringBuilder stringBuilder = new StringBuilder()
                .append("Welcome ").append(userName).append(" !\n");
        return stringBuilder.toString();
    }
}

As usual, you can find the source on github for rest-producer service.

rest-consumer microservice

This service exposes a simple API /api/greet/{username} which internally calls a service that will invoke the rest-producer service API /api/users/{username} using the restTemplate.

@Slf4j
@RestController
@RequestMapping("/api/greet")
public class GreetingController {
    @Autowired
    private GreetingService greetingService;
    @GetMapping("/{username}")
    public String getGreetingMessage(@PathVariable("username") String userName) {
        try {
            log.warn("Main Thread: {} ({})",
                    Thread.currentThread().getName(),
                    Thread.currentThread().getId());
            return greetingService.getMessage(userName);
        } finally {
            log.warn("Main Thread: {} ({})",
                    Thread.currentThread().getName(),
                    Thread.currentThread().getId());
        }
    }
}

RestTemplate Bean

@Bean
public RestTemplate restTemplate() {
   HttpComponentsClientHttpRequestFactory httpComponentsClientHttpRequestFactory
         = new HttpComponentsClientHttpRequestFactory();
   httpComponentsClientHttpRequestFactory.setConnectTimeout(5000);
   httpComponentsClientHttpRequestFactory.setReadTimeout(5000);
   return new RestTemplate(httpComponentsClientHttpRequestFactory);
}

GreetingService Implementation

@Slf4j

@Service

public class GreetingService {

    @Autowired

    private RestTemplate restTemplate;

    @HystrixCommand(fallbackMethod = "fallback_getMessage", 

            commandProperties = { 

            @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3000") 

    })

    public String getMessage(String userName) {

        log.warn("Hystrix Thread: {} ({})",

                Thread.currentThread().getName(),

                Thread.currentThread().getId());

        String greetMessage = restTemplate.exchange("http://localhost:9191/api/users/{username}",

                HttpMethod.GET, null,

                new ParameterizedTypeReference<String>() {}, userName).getBody();

        log.warn("Hystrix Thread: {} ({}), Msg: {}",

                Thread.currentThread().getName(),

                Thread.currentThread().getId(), greetMessage);

        return greetMessage;

    }

    public String fallback_getMessage(String userName) {

        log.warn("Hystrix Fallback Thread: {} ({})",

                Thread.currentThread().getName(),

                Thread.currentThread().getId());

        return "Fallback Message " + userName + " !";

    }

}

If you notice, I have given connection timeout as 5 secs, read timeout as 5 secs and hystrix thread timeout as 3 secs. The downstream API has been intentionally made to respond after 4 secs. So it will demonstrate that the call from rest-consumer will be served with fallback because the hystrix thread timeout is set to 3 secs. It’s time to hit the rest-consumer API either from cURL or Postman.

curl -X GET 'http://localhost:9090/api/greet/Rogers'

You should see the response as below

Fallback Message - Rogers !

Please take a look at the log messages on the rest-consumer service.

2021-02-13 12:12:34.945  WARN 28673 --- [nio-9090-exec-1] c.b.h.controller.GreetingController      : Main Thread: http-nio-9090-exec-1 (21)

2021-02-13 12:12:35.284  WARN 28673 --- [GreetingService-1] c.b.hystrix.service.GreetingService      : Hystrix Thread: hystrix-GreetingService-1 (42)

2021-02-13 12:12:38.279  WARN 28673 --- [ HystrixTimer-1] c.b.hystrix.service.GreetingService      : Hystrix Fallback Thread: HystrixTimer-1 (41)

2021-02-13 12:12:38.283  WARN 28673 --- [nio-9090-exec-1] c.b.h.controller.GreetingController      : Main Thread: http-nio-9090-exec-1 (21)

2021-02-13 12:12:39.394  WARN 28673 --- [GreetingService-1] c.b.hystrix.service.GreetingService      : Hystrix Thread: hystrix-GreetingService-1 (42), Msg: Welcome Rogers !

As you can see, the main thread ID is 21 which is working in controller and hystrix spawns a thread with ID 42 to make an external call to the downstream service. Now the thread 42 will be on waiting state till it gets response from the downstream service. But the thread timeout has been set to 3 secs and hystrix will launch another thread with ID 41 which will give a fallback message to the controller and you see that main thread with ID 21 returns the fallback response to the caller.

Please observe carefully, the last log statement which is printed after 4 secs which is nothing but the thread with ID 42 which was on waiting state receives the actual response from the downstream after 4 secs and it prints the response. But it’s of no use as the original request is already returned with fallback message. So it proves that the hystrix thread that’ s used to make an external call will be blocked until it gets response/exception from the downstream service. Hence it’s very important to setup the http read timeout and thread timeout as same so we can avoid blocking the hystrix thread and release it back to the pool, otherwise the hystrix thread will exhaust if we have high traffic.

As usual, you can find the source on github for rest-consumer service.

With great decentralization comes a great need of resilient fault tolerance. As Netflix says, fault tolerance is not a feature, it’s a requirement.

Conclusion

Simply wrapping Hystrix around external calls does not guarantee an effective fault tolerance. Generally, it has been observed that the hystrix configurations which we discussed above like thread-pool size, isolation strategy, max concurrency limit, read timeout etc are not configured explicitly and left to use their default values. As explained configuring them as per the requirements is very important to ensure an effective fault tolerance for an application.

Configuration Reference

• Hystrix

Alternate for Hystirx

• Sentinel
• Resilence4j

Read More