Understand and Analyze Java Thread Dump

Photo by Mel Poole

Microservices

Also known as the microservices architecture, is an architectural style that structures an application as a collection of services that are:

• Easily Maintainable and Testable
• Loosely Coupled
• Independently Deployable
• Organized around Business Capabilities
• Owned by a Small Team

The microservices architecture enables the rapid, frequent and reliable delivery of large, complex applications. It also enables an organization to evolve its technology stack.

The decentralization of business logic increases the flexibility and most importantly decouples the dependencies between two or more components, this being one of the major reasons as to why many companies are moving from monolithic architecture to a microservices architecture.

What is a Thread?

All of us have probably written a program that displays "Hello World!!" or given word is a palindrome or not etc. These are sequential programs that have a beginning, an execution sequence and an end, at any given point of time during the execution of a program, there is a single point of execution.

A single thread is also similar, as it has a beginning, an execution sequence and an end. However, a thread itself is not a program, a thread cannot run on its own, it runs within a program.

A program can consist of many lightweight processes called threads. The real excitement surrounding threads is not about a single sequence. It helps to achieve parallelism wherein, a program is divided into multiple threads and results in better performance. All threads within a process share the same memory space and might have dependency on each other in some cases.

Lifecycle of a Thread

For understanding a thread dump in detail, it is essential to know all the states a thread passes through during its lifecycle. A thread can assume one of these following states at any given point of time:

NEW

Initial state of a thread when we create an instance of Thread or Runnable. It remains in this state until the program starts the thread.

RUNNABLE

The thread becomes runnable after a new thread is started. A thread in this state is considered to be executing its task.

BLOCKED

A thread is in the blocked state when it tries to access an object that is currently locked by some other thread. When the locked object is unlocked and hence available for the thread, the thread moves back to the runnable state.

WAITING

A thread transitions to the waiting state while waiting for another thread to perform a task and transitions back to the runnable state only when another thread signals the waiting thread to resume execution.

TIMED_WAITING

A timed waiting state is a thread waiting for a specified interval of time and transitioning back to the runnable state when that time interval expires. The thread is waiting for another thread to do some work for up to a specified waiting time.

TERMINATED

A runnable thread enters the terminated state after it finishes its task.

Thread Dumps

A thread dump contains a snapshot of all the threads active at a particular point during the execution of a program. It contains all relevant information about the thread and its current state.

A new age application development involves multiple numbers of threads. Each thread requires certain resources, performs certain task related to the program. This can boost the performance of an application as threads can utilize available CPU cores. But we do have some trade-offs, for example, sometimes multiple threads may not co-ordinate well with each other and a deadlock situation may arise depending on the program. So, if something goes wrong, we can use thread dumps to identify the state of our threads.

As Java has been most popular language among application development, let's consider our application is built using spring-boot. If you want to take a snapshot of application threads, then we can go ahead with taking thread dump. A JVM thread dump is a listing of the state of all threads that are part of the process at that particular point of time. It contains information about the thread’s stack with other important information. The dump will be in a plain text format, the contents can be saved and analysis can be done either manually or using some UI that are available.

Analysis of thread dumps can help in following areas:

• Tweak JVM performance
• Tweak application performance
• Identify threads related problems within application.

Now we know the basics of thread and it's life cycle. Let's get into the next stage where we will explore how to take a thread dump from any running Java application. 

There are multiple ways to take a thread dumps. Am going to discuss about some JVM based tools and can be executed from the CLI or GUI tools.

Java Stack Trace

One of the easy way to generate a thread dump is by using jStack. jStack is a utility that ships with JVM, it can be used from the CLI and it expects the PID of the process for which we want to generate the thread dump.

jstack -l 1129 > thread_dump.txt

Java Command

JCMD is a command-line utility that ships with the JDK and are used to send diagnostic command requests to the JVM, where these requests are useful for controlling Java Flight Recordings, troubleshoot, and diagnose JVM and Java Applications. It must be used on the same machine where the JVM is running, and have the same effective user and group identifiers that were used to launch the JVM.

We can use the Thread.print command of jcmd to get a list of thread dumps for a particular process specified by the PID.

jcmd 1129 Thread.print > thread_dump.txt

Java Console

The jconsole GUI is a monitoring tool that complies to the Java Management Extensions (JMX) specification. It ships with the JDK and uses the extensive instrumentation of the Java VM to provide information about the performance and resource consumption of applications running on the Java platform.

Using the jconsole tool, we can inspect each thread’s stack trace when we connect it to a running java process. Then, in the Thread tab, we can see the name of all running threads. To detect a deadlock, we can click on the Detect Deadlock in the bottom right of the window. If a deadlock is detected, it will appear in a new tab otherwise a No Deadlock Detected will be displayed.

To launch the GUI tool, just type the below command on CLI.

jconsole

VisualVM

VisualVM is a GUI tool that helps us troubleshoot, monitor and profile Java applications. It perfectly fits all requirements of application developers, system administrators, quality engineers and end users.

As it's an external program, you need to download and install it on your machine. The GUI tool is very easy to use and lot of things you can monitor and troubleshoot things related to Java Applications.

Understanding Thread Dump Contents

Now, Let’s see what are the things we can explore using thread dumps. If we observe the thread dump, we can see a lot of information. However, if we take one step at a time, it can be fairly simple to understand.

1129:

2021-10-13 12:57:15

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode):

"Attach Listener" #142 daemon prio=9 os_prio=31 tid=0x00007f8dc7146000 nid=0x440b waiting on condition [0x0000000000000000]

   java.lang.Thread.State: RUNNABLE

"http-nio-8080-Acceptor" #138 daemon prio=5 os_prio=31 tid=0x00007f8dc7fab800 nid=0x9c03 runnable [0x000070000c59f000]

   java.lang.Thread.State: RUNNABLE

        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method

• The thread dump entry shown above, starts with the name of the thread Attach Listener whose ID is 142 thread (indicated by#142) created by the JVM after the application has started.
• The daemon keyword after the thread number indicates that it's a daemon thread, which means that it will not prevent the JVM from shutting down if it is the last running thread.
• After that we have are less important pieces of metadata about the thread like a priority, os priority, thread identifier, and native identifier.
• The last piece of information is the most important, the state of the thread and its address in the JVM. The thread can be in one of the states as explained earlier in thread life cycle.

I am sure that most of us may not want to analyze the thread dump in plain text file. One can use GUI tools to analyse thread dumps. 

• Fast Thread
• Java Stack Review
• Spotify Analyzer
• Site 24x7 Analyzer

Conclusion

Now you know, what's thread dump and how it can be generated. Also it's useful in understanding and diagnosing problems in multithreaded applications. With proper knowledge, regarding the thread dumps and it's structure, the information contained in dump etc,  can be utilized to identify the root cause of the problems quickly.

Read More

Redis Overview and Benchmark

Image Courtesy Morioh

ReDiS which stands for Remote Directory Server, is an open source in-memory data store, used as a database and as a cache. Redis provides data structures such as strings, hashes, lists, sets and sorted sets. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.

Redis is an open source, advanced key-value store and an apt solution for building highperformance, scalable web applications.

Redis has three main features that sets it apart from others:

• Redis holds its database entirely in the memory, using the disk only for persistence.
• Redis has a relatively rich set of data types when compared to many key-value data stores.
• Redis can replicate data to any number of slaves.
  

Following are certain advantages of Redis:

• Exceptionally fast − Redis is very fast and can perform about 110000 SETs per second, about 81000 GETs per second.
• Supports rich data types − Redis natively supports most of the datatypes that developers already know such as list, set, sorted set, and hashes. This makes it easy to solve a variety of problems as we know which problem can be handled better by which data type.
• Operations are atomic − All Redis operations are atomic, which ensures that if two clients concurrently access, Redis server will receive the updated value.
• Multi-utility tool − Redis is a multi-utility tool and can be used in a number of use cases such as caching, messaging-queues (Redis natively supports Publish/Subscribe), any short-lived data in your application, such as web application sessions, web page hit counts, etc.
  
  

Redis Monitoring

Availability, the redis server will respond to the PING command when it's running smoothly.

$ redis-cli -h 127.0.0.1 ping
PONG

Cache Hit Rate

This information can be calculated with the help of INFO command.

$ redis-cli -h 127.0.0.1 info stats | grep keyspace
keyspace_hits:1069963628
keyspace_misses:2243422165

Workload Statistics

The first two stats talks about connections and commands processed where last two stats talk about bytes received and sent from the redis server.

$ redis-cli -h 127.0.0.1 info stats | grep "^total"
total_connections_received:1687889
total_commands_processed:5602955422
total_net_input_bytes:198210899161
total_net_output_bytes:309040592973

Key Space

Anytime to know number of keys in the database, use this command. The size of the keyspace with a quick drop or spike in the number of keys is a good indicator of issues.

$ redis-cli -h 127.0.0.1 info keyspace
# Keyspace
db0:keys=3857884,expires=277,avg_ttl=259237

Clear Keys

We can clear all the keys from the Redis, using the below command.

$ redis-cli -h 127.0.0.1
127.0.0.1:6379> flushall

How to Perform Redis Benchmark?

Redis benchmark is the utility to check the performance of Redis by running n commands simultaneously.

redis-benchmark [option] [option value]

Option Description
-h Specifies server host name 127.0.0.1
-p Specifies server port 6379
-c Specifies number of parallel connections, default is 50
-n Specifies total number of requests, default is 100000
-d Specifies data size of SET/GET value in bytes, default is 3
-r Use random keys for SET/GET/INCR
-q Forces Quiet to Redis. Just shows query/sec values
-l Generates loop, Run the tests forever
-t Only runs the comma-separated list of tests
--csv Output in CSV format

$ redis-benchmark -h 127.0.0.1 -n 100000 -q
PING_INLINE: 57306.59 requests per second
PING_BULK: 57273.77 requests per second
SET: 56657.22 requests per second
GET: 57012.54 requests per second
INCR: 57240.98 requests per second
LPUSH: 57045.07 requests per second
RPUSH: 56657.22 requests per second
LPOP: 57142.86 requests per second
RPOP: 57175.53 requests per second
SADD: 56369.79 requests per second
HSET: 55679.29 requests per second
SPOP: 54704.60 requests per second
LPUSH (needed to benchmark LRANGE): 52798.31 requests per second
LRANGE_100 (first 100 elements): 35448.42 requests per second
LRANGE_300 (first 300 elements): 17618.04 requests per second
LRANGE_500 (first 450 elements): 12812.30 requests per second
LRANGE_600 (first 600 elements): 10036.13 requests per second
MSET (10 keys): 47281.32 requests per second

References

• Documentation
• Command Reference
• Benchmark

Read More

Retry or Stability Pattern

Photo by Brett Jordan

One of the key characteristic of microservices architecture is inter-service communication. We can split a monolithic application into multiple smaller applications called microservices. Each microservice is responsible for a single feature or domain and can be deployed, scaled, and maintained independently.

Since microservices are distributed in nature, various things can go wrong at any point of time. The network over which we access other services or services themselves can fail. There can be intermittent network connectivity errors or firewall issues. Individual services can fail due to service unavailability, coding issue, out of memory errors, deployment failure, hardware failure and etc., to make our services resilient to these failures, we adopt the retry pattern which is also known as stability pattern.

Retry Pattern

The idea behind the retry pattern is quite simple. If service A makes a call to service B and receives an unexpected response for a request, then service A will send the same request to service B again hoping to get an expected response.

Retry Pattern Representation

There are several retry strategies that can be applied depending on the failure type or nature of the requirements.

Immediate Retry

This strategy is the basic one. In this approach, calling service handles the unexpected failure and immediately makes the request again. This strategy can be useful for unusual failures that occur intermittently. The chances of success are high by just retrying in these cases.

Retry After Delay

In this strategy, we introduce a delay before retrying service call again, hoping that the cause of the fault would have been rectified. Retry after delay is an appropriate strategy when a request timeout occurs due to busy or failures or network-related issue.

Sliding Retry

In this strategy, the service will continue to retry the service call by adding an incremental time delays on each subsequent attempts. For example, the first retry may wait 500 MS, the second will wait 1000 MS, the third will wait 1500 MS until the retry count has not been exceeded. By adding an increasing delay, we reduce the number of retries to the service and avoid adding any additional load to a service which is already overloaded.

Retry with Exponential Backoff

In this strategy, we take the Sliding Retry strategy and ramp up the retry delay exponentially. If we started with a 500 MS delay, we would retry again after 1500 MS, then 3000 MS. Here we are trying to give the service more time to recover before we try to invoke it again.

Abort Retry

As we understand, we can't have a retry process happening forever. We need to have a threshold on the maximum number of retry attempts, we try for a failed service call. We need to maintain the counter and when it reaches the threshold value, our best strategy is to abort the retry process and let the error propagate to the calling service.

Conclusion

The retry pattern allows the calling service to retry failed attempts with a hope that the service will respond within an acceptable time.

With the varying interval between retries we provide the dependent service more time to recover and respond for our request.

It is recommend that, we need to keep a track of failed operations as it will be very useful information to find recurring errors and also the required infrastructure like thread pool, thread strategy etc.

At some point, we just need to abort the retry and we must acknowledge that the service is not responding and notify the calling service with an error.

References

• Resilence4J
• Spring Retry
• Spring Cloud OpenFeign

Read More

Understanding Load Balancer

Photo by Jon Flobrant

A load balancer is an important component of any distributed system. It helps to distribute the client requests within a cluster of servers to improve the responsiveness and availability of applications or websites.

It distributes workloads uniformly across servers or other compute resources to optimize the network efficiency, reliability and capacity. Load balancing is performed by an appliance either physical or virtual that identifies in real time which server [pod incase of kubernetes] in a pool can best meet a given client request, while ensuring heavy network traffic doesn't overwhelm any single server [ or pod]. Another important task of load balancer is to carry out continuous health checks on servers [or pods] to ensure they can handle requests. It ensures better use of system resources by balancing user requests and guarantees 100% availability of service.

Reverse Proxy/Load Balancer Communication Flow

During the system design, horizontal scaling is a very common strategy or solution to scale any system when the user base is huge in number. It also ensures better overall throughput of the application or website. Latencies should occur less often as requests are not blocked, and users need not to wait for their requests to be processed/served.

Availability is a key characteristic of any distributed system. In case of a full server failure, there won’t be any impact on the user experience as the load balancer will simply send the client request to a healthy server. Instead of a single resource performing or taking heavy load, load balancer ensures that several resources perform a bearable amount of work.

Categories of Load Balancer

Layer 4 Category Load Balancer

Load balancers distribute traffic based on transport data, such as IP addresses and Transmission Control Protocol (TCP) port numbers. Examples - Network Load balances in AWS and Internal Load balancer in GCP

Layer 7 Category Load Balancer

Load balancers make routing decisions based on application characteristics that include HTTP header information or the actual contents of the message such as URLs, Cookies etc. Examples - Applications Load balancer in AWS and Gloabl Load balancer in GCP

Types of Load Balancing

Hardware Load Balancing Type

Vendors of hardware‑based solutions load proprietary software onto the machine they provide, which often uses specialized components or resources. To handle the increasing traffic to the application or website, one has to buy specific h/w from the vendors. Example - F5 Load balancer from F5 networks 

Software Load Balancing Type

Software solutions generally run on regular hardware, making them economical and more flexible. You can install the software on the hardware of your choice or in cloud environments like AWS, GCP, Azure etc.

Load Balancing Techniques

There are various types of load balancing methods and every type uses different algorithms for distributing the requests. Here is a list of load balancing techniques:

Random Selection

As the name itself says, the servers are selected randomly. There are no other factors considered in selection of the server. This method might cause a problem, where some of the servers gets overloaded with requests and other might be sitting idle.

Round Robin

One of the most commonly used load balancing methods. It’s a method where the load balancer redirects incoming traffic between a set of servers in a certain order. As per the above diagram, we have have 3 application servers; the first request goes to App Server 1, the second one goes to App Server 2, and so on. When load balancer reaches the end of the server list, it starts over again from the beginning which is from App Server 1. It almost evenly balances the traffic between the servers. All servers need to be of same specification for this method to work successfully. Otherwise, a low specification server may have the same load as a high processing capacity server.

Weighted Round Robin

It's a bit more complex than the Round Robin, as this method is designed to handle servers with different characteristics. A weight is assigned to each server in the configuration. This weight can be an integer value that varies according to the specifications of the server. Higher specification servers get more weightage, which is the key parameter for traffic redirection.

Least Response Time

This algorithm sends the client requests to the server with the least active connections and the lowest average response time. The backend server that responds the fastest receives the next request.

Least Connections

In this method, the traffic redirection happens based on the server with the least number of active connections.

IP Hash

In this method, a hash of the source/client's IP address is generated which is used to select a server for redirection. Once the server is allocated, same server will be used for the client’s  consecutive requests. It becomes more like a sticky where requests of a client will be sent to same server irrespective of how busy the server with requests. In some use cases, this method will come very handy and even improve the performance.

Conclusion

Availability is a key characteristic of a distributed system. In case of a one server failure scenario, it won’t affect the end user experience as the load balancer will simply send the client request to another healthy server.

While designing a distributed system, one of the important task is to choose the load balancing strategy according to the application or website requirements.

HAProxy (High Availability Proxy) is open source proxy and load balancing server software. It provides high availability at the network (TCP) and application (HTTP/S) layers, improving speed and performance by distributing workload across multiple servers.

Nginx is a very efficient HTTP load balancer to distribute traffic to several application servers and to improve performance, scalability and reliability of web applications.

References

• Azure Load Balancer
• Google Load Balancer
• Amazon Load Balancer

Read More

Understanding Forward and Reverse Proxy

Photo Courtesy Unsplash

What is Forward Proxy Server?

A forward proxy, often called a proxy server or web proxy, is a server that sits in front of a group of machines within a same network. When these computers make requests to sites and services usually external to the network or on the Internet, the proxy server intercepts those requests and then communicates with web servers on behalf of those clients like a middleman. By doing so, it can control the traffic according to the custom policies, convert and mask client IP addresses, enforce security protocols, and block unknown traffic.

Direct Communication Flow

In the above diagram, the standard communication flow where Laptop/Phone would reach out directly to website AZ, with the client sending requests to the server and the  server responding to the client request.

Forward Proxy Communication Flow

When we have a forward proxy in place, Laptop/Phone/Desktop will instead send requests to FP, which will then forward the request to website AZ. Website AZ will then send a response to FP, which will in-turn forward the response back to Laptop/Phone/Desktop.

Reasons for Forward Proxy:

Restricted Browsing

Governments, Schools and Organizations use proxy server to give their users access to a limited version of the Internet. A forward proxy can be used to implement these restrictions, as they let the user requests to go through the proxy rather than directly to the websites.

To Block Specific Content

Proxies can also be set up to block a group of users from accessing certain websites. Our  office network might be configured to connect to the web through a proxy which enables content filtering, blocking the requests to social media or online streaming sites.

What is Reverse Proxy Server?

A reverse proxy is a server that sits in front of one or more web servers, intercepting requests from clients. This is different from a forward proxy, where the proxy sits in front of the clients. With a reverse proxy, when clients send requests to the origin server of a website, those requests are intercepted at the network edge by the reverse proxy server. The reverse proxy server will then send requests to and receive responses from the origin server. Unlike a traditional proxy server, which is used to protect clients, a reverse proxy is used to protect servers.

Reverse Proxy Communication Flow

A reverse proxy effectively serves as a gateway between clients, users, and application servers. It handles all the access policy management and traffic routing, and it protects the identity of the server that actually processes the request.

Reasons for Reverse Proxy:

Load Balancing

A reverse proxy server can act as a traffic cop, sitting in front of your backend servers and distributing client requests across a group of servers in a manner that maximizes speed and capacity utilization while ensuring no one server is overloaded, which can degrade performance. In the event that a server fails completely, other servers can step up to handle the traffic.

Global Server Load Balancing

In this form of load balancing, a website can be distributed on several servers around the globe and the reverse proxy will send clients to the server that’s geographically closest to them. This decreases the distances that requests and responses need to travel, minimizing load times.

SSL Encryption

Encrypting and decrypting SSL (or TLS) communications for each client can be computationally expensive for an origin server. A reverse proxy can be configured to decrypt all incoming requests and encrypt all outgoing responses, freeing up valuable resources on the origin server.

Protection from Attacks

With a reverse proxy in place, a website or service never needs to reveal their server identities. Also acts as an additional defense against security attacks. This makes it much harder for attackers to leverage a targeted attack against them, such as a DDoS attack.

Caching

Reverse proxies can compress inbound and outbound data, as well as cache commonly requested content, results in boosting the performance of traffic between clients and servers.

Superior Compression

Server responses use up a lot of bandwidth. Compressing server responses (e.g. with gzip) before sending them to the client can reduce the amount of bandwidth required, speeding up server responses over the network.

Monitoring and Logging Traffic

A reverse proxy captures any requests that go through it. Hence, you can use them as a central hub to monitor and log traffic. Even if you use multiple web servers to host all your website’s components, using a reverse proxy will make it easier to monitor all the incoming and outgoing data from your site.

Cloudfare, Amazon CloudFront, Akamai, StackPath, DDos-Guard, CDNetworks etc are some of the well known reverse proxies available in the market.

Read More

Template Method Pattern - Skeleton is Defined in Base Class

Photo Courtesy Unsplash

What is the Template Method Design Pattern?

The Template Method Design Pattern defines a sequence of steps of an algorithm and allows the subclasses to override the steps but do not allow to change the sequence. The Key to the Template Design Pattern is that we put the general logic in the abstract parent class and let the child classes define the specifics.

Template Method pattern falls under the behavioural design pattern, is one of the easiest to understand and implement. This design pattern is used popularly in framework development and also helps to avoid code duplication.

An abstract class contains the templateMethod which should be made final so that it cannot be overridden. This template method makes use of other operations available in order to run the algorithm but is decoupled for the actual implementation of these methods. Concrete class implements all the methods required by the templateMethod that were defined as abstract in the parent class.

Pattern Implementation

As am a coffee person, I will take an example of preparing the coffee using a template method pattern which should allow to understand the pattern easily instead of taking some framework implementations such as AbstractController, RequestProcessor, HttpServlet, InputStream, OutputStream etc.

public abstract class CoffeeMaker {
   protected boolean sugarFree = false;

   public void prepareCoffee() {
      boilWater();
      addMilk();
      if (!isSugarFree()) {
         addSugar();
      } else {
 System.out.println("- No Sugar")
      }
      addCoffeePowder();
      System.out.println("- Coffee is Ready!!!");
   }

   public final void boilWater() {
      System.out.println("- Boiling Water");
   }

   public boolean isSugarFree() {
      return sugarFree;
   }

   public void setSugarFree(boolean sugarFree) {
      this.sugarFree = sugarFree;
   }

   abstract void addMilk();
   abstract void addSugar();
   abstract void addCoffeePowder();
}

Now we will create 2 concrete classes which will provide different implementations.

public class CothasCoffeeMaker extends CoffeeMaker {

   @Override

   public void addMilk() {

      System.out.println("- Adding Milk");

   }

   @Override

   public void addSugar() {

      System.out.println("- Adding Sugar");

   }

   @Override

   public void addCoffeePowder() {

      System.out.println("- Adding Cothas Coffee Powder");

   }

}

public class BruCoffeeMaker extends CoffeeMaker {

   @Override

   public void addMilk() {

      System.out.println("- Adding Milk");

   }

   @Override

   public void addSugar() {

      System.out.println("- Adding Sugar");

   }

   @Override

   public void addCoffeePowder() {

      System.out.println("- Adding Bru Coffee Powder");

   }

}

Now it's time to write the main method to execute the above code and see the output of the two variations of the coffee.

public class CoffeeMakerExample {

   public static void main(String[] args) {

      System.out.println("Cothas Coffee Preparation")

      CoffeeMaker coffeeMaker = new CothasCoffeeMaker();

      coffeeMaker.prepareCoffee();

      System.out.println("Bru Coffee Preparation");

      coffeeMaker = new BruCoffeeMaker();

      coffeeMaker.setSugarFree(true);

      coffeeMaker.prepareCoffee();

   }

}

The CoffeeMaker class is an abstract class containing the algorithm skeleton. The prepareCoffee() is the method that contains the process steps. The boilWater() method is common step for the process of any coffee preparation and no customization is required, hence it has been made as final so the subclasses will not override and provide a different implementation. 

We have two subclasses CothasCoffeeMaker and BruCoffeeMaker which follows the same preparation process but implementation are given at each individual subclass.

If you execute the above program, you should see the following output.

Cothas Coffee Preparation

- Boiling Water

- Adding Milk

- Adding Sugar

- Adding Cothas Coffee Powder

- Coffee is Ready!!!

Bru Coffee Preparation

- Boiling Water

- Adding Milk

- No Sugar

- Adding Bru Coffee Powder

- Coffee is Ready!!!

Conclusion

• The pattern promotes the code reusability and decoupling, but at the expense of using inheritance.
• The pattern adhers to the Single Responsibility and Open/Closed principles of S.O.L.I.D - Design Principles.

Reference

• https://en.wikipedia.org/wiki/Template_method_pattern
  
  

Read More

Materialized Views in RDBMS - Is it a View or Table?

 

Google uses structured data to understand the content on the page and use that data to display in richer features in the search results.

Below you can see the difference in the search result data provided by Google for the OnePlus 8T product from official site vs amazon site. The amazon one has additional data rendered in the same search results.

Image Courtesy - Shushma

As we started to build the feature to feed the product details to Google, the data has to be retrieved from various tables from the catalog schema based on multiple  conditions.

Once we had the API ready, we did load testing, having 25 threads with a ramp-up time of 5 seconds and test duration of 30 seconds. Though we know the search engine bots won't put so much load on the site but the performance of the API was a concern to check.

If you look at the 95 percentile of time, it took 10 seconds to respond and the number of requests reached 430 only. Team members came up with the idea of caching the data on redis, so that next call will be faster. Caching at application layer for this use case will not be of that great help as we have a TTL of 600 seconds on Redis.

I recalled in one of my previous assignments, where we had a similar use case and we had utilised the caching of the data on the database (materialized view) side instead on the application layer which had given better results.

Understand Terminologies

Let's start with TABLE, it's basically an organized storage for your data in the form of rows and columns. You can easily query the table using predicates on the columns. 

To simplify the queries or maybe to apply different security mechanisms on data being accessed you can use VIEWs. Think of a view as glasses through which you can look at your data without knowing the actual tables and realtionship details etc.

If the table is a storage, a view is just a way of looking at it or a projection of the data as view doesn't store the data physically. If you query a table, you fetch its data directly. On the other hand, when you query a view, you are basically querying another query that is stored in the view's definition. But the query planner is aware of that and build a plan to merge the two together and give the results.

Between the table and view, we have the MATERIALIZED VIEW. It's a view that has a query in its definition and uses this query to fetch the data directly from the storage, but it also has it's own storage that basically acts as a cache in between the underlying table(s). 

Now you might have a question, if any data is updated in the underlying table(s) whether they will reflect in materialized view. The simple answer is NO as materialized view acts like a cache, it wont reflect the changes. We need to refresh the materialized view, through a process that would cause it's definition's query to be executed again on actual data and rebuilds the cache.

Materialized View Approach

We created a materialized view to cache the data on the database side, as we were retrieving the data from multiple tables. The required data is available in it, so we can avoid lot of computation at the real-time and also the fetched data won't change very frequently.

With the code changes, we did same load testing having 25 threads with a ramp-up time of 5 seconds and test duration of 30 seconds.

If you look at the 95 percentile of time it took 75 milli seconds to respond and the number of requests reached to 8376. The performance gain with respect to throughput is 19x and 95 percentile of time is 134x time faster. The results looks wonderful and it's without caching on the application layer side.

Query Plan

• The original query which used to join multiple tables with various conditions had the query planning time of 0.852 ms and execution time of 5.511 ms.
• The query which we fire on materialized view from the application layer had the query planning time of 0.087 ms and execution time of 0.052 ms.

Conclusion

• The Materialized View is supported by all relational databases like Oracle, MySQL, Postgres etc.
• The  Materialized View is a powerful tool enabling many performance improvements while providing another way of ensuring data consistency.

References

• Materialized Views by Oracle
• Materialized Views by PostgreSQL

Read More

Singleton Design Pattern - Build It, Break It and Fix It!

Photo Courtesy Unsplash

Singleton pattern is one of the simplest design patterns and falls under creational pattern as this pattern provides one of the best ways to create an object. This pattern involves a single class which is responsible for creating an object while making sure that the instantiation of a class to only one object. This class provides a way to access its only object which can be accessed directly without instantiating the object of the class.

Let’s see various design options for implementing the pattern. If you have a good hold on static class variables and access modifiers this is not a difficult task.

Lazy Instantiation

public class Singleton implements Serializable {

  private static Singleton singletonObject;

  //Adding a private constructor so that no one creates object
  private Singleton() {
  }

  public static Singleton getInstance() {
    if (singletonObject == null) {
      singletonObject = new Singleton();
    }
    return singletonObject;
  }

}

Here we have declared getInstance() as static so that we can call it without instantiating the class. First time, when getInstance() method is called, it creates a new Singleton object and later it just returns the same object reference. Note that singletonObject is not created until we call the getInstance() method, as we are using the lazy instantiation of the object.

The main problem with the above method is that it is not thread safe. If we have two threads T1 and T2 which invokes the getInstance() method at the same time, the execution sequence creates two objects for Singleton.

The other option is to make getInstance() method as synchronized. Here, using synchronized keyword makes sure that only one thread is allowed at a time to execute getInstance() method. The main disadvantage of this is method is, that using synchronized every time while creating the Singleton object is expensive and may decrease the performance of your program. However if performance of getInstance() is not critical for the application this method provides a clean and simple solution.

Eager Instantiation

public class Singleton implements Serializable {

  private static Singleton singletonObject = new Singleton();

  //Adding a private constructor so that no one creates object

  private Singleton() {

  }

  public static Singleton getInstance() {

    return singletonObject;

  }

  

}

Here we have created instance of Singleton with the help of static initializer. JVM executes static initializer when the class is loaded and hence this is guaranteed to be thread safe. Use this method only when the Singleton class is light and is used throughout the execution of the program.

Double Check and Locking Instantiation

public class Singleton implements Serializable {

  private static volatile Singleton singletonObject;

  //Adding a private constructor so that no one creates object

  private Singleton() {

  }

  public static Singleton getInstance() {

    if (singletonObject == null) {

      // Making thread safe

      synchronized(Singleton.class) {

        // Check again as object is still null

        if (singletonObject == null) {

          singletonObject = new Singleton();

        }

      }

    }

    return singletonObject;

  }

    

}

In the above implementation, we have declared the singletonObject as volatile. Using volatile is yet another way (like synchronized, atomic wrapper) of making class thread safe. Thread safe means that a method or class instance can be used by multiple threads at the same time without any problem. This method drastically reduces the overhead of calling the synchronized method every time.

Now that we know how to implement a Singleton pattern in 3 different ways. Let’s see whether we can break the Singleton pattern ?

Reflection

Reflection is a feature in the Java programming language. It allows an executing Java program to examine or introspect upon itself, and manipulate internal properties of the program. For example, it’s possible for a Java class to obtain the names of all its members and display them.

We will use Reflection and see whether we can break the Singleton or not ?

class Singleton implements Serializable {

  private static Singleton singletonObject = new Singleton();

  //Adding a private constructor so that no one creates object

  private Singleton() {

  }

  public static Singleton getInstance() {

    return singletonObject;

  }

  

}

public class SingletonExample {

  

  public static void main(String[] args) {

    try {

        Singleton obj1 = Singleton.getInstance();

        System.out.println("Singleton.getInstance().hashCode: " + obj1.hashCode());

        Constructor constructor = Singleton.class.getDeclaredConstructor();

        constructor.setAccessible(true);

        Singleton obj2 = (Singleton) constructor.newInstance();

        System.out.println("Reflection.newInstance().hashCode: " + obj2.hashCode());

    }

    catch (Exception e) {

        e.printStackTrace();

    }

  }

  

}

If you execute the above program, you should see in the output that the hashCode of both objects are different and it clearly violates Singleton pattern.

Singleton.getInstance().hashCode: 1418481495

Reflection.newInstance().hashCode: 303563356

Let’s fix this loop hole by making some simple code change in the Singleton constructor.

private Singleton() {

    if (singletonObject != null) {

        throw new IllegalStateException("Already Initialised!");

    }

}

Now if you execute the program with the above changes, the line 24 will call the constructor and it will throw an IllegalStateException. Thus prevents the creation of the second instance.

Caused by: java.lang.IllegalStateException: Already initialised!

What if I change the access level of the singleton field from private to public using the Reflection? Oops… it will allow me to set the singleton object to null and it will create a new object. Let’s modify our main method to hack the Singleton class.

public static void main(String[] args) {

    try {

        Singleton obj1 = Singleton.getInstance();

        System.out.println("Singleton.getInstance().hashCode: " + obj1.hashCode());

        Field field = Singleton.class.getDeclaredField("singletonObject");

        field.setAccessible(true);

        field.set(field, null);

        

        Constructor constructor = Singleton.class.getDeclaredConstructor();

        constructor.setAccessible(true);

        Singleton obj2 = (Singleton) constructor.newInstance();

        System.out.println("Reflection.newInstance().hashCode: " + obj2.hashCode());

    }

    catch (Exception e) {

        e.printStackTrace();

    }

}

Using Reflection, we are changing the value of the instance field to NULL after the object gets created. When we invoke the constructor, the condition fails as singleton instance is null hence it will allow you to create another instance.

We can prevent it by declaring the singletonObject instance as FINAL so that it’s value can’t be changed once assigned.

private final static Singleton singletonObject = new Singleton();

Now if you execute the program, it will throw an IllegalAccessException. Reflection can’t convert final field to non-final field. Please note, this approach only works with Eager Instantiation method.

Serialization

Java provides a mechanism, called object serialization where an object can be represented as a sequence of bytes that includes the object’s data as well as information about the object’s type and the types of data stored in the object.

After a serialized object has been written into a file, it can be read from the file and deserialized, i.e., the type information and bytes that represent the object and its data can be used to recreate the object in memory.

Most impressive is that the entire process is JVM independent, meaning an object can be serialized on one platform and deserialized on an entirely different platform.

We will use Serialization and see whether we can break the Singleton or not ?

class Singleton implements Serializable {

  private final static Singleton singletonObject = new Singleton();

  //Adding a private constructor so that no one creates object

  private Singleton() {

    if (singletonObject != null) {

      throw new IllegalStateException("Already Initialised!");

    }

  }

  public static Singleton getInstance() {

    return singletonObject;

  }

  

}

public class SingletonExample {

  

  public static void main(String[] args) {

    try {

        Singleton obj1 = Singleton.getInstance();

        System.out.println("Singleton.getInstance().hashCode: " + obj1.hashCode());

      

        // Serialize the object to file

        ObjectOutput objectOutput = new ObjectOutputStream(new FileOutputStream("file.txt"));

        objectOutput.writeObject(obj);

        objectOutput.close();

        // DeSerailize the object from file

        ObjectInput objectInput = new ObjectInputStream(new FileInputStream("file.txt"));

        Singleton obj2 = (Singleton) objectInput.readObject();

        in.close();

        System.out.println("Serialization.getInstance().hashCode: " + obj2.hashCode());

    }

    catch (Exception e) {

        e.printStackTrace();

    }

  }

  

}

If you execute the above program, you should see in the output that the hashCode of both objects are different and it clearly violates Singleton pattern.

Singleton.getInstance().hashCode: 1418481495

Serialization.getInstance().hashCode: 565760380

Let’s fix this loop hole by making some simple code change in the Singleton class. We have to implement readResolve() method and return the singletonObject reference.

protected Object readResolve() {

    return singletonObject;

}

Conclusion

• We saw how to implement the Singleton Pattern in different ways like Lazy Instantiation, Eager Instantiation and Double Check and Locking Instantiation.
• We saw how we can break the Singleton Pattern using the Reflection and Serialization methods.
• Also, we saw how it can be prevented by making few code changes so the Singleton Pattern won’t allow you to create more than one instance.

References:

• https://en.wikipedia.org/wiki/Singleton_pattern

Read More

Working With ThreadPoolTaskExecutor of Spring

Photo Courtesy Unsplash

ThreadPoolTaskExecutor is a java bean that allows for configuring a ThreadPoolExecutor in a bean style by setting up the values for the instance variables like corePoolSize, maxPoolSize, keepAliveSeconds, queueCapacity and exposing it as a Spring TaskExecutor.

One of the added Advantage of using ThreadPoolTaskExecutor of Spring is that it is well suited for management and monitoring via JMX.

The default configuration of core pool size is 1, max pool size and queue capacity as 2147483647.

This is roughly equivalent to Executors.newSingleThreadExecutor(), sharing a single thread for all tasks. And setting queueCapacity to 0 mimics Executors.newCachedThreadPool(), with immediate scaling of threads in the pool to a very high number.

If you are using XML file for configuring the bean, you can setup the ThreadPoolTaskExecutor like below:

<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
   <property name="corePoolSize" value="5" />
   <property name="maxPoolSize" value="10" />
   <property name="WaitForTasksToCompleteOnShutdown" value="true" />
</bean>

If you are using Java Annotation to define the bean, you can setup like below:

@Bean
public TaskExecutor threadPoolTaskExecutor() {
   ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
   executor.setCorePoolSize(5);
   executor.setMaxPoolSize(10);
   executor.initialize();
   return executor;
}

CorePoolSize

Is the minimum number of threads that remain active at any given point of time. If you don’t provide a value explicitly then it will have default value as 1. The TaskExecutor delegates the value to the underlying class ThreadPoolExecutor.

MaxPoolSize

Is the maximum number of threads that can be created. The TaskExecutor delegates the value to the underlying ThreadPoolExecutor. The maxPoolSize relies on queueCapacity because ThreadPoolTaskExecutor creates a new thread only if the number of items in the queue exceeds queue capacity.

Let’s test out the theory with some code so that it will be more clear for all of us. Here is a code snippet that will create new thread from TaskExecutor.

@Bean
public TaskExecutor threadPoolTaskExecutor() {
   ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
   executor.setCorePoolSize(5);public void createTasks(ThreadPoolTaskExecutor taskExecutor, int numTasks) {

    for (int i=0; i<numTasks; i++) {
       taskExecutor.execute(() -> {
       try {
          long sleepTime = ThreadLocalRandom.current().nextLong(1, 10) * 100;
          Thread.sleep(sleepTime);
       } 
       catch (InterruptedException e) {
          Thread.currentThread().interrupt();
       }
       });
   }
}

Now let’s create a main method where we can try out various use cases by setting values to the ThreadPoolTaskExecutor.

public static void main(String[] args) {
    // Creating ThreadPoolTaskExecutor
    ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
    taskExecutor.afterPropertiesSet();

    // Creating Tasks within ThreadPoolTaskExecutor
    createTasks(taskExecutor, 6);

    // Getting PoolSize
    System.out.println("Properties " +
        "- corePoolSize: " + taskExecutor.getCorePoolSize() +
        ", maxPoolSize: " + taskExecutor.getMaxPoolSize() +
        ", poolSize: " + taskExecutor.getPoolSize() +
        ", activeCount: " + taskExecutor.getActiveCount());

    // Shutting Down ThreadPoolTaskExecutor
    taskExecutor.setWaitForTasksToCompleteOnShutdown(true);
    taskExecutor.shutdown();
}

If execute the above code snippet, you will the below output on the console. You can see that corePoolSize is 1 by default and maxPoolSize is 2147483647. Since the QueueCapacity is also 2147483647, you will see that it will make use of the one thread to process all 6 tasks.

17:37:21.696 [main] INFO org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Initializing ExecutorService

Properties - corePoolSize: 1, maxPoolSize: 2147483647, poolSize: 1, activeCount: 1

17:37:21.747 [main] INFO org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Shutting down ExecutorService

Let’s modify our code to setup some of the parameters so that we can see whether it will have any behavioural change in the TaskExecutor.

ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(1);
taskExecutor.setMaxPoolSize(4);
taskExecutor.afterPropertiesSet();

You will still observe that the TaskExecutor will still make use of one thread only though the max pool size is set to 4.

17:42:43.396 [main] INFO org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Initializing ExecutorService

Properties - corePoolSize: 1, maxPoolSize: 4, poolSize: 1, activeCount: 0

17:42:43.447 [main] INFO org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Shutting down ExecutorService

Let’s modify our code to setup queue capacity parameter so we see the behavioural change in the TaskExecutor.

ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(1);
taskExecutor.setMaxPoolSize(4);

taskExecutor.setQueueCapacity(2);
taskExecutor.afterPropertiesSet();

Now when you run the same piece of code, you will see that the pool size increases but it will not exceed max pool size.

17:47:29.986 [main] INFO org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Initializing ExecutorService

Properties - corePoolSize: 1, maxPoolSize: 4, poolSize: 4, activeCount: 4

17:47:30.036 [main] INFO org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Shutting down ExecutorService

TaskExecutor creates a new thread only if the number of items in the queue exceeds queue capacity. As a result, you will observe that pool size increases.

To conclude, it’s always a good practice to define the core pool size, max pool size and queue capacity for the TaskExecutor explicitly from our end instead of leaving it to use the default values.

Read More