Java Virtual Machine Analysis and Debugging

Hello Folks whoever is reading this i am sharing my experience of Performance Testing i was involved in , in one of my projects in my company.

-->  Load Testing the API's written for a business model on top of a nosql graph database OrientDB . We used an open source XA transaction manager JCA adapter deployed on the Oracle weblogic server 12.1.3 . The API's were deployed as a restful web-service on the same application server as the adapter .

Client Tools used -
-> SoapUI
-> Jmeter

SoapUI and Jmeter are the best two tools i felt to fire http requests to the rest webservice , one can select the number of concurrent users / threads hitting the web service , in Jmeter and can loop such requests multiple times or forever . However SOAPUI provides the flexibility to send n number of concurrent clients and also fix the time for which the client will fire requests , also providing the lag between every next request .

Learn Jmeter @ JMeter Tutorial

However i often saw the JVM getting on slower and slower as the requests came , hence needed to get the Thread Dump periodically and analyze the thread dump , used few tools for this .

-> Jstack : This tool was used to get the thread dump , this tool can be found inside bin directory of jdk installation directory.

-> Samurai : This tool takes the thread dump and gives a gui representation of running , blocked and deadlocked threads. This tool will be helpful to analyze the threads that are going into blocked state slowing down the CPU and the JVM.

Further we found in the performance tab under Monitoring section under managed servers section of Oracle Weblogic server , that free heap percentage is growing lesser and lesser as the application serves the client requests .

-> Jmap : This tool was used to get a heap dump as and when we needed , whenever the free heap percentage went down and down. These tests were carried out on a 2GB heap so @ 5% free heap the HeapDump size came out to be 3.7G + .

Now how to analyse this heap dump , so there are a lot of tools to do that .

-> VisualVM : This is the best and free tool to anaylze the heap dump and will tell you the number of instances of each of the Objects with existing references and will also tell where the references are coming from.

-> Jprofiler : Jprofiler is not free but allows a 10 day free trial , again a great tool to analyze heap dump , provides the same details as above. However loads the entire heap initially , little slower than VisualVM.

-> MAT : Memory analyzer tool is an eclipse project , another alternative for heap dump analysis , however personally i didn't like this tool , much slower than the above two , but still worth a try.

-> Jhat : Jhat is again a tool provided in the bin directory of the jdk installation directory , its the oracle's provided tool to analyze heap dump . This will load the heap data as html and can be accessed on localhost port 7000 . A  very simple tool to use and provides the same data as Visual JM and Jprofiler.

This was all about thread dump and analysis and heap dump and analysis .

Finally we found the objects that were not garbage collected and accordingly modified the JCA adapter code to use the OrientGraphFactory and calling the getTx() method instead of manually creating OrientGraph objects which were never getting garbage collected.