I was working on a multithreaded application today that was deadlocking. In order to determine why it was hanging, I started debugging the application so I could suspend the worker threads and inspect the call stack and see what they were getting hung up on. Oddly enough, the program executed without a problem! Obviously, I was dealing with a race condition, and running the program with a debugger attached changed the timing just enough that the race condition failed to materialize.
In order to find out what the program was deadlocking on, I fired up one of the best debugging tools Java provides: VisualVM. You can get VisualVM from their website (http://visualvm.java.net/download.html), but it comes bundled with the Oracle JDK (since v6u7); you can find it in $JDK_HOME/bin/jvisualvm.
When you start VisualVM it will show a list of Java applications running on your machine; you can also list apps running on remote machines for remote debugging. To inspect an application, double-click on it in the list. This will open a new tab. In my case, I needed to find out where the worker threads were hanging, so I went to the threads tab. Sure enough, they were all blocking. In order to see exactly where they were blocking, click on the ‘Thread Dump’ button at the top of this pane.
This will open up a new tab with the thread dump as raw text. Here is an excerpt from my thread dump:
"pool-1-thread-8" prio=6 tid=0x00000000072fd800 nid=0x604 waiting for monitor entry [0x000000000b9de000] java.lang.Thread.State: BLOCKED (on object monitor) at org.hibernate.id.MultipleHiLoPerTableGenerator.generate(MultipleHiLoPerTableGenerator.java:204) - waiting to lock <0x00000007d7c060e8> (a org.hibernate.id.MultipleHiLoPerTableGenerator) at org.hibernate.event.def.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:121) at org.hibernate.ejb.event.EJB3PersistEventListener.saveWithGeneratedId(EJB3PersistEventListener.java:69)
This lists where and what the thread is blocking on. From here it is easy to open up the source and see what is going on (or consult Google). VisualVM is an excellent tool and is indispensable to Java developers.