Profiling Pentaho with the Java Interactive Profiler (JIP)Posted: October 6, 2011
For a variety of reasons you might want to use a profiler when running the Pentaho BI Server. This short post will show how to use the free Java Interactive Profiler (JIP) with Tomcat as the app server.
Getting the Code
The JIP project can be found at http://jiprof.sourceforge.net. The code and binaries are located at http://sourceforge.net/projects/jiprof/. The code is easy to compile with ant. Simply run $ ant dist.
Configuring the Profiler and Tomcat
There are six files you will need:
- profile.jar located in the profile/ directory.
- client.jar located in the client/ directory.
- The start, finish, and file scripts also in the client/ directory. There are .bat files for Windows and .sh files for *nix.
- A properties file that you create.
In my case I’m using a file named tomcat-profile.properties with the following properties. The ClassLoaderFilter properties are needed to profile the WebappClassLoader loaded classes. These classes are the ones of interest in the BI Server.
profiler=off remote=on port=15599 ClassLoaderFilter.1=com.mentorgen.tools.profile.instrument.clfilter.WebAppClassLoaderFilter ClassLoaderFilter.2=com.mentorgen.tools.profile.instrument.clfilter.StandardClassLoaderFilter
I created a directory under the root pentaho/ directory named jip/ and placed the profile.jar and tomcat-profile.properties files into into it to make it easy to reference. The absolute path to this folder is /home/pentaho/pentaho/jip on my Ubuntu box with Pentaho installed directly under my pentaho account. From here on out I’ll refer to the root directory as $PENTAHO where it equals /home/pentaho/pentaho.
The JIP documentation says to set the JAVA_OPT properties when using Tomcat. However, the start script for biserver sets this value, meaning you will need to modify the start file. Edit the $PENTAHO/server/biserver-?e/tomcat/ctl.sh file. The ? will either be ‘c’ for the community edition or ‘e’ for the enterprise edition. The file should be the same.
JAVA_OPTS is defined in two places. If you are starting by hand, then you really only need to modify the one under start_tomcat(). To the end of the string (inside the quotes) add the following:
Running the Profiler
Start the BI Server.
The settings above cause Tomcat to start with profiling turned off. This allows you to activate profiling when you are ready so that you more accurately capture your code rather than all of the startup of Tomcat. This is especially useful when starting Tomcat and loading the web app for the first time. You can verify the settings and profile start by viewing catalina.out in the tomcat/logs directory. You should see “profiler off” at or near the top of the file. This is also the spot to check for errors. The most likely error is a typo in JAVA_OPTS parameters the ctl.sh file.
From the jip/client/ directory use the scripts to set the log file and start and stop the profiler. Depending on how you want to run the scripts you may want to make them executable using $ chmod +x *sh
To set the file:
$ file.sh localhost 15599 profile-results.txt
Note that if you don’t give an absolute path for the file name, it is relative to where you started the server, not to where you are running the client file.
Start the profiler:
$ start.sh localhost 15599
Perform whatever actions you want to profile.
Stop the profiler:
$ finish.sh localhost 15599
Examining the Results
You should now be able to view the results of the profiling. See the JIP documentation for more information on how to interpret the profiling results and other parameters you can set.
There are a few things that can cause profiling to not work properly:
- If you are not getting data, you may not have the ClassLoaderFilter properly configured. By default only the app loader classes will be profiled and they are not the ones you are usually interested in.
- When running with debug=on, I would sometimes have the app server fail to stop. This shows up by not having “start” and “finish” in the catalina.out file. Check running processes if you shut down the server to make sure it really stops.
- After typing finish there is still a bit of work that JIP will perform. Monitor the size of the file until it stops growing to get all of the data.