Tutorial: Java Code Coverage with Emma

Last Updated: May 02, 2005

Overview
This document introduces how to generate Java code coverage reports of your tests using EMMA.

Contents

What is Emma? Before you Start Generating a Code Coverage Report Interpreting the Coverage Report How to setup and run Emma in QApache? References

What is Emma?
Emma is an open source Java code coverage analysis tool. It allows you to identify what code paths that have and have not been executed. It generates an HTML report which can help developers easily identify what code they have not tested. Managers can use the reports to judge how much of the overall code is covered by tests.

Before You Start

1. You should first download and install Emma. 1. Go to http://emma.sourceforge.net/, enter download page. It will forward you to sourceforge download page: http://sourceforge.net/project/showfiles.php?group_id=108932. Download the latest version. 2. Extract emma.jar from the Zip file and place it in your local disk in your JDeveloper classpath, say /home/alawang/jdevhome/jdev/myclasses. The rest of this document assumes you've placed emma.jar in /home/alawang/jdevhome/jdev/myclasses.
Note: If you are testing UI Controller code, we recommend using our modified

version of emma.jar.

2. Configure Emma In JDeveloper

1. Create a library in project settings for Emma. 2. Add -Demma.coverage.out.file=/home/alawang/jdevhome/jdev/myclasses/coverage.ec to your Java Options. 3. In Tools-> Preferences, change the OC4J Shutdown option to "Shutdown Gracefully First, Forcefully Subsequently". Emma will only dump code coverage stats when java exits gracefully. This step is required if you run OA or JSP pages during your code coverage testing. 4. Setup two external tools in JDeveloper to simplify the execution of Emma. Go Tools -> External Tools -> Add. Ext Tool 1 - Emma Instrumentation Display -> Menu Caption: Emma Instrumentation For detail tab, see the following screen shot. Make sure you add all the arguments in that order:

Ext Tool 2 - Emma Report Display -> Menu Caption: Emma Report For detail tab:

Note: Default project sourcepath is a predefined item. You should click Add button to choose it instead of Add Text like all others.

5. Create filters.txt. In this file, you can speicify the class files you will gather code coverage data. You can put one line for each class files' full class name, or you can use wildcard. Check Emma document for more details.

For more details about these setting and how to tailor them for your own project, please check Emma's website.

Generating a Code Coverage Report

To get code coverage report, you need to perform the following three steps. 1. Instrument your class files. 2. Run your java program: perform your API or UI tests 3. Generate code coverage report.

Instrument Your Class Files

This step basically takes your class files Rebuild your project. It is better for you to clean class files in /home/alawang/jdevhome/jdev/myclasses before you rebuild. That will guantee that you rebuild all. If for whatever reason, some classes files already instrumented found this directory, Emma may generate errors. Tools -> Emma Instrumentation. You would see a console window show up. Wait until it closes by itself. Done. You will see information like this in the output window of JDeveloper:
EMMA: EMMA: EMMA: EMMA: processing instrumentation path ... instrumentation path processed in 12308 ms [502 class(es) instrumented, 0 resource(s) copied] metadata merged into [/home/alawang/jdevhome/jdev/myclasses/coverage.em] {in 210

ms}

If you go to /home/alawang/jdevhome/jdev/myclasses, you will see a new file coverage.em was created. It will be used to generate reports.

Now Emma already instrumented your class files and embedded some additional information in it. This is similar with using -g with javac or gcc.

Run Your Program and Gather Code Coverage data

Simply run your application and execute whatever tests you want to run. Emma will gather the code coverage statistics as your application is running, and dump the statistics when the jvm exits (gracefully). Note, when running JSP or OA pages, you must terminate the OC4J server. If you look at your JDeveloper's output window, you will see:
collecting runtime coverage data ... at the beginning and: runtime coverage data merged into [/home/alawang/jdevhome/jdev/myclasses/coverage.ec] {in 30 ms}

You will see coverage.ec in /home/alawang/jdevhome/jdev/myclasses. This will also be used to generate reports. Generate Report Tools -> Emma Report. Wait for the console window to close. You will see the following in output window of JDeveloper:
EMMA: processing input files ... EMMA: 2 file(s) read and merged in 330 ms EMMA: writing [html] report to [/home/alawang/jdevhome/jdev/myclasses/coverage/index.html] ...

Interpreting the Coverage Report

Open /home/alawang/jdevhome/jdev/myclasses/coverage/index.html with your browser. A sample can be found here: http://qapache.us.oracle.com:25150/OA_HTML/coverage/coverage_cat_admin_api_20040827/

Overall Coverage
The first page is overall coverage. It tells you the coverage for all packages under investigation. For each package, it will should the following numbers: Column Meaning class % How many percent of classes were involved method % How many percent of methods were called block % How many blocks of code were executed? This is a proprietary gauge of Emma

line

% How many lines were executed? Excluding comment lines and those un-executable lines

Package Coverage
Click any package it will give you details about that package. Columns are the same.

File Coverage
Click any file it will give you details about that file. Columns are the same. On the top, you will see the statistics for each method. On the lower half, you will see the source code of that file. For lines covered, it will be in green. Uncovered will be in red. Partially covered will be in yellow. Developers can easily find out where is the blind spots for this execution.

How to setup and run Emma in QApache?

Set Up QApache Port
Before you can run Emma in QApache, you have to make the following configuration to your QApache port: 1. Add emma.jar to Prepend System Classpath field. 2. Add a JAVA Runtime Variables: emma.coverage.out.file=/home/alawang/jdevhome/jdev/myclasses/coverage.ec This runtime variable tells Emma where to dump the coverage statistics. The directory of that file must be open to everyone. Otherwise, QApache cannot creat or modify the file. You can call chmod 777 on the directory. . 3. Copy emmactrl.jsp to the APPL_TOP/html You can copy emmactrl.jsp from /home/alawang/bin/emmactrl.jsp.

Gather Code Coverage

The following is a typical process you can follow to gather code coverage from QApache: Step 1: Compile the code

Step 2: Instrument the code : Click Tools -> Emma Instrumentation For how to define 'Emma Instrumentation' external tool, please refer to the Configure Emma In JDeveloper section above. Step 3: Copy the instrumented class files to QApache. You can either copy the class files into APPL_TOP/java, or you can configure a prepend class path and put the instrumented class files there. You can also modify the command line to instrument class files and place them into the QApache APPL_TOP/java or prepend class path directly. Then you won't need Step 3. Step 4: Restart the QApache port This will force JServ to load new class files instead of keep using the old class files . Step 5: Run your test as usual. Step 6: Go to emmactrl.jsp and choose 'Save the coverage data' The emmactrl.jsp can be found at http://qapache.us.oracle.com:<my_port>/OA_HTML/emmactrl.jsp Step 7: Generate report: Run Tools -> Emma Report.

References
For more details about Emma, please visit: http://emma.sourceforge.net/