Instructions For Using the Emma Coverage Tool
For CISC 323, Assignment 5, Winter 2006
Part 1: General Information About Emma
Emma is a free, open-source code coverage tool. Its licence allows us to mirror a copy of the tool on this web site for your convenience. We appreciate the many people who spent time developing it, and their generosity in allowing us to use it for this course.
The instructions here are very specific to using Emma to test the SCorTrap program for Assignment 5.
If you would like to know more about Emma, please visit
http://emma.sourceforge.net.
If your computer is not a Windows computer, you should still be able to get Emma to work, but our batch files won't help you; you'll have to follow the link about and look at the instructions there.
Part 2: Setting Up Your Computer To Use Emma:
- Make sure you have a Java interpreter (JRE) and compiler (JDK). Even if you don't remember ever downloading
such a thing, they may have been pre-installed on your computer, or they might have come packaged with another
Java tool. Look for folders with names starting with "jdk" or "jre". Good places to look are under
the "Program Files" folder, or inside installation folders of Java tools such as JBuilder.
- If you don't have a JRE or JDK, you can install one from the
Sun web site; choose "Download JDK 5.0 Update 6".
- Make a note of your JRE and JDK installation folders.
- Download a copy of emma.jar
- Inside your JRE installation folder, there should be a sub-folder called lib and under that
a folder called ext. Put your copy of emma.jar into this folder.
- Create a folder for using Emma. Into that folder, download a copy of our solution to Assignment 4. The
folder should now contain seven Java files (SCorTrap.java, etc.) plus the initial database file
(SCorTrapBackup.dat). I will refer to this folder as your "Emma folder".
- Download a copy of the following three files into your Emma folder:
Part 3: Using Emma With SCorTrap:
- Go to your Emma folder with "Windows Explorer" or "My Computer" and double-click on emmaSetup.
This command compiles SCorTrap and modifies the class files to allow Emma to collect coverage data.
You will get a compilation warning about "unchecked or unsafe operations". You may safely ignore this;
it's basically a complaint that we're not using the most up-do-date features of Java 1.5 in SCorTrap. (We
choose not to use them so that students with older versions of Java could still run the program.)
-
It's possible that some of you will get a more serious error message, complaining that DOS could not find
the "javac" command. That means your computer is not set up to run the Java compiler and interpreter from
the command line. The simplest way to fix this is to edit the three batch files that you downloaded in
Part 2. Open emmaSetup.bat in a text editor such as Notepad. You will see a line beginning with
javac. Change that to the name of your JDK installation folder + "\bat\javac". For example,
if your JDK is installed in C:\Program Files\Java\jdk1.5.0_06, replace javac with
C:\Program Files\Java\jdk1.5.0_06\bin\javac. There is also a line beginning with java.
Change that to the name of your JRE installation folder + "\bat\java". For example, if your JRE is
installed in C:\Program Files\Java\jre1.5.0_06, replace java with
C:\Program Files\Java\jre1.5.0_06\bin\java. The other two batch files each contain a
java command; make the same change there.
-
Once you've run emmaSetup, you'll notice that you have two new sub-folders inside your
Emma folder. The classFiles
folder contains the original compiled version of the SCorTrap classes, and the emmaClasses
folder contains new versions modified by Emma. You will also see a new file called coverage.em,
which contains more information that Emma needs. Leave all of these alone.
-
Now you're finally ready to run SCorTrap. Double-click emmaRun in your Emma folder.
You'll see a few lines in the DOS window, and then SCorTrap will pop up. Run it as you usually would.
When you're done with it, you can close the DOS window. After this, Emma will have created another
file, coverage.ec, inside your Emma folder. This is where Emma records which statements
you covered in your run. It's not a file you can look at directly with Notepad or any other
standard Windows tool. In a few steps you'll be running
another script to create a readable report from the file.
-
You can repeat the previous step and run SCorTrap several times. Each time you run SCorTrap
(using the emmaRun script), Emma updates coverage.ec, so that it contains
coverage information from all of your runs.
-
When you'd like to see a coverage report, double-click emmaHtml. This command creates
a nicely-formatted HTML file from the information in coverage.emc.
It creates another
sub-folder called coverage inside your Emma folder. Inside the coverage folder will
be an index.html file, which
you can open with a browser. The first screen you see will display summary information, including the
percentage of the statements in the program that you covered in your test runs. This percentage will
not include statements in the calculation engine or the database; you don't have to test these and
so our Emma scripts excluded them. This first screen is all that you have to print out to hand in
with your assignment.
- To see more details, click on "default package" and you'll see coverage data for each class
individually. If you click on a class name, you'll see things broken down by methods. Finally,
if you click on the name of a method, you'll see a listing of the program source, with unvisited
lines highlighted in pink.
- Now that you've seen some lines your testing hasn't covered, you can use emmaRun to add
test cases that cover those lines. When you're done, run emmaHtml again and look at the
new version of the html report with your browser. You may need to refresh your browser to make sure
you're seeing the latest version. (Most browsers have an icon for refreshing the contents, or
look in the View menu for a "refresh" or "reload" command.)
- At some point, you may want to start over and reset all of Emma's coverage information. Just delete
coverage.ec. This erases all of the information Emma has gathered about line coverage from
your test runs.
- Suggestion: When you think you've gathered enough test cases to get the required coverage, it would be a very good idea to reset Emma, as in the previous point, and run your test cases over again. This will help
you spot any errors or confusing bits in your write-up of your test cases, and will verify that you've
written down every test case necessary for getting the coverage. The coverage reflected in the Emma report should correspond exactly to the testing instructions you hand in.
Finally, a Warning: We think Emma is a great tool, especially considering that it's free, but it does have one unfortunate bug. When you exit a program with System.exit, a necessity in Swing programs, Emma does not record the fact that you have covered the block of code including the call to System.exit. In our SCorTrap program, this means that Emma will never record coverage for the lines:
database.writeBackup(this);
System.exit(0);
These will be lines 330 and 331 in Emma's listing. These two lines will be part of the 1% of SCorTrap that you will be unable to cover.
back to Assignment 5 page