Assignment 5: SCorTrap Testing
CISC 323, Winter 2006
Queen's University, Kingston, Ontario
This Assignment is due by noon on Thursday, April 6.
Questions about Java or Emma should be directed to Prof. Lamb. Other questions regarding this assignment should be directed to Prof. Kelly.
Please make sure you have read the general
Assignment Policies
for
CISC 323 before beginning this assignment. Please note the following especially:
- you must hand in your assignments to the CISC 323 assignment box on the second floor of Goodwin Hall
- we do not accept late assignments
- you may work alone or with one partner
- we take academic dishonesty very seriously
Please print out a copy of the assignment cover sheet
(Word or PDF),
fill out the information, and attach it to your assignment.
For this assignment, you will use an open source tool to do a coverage analysis as you test SCorTrap.
The coverage tool you will use is called Emma. General information about Emma and instructions on how to use it with ScorTrap can be found here.
Jeff Beiko’s notes about Emma can be found here
(tutorial 7).
For this assignment, use the Java code solution
for Assignment 4, provided here. Make sure you use the official sample solution to Assignment 4 – not your own solution.
There are two documents that must be handed in for this assignment. The first is a description of the tests that you ran on SCorTrap. The second is the first page of the coverage report produced using Emma. The two documents are created as follows:
- Create a set of tests for SCorTrap and check the coverage reported by Emma. The goal is to reach 99% line (statement) coverage. Create your set of tests as follows:
- First, use robustness testing (the extension of boundary value testing) to direct your choice of test cases. Only include in your set of tests those tests that increase your line coverage.
- If you cannot reach 99% line coverage using robustness testing, then create special value tests (see the lecture on Testing) to increase the statement coverage analysis shown by Emma. Again, only include in your set of tests those tests that increase your line coverage.
- As a tester, you know that tools such as Emma are useful but not sufficient for adequate testing. Create one more test NOT suggested by robustness testing but would be useful to include in your set of tests to find possible bugs. The choice of this test should be based on your knowledge of the SCorTrap program and possible programming pitfalls.
- You must document your entire set of tests with clear, concise instructions so your tests can be repeated by the TAs. You do NOT need to document your tests to the detail you used in Assignment 3. For example, give directions such as the following:
- Log in as Swifp, with password JamesTKirk
- Type in user name "Worf", and enter the following exposure data items:
150,300,14,21,7,1,75000
- Type OK to the pop-up telling you that Worf can stay on the planet.
etc.
- In the documentation of your tests, clearly label which tests were created by using robustness testing and which tests were created by using special value testing (if required) to attain 99% coverage. For your final useful test, provide a short explanation of why you chose that test and what type of bug you are looking for in SCorTrap.
- Print out only the contents of coverage\index.html. That will tell us the line coverage that your set of tests has achieved. Don’t print out the detailed reports from Emma, especially the parts containing code listings.
- Hand in your testing instructions plus a copy of your coverage report from Emma; make sure you hand in only coverage\index.html. The TAs will check that, by following your testing instructions, they can generate your coverage report.
- Note that you will be looking at test coverage for three classes only: SCorTrap, LoginDialog and NewOperatorDialog. These are the three new classes written for Assignment 4. The code provided for Assignment 4 (the database and calculation engine) contains methods that can't be reached from SCorTrap, so we don't want you to have to have to worry about line coverage in those classes. In the instructions for using Emma, the batch files we've provided take care of filtering those classes out of the report.
Back to CISC 323 Home Page
This page maintained by
Margaret Lamb,
Queen's University, Kingston, Ontario. Last modified