Tool Mentor:  Evaluating Code Coverage Using Rational PureCoverage and Rational PurifyPlus (Windows)

Purpose

This tool mentor provides an overview of how to use Rational PureCoverage to quickly and easily identify the gaps in your testing of Visual C/C++, Visual Basic, Java, and VS.Net language applications. This tool mentor is applicable for use with systems running Microsoft Windows.

PurifyPlus is a Rational product that includes PureCoverage functionality. 

To learn more about PureCoverage, including how to filter coverage data, merge and compare runs, and fine-tune data collection, read Getting Started with Rational PureCoverage. 

For step-by-step information about using PureCoverage, see the PureCoverage online Help.

Related Rational Unified Process information:

• Activity: Perform Unit Tests, refer to the section titled "Execute Unit Test".
• Activity: Implement Test.
• Activity: Evaluate Test.

Overview

PureCoverage collects a detailed set of coverage data for your program and every component it uses. With this data, you can have confidence that the program you're testing has been exercised—every line, every function, procedure, or method—before you release it to your customers. 

Tool Steps

To assess the effectiveness of your code coverage and pinpoint the parts of your code that are not exercised:

1. Run a program using PureCoverage to collect coverage data
2. Use PureCoverage windows and tools to analyze the coverage data
3. Run the program again, and use PureCoverage windows and tools to find coverage changes

1. Run a program using PureCoverage to collect coverage data 

The first step in monitoring coverage is to collect the code-coverage data.

For Visual C++, instrument and run the program, either directly from Microsoft Visual Studio using the PureCoverage integration, or from PureCoverage. PureCoverage instruments copies of the executable and its associated modules. PureCoverage also inserts additional code to detect which functions and lines were executed in the program. PureCoverage shows you its progress as it instruments files.

For Visual Basic, run Visual Basic projects or p-code programs (Visual Basic 6.0 only), or Visual Basic native-code programs (Visual Basic 5.0 or later), either directly from Microsoft Visual Basic using the PureCoverage integration, or from PureCoverage. 

When you monitor projects or p-code programs, PureCoverage puts the Visual Basic for Applications (VBA) interpreter engine into a special mode that enables it to monitor the engine's operation and directly collect coverage data as your code runs. 

For native-code programs, PureCoverage instruments the program and then collects code-coverage data. PureCoverage instruments copies of the executable and its associated object libraries. It also inserts additional code to detect which procedures and lines were executed in the program. PureCoverage shows you its progress as it instruments files.

For Java, run Java applets, class files, or code launched by container programs from PureCoverage using the Run Program dialog or from the command line. 

When you monitor Java code, PureCoverage puts the Java virtual machine (JVM) into a special mode that enables it to monitor the JVM’s operation and directly collect code-coverage data as the applet, class file, or code runs.

For VS.Net language applications, run the application from PureCoverage using the Run Program dialog and specifying that you are monitoring a managed-code application, or from the command line. PureCoverage puts the common language runtime (CLR) into a special mode that enables it to monitor the runtime's operation and directly collect code-coverage data as the application runs.

When Rational PureCoverage starts monitoring, it displays the Run Summary window so you can see both numerical and graphical information reflecting the progress of your testing, and check other information about the run. As you exercise your code, PureCoverage records data about its code coverage. You can pause and resume data recording at any time, enabling you to check specific portions of code. You can also take a snapshot of the current data, enabling you to examine code coverage in stages.

When you exit the program, PureCoverage displays the coverage data it collected. Because there can be a lot of data, PureCoverage uses filters to automatically filter out non-critical data from system libraries and other modules before it displays the program's coverage details. As you analyze the coverage data, you can display more or less data.

Tip: In addition to using PureCoverage interactively, you can also incorporate PureCoverage into your test scripts for automated testing. 

For more information, look up the following topics in the PureCoverage online Help index:

• visual studio

• visual C++

• visual basic

• java

• run summary

• snapshots

• filters

• vs.net

• scripts

2. Use PureCoverage windows and tools to analyze the 
coverage data 

The second step in monitoring coverage is to analyze the code-coverage data that PureCoverage collected.

When you exit the program for which PureCoverage has been collecting data, it displays the Coverage Browser window, which provides information such as the number of functions, procedures, or methods (collectively referred to here as functions) hit and missed by your tests, and the number of lines hit and missed by your tests. In addition, you can see the percentage of functions and lines hit, and the number of times each function was called.

You can use PureCoverage's other windows to further examine the program's code coverage. To review coverage data for all functions in the current run, and to sort them by various criteria, use the Function List window. If debug line information is available, and you monitored functions at line level, you can use the Annotated Source window to analyze a specific function's coverage line by line. Line-by-line coverage data is not available when you monitor a Visual Basic 6.0 p-code program. For the most detailed coverage data, monitor your Visual Basic 6.0 project directly from within Visual Basic 6.0 using the PureCoverage integration.

PureCoverage provides several ways to display only the data you're interested in. For example, you can filter out data by module name, source file name, or function name. PureCoverage automatically filters out non-critical data from system libraries and other modules before it displays the program's coverage details. As you analyze the coverage data, you can display more or less data. You can also use the Show/Hide Columns dialog to display only the information you are currently interested in.

By default, PureCoverage creates an Auto Merge entry in the Navigator window so you can easily analyze the total coverage of the program over multiple runs. You can also merge specific runs yourself to analyze coverage.

For more information, look up the following topics in the PureCoverage online Help index:

• coverage browser window

• function list window

• annotated source window

• filters

• merging runs

• debug data 

3. Run the program again, and use PureCoverage windows and tools to find coverage changes 

The third and final step in monitoring coverage is to rerun your tests, to see whether changes have caused the coverage to improve or regress.

After you make changes to your code, you can rerun the updated program and compare the new results to the previous run.

Use the Navigator window to keep track of all the runs you're working with. You can save coverage data as a PureCoverage data file (.cfy) to use for further analysis or to share with other PureCoverage users. You can save data to a tab-delimited ASCII text file (.txt) to use outside of PureCoverage; for example, in test scripts or in Microsoft Excel. You can also copy data from the Coverage Browser window to use in a text editor or from the Function List window to perform further analysis in Microsoft Excel.

To track your code coverage from release to release, archive your saved .cfy files with your product releases.

Tip: If you have incorporated PureCoverage into your test scripts, you can automatically save coverage data to a text file (.txt). 

For more information, look up the following topics in the PureCoverage online Help index:

• scripts
• navigator window 
• saving data 

Copyright  © 1987 - 2001 Rational Software Corporation

Rational Unified Process