As mentioned previously, I've been updating a graphical application written 10 years ago, from one graphics library to a new major version of that graphics library. Over the interveening decade quite a bit has changed so in some places there are some fairly substantial rewrites required. In one place the changes meant that the main graphical display widget for the application didn't work correctly -- initially crashing with a string of NULL pointer dereferences and then simply failing to display anything. Each of these was relatively easily found by looking at the stack trace at the point where it crashed, and "Debugging by Thinking" -- that is, knowing what's supposed to happen in the code and working backwards from the error to why it must have occurred (the two main issues were a change in the way that graphics contexts need to be created, so the original converted code wasn't hanging onto the ones it created properly; and storing the points to draw into a copy of a vector that then got discarded rather than the original that got passed to the draw routine). While I'm not finished by any means, having an application that works well enough to use the main functionality is excellent progress in completing the update.

Now that I have a working (albeit incomplete) program with a large number of scattered changes to almost anything involved in interacting with the graphical interface, the obvious next thing to do is to try all those graphical interfaces out and make sure that they work correctly in the new version -- that the things they're supposed to change, actually change things, the settings are replicated, etc. Other than the tedium of doing so (this program has no GUI test suite, as its development predated easy availability of such tools), the most important thing is to have some idea that (substantially) all of the GUIs have been tested. Fortunately there are tools available to assist with this.

For C++ code, using gcc, the standard "code coverage" tool is gcov, the GNU coverage testing tool. It works in conjunction with some special GCC flags that annotate the code, and then records the code that is executed as the program runs -- after which you can run gcov to get a report on what was run.

Assuming you're using something like automake/autoconf with standard Makefiles then the approach to prepare the program for coverage testing is:

make clean
CXXFLAGS="-fprofile-arcs -ftest-coverage" ./configure # ...
make

(ie, recompile with a couple of extra flags). The compile seems to take substantially longer with those flags, but it only has to be done once providing you keep developing with those settings.

Then run the program, and do some testing of things.

After that you can run gcov SOURCEFILE to get a summary based around that source file of the testing coverage (including things that it calls or included). The main catch is that you seem to need to change to the appropriate directory containing the source file before running gcov, at least if you have a standard automake/autoconf recursive Makefile structure that changes into a subdirectory before compiling source files.

Also useful is:

gcov -a SOURCEFILE

which will produce a report down to individual "basic blocks" showing which (parts of) which functions got executed.

For my purpose the most useful thing to know is which functions got executed which is most easily obtained with:

gcov -b SOURCEFILE

and then searching for "^function" in the gcov file that gets output.
With C++ the mangled name is printed which makes the process a bit more difficult, but looking for "0%" reports (ie, the function was never executed) and then looking at the nearby source lines indicates what functionality didn't get tested.

Running gcov will leave a lot of .gcov files lying around; they can be removed with:

find . -name "*.gcov" | xargs -r rm

It also keeps the "call record" (.gcda) files around, and you can run the program repeatedly (eg, a set of tests) without removing those files to accumulate test coverage before running gcov. But when you want to start testing again you need to remove those files as well:

find . -name "*.gcda" | xargs -r rm

before running the program. (The contents is automatically ignored if the object file is recompiled; possibly if any object files are recompiled. When the application is rerun after compiling something it reports "Merge mismatch for summaries".)

(The "-r" to xargs prevents running the "rm" command if there were no files found to remove; which makes this run silently even if all the files have been removed already.)

The .gcno must be retained as the hold the information from the compile process that is used for determining what is executed; if they are deleted the relevant object files need to be recompiled in order to recreate them.

On a tangentally related note, Subversion (svn; the revision control system) stores "ignore" lists in a property. If you are using svn then it is useful to tell it to ignore the various files that gcov's compile and run process creates. The relevant property is "svn:ignore", and there can be one per directory. Assuming you already have a "svn:ignore" property, the appropriate command is:

svn propedit svn:ignore .

which needs to be done in each directory that is holding the files. You want to ignore ".gcda" and ".gcno" files. Something like:

for DIR in $(find . -type d | egrep -v 'svn|deps'); do 
    svn propedit svn:ignore $DIR 
done

will catch all of them.

ETA 2010-05-20: Better "remove temporary files" commands.