CI Unit Testing

Intro

CI (Continuous Integration) has a few parts - compiling, unit testing and communicating the results. This doc describes the command line implementation.

See IDE for the larger picture.

I separated commands between global and local where local is within a projects directory and operates directly on the project, where as global effect all the projects.

The workspace has a flat file structure, where each project has its own directory from the parent. This simplified file management and build tools.


       |--arcs

       |--bezier

proj---|--bsptrees

       |--bucket

       |--cetessD2

       ...

So global functions operate on all the directories under proj. And I would use local functions inside directories inside proj.

Options

The command options are given in angle brackets.

For example
projtest <update,release,silent,summary,clean>
can be called with relevant options
$ projtest release silent

Here are some common options.

silent suppresses command line output.
summary prints as small amount of information as possible.
debug generates debug code
release generates release code
update rebuilds the makefile
clean deletes object and executable code

The compilation is done generally in debug or release mode. This was reflected by testing in these modes as command line options. This restriction will be addressed at a later point in time as the client may want to supply other command line options to the compiler.

General Compilation(Local)

$ mkupdate generates the makefile, call after adding #include<>'s.

mkerrors <gdb release clean>
$ mkerrors compiles debug code.
$ mkerrors clean deletes object code.
$ mkerrors release compiles release code.

Global

These are functions that operate on many directories and generally take longer to run.

projtest <update,release,silent,summary,clean> - compiles and tests modules.

$ projdocreport - generate proj/report.html and proj/doc.html from previous tests.

$ projreport - all modules status is output to the command line.
For example.
zpr : module=Failure compiled=Success
which I call the amber state. There are three states which correspond with traffic light colours: red is compilation failure, amber has compilation success with no or failed unit testing, green has success on compilation and unit testing.

$ projreportneg - print in command line modules that failed to compile.

Local

Run within a projects directory, i.e. proj/xxx/
where xxx is the project/module.

projunittest <filename> <silent/summary> - run a test script in a module directory.
projunittests <silent/summary> - run projunittest on scripts testcripts*.txt in a module/directory.

projtest <update,release,silent,summary,clean> - compiles and tests a module.
projtest does both compilation and unit testing. The results are written to projreport.txt.
$ projtest release compiles release code.
$ projtest release silent compiles it without printing to the screen.
I generally delete the object files when changing the compiler flags.
$ projtest clean
The update option rebuilds the makefile.
$ projtest update summary

Test scripts are named testscript*.txt . Here is an example.
The text file testscript01.txt :
$ ./main prog=3 in=treeindexed01.txt $ ./main prog=2

To run this test script.
bsptrees]$ projunittest testscript01.txt

This outputs to the screen and writes a report to projreport.txt in the local directory. Other options can be supplied such as silent or summary to control output to the screen.
bsptrees]$ projunittest testscript01.txt silent


bsptrees]$ projunittest testscript01.txt summary

Report  testscript01.txt  Success

./main prog=3 in=treeindexed01.txt

  Success

./main prog=2

  Success

The plural projunittests does the same for each testscript in the current directory.

The power of the OS can be used against the problem by getting the OS to do unit testing. For example see proj/rsa/testscript03.txt.

projreport.txt

projreport.txt is a temporary or output file. After calling projunittest, projunittests, projtest, projtest the local project's report.txt file is written too.

An issue can occur when other programs use or expect information which is not there.

In bsptrees

$ projunittest testscript01.txt produced


Report  testscript01.txt  Success

./main prog=3 in=treeindexed01.txt

  Success

./main prog=2

  Success

./main prog=4

  Success

$ projunittests produced


Unit Tests  Success

Report  testscript02.txt  Success

./main prog=1

  Success

./main prog=6

  Success

Note error: no output from the first script, but the status of the test is OK.

$ projtest produced


Module Status  Success

Compilation Status  Success

Unit Tests  Success

Report  testscript02.txt  Success

./main prog=1

  Success

./main prog=6

  Success

Code

Writing unit tests for code is a matter of writing a function which returns a non-zero value to the OS if the unit test fails.

switch(proj) { ... case 133: return linetest::unittest01(); } return 0; }

Convention calling the function unittest*(.

The error strategy uses asserts in debug and runs the code in release. I have written a macro to handle both these cases so the program returns a non-zero value(program errors) when the assertion fails.

#include < typedefs.h > ... assertevalreturn( *pos == pti(pt2(0.5,2.0/3.0),197) );

The code is called when the testscript*.txt file is being parsed line by line calling each line at the terminal.

For example here is the line that calls the
case 133: return linetest::unittest01(); example.
./main prog=133