TDD and libcheck

2013-10-05

Recently I have been writing an interpreter for scheme r7rs-small using something resembling Test Driven Developement, for this I have been using libcheck which is "a unit testing framework for c", during this I have found a few things I find interesting about TDD and libcheck that I thought worth sharing.


My first amazement came from finding out how well parsing fits into the TDD model, you have a function that takes some input (in this case a string) and produces some ouput (a parse tree), this function has no side effects and the only influence over it's output is the input string.

Thus it is very simple to write an empty function, give it a valid input string, and then test the output matches what the end result should be (of course at this point it is just garbage); you then change the function until the tests stop complaining.

For (almost) every feature so far I have written the tests as soon as I had an idea of what the feature should do, I then know I have finished the feature whenever the tests stop complaining; as an aside I have found this very useful for my personal motivation, waking up to find a failing unit test from the previous night gives me a very easy point to 'jump back into'.


Parsing went very smoothly [1], but once I got to eval I began to worry; the idea of eval (at least in the model I am using) is:

input -> eval -> side effects + output

As an example of this; I began working on runtime errors, the current model is:

I couldn't think of a way of handling this 'within' my current unit tests [2], but as it turns out, libcheck has a function for this:

tcase_add_exit_test(tc_funcs, test_error_bad_args, 1);

tcase_add_exit_test takes a test to test case to attach to (tc_funcs), a test to run (test_error_bad_args) and an exit status to expect (1), this means the test will only be considered a 'pass' if it results in the program exiting with status 1 and it doesn't otherwise fail [3].

This is accomplished by libcheck running each test within a fork, this also means if the test fails horribly (e.g. a segfault) then later tests can still be run cleanly.


Eventually I ran into an error that seemed like a case of uninitialised memory, my usual approach of 'debugging via print' didn't work [4], and due to the forking nature of libcheck, firing up gdb didn't work as gdb is only monitoring the parent (and everything interesting happens in the forked off children).

After some digging I found out that the libcheck developers foresaw this problem, exposing the environment variable CK_FORK='no' will prevent libcheck from forking (at run time) thus meaning all the usual gdb-fu works as expected [5].

export CK_FORK='no'
# usual gdb-fu goes here


So far I have been very pleased with both TDD and libcheck; I feel any extra effort TDD caused has more than paid for itself with the many advantages gained, libcheck has performed amazingly and it's design reeks of thoughtfulness.



[1] minus one hiccup where my unit tests were insufficient and it took me a while to notice

[2] although I could easily have a shell script run something and check the exit status, this didn't fit cleanly into the current framework

[3] it is still possible for the test to fail for other reasons, and if so this will be considered a fail

[4] in this case adding the print statements made the error go away (damn undefined behavior)

[5] http://gnupdf.org/manuals/gnupdf-hg-manual/html_node/Using-gdb-to-debug-check-tests.html