Testing Part 2

A few weeks ago I wrote about introducing testing in CS classes, specifically using a testing framework. In that post I talked about the plan but now I can talk about the results.

My class interleaves with Hunter's CSCI 13500 - Software Analysis and Design I class. One day each week, my students have complete a hands on lab focusing on whatever is being covered in the 13500 class. I decided to use one of those labs as a first foray into testing.

I had each student use a classmates lab solution. They were to examine their solution and write a test suite for the lab using the doctest C++ testing framework.

I think this proved to be a valuable exercise.

Partway into the exercise, a few students independently had an issue - they couldn't even build the tests.

The code looked right. Something like this:

TEST_CASE("Lab 4 Tests"){
  CHECK(getValue("3/12/2016")==54.3);
}

It looked like at textbook example of a test case but it wouldn't even compile.

In all cases, the original student code compiled and ran and gave the correct result. All the students had code similar to this:

std::cout << "Testing getValue for 3/12/2016: " << getValue("3/12/2016") << std::endl;

This stumped everyone for a while - why wouldn't the test even compile.

The answer was "data types." The lab specification required that getValue was supposed to return a double but in all the problem cases, the student returned a std::string. The lab appeared to work since std::cout handles strings as well as numeric data types but once they put it in to a test case that actually checked types the problem was apparent and easy to fix.

As it turns out, I was pretty excited about this – even before finishign the tests, the act of writing tests revealed a problem. This is a good thing.

From there, it was pretty easy for students to finish writing the tests for their classmates labs.

Fast forward a week. This was the first lab where I required the students to submit their own test suite as part of their lab. This didn't seem to slow anyone down by an appreciable and time will tell if this leads to better development practices but it certainly made my life easier. I was able to quickly move from repo to repo and type:

make tests
./tests -s
... watch the tests run
make
./main
--- watch the program run

Easy peasy.

Not only did it make it faster for me, it also gave me insights into their programs by seeing their decisions in terms of test writing.

Is it a win?

At the very worst, they're learning an industry practice and at best, it's going to help them improve as software engineers. Add on the fact that it's making my life as a teacher easier both in terms of understanding the students work and in terms of speed of evaluation. Sounds like a win to me.