The assignment was simple – take our implementation of a toy cipher called baby rijndael, and adapt it to do CBC mode. Basically all the hard work was already done.
For some reason I felt the need to do the original assignment in C, which always inspires me to do all sorts of pre-mature optimization. Anyway I decided that my updated program would take arbitrary length inputs from the command line, so I had to start doing string processing – the original program just spit out encryptions and decryptions of some hex numbers. String processing in C is… much more of a pain than in Perl/Python/Ruby/everything except for assembly.
After spending way too much time getting it to work, I had a working program and went on to do the rest of the assignment. This was all done on my 64-bit desktop at home. When I tried running the program on my 32-bit laptop at my office, I got a segfault. To add to the joy of debugging, this is one hour before the assignment is due and during my lunch hour.
I fire up kdbg, and discover that the pointer to the output of my encrypt function, which is dynamically allocated, was not getting updated at all as the function looped through the input. After trying all sorts of fixes, including changing another local string variable to dynamic allocation, which actually partially fixed the problem, I noticed the real source. I was trying to set the 5th byte of a 4-character string buffer to null by addressing the 5th element. Classic off by one array indexing bug.
The point of this whole story is how errors like this cause confusing and inconsistent bugs. It just so happened that the byte after my string buffer was part of the pointer to the output string buffer, but only when compiled on my laptop. On my desktop, this bit of memory happened to be less important. When I changed a local variable to dynamic, a the error created a different bug.
The other point is that I need more practice in C… and now that Spring Break is approaching, it’s time for some NDS Homebrew fun!
