You see…in the ICPC, when you think your program is done, you submit your code to be judged and it can take a while for you to get back a response. During peak times it can be up to 30 minutes, which in a 5 hour competition is an eternity. To top it up, there’s only computer per team (of three), so, if after the wait the result that comes back is not “Accepted”, you have to stop a teammate who is already coding a solution for the next problem and sit down at the keyboard to debug your old code with no idea what’s wrong. If you’ve already mentally moved on to the next problem, it’s even more difficult.

As a result, one piece of advice for our teams would be to print out the code (I know, I cringed also… but it’s a competition, you play by the rules and make the best of what’s there) and start debugging it as soon as they hit the submission button. Consider it broken at all times, unless officially confirmed that it isn’t. The protest was always: but how do I know what’s wrong? Truth is you don’t… but you won’t know almost anything more from the judge’s response either! Except for crashes and time-limits, everything else usually comes back as a “Wrong answer”. You are not told what the input or output was. How useful is that! So you may as well assume it came in as a wrong answer and get started early.

Regardless, the biggest problem has nothing to do with not knowing what’s wrong. The problem here is getting yourself in the mindset that your code is broken. It’s very difficult, because if you genuinely believed that, you wouldn’t be submitting it in the first place! While you may agree to play this game and run through some scenarios, you *know* that your code works. The problem is done, finished, resolved. Ok, you’ll do a bit of this “fake” testing and debugging, since we all agreed to do it beforehand, but deep inside you know it’s pointless. You just built this thing, you’re in the maker’s mindset. Forcing yourself to assume that it’s broken requires you to be a different person, it requires you to be in a breaker’s mindset. And switching from one to the other, when it comes to something you’ve built, is far from easy. That’s why at ICPC, having another team member write test cases for your problem is sometimes a good idea.

The concepts transition unchanged from ICPC to real world commercial software development. Even with test-oriented developers and TDD (which everyone should be doing), you need “breakers”. You need people who were not directly involved in building the system that are motivated to try and break it. People that “know” that the system is broken and just need to figure out where and how. If you care about quality, you need a team of breakers.

Because as anyone that has ever written code knows… it’s always still a bit broken.

© anothem for Seconds & pixels, 2013. | Permalink | No comment |
Post tags: icpc, qa, tdd, testing

Feed enhanced by Better Feed from Ozh