Step 44: Build the CS Habit of Testing

One of the things I promised my students I would do – and which I finally did after coming back from spring break – is that I would limit how many times they can submit to the autograder. At the beginning of the semester, students had unlimited submissions. The idea was that, since they were just learning programming, it was more important that they get practice with the concepts of variables and branching and looping. At the time, it was not unheard of for a student to submit to the autograder ten times in as many minutes. They would wait until they see the first failed test case – they see the results come back live – then tweak their code (intelligently or not) and submit again. Not everyone did this, but many students did take seven or so submissions before passing all the tests.

About a month into the semester, I started making the students write test cases. This was mostly in lab, with instructions that they must write and submit the test cases first before they are allowed to work on the actual program. The test cases then become part of their grade, based on how many other students’ bad code the tests catch. I don’t have a good impression of the number of autograder submissions (and I’m too lazy to look now), but I do remember students actually running their own test cases to see if their code works. I didn’t tell the students how many test cases they should write, only that they should write enough so that, if there is a bug in their code, they should be confident that a test case will fail. It took half a lab before students experientially understood what I meant, and some students needed more prompting to use their own test cases than others, but in general students adopted the idea of running test cases first before submitting.

Then, for the homework due this week, I told them that they only had five submissions. Period. The assignment wasn’t a trivial one – it required a good dose of problem solving/sequencing, and also reasoning about loops. Like I said above, I have threatened to do so for a while now, and although students were initially apprehensive of when it would finally happen, I don’t think I heard a single complaint when I announced it. This submission limit was even more restrictive than Michigan’s, which was two submissions per day. But, as of this writing an hour away from the deadline, more than half the class has passed all the tests, and none of them used more than five submissions, with many doing it in their first or second attempt.

One time is not a habit, but for now I will say that I am really happy with how this turned out. The weird thing is that I’m actually not sure which components led to this result. I don’t feel like I have done anything special, and yet clearly something is different, because most Michigan students never took their initiative that far to test each function they wrote. I was talking to another faculty today, who had previously taught at UCLA, and he said that the students here were more driven than the undergraduates at UCLA, and I am inclined to agree with him. I feel as though the students here are more willing to take that extra step, and not just do what you literally tell them then stop. Maybe it has to do with building trust as well – much easier in a class of 40 than in a class of 140 – so students are more willing to try what you say and see if it works out.

Again, I’m not sure what I did, or if I did anything at all, but I’m elated that it is working.

Step 44: Build the CS Habit of Testing

One thought on “Step 44: Build the CS Habit of Testing

  1. […] One concern when I first proposed this approach to testing is that students won’t necessarily learn to write good testcases. That’s true, but I think it needs to be qualified. For one, I think we need to separate the habit of writing testcases at all from the ability to write good testcases. I think the autograder did a decent job at the former, as I’ve mentioned in a previous blog post. […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s