Step 48: Reflect on the Autograder

(Note: This is Step 48, which means I have hit the end of a 12 post cycle and will not be writing next week.)

As we near the end of the semester – and the beginning of the summer – I’ve started thinking about the changes that I need to make to the autograder. The autograder is the first complete piece of software that has been battle-tested; although I have written substantial sections of code before for research, that has only been used by ten or so people. In contrast, the autograder has been in heavy use for most of the semester. My 35 students have generated over 94,000 test results (!) from over 3,600 submissions. I’m somewhat surprised that the system has held up at all, given that I wrote most of it over three weeks.

With no classes over the summer, it’s time to think about updating the code. There are a number of technical challenges that I won’t describe here; I will instead talk about some uses of the autograder that I didn’t expect.

First, one thing I didn’t expect is that I would use the autograder less as the semester progress. It makes sense in retrospect, since I focused more on program design and less on implementation. When students started designing their own classes, it became impossible to autograde their work, even with the introspect offered by Python. This coincides with the shift towards assignments that allow more creativity, such as creating images. These assignments were difficult to grade in general, but that’s a topic for another time.

Aside from the creativity of the students, however, I did find uses for the autograder that are not just about testcases. The students’ most recent assignment is writing an AI for the dice game Pig, when the students only “passed” if they beat a random player more than half the time over 10,000 games. I suppose this is similar to assignments where students must write efficient data structures. A different use for the autograder is to include checks for coding style; I haven’t fully implemented this yet, but I do ensure that student code can be imported cleanly as a module. I intend to use pylint in the backend to provide style checking in the future.

Entirely separately, although the autograder was designed to allow student-written test cases – which does work – I only used that feature a small handful of times. This is related to the creative assignments above: I feel that there is only a short period of time when students were comfortable enough with code to write their own test cases, and when I want students to start thinking about code structure such that simple testing is not possible. This is of course a larger issue than the autograder – I will likely need to tweak my assignments to require both an autograded component as well as a creative component.

Mostly though, except for a couple of odds and ends with additional information that I need to capture – due dates for assignments, for example – I thought the autograder worked well. The general architecture is solid enough that I’ve been thinking about “hiring” students to redesign the frontend to make it more usable… which would be a cool way for students to continue their computer science education while bringing things full circle.

Step 48: Reflect on the Autograder

Step 47: Bargain with Constraints

I don’t know if I have expressed this explicitly before, but I love my job. I want to see my students try new things, to find that they enjoy a new discipline they previously know nothing about, then take enough pride in what they are learning to spend time on the assignments (…potentially to the detriment of their other classes). I see my students wanting to continue learning computer science, and I want to create courses to serve those needs.

Which is why it’s frustrating to hit constraints sometimes. Constraints such as class size, constraints such as how many courses I can teach in a year, constraints such as needing approval for new courses.

I mean, I get it. I know why these checks are in place. New courses needs to be approved so that they meet the standards for the rest of the curriculum. At Oxy, this approval comes from the Academic Planning Committee, which oversees all changes to curriculum and major/minor requirements, as well as pre- and co-requisite data. Even with the small size of Oxy, this committee is often swamped with work, and so approval often comes much later.

I also get the constraint on the number of courses I can teach, or probably more relevantly, the amount of time I should be spending on teaching, as opposed to research or service. I was recently approached by a senior colleague suggesting that I am trying to do too much – who was then promptly approached by a senior colleague of theirs and told the same. It’s amusing only in that we all want to hire people who throw themselves into their work, but we are then concerned that they will burn out or simply become overwhelmed.

And class size is the one I get the most. My course in the fall is again over-registered, and incoming freshmen have not even had a chance to do so. This one I do feel slightly guilty about, since even with my labs this semester, I felt I didn’t quite have time to spend with individual students and teams. If I had thought ahead or had more experience with registration, I would have reserved seats for incoming freshmen. As it is, since I do have a little wiggle room, I might expand the class just a little to get some freshmen in.

So I understand why the system is structured this way, and I even agree with the reasons. I just wish I or Oxy has more offer for students who want to continue with computer science, but are then hit by these constraints.

Step 47: Bargain with Constraints

Step 46: Breathe

Most of the time, come Tuesday when I need to write something for this blog, I already know what I will be writing about. I keep a running list of topics that I might one day explore, drawn from various incidents over the semester. But nothing looked appealing today – usually I will see some idea about teaching that I wrote down weeks before and find it interesting once again, but everything looks forced.

So let’s talk about research. I have two deadlines this week, and so getting the data analyzed and writing the papers have dominated my life in the last two weeks. This is the first real bit of research I have done since coming to Oxy, and things feel different. It is a little scary not to have an advisor making sure that your writing makes sense, or to spot errors in the logic, and so I spend a lot more time going back over what I have written to make sure everything that needs to be addressed is there. One of the papers is building off of work that a student has done, which is interesting in its own way.

Because of all this though, I have been pulling two however-many-hour weeks, and I just need to breathe. I have actually gotten less sleep not because I was working (well, that too), but because my mind is on too many things even while lying in bed, making decisions about whether I should prep for lecture in the morning or how I might better frame the paper. I haven’t written in my journal in two weeks – but strangely, it’s because my journal felt too structured. As with my papers, I write my journal by keeping notes on what happened during the week, then organize it into something of a narrative. But with all the writing I’m doing, I miss just sitting down and putting whatever thoughts I have into words – what is what I’m doing now.

Work life balance – or lack thereof – is actually something I intend to talk about at some point, and I won’t do it now. I will say, though, that being strict about writing this blog is actually part of it, because it’s a mandatory block of time for me to write about something I really enjoy. Giving up the time to write in my journal, something just for myself, is much easier than giving up the time for this blog, even though I probably would have written everything in my journal anyway, if not as structured.

I guess I’m just happy to be taking this break now, before diving back into things tomorrow. I promise next week’s post will discuss something more substantial.

Step 46: Breathe