Step 76: Discuss CS1 Grading Structure

When I designed the new CS1 last year, I made the explicit decision not to have midterms or finals, and instead have two in-lab “quizzes”. These 2-hour quizzes are done on the computer, with open notes and open internet (except communication), and account for a small percentage of the students’ grades (3% each). Teaching the same course my second time this semester, I mostly kept the same format, but increased the weight to 6% each. I write in my syllabus:

This course has no exams. This is deliberate; I believe that at the beginner level, computer science is more about the ability to produce working code (with a computer) than it is about answering questions about code (with pen and paper). For those who enjoy exams, rest assured that later computer science courses are more traditional.

I hesitate to draw conclusions from my first semester’s results, given that I was still calibrating what is possible. I believe, however, that this semester’s scores are more representative. The first quiz was before fall break and covers variables, functions, branches, lists, and loops, with three questions that each focused on one of the last three topics. The questions provided students with some example function calls and return values, but notably, their submission to the autograder does not give any feedback – they must write additional test cases themselves.

The results are decidedly mixed. A histogram with 10 bins results in a quad-modal distribution, with the two most extreme modes heavier than the middle two. The scores from the second quiz, on object-oriented programming (and in Java), had a similar distribution, although the two higher-grade modes were heavier. In both cases, the lowest mode contains exclusively students who scored 0%, and in both cases, there were students who scored 100%. There is some correlation between the two scores (correlation coefficient = 0.62), but there are students who did well on the first but tanked on the second, but also others who failed the first quiz but did decently on the second.

On a normal exam, this analysis would suggest that the exam is poorly written, and I’m not sure that this is not the case. The saving grace, however, is that a normal exam is worth 15-30+% of a students’ grade, which is more than the 12% of these two quizzes combined. In fact, the entire course is designed around frequent but low-stakes assignments. The vast majority (92%) of a student’s grade comes from

  • 8 “weekly” labs at 3% each, for 24% total
  • 6 individual homeworks at 4% each, for 24% total
  • 4 (optionally) group projects at 8% each, for 32% total
  • 2 quizzes at 6% each, for 12 % total

I would like to think this justified. First, frequent, low-stakes assessments allow students to evaluate their own progress. This is well-established in literature (here are two articles with references). Second, this allows students to stumble on a few assignments but still receive a good grade. This is necessary to accommodate the 12 homeworks, projects, and quizzes, spaced roughly once a week. Student evaluations suggest that students often spend 8+ hours a week on this course, and that’s outside of the 6 contact hours (3 hours of lecture and 3 hours of lab). Even taking late days into account, this is a significant amount of work, but is offset by the smaller weight of each assignment. Finally, I allow students to do the projects and many of the labs in groups. I half-follow Richard Felder’s ideas for cooperative learning in this, by assigning lab partners but not project partners, but definitely paying attention to minority status. As listed above, project groups are optional, which allows more advanced students to challenge themselves if they so choose.

In this grading structure, quizzes present the only timed, individual assignment. (I just realized that the projects are collaborative and untimed, the labs are collaborative and timed, the homeworks are individual and untimed, and the quizzes are individual and timed. I wish I could say I did this on purpose, and I’m not sure which is worth bragging about more.) In a way, quizzes serve as a reality check on the student’s abilities. Students can do well on the projects and homeworks by taking advantage of other students and multiple autograder submissions, but neither of those are available on a quiz. In terms of pure coding ability, writing real code with internet access under time pressure is one of the more realistic assessments.

That said, I am not convinced that coding quizzes are the right approach. While thinking about this issue, I came across Frank McCown’s blog post, where he summarized a thread on the SIGCSE mailing list on the topic. I am not convinced by the coding interview argument (being in an interactive environment dramatically changes the evaluation), nor by the cheating argument (the distributions above are not ones where cheating has occurred). The argument I am most swayed by is that pseudocode allows students to ignore syntax and focus on the problem solving. I don’t have a good idea of how much the syntax played into students who received low grades (although I should survey my students to find out), but it is a stumbling block that should not exist in an ideal class.

The SIGCSE thread makes it clear that there is no consensus on this issue, even if a majority of instructors prefer written exams. I am inclined to go deeper into the rabbit hole and ask whether exams/quizzes have a place in introductory computer science courses at all, but that will have to wait for another post.

Step 76: Discuss CS1 Grading Structure

Step 75: Share Governance of a Department

Oxy is in a time of transition – our Dean of Students stepped down not long after I started, and our Dean of the College recently started at his new position at Kalamazoo College. I was asked to be on the Dean of Students search committee, which just concluded last week. The process was time-consuming, with reading the applications, full days of interviews, and the on-campus interviews from the candidates. But it was also educational: I met people who I wouldn’t otherwise cross paths with, and the discussions on institutional priorities were some of the most meaningful, substantial conversations I have had since coming to Oxy.

One topic that stuck with me was a discussion of how Oxy students lack opportunities for leadership. What the speaker meant was that there were no obvious ways for student achievements to be recognized, nor for such students to take on additional responsibilities to guide future students down the same path. The most hierarchical organizations – the Occidental Weekly and the Orientation Team – has a visible impact on campus and do provide a path for advancement, but engages only a small portion of the community. The suggestion was that, as administrators and faculty, we should create channels for student merit to be nurtured, ultimately creating a culture where students recognize and encourage each other’s leadership potential.

The conversation was timely. Since GHC, I have thought hard about how to start a computer science peer mentorship program, especially one that is student-run and sustainable. As a leadership opportunity, I envision mentors coming to a consensus on goals to achieve for the year, as well as to interview and approve new potential mentors. The main reason I haven’t put this into action is that I am not clear on what the aim of such mentorship would be, if it should be network building, or include career path exploration, or extend into tutoring. I am also unsure of the role that existing student clubs should play or, in fact, how to best leverage my position as faculty to foster computer science at Oxy.

I had a second, much more radical, thought after the discussion: what would it mean for students to take partial responsibility for an academic program, to share governance of a department? This idea has been bubbling in my head for a while, from when students invited themselves to the faculty retreat, to earlier in the Dean of Students search process when I wondered why faculty meeting minutes are not available for students. The candid conversations about how the Dean is important to different constituents was not only meaningful to me, but also to the three students on the committee. It’s hard for me to say what exactly they experienced, if it’s simply having a voice at the table, or if it’s seeing faculty and administrators struggle with differing viewpoints. Whatever it was, I think they came out empowered with a deeper insight into how decisions are made at Oxy, and hopefully also more invested in the success of the new Dean.

Compared to Student Affairs, an academic department is one step removed from being directly responsible for and answerable to students, but that does not mean they should not be involved in the departmental decision. Students may not have the expertise to design a curriculum, but they could have a say in future course offerings, or who to bring in as speakers, or the structure of senior comps. In turn, these student representatives are expected to faithfully report the concerns and requests of their classmates and to bring them to the attention of the faculty, and conversely to relay the reasoning of the faculty back to other students.

Going back to student leadership, this role of student-faculty liaison is one that has clear privileges and responsibilities, and provides an obvious goal for students who want to excel and give back. Whoever steps into the role – and I am imagining two students, both to make scheduling easier and so they can support each other – must be respected by the other students, which requires leadership ability. And finally, the whole structure would strengthen student buy-in, while providing additional learning opportunities for students.

As far as I know, this model of departmental governance does not exist. Oxy, as with other colleges, assembles ad-hoc student committees for faculty searches. Students do sit on various administrative committees, but I do not know if they have a defined role; at least, I have not heard of student reports from committee meetings. The most relevant example I know of is for graduate students, but despite the stated responsibilities of the Graduate/Faculty Representative, that person has mostly relayed information out of faculty meetings and rarely provide input. Regardless, I do not know of any undergraduate departments where students participate in departmental affairs.

Secretly, I’m hoping that such a program will take off and help mend some of the mistrust between students and administration. The idea of departmental shared governance seems sufficiently obvious that I may be missing a critical flaw, but I’m not currently seeing one. And if starting a department is not the time to experiment, when is?

Step 75: Share Governance of a Department

Step 74: Address the Election

The big news from this past week is, of course, the election of Trump as the 45th president of the US. The atmosphere around Oxy has been subdued for most of last week, and there has been articles and blog posts about how colleges have responded to the unexpected turn of events.

For my part, Wednesday morning felt numb. My CS lecture was on parsing – normally a mind-bending topic due to the heavy mutual recursion, but which in context felt irrelevant. I hastily added a slide to my lecture about understanding if students can’t focus, offering to listen if they want to talk, but otherwise gave the lecture I had prepared before the election results came in. There were noticeably fewer students in class; one or two had emailed me saying that they were too distracted for class to be helpful, but many simply did not show up.

I co-teach a cognitive science class right after, where we made roughly the same announcement. Luckily, that class was dedicated to peer-review of papers, so students mostly worked in groups. Still, the conversations were hushed, and some students decided to pay with my colleague’s puppy (which she brought to class) instead.

What I did do was rethink some of my future lectures. One of them was the security and privacy lecture yesterday, which was depressing already. In addition to the standard narrative about strong passwords and limiting information leaks, I also added a slide about the privacy implications of corporate acquisitions. The reading was about Microsoft and LinkedIn, but I also brought up the NSA and government transitions. I will be changing one more lecture as well: the last lecture of the semester was originally going to be on version control, but instead I decided to more generally address the role that technology might have played in the election, specifically job loss due to automation. The plan to conclude on the message that, as students who how computers work and what technology can do, they must be deliberate in applying that knowledge to the social good.

All that, and I am not sure what else I can do for my students. I know some of my other colleagues have had students message them, even the night of the election as Trump pulled ahead of Clinton, seeking both advice and comfort. I have not had students come to me, and to be honest I am relieved for that. I have never been great at comforting people, and I have never learned to navigate that especially in the context of student-teacher relationships. I guess, for students reading this: you should know that many of your professors are feeling as helpless, and as lost for words, as you are. We don’t know what the next months or years will entail, but if you need to rant, or just a place to sit and take a deep breath, we’ll be here.

Step 74: Address the Election

Step 73: Reflect on the Grace Hopper Celebration

It’s been two weeks since GHC, and I want to reflect on the experience a little. I have been other smaller conferences – namely, the Wonder Women Tech conference/expo in July – and to be honest, that event left a bad taste in my mouth. There were too many “motivational” stories and not enough dissection of the challenges of recruiting women and minorities. There was a panel of director-level women which I enjoyed, and I got to ask Nicole Stott a question about sharing our passions with others, but those were the best moments.

Still, GHC is the largest and best-known event for a reason, and although it didn’t blow my mind, I also didn’t think it was a waste of time. I share the other faculty’s view that the event is heavily focused on industry, and I would even characterize GHC as a giant career fair with talks on the side. Most of the suggested policies in the talks are also aimed at companies, not schools – a flex work policy to retain women, for example, is a moot point for students who can already work whenever they want. Similarly, seeking out technical opportunities is much easier for students, who get to choose their classes every semester, than for early-career professionals.

Nonetheless, I think it is worthwhile for me to go, if only to have a better idea of what I will bring students to. I did learn a few things from the Redefining Mentorship panel, have some met people who I might keep in contact with. But let me conclude with one experience I didn’t blog, which occurred minutes after arriving at the convention center. I tweeted,

“Most salient feeling after five minutes at #GHC16: not belonging at a women’s celebration. …Except that’s the default for women in tech.”

I should tape that on my wall, because it’s so easy to forget that the lack of belongingness may have any single cause, but as an emergent property of the accepted culture. I’m not sure I know how to fight it, but at least I have a small taste of what it might feel like.

Step 73: Reflect on the Grace Hopper Celebration