Step 77: Practice Practicum

The semester is wrapping up, and the first course to “conclude” is the Practicum course. There was no final presentation; instead, the last community partner meeting was on Saturday, where the students demo’ed their work… and that was that. There are some small changes we still have to do, and I will be back with some students next week to set up a production system, but the course is over.

Since this is the first time I’ve taught the course, I thought I would share some lessons learned. As a reminder, we were building a system that allowed volunteers with no tech experience (as in, may need help creating an email account) take pictures, then have it be OCR’ed and made searchable online.

  • Setting up expectations is (as expected) key. I knew this going in, and still failed to convey that we were not building a mobile app for the pictures. This mismatch was discovered three months into the four-month project. Lucky for us, this expectation mismatch did not require starting over, and I think the conversation we had in fixing this led to a better program at the end. Given that I was already watching for this problem and it still happened, I’m not sure what I should do next time to avoid it.
  • Students need help with organizing larger code bases. This lesson may be specific to me, because I was working with students who have only finished CS1. The insidiousness of this lesson is that the code works – it just repetitive, with multiple functions that do similar things, and the entire codebase difficult to extend for new functionality. I ended up re-writing a significant portion of the code about halfway through the semester, but in the future I will require code reviews and refactors. Similarly…
  • Force students to write documentation. By documentation here I don’t mean comments or APIs, but a narrative of the design process. I would like records of what ideas they have considered and why they ultimately decided on their current solution, but getting them to write this report was like pulling teeth. I assigned weekly reflections, which worked at the beginning of the semester when they were still understanding the problem, but were less useful as the coding took precedence. Even then, the reflections do not address the technical decisions. Both the previous point and this one lead to…
  • Breaking down grades is futile. I started the semester with a grade breakdown, with some percentage of student grades for peer evaluations, community partner evaluations, report, code, etc., but it’s unlikely I will grade based on anything but instinct. I talked to a colleague about their community-based learning class, one that is similarly product-focused, and they told me that they start with a baseline of A’s and subtract from there. I’m not opposed to this strategy, but also find it unsatisfactory. At the same time, it’s hard to devise any objective measure of goodness, even if the grade is broken down, so it seems like it comes down to gut judgments regardless.
  • Be prepared to provide ongoing support. This is tricky; students have no formal ties with the project after the semester, and even if they are willing to support the code, I feel bad putting them on the hook. As a trial run of the course, this current project is simple enough that I can support it, but I’m not sure how it would scale up when projects get more complicated or when there are more projects to be supported.

I deliberately imposed as little structure as possible on the students this semester, and it has helped me see where things break down. I was able to pick my students, and having their trust helped prevent the class from failing as I figured things out. If I teach this course again next year, it will be with more students and more projects. The library we worked with this semester agreed to work with us again (which is honestly very validating), but I will need to find new community partners as well. In the meantime, let’s see if I can put together a more coherent and structured course.

Step 77: Practice Practicum

Step 76: Discuss CS1 Grading Structure

When I designed the new CS1 last year, I made the explicit decision not to have midterms or finals, and instead have two in-lab “quizzes”. These 2-hour quizzes are done on the computer, with open notes and open internet (except communication), and account for a small percentage of the students’ grades (3% each). Teaching the same course my second time this semester, I mostly kept the same format, but increased the weight to 6% each. I write in my syllabus:

This course has no exams. This is deliberate; I believe that at the beginner level, computer science is more about the ability to produce working code (with a computer) than it is about answering questions about code (with pen and paper). For those who enjoy exams, rest assured that later computer science courses are more traditional.

I hesitate to draw conclusions from my first semester’s results, given that I was still calibrating what is possible. I believe, however, that this semester’s scores are more representative. The first quiz was before fall break and covers variables, functions, branches, lists, and loops, with three questions that each focused on one of the last three topics. The questions provided students with some example function calls and return values, but notably, their submission to the autograder does not give any feedback – they must write additional test cases themselves.

The results are decidedly mixed. A histogram with 10 bins results in a quad-modal distribution, with the two most extreme modes heavier than the middle two. The scores from the second quiz, on object-oriented programming (and in Java), had a similar distribution, although the two higher-grade modes were heavier. In both cases, the lowest mode contains exclusively students who scored 0%, and in both cases, there were students who scored 100%. There is some correlation between the two scores (correlation coefficient = 0.62), but there are students who did well on the first but tanked on the second, but also others who failed the first quiz but did decently on the second.

On a normal exam, this analysis would suggest that the exam is poorly written, and I’m not sure that this is not the case. The saving grace, however, is that a normal exam is worth 15-30+% of a students’ grade, which is more than the 12% of these two quizzes combined. In fact, the entire course is designed around frequent but low-stakes assignments. The vast majority (92%) of a student’s grade comes from

  • 8 “weekly” labs at 3% each, for 24% total
  • 6 individual homeworks at 4% each, for 24% total
  • 4 (optionally) group projects at 8% each, for 32% total
  • 2 quizzes at 6% each, for 12 % total

I would like to think this justified. First, frequent, low-stakes assessments allow students to evaluate their own progress. This is well-established in literature (here are two articles with references). Second, this allows students to stumble on a few assignments but still receive a good grade. This is necessary to accommodate the 12 homeworks, projects, and quizzes, spaced roughly once a week. Student evaluations suggest that students often spend 8+ hours a week on this course, and that’s outside of the 6 contact hours (3 hours of lecture and 3 hours of lab). Even taking late days into account, this is a significant amount of work, but is offset by the smaller weight of each assignment. Finally, I allow students to do the projects and many of the labs in groups. I half-follow Richard Felder’s ideas for cooperative learning in this, by assigning lab partners but not project partners, but definitely paying attention to minority status. As listed above, project groups are optional, which allows more advanced students to challenge themselves if they so choose.

In this grading structure, quizzes present the only timed, individual assignment. (I just realized that the projects are collaborative and untimed, the labs are collaborative and timed, the homeworks are individual and untimed, and the quizzes are individual and timed. I wish I could say I did this on purpose, and I’m not sure which is worth bragging about more.) In a way, quizzes serve as a reality check on the student’s abilities. Students can do well on the projects and homeworks by taking advantage of other students and multiple autograder submissions, but neither of those are available on a quiz. In terms of pure coding ability, writing real code with internet access under time pressure is one of the more realistic assessments.

That said, I am not convinced that coding quizzes are the right approach. While thinking about this issue, I came across Frank McCown’s blog post, where he summarized a thread on the SIGCSE mailing list on the topic. I am not convinced by the coding interview argument (being in an interactive environment dramatically changes the evaluation), nor by the cheating argument (the distributions above are not ones where cheating has occurred). The argument I am most swayed by is that pseudocode allows students to ignore syntax and focus on the problem solving. I don’t have a good idea of how much the syntax played into students who received low grades (although I should survey my students to find out), but it is a stumbling block that should not exist in an ideal class.

The SIGCSE thread makes it clear that there is no consensus on this issue, even if a majority of instructors prefer written exams. I am inclined to go deeper into the rabbit hole and ask whether exams/quizzes have a place in introductory computer science courses at all, but that will have to wait for another post.

Step 76: Discuss CS1 Grading Structure

Step 75: Share Governance of a Department

Oxy is in a time of transition – our Dean of Students stepped down not long after I started, and our Dean of the College recently started at his new position at Kalamazoo College. I was asked to be on the Dean of Students search committee, which just concluded last week. The process was time-consuming, with reading the applications, full days of interviews, and the on-campus interviews from the candidates. But it was also educational: I met people who I wouldn’t otherwise cross paths with, and the discussions on institutional priorities were some of the most meaningful, substantial conversations I have had since coming to Oxy.

One topic that stuck with me was a discussion of how Oxy students lack opportunities for leadership. What the speaker meant was that there were no obvious ways for student achievements to be recognized, nor for such students to take on additional responsibilities to guide future students down the same path. The most hierarchical organizations – the Occidental Weekly and the Orientation Team – has a visible impact on campus and do provide a path for advancement, but engages only a small portion of the community. The suggestion was that, as administrators and faculty, we should create channels for student merit to be nurtured, ultimately creating a culture where students recognize and encourage each other’s leadership potential.

The conversation was timely. Since GHC, I have thought hard about how to start a computer science peer mentorship program, especially one that is student-run and sustainable. As a leadership opportunity, I envision mentors coming to a consensus on goals to achieve for the year, as well as to interview and approve new potential mentors. The main reason I haven’t put this into action is that I am not clear on what the aim of such mentorship would be, if it should be network building, or include career path exploration, or extend into tutoring. I am also unsure of the role that existing student clubs should play or, in fact, how to best leverage my position as faculty to foster computer science at Oxy.

I had a second, much more radical, thought after the discussion: what would it mean for students to take partial responsibility for an academic program, to share governance of a department? This idea has been bubbling in my head for a while, from when students invited themselves to the faculty retreat, to earlier in the Dean of Students search process when I wondered why faculty meeting minutes are not available for students. The candid conversations about how the Dean is important to different constituents was not only meaningful to me, but also to the three students on the committee. It’s hard for me to say what exactly they experienced, if it’s simply having a voice at the table, or if it’s seeing faculty and administrators struggle with differing viewpoints. Whatever it was, I think they came out empowered with a deeper insight into how decisions are made at Oxy, and hopefully also more invested in the success of the new Dean.

Compared to Student Affairs, an academic department is one step removed from being directly responsible for and answerable to students, but that does not mean they should not be involved in the departmental decision. Students may not have the expertise to design a curriculum, but they could have a say in future course offerings, or who to bring in as speakers, or the structure of senior comps. In turn, these student representatives are expected to faithfully report the concerns and requests of their classmates and to bring them to the attention of the faculty, and conversely to relay the reasoning of the faculty back to other students.

Going back to student leadership, this role of student-faculty liaison is one that has clear privileges and responsibilities, and provides an obvious goal for students who want to excel and give back. Whoever steps into the role – and I am imagining two students, both to make scheduling easier and so they can support each other – must be respected by the other students, which requires leadership ability. And finally, the whole structure would strengthen student buy-in, while providing additional learning opportunities for students.

As far as I know, this model of departmental governance does not exist. Oxy, as with other colleges, assembles ad-hoc student committees for faculty searches. Students do sit on various administrative committees, but I do not know if they have a defined role; at least, I have not heard of student reports from committee meetings. The most relevant example I know of is for graduate students, but despite the stated responsibilities of the Graduate/Faculty Representative, that person has mostly relayed information out of faculty meetings and rarely provide input. Regardless, I do not know of any undergraduate departments where students participate in departmental affairs.

Secretly, I’m hoping that such a program will take off and help mend some of the mistrust between students and administration. The idea of departmental shared governance seems sufficiently obvious that I may be missing a critical flaw, but I’m not currently seeing one. And if starting a department is not the time to experiment, when is?

Step 75: Share Governance of a Department

Step 74: Address the Election

The big news from this past week is, of course, the election of Trump as the 45th president of the US. The atmosphere around Oxy has been subdued for most of last week, and there has been articles and blog posts about how colleges have responded to the unexpected turn of events.

For my part, Wednesday morning felt numb. My CS lecture was on parsing – normally a mind-bending topic due to the heavy mutual recursion, but which in context felt irrelevant. I hastily added a slide to my lecture about understanding if students can’t focus, offering to listen if they want to talk, but otherwise gave the lecture I had prepared before the election results came in. There were noticeably fewer students in class; one or two had emailed me saying that they were too distracted for class to be helpful, but many simply did not show up.

I co-teach a cognitive science class right after, where we made roughly the same announcement. Luckily, that class was dedicated to peer-review of papers, so students mostly worked in groups. Still, the conversations were hushed, and some students decided to pay with my colleague’s puppy (which she brought to class) instead.

What I did do was rethink some of my future lectures. One of them was the security and privacy lecture yesterday, which was depressing already. In addition to the standard narrative about strong passwords and limiting information leaks, I also added a slide about the privacy implications of corporate acquisitions. The reading was about Microsoft and LinkedIn, but I also brought up the NSA and government transitions. I will be changing one more lecture as well: the last lecture of the semester was originally going to be on version control, but instead I decided to more generally address the role that technology might have played in the election, specifically job loss due to automation. The plan to conclude on the message that, as students who how computers work and what technology can do, they must be deliberate in applying that knowledge to the social good.

All that, and I am not sure what else I can do for my students. I know some of my other colleagues have had students message them, even the night of the election as Trump pulled ahead of Clinton, seeking both advice and comfort. I have not had students come to me, and to be honest I am relieved for that. I have never been great at comforting people, and I have never learned to navigate that especially in the context of student-teacher relationships. I guess, for students reading this: you should know that many of your professors are feeling as helpless, and as lost for words, as you are. We don’t know what the next months or years will entail, but if you need to rant, or just a place to sit and take a deep breath, we’ll be here.

Step 74: Address the Election

Step 73: Reflect on the Grace Hopper Celebration

It’s been two weeks since GHC, and I want to reflect on the experience a little. I have been other smaller conferences – namely, the Wonder Women Tech conference/expo in July – and to be honest, that event left a bad taste in my mouth. There were too many “motivational” stories and not enough dissection of the challenges of recruiting women and minorities. There was a panel of director-level women which I enjoyed, and I got to ask Nicole Stott a question about sharing our passions with others, but those were the best moments.

Still, GHC is the largest and best-known event for a reason, and although it didn’t blow my mind, I also didn’t think it was a waste of time. I share the other faculty’s view that the event is heavily focused on industry, and I would even characterize GHC as a giant career fair with talks on the side. Most of the suggested policies in the talks are also aimed at companies, not schools – a flex work policy to retain women, for example, is a moot point for students who can already work whenever they want. Similarly, seeking out technical opportunities is much easier for students, who get to choose their classes every semester, than for early-career professionals.

Nonetheless, I think it is worthwhile for me to go, if only to have a better idea of what I will bring students to. I did learn a few things from the Redefining Mentorship panel, have some met people who I might keep in contact with. But let me conclude with one experience I didn’t blog, which occurred minutes after arriving at the convention center. I tweeted,

“Most salient feeling after five minutes at #GHC16: not belonging at a women’s celebration. …Except that’s the default for women in tech.”

I should tape that on my wall, because it’s so easy to forget that the lack of belongingness may have any single cause, but as an emergent property of the accepted culture. I’m not sure I know how to fight it, but at least I have a small taste of what it might feel like.

Step 73: Reflect on the Grace Hopper Celebration

Step 72.4: Attend the GHC Faculty Lunch

The GHC schedule leaves the lunches free for attendees, but also defines events for particular groups to socialize. One of these is the faculty lunch on Friday, which I attended. I sat at a table with Linda Sellie (NYU Tandon School of Engineering and, to my surprise, Occidental alum; she’s the second alum I ran into), Deb Richardson (UC Irvine), Soha Hassoun (Tufts), Marie desJardin (University of Maryland, Baltimore County), Aaron Gember-Jacobson (Colgate), Alexa Sharp (Oberlin), David Liben-Nowell (Carleton), and several others whose names I have forgotten.

There were two main topics of discussion at my table. The first was how GHC could be improved for faculty. All of us felt that the GHC was heavily industry focused, and that there were not many events for faculty to learn something. Accord to Deb, who has been at all but the first two GHCs, this trend towards commercialization is not new. We were surprised to hear that only about a third (5000 of the 15000+ attendees) were from academic (ie. faculty and students). Some thought that the number of company recruiters made up for the rest, but having been in some of the more professional-development-like sessions (like the technical career session), I actually believe that a significant number of attendees are early-career workers, looking to switch into a more technical position. I think this is evident in the sessions as well: some are entirely non-technical, otherwise require only an undergraduate-level of knowledge. Very few sessions are targeted at the PhD level, however, and Deb suggested (although she said it was not her idea) that Anita Borg should create a technical track taking female-authored best-papers from ACM and IEEE conferences.

It’s a good idea, but it doesn’t help the second problem: GHC is too crowded. The most popular talks had lines that wrap around the hallways, and apparently this was already happening last year when there were 4000 fewer attendees. (GHC was also at the same location last year.) None of us had good solutions to this problem: there was the suggestion of separating into multiple local conferences, but I suspect the West Coast (Silicon Valley) celebration would remain equally well-attended. One good thing about local conferences is that it would reduce travel costs for students and schools, especially since GHC is a major factor in attracting and retaining female CS students.

From there, the table turned to discussion strategies for teaching CS1 and CS2. There was a general consensus that we need to put the most personal, least scary professors in CS1, so that beginners are not scared off. Most of the other pedagogical strategies, however, are already well-known (to me, at least): encouraging in-class group discussions, soliciting multiple answers first (potentially then asking students to convince each other who is correct), and incorporating real-world scenarios and datasets into assignments. I got the feeling that none of us are satisfied with the literature, and are hoping that others will have additional ideas that would help.

I enjoyed the short hour that I got to talk to other faculty, and I wish there were more events where faculty can learn from each other. But, I understand that GHC is mostly an event aimed at students, and I’m glad to have any formal opportunity to meet faculty at all.

Step 72.4: Attend the GHC Faculty Lunch

Step 72.3: Help Non-CS Students Start Technical Careers

One of the most promising sessions – which I found myself, but also which several students caught and recommended I attend, is titled Pursuing a Technical Career without a Computer Science Degree. Clearly it was relevant for Occidental College, where students can’t get a CS degree even if they wanted to! Victoria Wobber (People Analyst at Google), Alison Song (People Analyst at Google), Gabriela Alcala Murga (Data Scientist, Facebook), and Manizeh Khan (Data Scientist at Amazon) hosted the session. Although they called themselves a panel, most of the session was interactive, with only one panelist on stage while the others walked around to talk to the audience. The session started with stories from each panelist of how they ended at their current positions, but then turned into pair/group discussions of our individual aspirations for a technical career.

Two things struck me about the introduction of the panelists. First, although they may not have any programming background, most of them were already in a STEM (or STEM-ish) field before switching. Tori has a doctorate in biology, Manizeh has a doctorate in psycholinguistics, Allison majored in 2012, and Gabi majored in math and stats. Second, the focus of the workshop was decidedly on data analyst positions: all four panelists mentioned learning SQL, and at least two of them mentioned R. My original interpretation of “technical career” was about software engineering, so I thought the title was ambiguous in that way. (I wonder if there are technical careers that are neither programming nor data analysis; perhaps lab-technician like positions might qualify?)

I was surprised a second time to find out that the main interactive portion of the session was about concrete steps to learning skills. First, our hosts helped us identify what each person wanted to learn, what resources they had to learn it, and how it might tie into their current or future jobs. The second half was then about setting short term (next week) to intermediate term (next year) goals, and how they might make themselves accountable. It was obvious that the session was a success, as the specific topic and the concrete steps meant most people walked away with an action item.

My main thought, as I was leaving, was how I could apply what I learned to Oxy’s students. When I decided to attend this session, I was hoping for more perspective on what careers exist that mix technical knowledge with other fields, or perhaps stories of transitions, or tips for navigating the job search and interviews. As it stands, the session was targeted more at early career professionals than at current/graduating students, who have a lot more opportunity to learn a technical skill while not juggling other responsibilities. So, although I thought the presenters did a great job leading us through the exercises, culminating in a concrete plan for improvement, I’m not sure it would be as helpful to my students as I want the session to be.

Step 72.3: Help Non-CS Students Start Technical Careers