Step 57: Publish Papers

I just got back from a four-day trip to Chicago, for my first conference after starting at Oxy, so I thought it’s a good time to talk about the research side of my job. My co-authors and I wrote most of the paper during the spring semester, which required multiple late nights on my part. Despite that sacrifice, I found that I actually enjoyed sitting down and organizing my thoughts through writing. Maybe it’s because it’s the first piece of academic writing I have done since my thesis and it was a relief to have to write only six pages of text. My paper with a student was particularly enjoyable, since they did half the experimental work and I didn’t have to deal with the analysis (at least, not the initial stab) and could focus on the writing.

Still, when I think ahead to establishing a broader research program, I can’t see the path from here to tenure. The easiest obstacle to overcome is computational resources, but I plan on using my startup to get computers before the summer’s over. More worrisome is the lack of students with advanced computational skills. The students may only be able to do conceptual work at a high level, while I try to convey the structure that I need from their work. It has been working well thus far with my summer student, but it’s hard to say whether I will continue to have student-completable work. My teaching and research mingle in the case of student researchers; the better I do attracting students to computer science, the more potential recruits there are. For the next year or two, however, I’m not sure I will be able to maintain a research group with any continuity.

On an even longer horizon, one piece advice I received is to develop a five-year research plan for topics for papers and venues for publishing them. I suspect this is one area where computer science differs from many fields, since papers are published on an annual conference schedule. While I have a broad research goal to pursue and projects within that goal, I do not know when each project will be complete or reach a publishable state. Maybe this also speaks to my own inexperience, but as long as I keep moving multiple projects at different stages of the pipeline, I don’t feel as though a concrete long-term plan is necessary.

It is already the end of June, halfway through the summer, and I feel I have only taken a breather and caught up on all the non-teaching work that I needed to do. I will be presenting at another conference in August, and hope to submit at least one paper in the fall, all while starting to prepare for classes again, this time with added committee service on top.

I’m excited.

Step 57: Publish Papers

Step 56: Bring the City to the Classroom

One thing Oxy prides itself on – or at least, one that that Oxy mentions in its advertising – is that the college is located in urban Los Angeles. The standard narrative is that theater students can read about the history of a playwright one day, then watch professionals perform the opera the next day; that economics students can learn about globalization and immigrant one day, then interview people who have felt their effects the next day. Oxy has a Center for Community-Based Learning specifically to help faculty engage organizations in Eagle Rock and LA. Multiple faculty, in fields ranging from kinesiology to critical studies to philosophy, have worked with the CCBL to create courses where the community plays a crucial role.

I decided to join in the fun in the fall when, while drafting the new computer science minor curriculum, I added a Practicum course as one I would like to develop. The idea is that students in this course would work in groups on a single project (per group) for the entire semester. The actual problem they were solving would depend on which member of the community they were working with. (CCBL includes the faculty and offices of Oxy in their definition of community.) Pedagogically for computer science, this would essentially be a Software Engineering course, for students to learn about working in teams, agile development, user-centered design, and so on. The twist is that CCBL works with many non-profit organizations that tackle specific social justice issues in LA: gentrification, police abuse, intersectional public health, urban education, and many others. By working closely with these organizations and the population they serve, the goal is for students to also experience first-hand how technology intersects with other disciplines.

Due to my existing teaching responsibilities, I didn’t think I would have a chance to implement this course for several years. But then several students approached me about doing an independent study, and I realized that having one group of students and one community partner would be the perfect way to pilot what could be an unmanageable course. With help from the CCBL, I got in touch with the Southern California Library (SCL), which has a physical collection of documents from local non-profits but no funding to digitize them into a form easily usable by archivists. If everything goes to plan, my students will be dipping their toes into image manipulation and optical character recognition, while figuring out how to create something that can be used by both the library’s volunteers (who will provide the labor for scanning/photographing the documents, but who may be technologically illiterate) as well as the archivists (who may want documents to be organized and presented in specific ways).

A course like Practicum seemed so natural to me, that only after I have talked to SCL did I realized what my influences were. The name Practicum came from a computer science course I took as an undergrad. The course was a quarter-long project, but all the projects were about improving the performance of some kind of AI. Since the AI was part of the professors’ research, the course was basically group-based directed research (with some software engineering).

The other influence which I had forgotten about, however, were my undergraduate engineering design courses. My first design course was an engineering requirement, but I continued in the curriculum (ultimately getting a design certificate). All the courses were quarter-long projects (and one spanned two quarters), but crucially, we were working with organizations like the Rehabilitation Institute of Chicago and the Shedd Aquarium. It was in these courses that I learned about user-centered design, but it fits so seamlessly into computer science (especially human-computer interaction) that I had forgotten its separate disciplinary origin.

To be honest though, increasingly I have been thinking about teaching computer science as engineering design. While I doubt upper-level computer science courses can be taught this way, I see multiple parallels at the introductory level that may guide students into the right mindset. It’s not just in the process of iterating and testing, but also the consideration of goals and the breaking down of functional requirements and components. I suspect courses like Practicum are also great ways to tie computer science into all the other disciplines in the liberal arts, and I fully intend for the course to be required for computer science majors.

Step 56: Bring the City to the Classroom

Step 55: Gamify Computer Science

I apologize for the late post – I was sick yesterday and couldn’t finish this post in time.

I’ve been thinking about the idea of gamification and education. I was inspired by two episodes of Extra Credits, a vlog about video game design. The first video is about the separation between entertainment and education (or “edutainment”) games, and the former could still include “tangential education” elements. One good example is the Civilopedia (an in-game encyclopedia in the Civilization games), which explains the culture, technology, and people that exist in the games. Age of Empires II had a physical, multi-hundred-page manual that included the history of each of the units in the game. On the more explicitly educational side, I remember pouring over the “world almanac” that was included with Where in the World is Carmen Sandiego when I was younger. Although, I was also the kind of kid who spent his allowance on Encarta ’95… and spent days on the MindMaze trivia game.

These products start with a fun game and inserts education, but in colleges we go the other way – start with some concepts or skills that we want students to master, then motivate students to learn it. This is the second Extra Credits video, although many ideas are targeted at younger students who don’t mind more heavy-handed gamification such as skills and levels. Still, some ideas have entered college-level material. For example, providing students with immediate feedback is already pervasive in computer science, and is spreading to other subjects in the form of online courses. The interactive online textbooks from zyBooks takes it a little further, by displaying stars that light up when students complete an activity, bringing to mind games such as Angry Birds.

I’m not a gamer, and I don’t particularly like overt external reward systems, but four ideas from the video still jumped out:

  • One is simply the idea of framing progress in a different way. The video talked about grades that add up from 0 instead of being subtracted from 100. zyBooks uses stars, which is kind of corny, but my autograder actually uses the same idea, by including what I call the “scoreboard” that displays whether the last submission passed or failed each testcase. In theory, I am not giving more information than a number – “43/76 testcases passed” – but seeing each test turn red or green in real time has a psychological effect. I don’t remember if I’ve mentioned it before, but in the middle of the semester I was concerned by how much time students were spending on my assignments. I realized that students were not used to getting immediate feedback and got into an “okay, just one more try…” mindset. I’m ambivalent about this effect, since I don’t want students to neglect their other classes, but I won’t be changing the autograder for now.

scoreboard

  • Another idea, not as explicit in the video, is putting exercises in a narrative. The most recent example I can think of is the Advent of Code, a set of 25 exercises with a weak story about Santa and his elves. But I am slightly curious what happens to Santa, even though I know it is only the barest of a narrative. For computer science, I think the benefit here is not so much engagement/motivation, as it is demonstrating how computer science may be used in context, or perhaps as worked examples of how to represent real problems in code.
  • The third idea – the first of two that I’m more excited about – is unlockable content, or more generally the recognition of an “achievement”. The obvious reward here is extra credit, but there are likely other possibilities. For example, maybe if a narrative is involved, doing well in the assignment would unlock the ending to the story. If that sounds clich├ęd, what about unlocking real data on which students can test their code, ideally coming from that narrative? We can also talk about what triggers these achievements, which could range from passing the autograder on the first submission, to a testcase that reveals a bug in every student’s code. Most achievements will probably only be reached by a small number of students, but maybe with enough achievements at enough difficulty levels, every student will get at least one achievement.
  • The last idea is that of collective goals: rewards that depend on everyone in the class doing well. I am reminded of positive inter-dependence, which is the idea of structuring assignments so that members of a team must depend on each other for the whole team to succeed. Extra Credits talked about rewards depending on everyone in the class getting B’s on a test, but again we can generalize to other achievements. The important thing here is cooperation between students and incentivizing better students to help others who may be struggling. This past semester I had a 1% extra credit for “general helpfulness”, which I never assigned to anyone because it was near-impossible to assess. Collective goals have other problems – there will always be students who don’t help others but get the reward – but this may be a better motivation for students to help each other, or even to publicize those who are pro-social.

Again, I’m not a gamer, and in the past I have been generally against the idea of gamification, or offering external rewards in general. (Grades are also external rewards, and I’m also iffy about them, but that’s a whole different topic.) What I think is different about the ideas here is that they are not arbitrary rewards for acing a test or writing code. Okay, progress bars is sort of in this realm, but “rewarding” students with real-world data or motivating students to help others are valuable experiences in their own right. You might say that this is making explicit some of the periphery skills that computer scientists should have.

I think video games have gotten complex enough, and importantly there are enough examples of clever incentive mechanisms, that gamification is worth a second look. Of course, not everything in video games will transfer, and gamification can have its dark sides, but I suspect there are many ideas which could be usefully imported into education.

PS. Also, go watch Extra Credits. To start you off, here’s an episode about how one game forced players to think about racial representation.

Step 55: Gamify Computer Science

Step 54: Review CS1 Goals (Part 4)

This is the last post in this series of reflection about teaching CS1. For the last couple weeks, I have written about my goals for the course and what I did (deliberately or not) to meet those goals. Today I will be addressing changes for future iterations of the course.

Let me start with two particular goals I would like to improve on. One is the encouragement of test-driven development. Although students proved capable of writing good tests for highly visual problems, there is room for improvement in developing their general testcase writing skills. The simple way to do this is to increase the use of peer-submitted testcases, introducing them earlier and requiring students to submit testcases more often. (Several assignments in the later half of the semester allowed for significant creativity for the students, such as creating a picture of their choice, which could not be easily autograded.)

The obstacle here is mostly in logistics, as I don’t want to impose deadlines for the testcases in addition to the actual assignment. What I might do is only allow students to use their own test cases to begin with – essentially turning the autograder into a single-user testing platform – then do a batch regrade afterwards. This requires slightly more work on my part, but should help students build the testing habit. I am also considering making students reflect on the coverage of their testcases, but that may be too much.

The second goal I would like to work towards is functional decomposition. This was not something I explicitly addressed this semester, which is an oversight I intend to fix. More generally though, I suspect a greater focus on the design and structure of programs will help. For now, my plan is to require pair programming during the labs; since students are already working in pairs (but each with their own laptop), this is a relatively small change. I’m also thinking that students must be able to explain how their code will work to me (at first) and to each other (later) before actually coding.

There are other changes as well. The most general one is, as has been apparent on this blog, that I badly over-estimated what students were capable of. I think the root cause of this is that I had too rosy a view of CS1 at Michigan. It’s only now that I can see how I may have been blinded by my positive experience teaching the course and not realized that students may not have learned as much as I thought. The pedagogical content knowledge I gained this semester will be used to tweak everything next time.

The remaining changes all come from the teaching evaluations. The first is changing how I teach multiple programming languages. This semester, we spent the majority of the course on Python, then two weeks (and a lab and a homework) on Java. This was not well received:

I think the last few classes spend on Java & Javascript were unhelpful and could have been made more effective by focusing more on classes and more Python/HTML related topics. I think the web app would be better off being focused more towards the last couple weeks instead of just cramming it in (which is what it felt like). Java and Javascript were not smart uses of our time (in my opinion).

The intention was to prepare students for the next course (which is in Java), and also to blur the trees (syntax) to see the forest (the computational ideas). While using multiple languages is not uncommon, I don’t know of any literature on how/when to present the second languages. Introducing every concept in Python first was the obvious first approach, but next time I might introduce OOP in Java, then later reintroduce the same concepts in Python. Since OOP is right after spring break, this might help reinforce the old concepts (branches, loops) while forcing students to engage with Java. The JavaScript, on the other hand, I might remove from the syllabus entirely.

The last major improvement to make is about the ability for individual students to do work. I mentioned that the students work in pairs during lab; they also work in groups for their large projects, leaving just their homework which is done independently. This organization is good for enthusiasm…

This class was overall a good experience for me. It really helped that group work was encouraged so much, and I loved having the work sessions in the library.

…but perhaps not so good for confidence:

I am ending this class feeling pretty dependent on others to help with my programming, so if there was a way to make the class slightly more independent, I think that could be good.

I know programming is a collaborative effort, but it often times feels like the only way I can complete an assignment is with help from my peers or the professor. Perhaps we are allowed to collaborate too much, which leaves us reliant on our peers.

I don’t have a good general solution, although several students suggested that I hold individual check-ins every few weeks, which I think is a great idea. This would also help mitigate the mixed skill levels of students, something I’ve already noticed, while also encouraging a more diverse group of students to continue in computer science.

The devil is in the details of all of these changes, of course, but I’m making the plans now and will see how they work in the fall.

Step 54: Review CS1 Goals (Part 4)

Step 53: Review CS1 Goals (Part 3)

Let’s get straight to it.

Debugging

My approach here is probably frowned upon. Although students were using an IDE (PyCharm for Python and IntelliJ for Java), I never taught them to use the debugger; instead, I relied entirely on printing. There’s really no good reason for this, except perhaps for the rustication that students should be able to debug even without a debugger. At Michigan, where we did teach the debugger (in both Visual Studio and Xcode), students were often confused by the various stepping functionalities, and especially when loops are involved, they get tired of continuing until some particular state. Conditional breakpoints could solve the problem, but at that point using print statements were easier.

I think the biggest obstacle in debugging was not seeing what values the variables contained, but matching those values with the error descriptions. While Python error messages were relatively clear, they often required a deeper understanding of what is going on. One error that stumped many students on a quiz comes from a line such as

>>> a, b = '1 2 3'.split()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)

You can argue that I shouldn’t be teaching multiple assignment (… which is a valid point), but the general point here is that error messages can be cryptic even if they identify the buggy line in question. I don’t know if there’s any good way to teach that.

Test-Driven Development

The main way I have approached this is through crowd-sourced testing using the autograder. For grading, students’ test cases accounted for 10% of their grade for an assignment, based on their two testcases that the most students failed. The testing score for the entire class is curved, so the student whose tests failed the most submissions always get 100%; this means that everyone will get 100% for the testing portion in the unlikely event that everyone submitted perfect code. Incorrect testcases subtract from their score.

One concern when I first proposed this approach to testing is that students won’t necessarily learn to write good testcases. That’s true, but I think it needs to be qualified. For one, I think we need to separate the habit of writing testcases at all from the ability to write good testcases. I think the autograder did a decent job at the former, as I’ve mentioned in a previous blog post.

I also think that whether students understood what “good” testcases are depends greatly on the domain. I did a lab where students had to determine whether a latitude-longitude pair was in the State of California (as defined by seven corner points). The lab was too math-heavy, but the heavily visual problem allowed students to use Google Maps to identify points just inside or just outside the boundary. These testcases actually identified a fundamental problem with the lab (namely, that we assumed the coordinates were on a plane and used straight lines instead of using great-circle distance) and, in that sense, they were exactly the right testcases that students should be writing. I don’t think it’s a strong habit by the end of the semester, but I was not pushing on testing particularly hard either.

This acumen, however, did not extend to more abstract problems. For a function that determines whether a date is part of the Gregorian calendar (ie. after 1752-09-14), most students were unable to come up with a comprehensive test suite. Similarly, for some of the more data-driven assignments, most students didn’t think of creating simpler fake data for testing. This second oversight can probably be amended, but I think the fundamental problem is that writing good testcases is hard. As with debugging, some of the knowledge is not actually about the domain, but about more complicated aspects of computer science; in this case, what errors a stereotypical programmer might make. I can walk students through my own testcases (and in fact, I should do exactly that next time), but I suspect a lot of it comes down to experience.

Version Control

We only used Version control for the final project, where it was the easiest way to put a webapp on Heroku. I only taught the basics of version control (committing, pulling, merging, and pushing in git), and only in the GUI provided by PyCharm, and for the most part students had little trouble. To be fair, I mostly had students follow scripts of how to use git, but I think exposure is the right goal to aim for in CS1.

Structural/Procedural Abstraction

While I understand what I mean here at an, ah, abstract level, it is hard for me to say what this means concretely in a CS1 class. Perhaps the best example is one where the student did not abstract. For the course-directory webapp, one student wanted to list courses by the core requirement that they satisfied. Instead of writing a function that takes as argument the requirement, however, they wrote one function for every single requirement, duplicating the same several lines of code about a dozen times. I was honestly surprised by this, as the student had otherwise a good understanding of the concepts in the class.

I can tell a similar story about structural abstraction, about a different student who create multiple copies of a class that differed only in one hard-coded constant, which should instead have been an argument to the constructor.

I will be honest: this kind of abstraction never had a prominent place in my curriculum, mostly because I’m not sure what I have to say about it. The idea of abstraction itself is sufficiently abstract that I’m not sure what lecture to provide. I will try to give more worked examples of generalizing code, but I would appreciate any ideas for more directly getting students to thinking about/practice abstraction.

Functional Decomposition

This is another area where I struggled. I think there are several types of functional decompositions. One is actually with functions, where you find a part of the problem that seems self-contained and make it a function. I never wrote an assignment that emphasizes this. What my students did have trouble with is functional decomposition at the line-by-line level, where they can’t figure out what variables they need and how they should change. This was the main conceptual obstacle that students faced with loops, and I tackled that through worked examples and a much more scaffolded assignment. Students got it by the end of the semester, although next time I will just start with the scaffolded examples.

Going back to the larger scale functional decomposition, some of that is part of the functional abstraction I’ve already talked about. Students did pick up on the fact that they would have trouble breaking down an intermediate into the functions that I did. Looking back, however, the benefits to those functions in the assignment was mostly conceptual. Each function is only used once in the execution of the program, so putting everything in one block would not have drastically increased code size. I’m not saying that reducing cognitive load is pointless, but I’m wondering if decomposition is best motivated if the code would otherwise be inefficient.

Space-Time Tradeoff

I don’t think I talked about the space-time tradeoff specifically, other than a throwaway mention when I answered a student’s question. Tradeoffs in general, however, I talked about a decent amount, most prominently in my representation lab. I am not sure what CS1 students need to take away regarding tradeoffs – should they only know that they exist? Maybe the goal here should be more specifically that students should be deliberate in selecting the compromise, which implicitly includes identifying tradeoffs. This general lesson is not one that I worked into my course at all, although I can now see places where it could be inserted. (For example, selecting between two lists, a dictionary, or a list of instances is a tradeoff in data representation.)

Connections with Other Disciplines

At first I thought interdisciplinary connections is something I emphasized, but look back now there are not a lot of places where I make these connections explicit. Students did spend two or three weeks on graphics as they learned object-oriented programming, but I would hesitate to call it art (although I would make an exception for some pieces). I hinted at some cognitive science/HCI/design work for the webapp as well, but I didn’t force students to go through the design process.

I was originally going to do a lab on music visualization, which would have included a primer in the physics of sound, but I got swamped and canceled the assignment. For next semester, I’m currently planning a cellular automata/agent-based modeling assignment, in the spirit of the Parable of the Polygons (aka. the Schelling serration model), which could pull in sociological and economic concepts.

Connections with the Non-Technological Real World

I mean, more than usual. When I wrote this goal, I was thinking of identifying how computer science concepts are embodied in existing infrastructure, like how library book stacks are an unrolled linked list. These examples are harder to find for CS1, and I don’t think I included any this semester. I will have to work harder to find cases where students can appreciate computation as a perspective and not just as a tool.

Step 53: Review CS1 Goals (Part 3)