Step 21: Use Readings Differently

One thing I’ve been discovering in the last couple weeks, co-teaching Intro to Cognitive Science with a philosopher and cognitive scientist, is that we use the class readings differently, and expect students to get different things out of them.

  • Philosophy readings tend to be original texts from the philosophers themselves – Rene Descartes, David Marr, Daniel Dennett, and so on. The focus is for students to understand exactly what the authors are saying, the new ideas that they are trying to get across. So students need at least some historical perspective – one might say this is true of most philosophy – then in class we use it to frame other topics of the course in philosophical terms.
  • Cognitive science readings tend to be “second-hand”, in the sense that while they may be written by the original researchers, we tend not to read research papers, but more layman’s summaries that cover multiple studies and experiments – Oliver Sacks being a good example. For students, we want them to understand not just the phenomena that is being studied, but also the experiments that allowed researchers to isolate causes, which they must later use to justify their explanations.
  • My own readings vary between original research papers and interactive explanations of computer science concepts. Since students in the course are not programmers, the readings tend to be older papers, and even then ones that are closer to surveys than to technical advances. My desire for the students to is skim over the details while getting the gist of what is going on, to gain just a little understanding of why computers and AI is important in cognitive science.

It would never have occurred to me before this class, but these differences make sense given their respective disciplines. That is, the way we choose readings is based on how we were given readings when we were students, which in turn reflects the tradition of philosophy, cognitive science, and computer science respectively. Philosophy is (at least partially) about schools of thought, and so it makes sense to study the thinker and the context in which that thinker lived. Cognitive science, as a semi-physical science, is about creating theories that describe the world, and so it’s necessary to know the experiments that separate the good hypotheses from the bad. And finally, since (for me) computer science is about building and doing things, I choose papers that demonstrate new representations and algorithms, but where the details may not matter as much.

And I love this. About three years ago, I started a personal project to understand what constitutes progress in various academic disciplines. As part of that project, every time I met a graduate student, I asked them what counted as “evidence” in their field and how contradictory theories are resolved. It’s fascinating to learn what counts as progress in philosophy – something that is often still debated by philosophers. (A philosophy friend suggested that although philosophical theories for, say, what constitutes “knowledge” doesn’t have direct supporting experimental evidence, we can still prefer theories that have fewer “exceptions” – much as we use Ockham’s Razor to choose between scientific theories.) One of the more interesting things I learned from the project is that computer science and statistics are unique in that we generate our own data to test our methods – think recovering the parameters of a distribution.

I wonder if it would be useful for students to understand these differences between disciplines, and if so, what would be the best way to guide them towards discovering it.

Step 21: Use Readings Differently

Step 20: Define Curriculum User Stories

Short post today. We (a group of faculty at Oxy who advises on the development of the computer science program) have started thinking about what the ideal computer science curriculum should be. I am still learning about the material that is taught in the current computer science courses here, but have also started thinking about the (kinds of students) that we want to attract.

One of the more interesting things I have been learning is where computer science is being used in other disciplines. I’ve always been interested interested in the intersections of computer science with other things, almost more than what might be “pure” computer science, and this is something I want to encourage in students and enable them to understand as well. Part of the discussion that is going on right now is about where faculty want computer science in their own courses. I am aware of faculty who do computational research, or who teach classes in computational methods, but was surprised by faculty who wants to use programming as optional material to engage advanced students. Other faculty wants students to not just understand how to create computer models, but how to evaluate whether other models are valid – something which I wouldn’t have thought of in a computer science course, but definitely something that I think students should learn as well.

Beyond this, however, I was thinking about the different kinds of students that would take a computer science course. I’m not talking about their skill level when they come in, which is often discussed in curriculum design, but what students want to do what they learn. Here’s a list of different types of students I can think of:

  • students who want to go to computer science research
  • students interested in the tech industry
  • students interested in programming for other disciplines
  • students interested in programming for fun
  • students taking the course out of boredom

Making this list reminded me of user stories in software development. It’s the practice of writing short vignettes about fictional users, with enough details to explain what they want to accomplish and how they would do so using the developed product. I feel this must have been used by departments to plan their curriculum, but I also have not heard of classes being discussed in this manner. Of course, this must be integrated with what faculty think are the required fundamentals. I will be thinking about both of these in the coming week, and see if insights pop up.

Step 20: Define Curriculum User Stories

Step 19: Join a Co-Teaching Team

The other class I’m teaching this semester is introduction to cognitive science (hereby referred to as CogSci 101), which I co-teach with two other professors. This is the second time I’ve joined a co-teaching team – I was also one of three instructors this past string, teaching an introductory computer science course (hereby referred to as EECS 183). Co-teaching is a fairly superficial way of clustering courses, but the teams are organized differently enough to warrant some thought. It’s obvious in retrospect that organization of the team depends on the structure of the course, but somehow I’ve never thought about how one influences the other.

To start, the two courses could not be more different. EECS 183, being an introductory course at a large state university, had close to 700 students, who were split into four parallel lectures of hundred-plus students each. In contrast, CogSci 101 has maybe 50 students, who all fit into one medium-sized lecture hall.

These enrollment numbers themselves dictate the organization of the co-teaching teams. With 700 students in parallel lectures, each instructor covers the same material, and must therefore coordinate their presentations; for EECS 183, this was done by rotating who prepares the slides, which then go through edits by the other instructors. The University of Michigan, and the computer science department in particular, is fond of using undergraduates as teaching assistants (TAs), and for EECS 183 we had over twenty of them, who not only held office hours (in addition to the ones the instructors held, of course), but also helped prepare upcoming assignments. Aside from the infeasibility for only three instructors to deal with 700 students, the class size also meant that projects need to be thoroughly proofread and tested before dissemination. It is not an exaggeration to say that without the TAs the course would have fallen apart.

CogSci 101, on the other hand, doesn’t face any of these challenges. The class is co-taught not because there are too many students, but because we each bring a different perspective on cognitive science. (I enjoy this tremendously; it’s fun to dig into parts of cognitive science and philosophy that I haven’t touched in a while.) This also means that we co-teach in serial instead of in parallel, with each class covering a different topic. It’s the fourth week of the semester, but I have only given one lecture – although, to be fair, I will have given two more by the end of the week.

One major difference that initially surprised me is the lack of any regular course planning meeting. Compare this to the weekly meetings in EECS 183, where all the instructors and TAs would talk about what is happening in the class and what needs to be done in the future. At first I thought it was a consequence of me no longer being a graduate student, but even the full-time lecturers for EECS 183 had to attend the meetings. I now understand why this difference exists – with 700 students this kind of check-in is crucial, especially since different instructors must teach the same material (including the TAs during discussion). For 50 students, with each instructor teaching something different, the only coordination is at the level of how one topic flows into another – which was all decided in a single meeting at the beginning of the semester. I have no doubt that when homeworks and exams come around we’ll have more meetings, but this level of coordination is sufficient for just teaching.

One problem that I do have with the current arrangement with CogSci 101 is that I have yet to know any of the students really well. Only teaching one in three classes makes it difficult to build up rapport, and because the course does not yet have any work due, no students have come to office hours yet either. I don’t think the distribution of students coming to office hours is different from the beginning of EECS 183, but with 700 students I at least got to know some of them.

Which brings me to my last point. A class of 50 students is in the awkward range where you can’t interact with every student every class, but also can’t justify students buying and using remote clickers (which we did in EECS 183). The only work we get from students on a weekly basis are what we call “thought questions”, which are usually about applying a theory to some real world phenomena or to a case study. This is in line with the course goal of introducing cognitive science ideas, as opposed to having students memorize the specific results of specific experiments (or researchers).

What I realized is that this is almost never the case in computer science – there is usually some technical component that students have to learn, which can be adapted for in-class concept questions. It struck me that most of my pedagogy education is in STEM (and especially computer science), and that I’m not sure what low-stake, formative assessment techniques are used in the social sciences or the humanities. I think the problem that the subject is less stringent on the correct answer, which makes it harder to show mastery in a short period of time. But there must be some way of finding out whether students are on the right track, beyond having them self-report what they understand and what they still find confusing.

Do people have any ideas?

Step 19: Join a Co-Teaching Team

Step 18: Teach Your Home Subject Outside Your Home Department

It’s a little dangerous writing about a class that I’m currently teaching, but this has been weighing on me and writing is usually cathartic. What I want to talk about is some of the challenges and frustrations I’ve encountered with my Topics in AI course, and both how I have reacted to it, as well as some remaining uncertainties. As background the course is called Topics in Artificial Intelligence, is is being offered in the cognitive science department. It is listed by the registrar as requiring either “CogSci 101” or “CompSci 201”, in quotes because that’s not the real course numbers but are representative of the department and the level.

I think the fundamental problem that I have faced is one of different expectations between my and my students, not only on the material that will be covered in the course, but also the way in which skills and abilities needed to succeed. There are many reasons for this, but I think the biggest one is simply the change in context for me: going from the computer science department of a large research university to the cognitive science department of a small liberal arts college. I simply didn’t understand the students’ interests and motivations, and in doing so had integrated assumptions into the design of my course that, in retrospect, was counter to what I wanted to achieve.

Before I go any further, I want to emphasize that I’m not blaming anyone for this. Part of me wants to even excuse myself, and suggest that perhaps I couldn’t have understood it even if I had tried, since there is no way to communicate what I now know to who I was even a month ago. But I would be damned if I didn’t try, and for other people out there who are going into departments other than their original, I hope that something can be learned from this.

Let me start with what I had planned. A lot of this was written into previous posts, but I think I need to make explicit the assumptions I made in those posts. For one, my planning revolved around students being interested in AI as a subject, and while it’s connections to cognitive science are also worth knowing, it was secondary to understanding what computers are doing when we make them smart. As a result, my syllabus is organized such that students first learn the big idea of a technique, then some details of how it works, before going back out to applications and the challenges that face the researchers and consumers.

This assumption is about as far from the truth as I can get. The majority of students in the class come from a cognitive science background, and am much more interested in how artificial intelligence can shed light on human cognition than they are in how computers can be smart in general. This means that my sequence of presentation is backwards – I should have started with the computational aspects of human cognition, then presented their AI counterparts, then address how they are inadequate. By starting with the algorithm – even when I explained how people solve the same general problem every day – I inadvertently alienated students who came in seeking the connection to things they learned in their cognitive science courses.

I think there’s a more general lesson here regarding teaching your home subject in a course from a different department, which faculty in a new department would likely face. The question is: to what extent should the course focus on your home subject as opposed to the subject of the hosting department? This is not just a question about balancing the amount of content in each, but also how topics should be introduced (as suggested above), the depth to which they should be studied, and whether what you want students to learn is the same as what the students actually want to learn. It’s all too easy to bring in preconceptions from your home subject, and do it in a way that you are not even aware the students could see things differently. I am not even talking about the “AI for AI’s sake” thing here, but things like what you expect prerequisites to cover.

Which brings me to the next point: it is really, really hard to know what students are and are not capable of. One lesson here for me is that even talking to other faculty may mot give an accurate picture of student’s skills. One reason for this is that other faculty may have a different competence scale regarding your subject – by definition, since you are starting a new department, the student skills levels are necessarily based on students who do not have extensive experience in your subject. But the other reason that assessment of students from other faculty may not help is that they may have in mind a particular way of teaching, one that they know students will find approachable. So while they understand how having students of a particular level translates to differences in the classroom, you as the new faculty do not. This difference in common ground is obvious in retrospect, but I wish I had realized it sooner.

There is one more challenge that I have yet figure out how to meet, and that’s how to take advantage of students who do have a technical background, without simultaneously confusing students who don’t have it. In principle it should be as simple as “if you know about X, then this is like…”, but somehow that felt insufficient. I did try it, but it almost felt it had a negative effect. I worry that the resulting explanation didn’t make sense to those students, and that it has led to them doubting their own ability. It’s likely that this is an unfounded fear, and that as long as I make sure that even students who don’t know X can get the concept, it doesn’t matter what students knew before.

So what have I done in response to these challenges?

The biggest change I have made is to almost entirely change the assignments. My original syllabus was heavily focused on implementation of algorithms – that is, on programming. I was lucky to find out, three hours before my first class, that only about a third of the class has programming experience. I admit I panicked, but ultimately figured out how I could get the same ideas across with a minimal amount of programming experience. I ended using IPython as the platform on which students can run code, and be able to do small-scale explorations of algorithms by only changing the numbers. Luckily, I designed the course to be driven by applications, it was sufficient to have students specify how to represent a problem for the algorithm, and the likely difficulties that someone doing so would encounter.

On a larger scale, I am reconsidering the order in which I present present material. Because the course consists of four mostly-unrelated topics, the lessons I learn from the first topic can be applied for the remaining topics without much difficulty. Within the first topic, I am also moving the discussion of application earlier, and will use that to as the backdrop for the discussion on challenges. I do think the course is better for it, and I hope that it’s not too little too late.

Finally, I am also addressing the expectations of students. My original syllabus and the belatedness of the realizations I’m having now meant that I went with the algorithmic details for the first two lectures. The technicalities are hard for students, and I’ve been trying to make sure students know it’s okay to only get the gist of the readings (and to have them cleared up in class), as well as how much I want them to come to office hours. I did emphasize these things during the first class as well, but I don’t think I sufficiently conveyed how challenging they might find the material.

I will write another post in a couple of weeks on how this turns out.

Step 18: Teach Your Home Subject Outside Your Home Department

Step 17: Gather Potential Resources for Contact

Oxy’s first faculty meeting of the year occurred last Monday. It was the first time I’ve been to a faculty meeting, and it was interesting to see the things that go on, especially since these were college level meetings, that I assume rarely or never happen at larger schools like Michigan. As a result, the issues that were announced/discussed were college-wide issues. What I noticed during the meeting – and really, during the new faculty orientation as well – is that I am starting to care about things I didn’t before.

Take, for example, issue of diversity. Oxy is currently looking for a Chief Diversity Officer, and my first thought on hearing that was whether the position would also be available as a resource for helping individual programs and departments address diversity issues. I was curious enough about the position and the hiring process that I went to the town hall meeting later in the week. While diversity in computer science is something I paid attention to before – it’s presence in my head has grown in the last several years – I don’t think I would have taken quite so active an interest even a year ago. Similarly, at the new-faculty orientation I learned about the office responsible for keeping track of student demographics, and I actually called them to see if I could use them for data about students in my own classes instead of collecting it myself. We ended up having a short conversation about FERPA concerns (I’m still confused about the extent to which FERPA applies to faculty of the school), which is definitely something I now have to keep in mind.

A similar thing happened when the various student resources are mentioned on campus, including the Career Development Center that is currently under renovation, as well as the Academic Excellence Center (AEC) that offers peer tutoring for a wide variety of subjects with the help of faculty. My immediate thought on hearing about these resources is whether computer science is represented (unlikely, given the scant course offerings), and whether I can help them develop resources for people interested in or working on computer science.

Then there are entirely student-run organizations, like Oxy Open Source and Oxy Engineering Club, that I would at least like to check out and see what they do. I almost feel like a freshman going to college.

I think the point here is that there are a lot of places to which a computer science department can reach out and build a relationship with. I know the AEC has peer tutors for a number of subjects, but I wonder about the degree to which other departments are involved in selecting those people – I assume that is at least some involvement. On the other hand, I would be mildly surprised if faculty work as closely with the job search process, especially for students not looking to stay in academia or research. I do think computer science is different in that academia has closer ties to both big companies and the industry in general (eg. startups), and so we may have a bigger role to play there.

Step 17: Gather Potential Resources for Contact