How do you measure the success of a computer science program?
I am this because I have been thinking about data collection, about what longitudinal statistics I would need to measure the health and growth of a department. Is surprised to learn several months ago that Michigan does not have comprehensive records of their students. They may have the number of students in computer science but year, but that’s already the lowest granularity. There has been talk of starting a program to track what courses students take and why they continue or not continue, which I think I’d necessary to any more sophisticated diagnosis and improvement.
In the introductory class I lectured for this past semester, for example, we not only kept track of students’ anticipated next CS course, but also their attitudes about computer science. For example, here are some questions that we asked them at both the beginning and the end of the semester:
- I believe computer science can make a positive impact on the world.
- I believe computer science will help people.
- After graduation, there are equal opportunities for a career in computer science for males and females alike.
- I believe that knowledge about computer science will be more important in the future than it is now.
- I believe that having a career in computer science is as compatible as any other career with having a quality family life.
- My opinion of computer science is representative of those of my gender.
- I find computer science intimidating.
- I can see myself in a computing-related career in the future.
- I can see myself as a computer scientist in the future.
- Someone who takes future CS classes (e.g. 280) will be a coder for the rest of his or her life.
- I believe that other students in computer science will be welcoming to me.
I think these are good questions, especially if they are asked every semester. I wonder, though, if there are questions which would allow comparison between colleges – I don’t know of any effort to have something standardized questionnaire that would allow that kind of analysis. Short of that, perhaps the questions from national surveys could be adopted, which would allow comparison with national trends.
Other obvious data to keep include course evaluations, the jobs that graduates get (perhaps even over multiple years after graduation), their course trajectory and grades, etc. Fit the last one, I wonder if it’s possible to build some kind of model that uses future course grades to induce the quality of a previous course.
But there are also other statistics to keep, which do not have to do with the student population. One metric if diversity that occurred to me, for example, is the race and gender of invited speakers. While not as direct as measuring the diversity of the students, it is seems like a plausible symptom of a department that is not thinking sufficiently about diversity, much like looking at the demographics of professors.
Along similar lines, a non-student statistic to track is the type of assignment given in class. Is the assignment of interest only to computer scientists, or does it involve some real world use case? Is it text based or is it a media computing project, or is it purely algorithmic with no output? It would be interesting to then correlate these projects to student performance or opinion, to get insight on what projects excite students.