Thursday, May 10, 2012

About UC Berkeley CS169 “Software Engineering”

Since I find myself periodically explaining our “reinvented” CS169 to my colleagues, and since ourSaaS MOOC is based on it, I thought I’d write up this short description.  (The official link to the Berkeley course homepage is here, but it changes each semester depending who is teaching it.)

Background. CS 169 is Berkeley’s upper-division (seniors and some juniors) software engineering course.  The way it’s taught varies widely depending on the instructor. This post describes how I teach it, often with the help of Dave Patterson and recently also Koushik Sen.  It’s not a required class in the major, but rather one of several classes that satisfies specific requirements such as a design project, technical communication, etc.

Prerequisites. This is the only undergrad course at Berkeley that claims to address the topic of Software Engineering.  As such, it’s ambitious and fast-paced, with 5 1-week programming assignments, 5 quizzes, and a significant team project with an external customer, all in a single 14-week semester.  Students should be comfortable with at least 1 other language and with basic programming concepts such as object orientation, classes and inheritance, recursion, and higher-order functions.  Prior to this course, Berkeley CS students take CS61A Structure & Interpretation of Computer Programs, which introduces the four major programming paradigms (until recently using Abelson & Sussman’s awesome “wizard book“,now using Python); CS61B Data Structures using Java; and (usually) CS61C Great Ideas in Computer Architecture (aka Machine Structures), using C, MIPS assembly, and more Java (for a Hadoop assignment).

History. While working on the RAD Lab project (2006-2011), we needed SaaS apps to show off our machine-learning-based technology for automating various aspects of cloud operations.  Since no Berkeley course taught SaaS, we created an informal seminar course in 2007 to bootstrap a cohort of undergraduates to create showcase apps using Rails.  It was so popular that we offered it again, increasing the focus on TDD and good software practices, when our colleague Paul Hilfinger observed that we were well on the way to teaching the basics of Software Engineering in a format that students were enthusiastic about, so why not go all-out and teach CS 169 this way?  We agreed, and we did a “dry run” of the beefed-up course in Fall 2009, debugged it, and offered the “SaaS version” of CS 169 for the first time in Fall 2010.  Enrollments have been increasing by nearly 50% per offering.

CS 98/198 Spring ‘0725
CS 194 Fall ‘0835
CS 194 Fall ‘0950
CS 169 Fall ‘1075
CS 169 Spring ‘12115
CS 169 Fall ‘12180

Practices taught and course format. The course teaches software engineering techniques based on the Software Engineering Body of Knowledge (SWEBOK) using SaaS+Agile+cloud as the vehicle and Rails as the framework.  A partial list of what we cover includes test-driven development, behavior-driven/user-centric design, design patterns, legacy code and refactoring, deployment (including “SaaS Performance & Security 101”), and working effectively as part of a small team (using version control with branches, estimating progress toward customer-driven goals, work planning, etc.)  Our recent  explains why we believe these choices bridge the gap between what academics believe should be covered in software engineering courses and what industry wants to see in graduates of those programs.  (Contrary to what one might expect, leading software companies donot want us to become trade schools teaching specific tools, languages or frameworks; they want skills that transcend these, including dealing with legacy code and working in a team serving a nontechnical customer.)  We chose Rails because it has the best testing and code-grooming tools and its developer community places high value on beautiful code and thorough testing.  There are two lectures and one small-section meeting (~30-40 students) per week, weekly programming assignments, bi-weekly short-answer quizzes, no “big” midterm or final exam, and a 6 to 8 week course project.  We are experimenting with pair programming as well.

Course project. We work with on-campus organizations including The Berkeley Group to identify external customers—some nonprofits, some on-campus units, some others—whose needs could be addressed in part by a SaaS prototype.  ”Two-pizza” teams of 4-6 students bid on the projects they’re most interested in and we match them up.  During each of four 2-week iterations, students meet with their customer, use lo-fi mockups and user stories to agree on goals, use BDD and TDD to develop new functionality and tests, and deploy to the public cloud on Heroku.  Per-iteration design/progress reviews with course staff (TA’s) help identify problems and provide technical guidance where needed; we have found no substitute for this critical part of the software craftsmanship apprentice process.  (In Spring 2012, two full-time TA’s monitored 25 teams; we hope to improve this ratio in Fall 2012.)  Each team’s progress is publicly tracked and estimated (and visible to customer and course staff) using the free Pivotal Tracker throughout, and grading is based heavily on (a) demonstrated responsiveness to customer feedback on deployed functionality, (b) demonstrated improvement in ability to estimate how much work will be completed by end of iteration, (c) sound use of agile processes as demonstrated by BDD scenarios (which Cucumber turns into integration tests), good test coverage, and reasonable complexity and beauty metrics (cohesion, lack of code smells, etc.) on code, which is publicly accessible on GitHub for review by course staff at any time.  At the end of the course, students present their work in a poster/demo session attended by course staff, the external customers, and invited guests such as industry practitioners and VCs.  Many students reported that their customers were so delighted that they were trying to hire the students to continue the work over the summer.  Two projects from Spring’12 were already deployed in production with real users by the time the poster was presented.

We’ve started gathering screencasts and customer interviews highlighting representative projects; more are being added all the time.

(Coming soon: aggregate code statistics for Spring 2012 projects, including test coverage & code cleanliness metrics)

Scalable grading. Given the growth in popularity of the course and CS courses in general here, we had already been thinking of ways to scale the grading by repurposing testing and code grooming tools such as RSpec, Cucumber, Mechanize, reek, flog/flay, and Webdriver (Selenium) both to assess correctness of student code at a fine grain and give nontrivial feedback on code quality.  When we agreed to offer the first 5 weeks of the course as a massive open online course in Feb/Mar 2012, it forced the issue and made us sit down and write the autograders.  Of course, these are no substitute for actual interaction with an instructor; indeed, the autograders have freed up our teaching staff to focus on creating additional review material and holding design meetings.  (If you’re an instructor interested in participating in our in-classroom beta program, we’ll even run the autograders for you.)

Teaching assistant duties & prereqs. (Thanks to head TA Richard Xia for this info.)  The course is approximately split into two halves, with homeworks and quizzes dominating the first half and the project dominating the second half. During the first half, each TA runs a discussion sections of ~30 students (1-2 hours/week + 2-4 hours to prep and review material), holds office hours (2 hours/week), monitors the online question forums on Piazza (4-6 hours), and miscellaneous tasks such as individual emails and handling regrade requests (4 hours).  In addition, for the first offering of the course the content creation included not only homeworks, quizzes, and section material, but the grading rubrics for the autograders for each type of evaluation.   For the second half of the course, we converted most of the discussion sections into project meetings with the students in which we met with each group for about 10 minutes each week, so less time was spent preparing homework/quiz material and the section-prep time was replaced by evaluating project checkpoints.  A few additional hours per week were spent managing the online course, but as we fine-tune the material and autograder logistics, we expect that the online course can be managed by a single 10-hour-per-week TA, leaving the CS169 TAs free for for direct interactions with the students, especially during the project.

Book. Modern software touches many subsystems of different types, each of which has historically been the focus of some CS subspecialty.  For example, SaaS encompasses datacenter computing, databases, OO programming, network security and performance, and user-centric design, plus nontechnical topics such as how to work with nontechnical customers and deliver a user-centric design. While there are many great (and not-so-great) books and online resources on each of these topics, a reading list cobbled together from them is impractical, lacks a “through-narrative”, and is very hard to get students to take seriously.  We finally decided in early 2011 to create our own book that would introduce enough of each topic to function as a SaaS engineer and weave them together in a way that both made sense for a one-semester (or shorter) course. Our division of topics into largely-standalone subgroups allows instructors with less time or a less-sophisticated audience to select subsets of material appropriate for their needs.  We decided very early to self-publish to keep the price low (currently $10 ebook/$20 print book).   is now in its beta draft, and we expect the First Edition to be ready by Spring 2013.

No comments:

Post a Comment

Comments are disabled because the only commenters are spammers, despite Google's best efforts. But I welcome actual comments: Google my name and you can easily direct an email to me, and I'll publish your comment here.

Note: Only a member of this blog may post a comment.