Armando Fox » Teaching

Meeting some MOOC students in Paris

fox — Sun, 05 May 2013 22:06:58 +0000

Last week I traveled to Paris for CHI 2013 to speak on a panel about online education.

I’m getting to do that a lot these days, and while the travel can be tiring, it’s certainly more fun when I get the chance to meet up with MOOC students face to face!

In a previous post I described my experience meeting up with two students in Singapore and Indonesia. This time around I got to meet Julien Coupez (left) and Daniel Kindler (back, second from left) the evening I arrived in Paris. We had a round of drinks near the Place de Clichy with my colleague Prof. Marti Hearst (right), an accomplished researcher in human-computer interaction and information sciences at Berkeley with whom I’m co-advising graduate research on learning in MOOCs.

It was fun to hear about how Julien’s “social bookmarking” startup NexBoo already uses some of the Agile techniques covered in CS 169.1x, while Daniel wanted to learn about software practices that are not widely used by his current employer (a large financial services company).

As always, connecting with colleagues around the world is a way of reminding myself that at some level we’re all playing on the same team. Thanks to Daniel and Julien for making the time to get together, and for making this visit to one of my favorite cities even more enjoyable!

À bientôt, folks!

The traveling MOOC professor

fox — Sun, 10 Mar 2013 17:29:30 +0000

I just returned from a short but much-needed vacation to Thailand and Indonesia, with a brief stopover in Singapore. (My wife and I are scuba divers, so we were diving in the Raja Ampat area of Indonesia, which was spectacular; and since we were going to be on the other side of the world, we decided to also check out Chiang Mai, since we’d never been to Thailand, and Singapore, since everyone has said it’s a great place to visit.)

Since I periodically receive nice emails from students in our BerkeleyX MOOC on Software-as-a-Service, in a fit of inspiration while traveling, I wondered if any of those students might live in Jakarta or Singapore, two unfamiliar cities where we had stopovers. With EdX’s help, I sent an email to the MOOC students, and I was pleased to receive numerous replies!

In Singapore, I met Tony Luong (left) and Hung Mai (right), Vietnamese nationals currently living and working for IBM in Singapore. We had a nice sushi dinner with them (at Sushi Tei in the City Square shopping center near Little India, if you know Singapore), during which we talked about the differences between living and working in IT in the USA vs. Asia, professional plans for the future (Hung is coming to the US to work soon!), software engineering, and life.

In Jakarta, I was greeted at the airport by Reza Anwar and his wife Tanti Ruwani. Both are Indonesian nationals who had had part of their higher education in the West, so they spoke perfect English and were lovely hosts, giving us a brief driving tour of downtown Jakarta and a wonderful traditional Indonesian dinner at a very nice restaurant housed in a former Dutch-colonial mansion. Trading stories about life, business, and career aspirations with students from other countries was inspiring.

I’m keenly aware of my role as an ambassador not only of Berkeley, and not only of the USA, but of higher education in general when doing visits like this. I was very fortunate to get in touch with students who had enjoyed taking my course (and had very kind things to say about it) and I’m now thinking I’ll try to find students to connect with whenever I travel abroad! It’s one more way to realize that despite whatever other differences people may have, at some level, we are all indeed in this together.

Thanks to Tony, Hung, Reza, and Tanti for turning what could have been unremarkable stopovers into a rewarding social and professional experience!

What was it like to teach a MOOC?

fox — Tue, 15 Jan 2013 01:22:40 +0000

Since I get asked this question a lot, including by the media, I thought it’d be useful to blog it.

What MOOCs have I taught?

I split our 15-week on-campus software engineering course into two pieces, an “introduction” and “advanced” MOOC. The Intro MOOC was offered three times on Coursera and is now in its second offering on EdX, with whom UC Berkeley has an institutional partnership. The Advanced MOOC has been offered just once so far on EdX. (The course number is CS 169.1x for the Intro and Cs 169.2x for Advanced.)

What was it like to adapt the campus course?

It has been a lot of work, but it has greatly improved the on-campus course as well as making it available worldwide.

Because of high enrollment demand in the on-campus course, we had already started thinking about automatic grading of programming assignments. We and our on-campus TAs spent hundreds of engineer-hours creating sophisticated automatic graders for programming assignments since we believed neither MOOC nor on-campus students would be well served by a software engineering course whose assessments were based solely on multiple choice or short-answer questions.

While this was grueling, the autograders are reusable and customizable, and have graded hundreds of thousands of assignments so far, which would require an army of TAs to do manually. Thus it helped us expand access to the course on-campus as well as online.

I also reorganized my 90-minute lectures into 8-12 minute “lecturelets”, each covering a specific topic and accompanied by self-check questions. Our colleagues, the founders of Coursera, suggested this format worked well for MOOCs, but we found that it also made the on-campus lectures livelier and better-attended.

We live-captured our lectures and postprocessed them afterward. In general, we were unwilling to spend time on the MOOC that would not also improve the on-campus course. My teaching ratings and the on-campus course’s ratings (given anonymously by the students) both went up when the MOOC technology was integrated.

What worked well?

The automatic grading allowed vastly more students to practice the material, and modulo some glitches, was widely praised.

The online students took the same quizzes and did the same programming assignments as the Berkeley students, on comparable deadlines, and the best of them did just as well.

Intelligence is uniformly distributed worldwide: students from 130 countries enrolled, and thanks to lecture subtitling provided by EdX, were able to follow the lectures despite not being native English speakers.

More important than the number of students reached was the smaller number (hundreds) of truly motivated students to whom we were able to give an opportunity otherwise unavailable to them, such as a student in the Gaza Strip who only received 6 hours of electricity each day and used part of it to complete our course.

What challenges did you encounter?

This is an enormous amount of work. We did it on our own time and without extra compensation, but not all instructors will have that luxury. We also had discretionary money we used to pay TAs to help support the online course; there will have to be more systematic sources for those funds eventually.

Online students don’t get face-to-face time with instructors and TAs, as on-campus students do. The discussion forums work pretty well, but despite being monitored by TAs, they’re basically self-service.

The on-campus students get to do team projects in small groups working with external customers. In their evaluations they indicated this was the most valuable aspect of the course. Yet the online students don’t get to do this part, as we haven’t figured out how to scale up the process of forming project teams and matching them up with external customers.

Discussion-oriented learning doesn’t work well at large scale. Some parts of our course dont’ require it, but for the parts that do, such as office hours and TA design reviews of team projects, there was no way to give the online students this experience.

Nonetheless, our goal was not to replicate every aspect of the on-campus course, but rather to identify which aspects, if converted to MOOC format, would work well for both online and on-campus students and focus on making those things great.

There is lots of cheating in MOOCs. We have hard as well as anecdotal evidence from our courses and colleagues’ courses. But since no credit is offered, we’ve chosen to focus on improving the course for the vast majority of honest students rather than try to catch the cheaters.

A very small fraction of MOOC students are vocal jerks or have a disproportionate sense of entitlement about everything being free.

Would you do it again?

Absolutely, I’m planning to. And as I develop new on-campus courses, I’ll be blending in MOOC technology, such as shorter lectures with self-check questions and extensive autograding. That allows the TAs and me to shift our scarce on-campus student contact time from a lower-value activity (repeating the same lectures) to a higher-value activity (face time with students who’ve watched lectures and/or done the prep work using MOOC resources).

Viewpoint: MOOCs can strengthen academia

fox — Wed, 28 Nov 2012 22:06:48 +0000

Now that I’m the Academic Director for online learning at Berkeley, I figured it’s OK to raise my voice a little about what I think is good about MOOCs. The following was submitted as a “Viewpoint” (op-ed) to Communications of the ACM, in response to editor-in-chief Moshe Vardi’s article on “Will MOOCs Destroy Academia?” in the November 2012 issue.

Viewpoint: MOOC Mythbusting

[Draft version. Submitted as an opinion piece to Communications of the ACM, cacm.acm.org. Version: 1/30/13 4:42 PM]

As the media’s infatuation with MOOCs continues unabated, some academics seem to be succumbing to the hand-wringing about whether MOOCs will destroy higher education as we know it (Will MOOCs Destroy Academia?, Moshe Vardi, CACM 55(11), Nov. 2012). Should we want to “put the MOOC genie back in the bottle,” as Dr. Vardi suggests? I argue that a close, systematic, and sustained look at how MOOCs are actually being used should persuade the careful observer that MOOCs can strengthen academia.

Note that I don’t say “MOOCs will strengthen academia”. They certainly can, but whether they do depends on how they are received and used by academics. Full disclosure: besides being a MOOC instructor myself, I’m the newly appointed co-director of Berkeley’s online education programs, which have recently been extended to include MOOCs. But I’m not cheering for MOOCs because I have this position; rather, I agreed to take the position because I’m excited about the possibilities of MOOCs and other online education.

To help my colleagues explore the potential benefits to our students and ourselves, let me offer counterexamples to some “MOOC myths” in recent media coverage. While most myths are based on a kernel of truth and may be true of at least some MOOCs, they are just as often untrue and it’s a disservice to interested readers to present them as foregone conclusions or faits accomplis.

Myth: Universities will use MOOCs to lower costs by firing faculty and TAs, thus sacrificing educational quality.

In a recent pilot program at San José State University in California, students in an analog circuits course used MIT-authored MOOC lectures and homework assignments created by Prof. Anant Agarwal . The students’ in-classroom time was spent working on lab and design problems with local faculty and TAs. The students in this SPOC (Small, Private Online Course) scored 5 percentage points higher on the first exam and 10 points on the second exam than the previous cohort that had used the traditional material. Even more strikingly, the proportion of students receiving credit for the course (“C” or better grade) increased from 59% to 91%. So educational quality arguably increased, and costs were lowered by helping students graduate more quickly, rather than by firing people. Productivity was enhanced because the on-campus instructors shifted their time from what they perceived as a lower-value activity—creating and delivering lectures on content that hasn’t changed much—to the higher-value activity of working directly with students on the material. Several of my colleagues in the California State University system and the community college system have expressed similar enthusiasm. And at the risk of alienating colleagues, if a particular instructor really offers no value at all over a digitally-delivered MOOC, it is worth asking some questions about the value proposition of the traditional course in that case.

Myth: MOOCs will fail because many aspects of traditional classes, such as small-group discussions and face-to-face time with instructors, do not work in the MOOC format.

This assertion is true, but it implicitly and incorrectly assumes that replicating the classroom experience is the proper goal for an online course. As educators, a better question for us to ask is this: What can be delivered effectively through this medium in a way that helps our on-campus students, and has the valuable side effect of helping the hundreds of thousands who won’t have the privilege of attending our universities in person? (Indeed, many of our MOOC students reported that our course was better than anything available at the brick-and-mortar campuses to which they had access.)

For example, rather than asking whether automatic graders (which, by the way, have been around since at least 1960 ) can replace individual instructor attention, we can ask: When can they relieve teaching staff of drudgery, allowing scarce instructor time to focus on higher-value interactions such as tutoring and design reviews? Rather than worrying whether MOOC-based social networking will replace face-to-face peer interactions, we can ask and experimentally answer: Under what conditions and with what types of material do online communities help foster learning, and how can social networking technology help foster both online and in-person community building? Rather than ominously predicting a “winner take all” effect in which one particular MOOC comes to dominate all instruction in a given subject, we can, like Prof. Doug Fisher and others , selectively adapt the content for use in our own on-campus courses, as we do with textbooks. And learning activities that don’t appear to be “MOOCable”—discussion-based learning, open-ended design projects, and so on—can just be omitted, as we’ve done in our software engineering course, whose MOOC version lacks the on-campus course’s open-ended design project. Indeed, at universities on the quarter system, it’s common to offer a two-quarter sequence in which the first quarter focuses on well-circumscribed assignments and the second quarter focuses on a design project, since a single quarter can’t cover both. The first course clearly has value despite lacking a design project. By analogy, MOOCs that don’t offer “the same” experience as a complete residential course also have value, and our job as educators is to make judgments about where that value lies and how to combine it with the other education modalities we offer our students.

Myth: MOOCs distract faculty who should be focusing on improving their on-campus pedagogy.

The very scale of MOOCs offers us new and unprecedented opportunities to improve our on-campus courses using techniques that just don’t work at smaller scales, and were previously reserved for ”high stakes” exams such as the GRE or SAT . Exploratory factor analysis lets us identify questions that test comparable concepts, giving instructors a way to vary exam content. Item response theory allows us to discover which questions are more difficult (in the statistical sense that higher-performing students are more likely to get them right). A/B testing gives us a controlled way to evaluate which approaches have better effects on learning outcomes, just as high-volume e-commerce sites evaluate which user experience results in more purchases. None of these techniques works on classroom-sized cohorts (say, 200 or fewer students), but we are applying all of them to our current MOOC. Indeed, not all instructors will be eager to receive the avalance of MOOC data telling us what’s not working in our courses and how we can improve them, but our sense at Berkeley is that MOOCs may well raise the bar for acceptable teaching on campus, as well as improve the recognition of good teaching, perhaps bringing the era recycled PowerPoint slides finally to a close.

In addition, in each of four offerings of our software engineering MOOC totalling over 100,000 enrollees, about 8%, or nearly 32,000 total, identified themselves as instructors, suggesting that MOOCs may be even more effective than traditional textbooks at “teaching the teachers” and getting innovative new pedagogy out to a large audience. In fact, our faculty colleagues who are classroom-testing our unconventional new textbook Engineering Long-Lasting Software: An Agile Approach Using SaaS & Cloud Computing are all doing so in conjunction with our MOOC (EdX CS 169.1x), so that they can take advantage of the autograders, screencasts and other materials.

Myth: Private capital will pursue MOOCs to make money at the expense of educational quality.

How much will students pay for certificates and proctored exams? How much will employers pay to recruit top MOOC students? The answers are still uncertain, yet it would hardly be the first time that venture capital was poured into enterprises before the business models were well understood. And nonprofits such as EdX provide an alternative for institutions uncomfortable with a deep partnership with a for-profit entity. An important decision for universities to make is how and with whom they want to structure a partnership around online education if they believe it is of long-term strategic importance.

Forunately, at the moment the uncertainty about revenue models doesn’t seem to bother those who have been driving MOOC creation: passionate faculty, most of whom are receiving little or nothing for the hundreds of hours of extra work required to create a MOOC. They are driven instead by the kind of recognition that comes from creating a textbook—another educational instrument with uncertain revenue generation potential.

Myth: MOOCs will “sterilize” courses by separating the course material from instructor and by promoting uniformity rather than diversity in course content.

Setting aside the fact that most of us who teach already use textbooks written by others, one could raise a similar complaint about the printing press. Books used to be laboriously and lovingly crafted by hand; the printing press made them cheap to manufacture but generic in appearance. Receiving a book used to be part of a social ritual that included interactions with learned persons; the printing press gave rise to bookshops where anyone could transact in books without such an interaction. Yet these objections don’t make the printing press a bad idea: unbundling the book’s content from its context benefited all of us by creating vastly more readers and by giving voices to authors who would never have had one. (And while the printing press certainly made it easier to get bad writing published, handcrafted books were also far from uniform in the quality of their content!)

In a similar way, MOOCs won’t replace high-quality face-to-face instruction, but we can reach many more learners, leading to a net social and economic benefit, and we can give great teachers a more prominent voice than they have had since Socrates.

Conclusion

MOOCs represent a new technology opportunity whose potential pedagogical impact needs to be researched. To be sure, many bad experiments will be tried—some are probably already underway—and many worthy experiments will fail or have a different outcome than desired. But if failed experiments were an obstacle to doing world-changing research, we academics would probably choose a different job.

[1] The Graduate Record Exam (GRE) and Scholastic Aptitude Test (SAT) are standardized tests that are part of most students’ applications to American graduate and undergraduate programs respectively.
[2] Lawley, D., Estimation of factor loadings by the method of maximum likelihood. Proc. Royal Soc. Edinburgh, 60A, 1940.
[3] Lord, F.M. Applications of item response theory to practical testing problems. Mahwah, NJ: Erlbaum, 1980.
[4] Davis, Barbara Gross. Tools for Teaching. Jossey-Bass Publishers, 2009.
[5] Hollingsworth, J. Automatic graders for programming classes. CACM 3(10), Oct. 1960

Thanks, Mr. Ambani, for thwarting free online education in India

fox — Thu, 24 May 2012 21:59:01 +0000

Given our efforts to provide free education in Software Engineering—an area where India is known to be a strong player—it was particularly disconcerting to read that numerous Web sites including Vimeo and Pastebin have been blocked in India at the ISP level due to a court order.

According to articles in the New York Times and the Wall Street Journal, it sounds like you should make your displeasure known to Reliance Entertainment, a media company largely controlled by media mogul Anil Ambani.

Other news sources report that some ISPs are still allowing Vimeo and Pastebin through, and that proxies and VPNs can be used to circumvent the blockage.

We don’t plan to move our content, and we’re confident the world’s largest democracy will respond appropriately to what amounts to censorship, as we said in our SaaS TV Chat this week.

About UC Berkeley CS169 “Software Engineering”

fox — Thu, 10 May 2012 18:20:06 +0000

Since I find myself periodically explaining our “reinvented” CS169 to my colleagues, and since our SaaS MOOC is based on it, I thought I’d write up this short description. (The official link to the Berkeley course homepage is here, but it changes each semester depending who is teaching it.)

Background. CS 169 is Berkeley’s upper-division (seniors and some juniors) software engineering course. The way it’s taught varies widely depending on the instructor. This post describes how I teach it, often with the help of Dave Patterson and recently also Koushik Sen. It’s not a required class in the major, but rather one of several classes that satisfies specific requirements such as a design project, technical communication, etc.

Prerequisites. This is the only undergrad course at Berkeley that claims to address the topic of Software Engineering. As such, it’s ambitious and fast-paced, with 5 1-week programming assignments, 5 quizzes, and a significant team project with an external customer, all in a single 14-week semester. Students should be comfortable with at least 1 other language and with basic programming concepts such as object orientation, classes and inheritance, recursion, and higher-order functions. Prior to this course, Berkeley CS students take CS61A Structure & Interpretation of Computer Programs, which introduces the four major programming paradigms (until recently using Abelson & Sussman’s awesome “wizard book“,now using Python); CS61B Data Structures using Java; and (usually) CS61C Great Ideas in Computer Architecture (aka Machine Structures), using C, MIPS assembly, and more Java (for a Hadoop assignment).

History. While working on the RAD Lab project (2006-2011), we needed SaaS apps to show off our machine-learning-based technology for automating various aspects of cloud operations. Since no Berkeley course taught SaaS, we created an informal seminar course in 2007 to bootstrap a cohort of undergraduates to create showcase apps using Rails. It was so popular that we offered it again, increasing the focus on TDD and good software practices, when our colleague Paul Hilfinger observed that we were well on the way to teaching the basics of Software Engineering in a format that students were enthusiastic about, so why not go all-out and teach CS 169 this way? We agreed, and we did a “dry run” of the beefed-up course in Fall 2009, debugged it, and offered the “SaaS version” of CS 169 for the first time in Fall 2010. Enrollments have been increasing by nearly 50% per offering.

CS 98/198 Spring ‘07	25
CS 194 Fall ‘08	35
CS 194 Fall ‘09	50
CS 169 Fall ‘10	75
CS 169 Spring ‘12	115
CS 169 Fall ‘12	180

Practices taught and course format. The course teaches software engineering techniques based on the Software Engineering Body of Knowledge (SWEBOK) using SaaS+Agile+cloud as the vehicle and Rails as the framework. A partial list of what we cover includes test-driven development, behavior-driven/user-centric design, design patterns, legacy code and refactoring, deployment (including “SaaS Performance & Security 101”), and working effectively as part of a small team (using version control with branches, estimating progress toward customer-driven goals, work planning, etc.) Our recent CACM article explains why we believe these choices bridge the gap between what academics believe should be covered in software engineering courses and what industry wants to see in graduates of those programs. (Contrary to what one might expect, leading software companies do not want us to become trade schools teaching specific tools, languages or frameworks; they want skills that transcend these, including dealing with legacy code and working in a team serving a nontechnical customer.) We chose Rails because it has the best testing and code-grooming tools and its developer community places high value on beautiful code and thorough testing. There are two lectures and one small-section meeting (~30-40 students) per week, weekly programming assignments, bi-weekly short-answer quizzes, no “big” midterm or final exam, and a 6 to 8 week course project. We are experimenting with pair programming as well.

Course project. We work with on-campus organizations including The Berkeley Group to identify external customers—some nonprofits, some on-campus units, some others—whose needs could be addressed in part by a SaaS prototype. ”Two-pizza” teams of 4-6 students bid on the projects they’re most interested in and we match them up. During each of four 2-week iterations, students meet with their customer, use lo-fi mockups and user stories to agree on goals, use BDD and TDD to develop new functionality and tests, and deploy to the public cloud on Heroku. Per-iteration design/progress reviews with course staff (TA’s) help identify problems and provide technical guidance where needed; we have found no substitute for this critical part of the software craftsmanship apprentice process. (In Spring 2012, two full-time TA’s monitored 25 teams; we hope to improve this ratio in Fall 2012.) Each team’s progress is publicly tracked and estimated (and visible to customer and course staff) using the free Pivotal Tracker throughout, and grading is based heavily on (a) demonstrated responsiveness to customer feedback on deployed functionality, (b) demonstrated improvement in ability to estimate how much work will be completed by end of iteration, (c) sound use of agile processes as demonstrated by BDD scenarios (which Cucumber turns into integration tests), good test coverage, and reasonable complexity and beauty metrics (cohesion, lack of code smells, etc.) on code, which is publicly accessible on GitHub for review by course staff at any time. At the end of the course, students present their work in a poster/demo session attended by course staff, the external customers, and invited guests such as industry practitioners and VCs. Many students reported that their customers were so delighted that they were trying to hire the students to continue the work over the summer. Two projects from Spring’12 were already deployed in production with real users by the time the poster was presented.

We’ve started gathering screencasts and customer interviews highlighting representative projects; more are being added all the time.

(Coming soon: aggregate code statistics for Spring 2012 projects, including test coverage & code cleanliness metrics)

Scalable grading. Given the growth in popularity of the course and CS courses in general here, we had already been thinking of ways to scale the grading by repurposing testing and code grooming tools such as RSpec, Cucumber, Mechanize, reek, flog/flay, and Webdriver (Selenium) both to assess correctness of student code at a fine grain and give nontrivial feedback on code quality. When we agreed to offer the first 5 weeks of the course as a massive open online course in Feb/Mar 2012, it forced the issue and made us sit down and write the autograders. Of course, these are no substitute for actual interaction with an instructor; indeed, the autograders have freed up our teaching staff to focus on creating additional review material and holding design meetings. (If you’re an instructor interested in participating in our in-classroom beta program, we’ll even run the autograders for you.)

Teaching assistant duties & prereqs. (Thanks to head TA Richard Xia for this info.) The course is approximately split into two halves, with homeworks and quizzes dominating the first half and the project dominating the second half. During the first half, each TA runs a discussion sections of ~30 students (1-2 hours/week + 2-4 hours to prep and review material), holds office hours (2 hours/week), monitors the online question forums on Piazza (4-6 hours), and miscellaneous tasks such as individual emails and handling regrade requests (4 hours). In addition, for the first offering of the course the content creation included not only homeworks, quizzes, and section material, but the grading rubrics for the autograders for each type of evaluation. For the second half of the course, we converted most of the discussion sections into project meetings with the students in which we met with each group for about 10 minutes each week, so less time was spent preparing homework/quiz material and the section-prep time was replaced by evaluating project checkpoints. A few additional hours per week were spent managing the online course, but as we fine-tune the material and autograder logistics, we expect that the online course can be managed by a single 10-hour-per-week TA, leaving the CS169 TAs free for for direct interactions with the students, especially during the project.

Book. Modern software touches many subsystems of different types, each of which has historically been the focus of some CS subspecialty. For example, SaaS encompasses datacenter computing, databases, OO programming, network security and performance, and user-centric design, plus nontechnical topics such as how to work with nontechnical customers and deliver a user-centric design. While there are many great (and not-so-great) books and online resources on each of these topics, a reading list cobbled together from them is impractical, lacks a “through-narrative”, and is very hard to get students to take seriously. We finally decided in early 2011 to create our own book that would introduce enough of each topic to function as a SaaS engineer and weave them together in a way that both made sense for a one-semester (or shorter) course. Our division of topics into largely-standalone subgroups allows instructors with less time or a less-sophisticated audience to select subsets of material appropriate for their needs. We decided very early to self-publish to keep the price low (currently $10 ebook/$20 print book). Engineering Long-Lasting Software: An Agile Approach Using SaaS & Cloud Computing is now in its beta draft, and we expect the First Edition to be ready by Spring 2013.

Crossing the software education chasm

fox — Mon, 23 Apr 2012 18:03:03 +0000

Dave Patterson and I wrote an extended op-ed piece that appears in this month’s Communications of the ACM talking about how and why we reinvented UC Berkeley’s undergraduate software engineering course to bring it more in line with modern development methodologies.

Although at the time of writing we didn’t even know we were going to offer this content in an online course, as it turns out, the same reasons we believe the course worked well at Berkeley also allowed us to offer it as a MOOC.

What are your thoughts? Are you doing something similar at your institution? Have suggestions for us? Leave comments here or on the CACM article!

CACM article
Short video expanding on the article and demonstrating autograding tools we used in MOOC
Our inexpensive textbook (print/ebook) specifically designed to complement this course—we’re looking for interested instructors, especially in North America, who want to try out these ideas in the classroom in Fall 2012

Made it to Spring Break, things still holding together!

fox — Fri, 23 Mar 2012 22:30:12 +0000

We made it to spring break without the online course imploding! Woohoo!

We were concerned (especially after Homework 2, which we knew was tough) that we were just going to lose everybody. But no—over 5,000 people are still with us, comparable to Prof. Jennifer Widom’s database course, which I hold as the benchmark.

Dave and I are impressed with the perseverance of the online students, especially given that most have full time jobs. The next time around we will probably expand the course by a week or two in order to space out the deadlines a bit. (We just extended most of the homework deadlines by a week for the online course.) We are also hiring local help (Berkeley students!) to help further debug the homeworks, improve the autograders, create new assignments and quizzes, monitor the forums more regularly, and fix the A/V quality (we know, we know).

We definitely plan to offer this course for free at least a couple of more times over the summer. Beyond that it’s unclear what will happen–it’s no secret that Coursera is a for-profit company, and we don’t know how long they will make courses available for free. However, we’re excited enough about this that we will make sure to find some way to keep getting the material out there, and if we can do so for free, we will. We’ll even get a chance to meet a few of the online students in person—they formed a study group that meets at a Starbucks close to UC Berkeley!

A big thanks to those who had faith in us and have continued to persevere through the course! And especially those who bought the book—we have 2 new chapters just about ready to go and they’ll be available as a free upgrade around two weeks from now. As promised, anyone who bought the Kindle edition will continue to get free updates until the book is content-complete.

So…as we get close to the end of the first iteration…

To those who hung on until the end despite the problems: THANK YOU for your patience, for the constructive feedback you provided via the forums, personal emails, comments on these blog posts, and however else you reached out to us. You’ve made it worthwhile for us by letting us know you were getting something out of the course!

To Instructors: if you’d like to beta test this material in your classrooms, we’ll even offer to run the autograders for you. See our beta program description if you might be interested.

To Griefers who tried to poison the forum atmosphere early on: see ya, and don’t let the door hit your a** on the way out. The smart money says that a whole bunch of companies besides Coursera are about to start trying to monetize courses like this (you can read about it in the latest Wired, in the NY Times, and elsewhere), so next time you’ll at least be paying for the right to complain.

And now…Spriiiing Breeeeaaaaak!!! Wooooo!!! (For the next week I can work from home instead of going to campus.)

P.S.: I had some technical video difficulties with “SaaS TV #2″ (salesforce.com) but it should be posted by tomorrow. SaaS TV #3 (with GitHub) should be available by April 4. It’s must-see TV.

UPDATE: I got an unexpected opportunity to speak with Raffi Krikorian, Director of Engineering at Twitter! That conversation is the basis of the just-posted SaaS TV #3. Next week we hope to post #4 (with Zach Holman of GitHub) and, hopefully, #5 (Dan Webb of Twitter talking about the present and future of JavaScript)!

The downside of online education

fox — Thu, 08 Mar 2012 21:36:31 +0000

In the 90s, the joke was “On the Internet, nobody knows you’re a dog.” The anonymity of online interaction allowed you to reinvent yourself.

One aspect of offering the online course has been to remind me that “On the Internet, nobody can confront you for being an a**hole.”

I just have to copy-and-paste (verbatim) a recent posting from a student [sic] in the online course forums, because paraphrasing just won’t do it justice:

As you don’t take care of us, the students, due the quality of the course I decide to quit this one

If you (the organizers) realize how important is our time and you decide to take this course seriously then perhaps I will return

Do you (the organizers) realize how many courses like this one are there? Do you realize how serious are them (see udacity for example) and how they care about us?

Is this the quality of the whole university? I now understand why stanford or others are more reputated than cal tech: because they take care about what they are doing but not you, you are making a fudge here and I have no time for fudges

…it goes on for awhile like this. I don’t even know who this person is—he or she posts as “Garito” with no other information.

I’ve dealt with whiny students before, but this level of entitlement is, frankly, stunning. Besides the fact that “Garito” provides no actual suggestions and confuses UC Berkeley with Cal Tech, what gets me is the downright nasty and ad hominem assertion: You don’t care about what you’re doing, or about the students.

We’ve already acknowledged a number of technical glitches that have slowed things down and that we’re working to fix, but “Garito’s” statement is just injurious and insulting.

Perhaps I shouldn’t be surprised. I volunteer at a nonprofit community theater, and for years it’s been our consistent experience that the worst customers are the ones who got free tickets. As a group, they complain more, are more likely to cancel at the last minute or walk out of the performance, write the nastiest reviews, and rarely turn into repeat customers even when they said they loved the show. That is, they’re more motivated by getting a ticket for free than by the product offered. (That’s also why our theater doesn’t do deals with GroupOn and similar outfits.)

I strongly suspect that online education purveyors will reach a similar conclusion, and hopefully that’ll happen before instructors get sufficiently turned off by attitudes like these that they decide to stop donating hundreds of uncompensated extra work hours for the “Garitos” of the world.

I know it’s human nature, when you’re trying to do a good job at something, to focus disproportionately on negative feedback; very few students, even those with legitimate grievances, have been anything like “Garito” and many more have been very positive in their comments. So in this case, sorry, “Garito”, but all I can really say, to use your own terminology, is “fudge you.”

Week 3: no disasters yet

fox — Mon, 05 Mar 2012 22:47:21 +0000

Our free online SaaS course is now entering its third week, with no major disasters to report!

Our autograder technology is working, thanks to heroic efforts by our on-campus graduate student instructors (i.e. TA’s), especially Richard Xia’s strong Ruby chops, and to Amazon Web Services’ generous donation of EC2 credits to run the autograder processes. No doubt we will further improve, streamline and fortify the autograders for future course offerings, but without Richard’s efforts we couldn’t have had them running in time. The autograders give detailed feedback on which tests passed and failed in the homework submission, and we allow students to resubmit homework to improve their scores based on the autograder’s feedback right up until the deadline.

The students are about to submit HW#1, completion of which entitles each of them to a $10 Amazon AWS credit and 90-day GitHub Micro account (thanks to Jinesh Varia at AWS and Kami Lott at GitHub for these very generous donations!). In fact, based on the activity in the autograder processes (another big thanks to AWS for donating EC2 credits that we can use to run autograders for 39,000 students), thousands of students have already submitted it.

We’ve been humbled by the number of countries represented, including many where students cannot buy Kindle books, or in some cases cannot easily complete a financial transaction to the US. We’ve made separate arrangements with many of these students so they can continue the course, but obviously we’ll have to find a sustainable and scalable solution to the distribution problem for next time. We’ve even had a couple of generous offers from students willing to subsidize the book purchase for others who are in financial hardship. We’ve fielded emails about this issue from the Gaza Strip, Tunisia, Nepal, Bangladesh, Pakistan, Saudi Arabia, Iran, Turkey, Nigeria, Georgia, Indonesia, Macedonia, Malaysia, Serbia, and Singapore, in no particular order.

Last but certainly not least, we’ve seen a handful of VERY positive and gracious emails and forum posts from people who are clearly appreciative of our efforts and feel they are getting a lot out of the course, despite its imperfections and the inevitable glitches that happen on a first offering. To those who reached out to us in that way—you know who you are—thank you for both your gratitude and your understanding that this is a brand-new experience for us and that problems will happen despite our best efforts. I’m spending most of my waking hours focused on this course, and hearing from people like you renews my energy to improve it further. Originally I was excited about being able to reach 35,000 students, but the truth is that the real reward is hearing from the few hundred for whom the opportunity has had such a positive impact.