Skip to main content
SearchLoginLogin or Signup

World of EdCraft: Teaching Healthcare Finance at MIT

Published onMay 02, 2022
World of EdCraft: Teaching Healthcare Finance at MIT
·
key-enterThis Pub is a Supplement to

Abstract

In this article, I describe my approach to dealing with the challenges and opportunities of synchronous online teaching during the Fall semester of 2020 in the specific context of a 90-student graduate course in Healthcare Finance at the MIT Sloan School of Management.

Keywords: online teaching, distance learning, live streaming, massive online open course (MOOC), gamification, healthcare finance


1. Introduction

In Lo, Stevens, and Willems (2022), we presented our approaches to synchronous online teaching and gave readers a summary of the hardware, software, and design principles involved in each of our home studios. In this companion article, I provide a case study of how I used my setup in the Fall 2020 semester to teach a Master of Business Administration (MBA) elective course on Healthcare Finance at the MIT Sloan School of Management (Sloan).1

This is a course I developed several years ago to focus on the role of finance in the health care industry, with particular emphasis on the application of novel financing methods to facilitate drug discovery, clinical development, and greater patient access to high-cost therapies. No prior knowledge of the health care industry or biomedical sciences is required or assumed, but this course is designed for students who wish to pursue careers in the health care and health care-investments industries, broadly defined.

The teaching slot assigned to me, Thursdays from 6:00 p.m. to 9:00 p.m. ET, was not ideal, especially for a synchronous online-only format. However, it was a necessary consequence of the complex scheduling challenges of Sloan’s combination of in-person, online-only, and hybrid courses, all delivered in a socially distanced manner to keep students, faculty, and staff as safe as possible.

This time slot yielded 13 three-hour sessions, and the topics covered included: basic financial analysis for the life-sciences professional, the historical financial risks and returns of the biotech and pharmaceutical industries, the mechanics of biotech startup financing, capital budgeting for pharmaceutical companies, estimating probabilities of success for clinical trial outcomes using machine learning, and applications of financial engineering tools (e.g., portfolio theory, risk management, real options) and new business models (e.g., venture philanthropy, special purpose acquisition companies, etc.) to various biomedical challenges including cancer, Alzheimer’s, rare diseases, infectious diseases like COVID-19, and other illnesses.

2. The Challenge and Course Objectives

An online version of this course already existed prior to the pandemic. I taught this course for the first time in 2017, and all the lectures were videotaped and made into a massive online open course (MOOC) on the MITx/edX platform.2 Therefore, in principle, it would have been possible to have students enroll in this MOOC. However, given the overwhelmingly negative feedback we received from students during the second half of the Spring 2020 semester when all classes were switched to an online-only format, offering MOOCs for their Fall courses was simply not an option. The average tuition for a typical U.S.-based MBA program is approximately $75,000, and scholarships are exceedingly rare. Not only are these students paying full tuition with after-tax dollars, but most of them have given up well-paying jobs to enroll, hence the opportunity cost of their forgone salaries must be included in the cost of their education. Therefore, it is understandable that they have higher expectations for their curriculum. As one student put it: “I didn’t give up my consulting job [to] pay $75,000 in tuition for Khan Academy.”

But how can we meet their expectations with an online-only course, especially given the behavioral factors of live-version remote interactions discussed in Lo, Stevens, and Willems (2022, section 2) and Sarma (2020)?

To solve this conundrum, I first asked what students find most valuable about business school besides academic subject matter, which they can access via Khan Academy and other online platforms. Why would they willingly give up their jobs and pay $75,000 to come to business school? The answer to this question laid the foundation for my online version of Healthcare Finance and the specific design of my home studio.3

I concluded that business school students have at least five additional objectives beyond academic training: (1) networking opportunities with their classmates, faculty, and business leaders; (2) exposure to the unique perspectives of faculty members and classmates through live interactions; (3) exposure to new industries and business leaders through action-learning opportunities; (4) assistance with career development; and (5) certification of a business degree. Given that the last two issues are not directly related to teaching, my focus turned to the first three objectives, but kept in mind the possibility of using the class to help develop new career opportunities for the students.

Would it be possible to meet, and perhaps even exceed, student expectations using the technology of online delivery platforms? In fact, I had a more ambitious goal: could we create an online-only course that was better than live, at least in these dimensions?

3. Course Design

I believe that the course design I finally settled on accomplished this goal. The four specific objectives were to exploit online technology to:

  • Maximize networking opportunities for students with classmates, faculty and staff, and business leaders and mentors.

  • Take advantage of the lower cost of participating in online classes to attract a larger group of outside speakers who otherwise would not have been able to participate in an in-person class.

  • Provide students with unparalleled access to action-learning opportunities with mentors.

  • Create new career opportunities for students interested in the health care industry.

To achieve these goals, the first challenge was to keep students engaged with the course despite the online-only format, so the first decision was to offer synchronous rather than prerecorded lectures.4 Of course, all lectures were automatically recorded through Zoom (the platform chosen by MIT for delivering online-only classes), so if students missed a lecture, they had the option of viewing it afterwards, but the interactive nature of the class made it difficult to enroll in this class on a consistently asynchronous basis.5 Despite time zone challenges—including students in Asia and Europe—almost all of the 90 students taking the course for credit (and approximately 10 listeners) attended every lecture live.

Other methods for keeping students engaged were incorporated into various aspects of the course requirements:

  1. Regular attendance and participation, in-class group exercises with assigned teams and peer evaluation after each class (20% of the total course grade)

  2. Weekly online pre-lecture questions, through the MITx platform (20% of the total course grade)

  3. Six problem sets, also through MITx (60% of the total course grade, 10% for each problem set)

  4. Volunteering to serve as moderator for a guest speaker (which allowed the student to be excused from one problem set among the first four)

  5. An optional ‘practicum,’ which involved working in teams on projects proposed by practitioners who served as mentors to the teams. If selected, this practicum excused the student from problem sets 5 and 6, hence it comprised 20% of the total course grade for students choosing this option.

These requirements engaged students before class (through the reading assignments and the pre-lecture questions), during class (through the synchronous lectures), after class (through informal after-class discussions), and outside of class (through the problem sets, practicums, and informal meetings, office hours, and lunches). Let me describe each of these four components in more detail.

4. Before and During Class

Prior to each lecture, readings were assigned6 and students were asked to answer two to four pre-lecture questions on MITx before class. These questions were generally nontechnical true/false or multiple-choice questions, and meant to familiarize students with the general theme of the upcoming lecture. They sometimes referred to the concepts covered in the reading materials so as to motivate students to prepare in advance for class, and were automatically graded by the MITx platform so students learned the correct answers (as well as their performance) immediately upon submitting their answers.

Each class began promptly at 6:05 p.m. ET, and I typically started my preparations an hour beforehand by rebooting my workstation. I found this to be the most reliable way of avoiding Open Broadcaster Software (OBS) and Zoom crashes, and for ensuring that the other software components would run as expected. I then followed a checklist I developed for initializing my equipment that has become invaluable because of the sheer complexity of the set-up (Figure 1). One example of this complexity is that the order in which certain components are turned on actually matters—in my setup, launching Zoom before OBS or switching on my webcam often causes Zoom to freeze and eventually crash.

Figure 1. 15.482 Fall 2020 online class broadcast checklist.

To build anticipation, I followed the typical gamer’s livestream approach of displaying a countdown timer on the course splash page about 30 minutes before class (see Figure 2). During this time, I would run through my slides one last time to check the animations and positioning of the text relative to my image, and then do a final check of the audio and video outputs. Also during this time, I would usually play music from my iTunes library for my own benefit, but students signing in early enjoyed this aspect of the class so it became a weekly tradition. I would choose a particularly upbeat piece of music during the final minutes before class to increase the energy level of the class, much the way performers use warm-up acts to prepare the audience. I have no idea whether this made a difference, but it did have a positive effect on my energy level so I can recommend it on that basis.

Figure 2. Splash page for 15.482 Fall 2020 using Snaz countdown timer and gif background image obtained from Shutterstock.

The three-hour class was divided into two parts—my lecture and then one or more outside speakers—separated by a 10-minute break. For classes with only one or two speakers, the lecture period ran from 6:05 p.m. to 7:55 p.m., leaving approximately 50 minutes for the speakers. For classes with three or more speakers, the lecture period would typically conclude at 7:25 p.m., leaving a total of about 80 minutes to accommodate the additional speakers. However, this schedule varied from week to week because of scheduling issues with certain speakers (e.g., one speaker was based in Europe and requested an earlier speaking slot, so for that class, he and others spoke during the first half and I lectured during the second half).

Because of these idiosyncrasies, it was critical to provide students with detailed schedules for each class in advance so they could plan accordingly—and to keep to that schedule as closely as possible—especially given the fact that a number of students were taking the class synchronously despite being based in Asia or Europe. An example of the posted schedules for classes 11–13 is given in Figure 3.

Figure 3.  Schedules for 15.482 Fall 2020 classes 11–13 posted in advance on the course website.

To keep students engaged throughout the lecture portion of class, I used three features of Zoom in addition to encouraging students to ‘raise their hands’ for questions and comments (available through the ‘Reactions’ feature): polls, breakout-room exercises, and the chat window.

Poll questions ranged from students’ career goals (e.g., ‘I plan to work in health care after graduation: (a) yes; (b) no; (c) undecided’) to illustrative examples to highlight certain concepts (e.g., choosing among four investments with different risk/reward profiles). Each lecture contained at least one and as many as four polls, each lasting about 30 seconds, and although they were all constructed in advance, the teaching assistant (TA) entered each of them manually at the start of the Zoom session.7 This required further coordination between the TA and I prior to each class, which was part of the 30-minute preparation beforehand.

Breakout-room exercises typically involved applications of a concept just covered in lecture, for example, how to compute the net present value of a drug candidate undergoing clinical trials, and took place in two parts. The first part involved sending students into preassigned breakout rooms of four to six students each for a 5-minute ‘icebreaker’ session where they introduced themselves to each other. Then the students returned to class for additional lecture content, after which the breakout exercise was announced and students were sent back into a 10-minute session with the same preassigned teammates to work on the problem. At the end of this session, students rejoined the main class and I ask for volunteers to present their team’s solution to the rest of the class. These two parts are clearly delineated in the lecture slide deck (Figure 4).

(a)

(b)

Figure 4. Breakout-room exercise slides for 15.482. Slide (a) announces the 5-minute icebreaker session and slide (b) provides the exercise itself, on which teams of 4–6 students collaborated in a 10-minute breakout-room session.
The purpose of separating the icebreaker from the exercise was to give students an opportunity to meet each other first and get introductions out of the way, so as to be able to devote full attention to the exercise during the second breakout-room session. The purpose of preassigning breakout-room membership was to maximize the number of new contacts each student made during the semester. Assignments were done via an integer-programming algorithm that “samples without replacement” (Xu, 2020) so each week’s breakout room assignment allowed students to meet as many new classmates as possible (Figure 5).8

Figure 5. Optimization problem to generate breakout-room assignments so as to maximize the number of networking opportunities among students during the semester. See Xu (2020) and corresponding Julia code for details, both of which are available at https://bit.ly/38vlRdb.

And the chat window is perhaps the most novel aspect of learning technology from the perspective of traditional in-person teaching. At first, I was taken aback by how much chatter was going on while I was speaking. But I quickly realized that, while some of the chats were social—which helps build camaraderie among the students, a particularly valuable thing for an online-only class—much of it was questions and answers between the students. Because of heterogeneity in the students’ educational and career backgrounds, some students had deep expertise in health care while others were totally inexperienced. The chat window allowed tremendous amounts of information to be communicated from one student to another and, in most cases, nearly instantaneously. In an example I provided in my video overview of this class, I referred to a ‘pharmacy benefit manager’ by the acronym, ‘PBM,’ without first defining it, prompting one student to ask his classmates via chat what this term meant, and he received four replies—each with different and useful aspects of the definition—within the span of 229 seconds.

Once I realized the power of the chat window in crowd-sourcing information that usually supplements my lecture, I encouraged students to make use of this important feature. Moreover, during presentations by outside speakers, I would make use of the chat window as well, pointing out connections between speakers’ comments and course content, or posting articles to support points made by speakers or students. To develop a sense for the value of these chats, I have included in Appendix A a portion of the chat transcript for one of my classes (de-identified to preserve my students’ privacy).

On occasion, discussions in the chat would become so relevant to the topic I was lecturing on, or, in rare cases, so confused, that the TA—who was asked to monitor the chat window throughout the lecture—would interrupt me to bring the issue to my attention so that we could engage the entire class in its discussion. This is a critical role of the TA, and its importance to maintaining an active and productive class dynamic cannot be overemphasized.

The combination of raised hands, polls, breakout sessions, the chat window, and interactions with speakers (see section 5) implies that students are being engaged in some specific task requiring their active participation once every 15 minutes or so, which achieves the goal of maintaining their attention and interest level throughout the lecture portion of class. Figure 6 illustrates the duration between distinct activities in a typical class. In the context of an in-person lecture, these shifts in activities every 15 or 20 minutes would be considered by most teachers as highly disruptive to the flow of a standard lecture. This is due to the fact that the physical presence of a lecturer, and their ability to ‘read the class’ and change the dynamics by stopping to take comments or cold-calling on students, allow them to engage students with very little effort. As discussed in Lo, Stevens, and Willems (2022), replicating that same level of engagement in an online class requires a great deal more effort, which includes periodic shifts in the activities of otherwise passive participants.

Time

Delta (min)

Activity

6:05 p.m.

Class begins

6:20 p.m.

15

Poll #1

6:30 p.m.

10

Breakout Session (icebreaker)

6:35 p.m.

5

Class resumes

6:50 p.m.

15

Poll #2

7:10 p.m.

20

Breakout Session (exercise)

7:20 p.m.

10

Teams report

7:30 p.m.

10

Class resumes

7:55 p.m.

25

10-minute break

8:05 p.m.

10

Introduce student moderators

8:10 p.m.

5

Students introduce speaker #1

8:55 p.m.

Class ends

Figure 6. Schedule of activities of a typical 15.482 Fall 2020 class, with activities spaced as evenly as possible throughout the session to maintain student attention and engagement.

5. Guest Speakers

To increase student engagement during the speaker portion of the class, I chose a ‘fireside chat’ format for all speakers rather than asking them to give prepared presentations. This had the dual purpose of minimizing the burden to speakers as well as maximizing the interactions between speakers and students, especially since I delegated the responsibility of introducing each speaker and moderating the fireside chats to student volunteers. For the first class, I took on the role of introducing the speaker and moderating the fireside chat, just to give students an example of what was expected, and then asked for volunteers for subsequent weeks.

The moderator role involved scheduling and preparing for a 30-minute prep call with the speaker the week before their visit to go over the topics to be covered, and to settle on the first two or three questions that the student moderator would ask the speaker before turning to the class for general Q&A. In addition to facilitating the fireside chat, these prep calls served another purpose: to give students opportunities to network with industry leaders, and to allow those leaders opportunities to mentor students. This second purpose was embraced not only by the student volunteers but also by every single speaker.

Initially, I expected to assign one moderator per speaker, but due to popular demand, I assigned two to three moderators per speaker in most cases and this worked out very well. Without exception, all speakers said they thoroughly enjoyed interacting with their moderators during the prep calls, and the students were thrilled to have the opportunity to meet such extraordinarily accomplished individuals. Moreover, the class discussions generated by this format—and thanks in no small part to the coordination between speakers and moderators in advance—were lively, uniquely informative, and truly inspiring. To acknowledge the time and effort involved in this role, moderators were excused from one out of the first four problem sets of their choosing.9

6. After Class

To replicate the common practice of speaking with students informally after an in-person lecture, I offered to meet with students after each class in a separate Zoom session that was posted in the chat window by the TA at the end of class. I decided to hold my informal meeting in a separate Zoom session after a 5-minute break for several reasons. The first was a desire to have a clear end to the formal class so students wouldn’t feel obligated to stay for the informal discussion (this was especially important given the late hour for students in the local time zone, not to mention those in Europe and Asia).

The second reason is that the class typically ended with guest speakers and I wanted to give students the opportunity to interact with them informally, without my presence, much like how an in-person class with guest speakers would conclude. My moving to a different Zoom session was less disruptive to these after-class interactions than asking our speakers and students to do so.

And third, moving to a different Zoom session after a short break gave students an opportunity to stretch their legs or get a drink, and gave me the opportunity to change the look and feel of the setting so that students would, in fact, feel less formal and more relaxed. Although this could have been accomplished at the touch of a button (thanks to the Stream Deck), having a separation in time and space between the formal setting of class versus the informal setting of my after-class discussion—complete with a slight wardrobe adjustment—served to underscore this shift (see Figure 7).

(a)

(b)

Exhibit 7. Using different virtual backgrounds in distinct Zoom sessions to separate (a) formal class time from (b) informal after-class discussion in 15.482.

The last after-class task was to request that students fill out a survey form (provided in Appendix B) consisting of an evaluation of the performance of: (1) their breakout-room teammates, (2) the instructor, and (3) each guest speaker. Students were also given an opportunity to provide comments and suggestions for improving the class (see Appendix B, Question 4). Students were given 24 hours to complete the survey so as to collect feedback while the impressions from class were still fresh in their minds. Requesting feedback after each class may seem excessive, but it was particularly important given the experimental nature of this online class. And while immediate feedback can be somewhat tough on the instructor’s ego, it seems only fair, given that we instructors evaluate students on participation after each class. Moreover, some of the most effective components of the course were suggested by students through these surveys, for example, having a separate ‘icebreaker’ session prior to the breakout exercise, providing students with class schedules prior to each class, and allowing more than one student to serve as moderator for a given speaker.

All evaluations were scaled from 1 to 5, with the following definitions for each score: 1-ineffective, 2-somewhat ineffective, 3-neutral, 4-somewhat effective, and 5-effective. In evaluating their breakout-room teammates, we asked students to also include their own name in the list to preserve their anonymity.10 These breakout-room performance evaluations were averaged across all classes and incorporated into the students’ class participation grade, hence the students took these collaborative exercises—and the evaluations—seriously.

7. Outside of Class

I used several means to maintain student engagement outside of class. The first involved 15-minute informal meetings with groups of three to five students during the first 2 weeks of class. These meetings were extremely helpful in putting names to faces, learning more about the students’ backgrounds and career objectives, and getting students to meet each other. What I learned about the students through these meetings also allowed me to draw upon the rich pool of their experiences during class discussions. For example, in reviewing a specific pharmaceutical company’s clinical trial outcome, I was able to ask one of the students—who, I learned, worked there as a summer intern—to describe the mood at the company during the period leading up to the trial outcome.

These informal meetings did require a significant investment of time at the start of the semester, and were logistically challenging to schedule (reserving a block of hours and posting sign-up sheets helps), but it was well worth the investment and paid many dividends during the rest of the semester.

The second means for maintaining student engagement was holding informal virtual drop-in lunch hours with my TAs each week. Students were asked to sign up for a 15-minute slot in advance if they wanted to discuss a specific issue, but were also invited to drop in for informal discussion if they wished. During periods when no students showed up, I would use the time to discuss various aspects of the class with my TAs. These informal gatherings were similar to the meetings during the first two weeks, but as the semester wore on and I got to know the students better, these informal lunch meetings became more engaging and lively.

8. The Practicum

The third, and probably most valuable, component for maintaining student engagement outside of classes was the optional ‘practicum.’ This was a project-based exercise that took place over a particular three-week period in November 2020 in which teams of three to seven students worked on live problems posed by health care practitioners and researchers. Projects ranged from developing a proposal to commercialize therapeutics for ultra-rare diseases to determining the cost of capital for a gene-therapy reinsurance company. Students were given a list of these projects—each described in a one-page summary by the mentor associated with it (a sample one-pager is given in the course syllabus)—and then allowed to sign up for their top choice. Teams were formed on the basis of these sign-ups.

Each team hosted one 45-minute Zoom meeting with the mentor at the start of the three-week period, during which the mentor would provide more background for the project and offer some guidance as to possible directions to explore. However, it was up to the team to decide how best to address the challenge. Mentors were not expected to have further contact with the team until the end of the three-week period (although almost all of them did have additional contact, voluntarily and enthusiastically), at which point each team was to present their findings to their mentors in a 1,500-word memorandum and a 30-minute Zoom presentation.  These final presentations were recorded and made available to the rest of the class, and the practicum was graded on the quality of both the memorandum and the presentation.

Given the workload involved in this project, students opting to participate in a practicum were excused from Problem Sets 5 and 6, which were assigned and due during this same three-week period in November 2020. Of the 90 students enrolled in the course, 50 participated in 10 different practicums, and the outcomes were nothing short of phenomenal. All the mentors commented on how impressed they were with the memoranda and presentations, and indicated that they planned to make use of some of the students’ recommendations. They also enjoyed their interactions with the teams, and several have followed up with students in various ways, including continued collaboration on their projects, exposure to current industry practices and issues, and, in several cases, job interviews.

9. Did It Work?

Recall that the four objectives for the online edition of 15.482 were:

  • Maximize networking opportunities for students with classmates, faculty and staff, and business leaders and mentors.

  • Take advantage of the lower cost of participating in online classes to attract a larger group of outside speakers.

  • Provide students with unparalleled access to action-learning opportunities with mentors.

  • Create new career opportunities for students interested in the health care industry.

Based on the final course evaluation for 15.482 administered by MIT (not the weekly evaluations administered by me),11 and qualitative feedback from speakers, mentors, and teaching staff, the format and structure of the online version coupled with the delivery technology was a success from the perspective of students, faculty, staff, and speakers and mentors.12

Although students acknowledged that online interactions were inferior to in-person meetings, the lower cost of conducting meetings online provided much greater opportunities for networking, mentorship, experiential learning, and career development, especially with the extraordinary group of business leaders and academic researchers who participated in the class. The practicum was perhaps the most powerful illustration of the potential benefits that an online platform can provide, but even the informal lunches showed how technology can give both students and faculty new ways of building community in the absence of in-person meetings.

To quantify the degree of networking facilitated by class interactions, we tracked all class-facilitated first-time meetings throughout the semester between every pair of students. These first-time meetings are displayed in Figure 8, which is a  matrix where the (i,j)(i, j)-th element is colored green if students met during a breakout room, practicum, moderator meeting, or informal meetings and office hours with me.13 Based on this data, the average number of new contacts made by students in the class was 43, the minimum was 35, and the maximum was 51.

Of course, first-time meetings do not necessarily translate into meaningful connections between peers, but this does quantify the degree to which the online platform and our class activities can boost networking objectives. It would be useful to conduct surveys of in-person classes to assess the number of first-time meetings in traditional classroom settings to see whether the online platform truly dominates. If we had included first-time meetings with speakers and mentors as part of our metric, there is no doubt that the online platform yields an absolute advantage over in-person classes. These observations lead to the natural hypothesis that a hybrid course—one with in-person lectures as well as some online components—could be superior to both alternatives.

Figure 8. First contact matrix for 15.482 Fall 2020 students. Cells (i, j) highlighted in green represent class-facilitated contact between students i and j at some point during the semester.

10. Lessons Learned

There are two main issues with the online edition of 15.482 that instructors should be aware of when thinking about using this format. The first issue is that the use of so many outside speakers—we hosted 32 over the course of 13 required classes and one optional class—necessarily required reducing the amount of lecture material I would have normally covered in this course. The content I chose to omit in favor of speakers were of two types: (1) additional illustrative examples, including extended case studies, that show how to apply concepts such as net present value calculations, cost of capital estimation, and capital budgeting decisions using real options, decision trees, and Monte Carlo simulation and (2) more technical content such as portfolio optimization for biomedical assets, estimation of probabilities of success for clinical trials, and applications of machine learning to forecasting clinical trial outcomes.

This decision was motivated by the four objectives described earlier, especially the desire to give students unique access to extraordinarily accomplished business leaders and academic researchers who would ordinarily not be available to us in an in-person setting during normal times. Moreover, if the primary purpose of the omitted content was to provide students with business context and exposure to best practices, that was achieved much more effectively by our speakers, many of whom pioneered the very business models I would have lectured about. Finally, the omitted material was, in fact, available online through the 15.482x MOOC that I developed a few years ago. Links to that material were posted on the course website.

The second and more important issue with the online version of 15.482 is the heavy workload for students, faculty, and staff. The increased workload for students came primarily from the practicum, which ended up involving far more time and effort than two problem sets’ worth. However, there is no easy solution to this problem because allowing students to substitute three or four problem sets for the practicum—which may be a more equitable trade-off in workload—implies that those students will miss out on important training covered by those problem sets. Also, my sense from working with the practicum teams and their mentors is that the extra effort students expended in the practicums was largely the result of their passion for their chosen problems (which is one important reason to allow them to choose projects rather than to assign them), and their own commitment to the highest standards of excellence. Perhaps the most balanced solution for future renditions of the practicum is to inform students in advance that the workload will be greater than that of two problem sets, but what they receive in return is practical experience that they can’t get any other way.

The workload for the teaching staff was also greater. The role of the TA during class was critical, as discussed, but other important responsibilities of the TA included offering recitations, office hours, updating problem sets and providing answer keys, and supporting practicum teams. I was fortunate to have recruited one of my top PhD students, Qingyang Xu, to TA the course, and he was exceptionally committed to this position, as well as my unique goals for the course. For example, he almost always responded to student emails within 30 minutes so students seeking help would not be kept waiting and lose engagement. He set up dozens of individual Zoom meetings—some taking an hour or longer—outside of his regular office hours and weekly recitations to help students with problem sets and practicum problems, and quickly became an integral part of the students’ learning experience. Such responsiveness is, of course, highly dependent on the particular personality of the TA, but it would have been impossible without the online technology. Given this workload, one TA was insufficient for 90 students, so I asked a former TA for the in-person version of 15.482, Zied Ben Chaouch, for help and he graciously agreed to volunteer in a supporting role.

The role of the course assistant was also critical, especially for managing the logistics of meetings and communications with all the speakers, mentors, and students. While many of these meetings were scheduled by the students themselves, initial contact with speakers and mentors was my responsibility. Subsequent contact to provide Zoom links and other logistical information was the responsibility of the course assistant, Lena Ngor, who did an excellent job managing the logistics of the entire class.

Finally, my own workload was considerably higher than any in-person course I have ever taught.  Part of this increased workload was due to my own lack of experience with online technology, and the lack of institutionalized support for the type of lecture style and classroom experience that I wanted to achieve. That may change over time as MIT invests more resources into supporting online teaching, and as I develop more experience with this platform. But another part of the increased effort is to be expected for any form of online teaching. As discussed in Lo, Stevens, and Willems (2022), substituting in-person interactions with online technology is extremely difficult and requires many more person-hours to achieve anything close to comparable impact.

To differentiate between these two types of increases in workload, I have separated the sources of incremental effort required for the online version of 15.482 into two categories in the following lists:14

One-Time Incremental Start-Up Effort:

  • Build home studio (one week, given the resources provided in Lo, Stevens, and Willems (2022) and the corresponding videos, and once all the parts have arrived)

  • Learn how to use and maintain each component (three weeks, once the studio is fully operational)

  • Convert preexisting conventional PowerPoint slide decks to an online format (5-10 hours per 90-minute lecture, depending on the degree of animation, sound effects, polls, and breakout exercises)

Recurring Incremental Effort:

  • Maintain studio; for example, update software, replace failed components, and so on (1–5 hours per week)

  • Invite speakers, explain fireside-chat format of the class (15–30 minutes per speaker)

  • Invite practicum mentors, discuss possible projects with each mentor, edit one-page descriptions provided by mentors, view and grade final presentation of each team, follow-up with mentors after the final presentation (3–5 hours per project)

  • Rehearsals alone and with TA(s) to ensure that the PowerPoint presentations work as expected on OBS/Zoom, and to practice the timing of the animations, sound effects, polls, and breakout sessions (1-2 hours per 90-minute lecture, not including time spent updating lecture contents)

  • Initial informal 15-minute meetings with students in groups of three to five students during the first two weeks (three blocks of 2-hour meetings for 90 students)

  • Informal lunches/office hours (1–2 hours per week for a class of 90 students)

Such incremental effort is currently not typical among faculty at universities where hiring and promotion decisions are based largely on research impact and productivity rather than teaching. But even at research-oriented institutions, faculty may still be drawn to invest in their teaching under certain conditions. The desire to have impact on the next generation of business leaders is compelling motivation for many faculty. The need to ‘step it up’ for our students in the midst of a pandemic-induced lockdown has been another source of motivation for my colleagues and me. Also, if it’s possible to design a course that aligns with a faculty member’s research agenda—as was the case for me with 15.482—then faculty can and will devote significantly more time and effort to course development and teaching without hesitation or regret.

In my case, the opportunity to interact with and learn from highly motivated and engaged students, guest speakers, and mentors has contributed significantly to my own research, and by bringing some of that research into the classroom, I believe that students will be better prepared for careers in the health care industry. To the extent that universities can craft course offerings to match more closely the research interests of their faculty, there’s no limit to the innovations that faculty will bring to their classrooms.

11. Conclusion

The bottom line is that, to achieve a pedagogical outcome that approximates the value-added of in-person teaching, synchronous online teaching requires much more time and effort. Regardless of how much experience an instructor has with their preexisting course contents, turning an in-person course into a synchronous online course is an entirely new prep, and also involves an entirely new set of technologies that require a significant amount of training and experimentation for instructors unfamiliar with them. However, once the upfront investment has been made—and it really does have to be viewed as an investment, given the resources needed to become proficient—synchronous online teaching can be extremely rewarding, and actually outperform in-person teaching in several dimensions.

Based on my experience during the Fall 2020 semester, I plan to adopt a number of components for in-person teaching after the pandemic is over. Using videoconferencing platforms to meet with small groups of students outside of class is an extremely efficient way of developing rapport with the class at the start of the semester. Incorporating guest speakers via videoconferencing is also another aspect that can be adopted with in-person teaching. And giving students access to unique networking opportunities and practical training through practicums facilitated by online interactions with practitioners is yet another component. Finally, I plan to adopt some technology that can replicate the chat window so students have a means for interacting with each other in real time while I lecture, but where the TA can moderate the discussion and maintain the desired culture of the class. The broader implications for online education are discussed in Lo, Stevens, and Willems (2022).


Acknowledgments

I thank Brian Stevens and Sean Willems for many helpful and inspirational discussions and feedback, Qingyang Xu for helpful comments and outstanding contributions as the TA for this course, and Jayna Cummings, Amara Deis, and Rebecca McLeod for editorial assistance. Thanks also to the HDSR reviewers and editors, especially Xiao-Li Meng and David Parkes, for their many helpful comments on the manuscript.

Disclosure Statement

The views and opinions expressed in this article are those of the author only, and do not necessarily represent the views and opinions of any institution or agency, any of their affiliates or employees, or any of the individuals acknowledged below.


References

Lo, A. W., Stevens, B., & Willems, S. P. (2022). World of EdCraft: Challenges and opportunities in synchronous online teaching. Harvard Data Science Review, 4(2). https://doi.org/10.1162/99608f92.73a1c910

Sarma, S. (2020). Grasp. Doubleday.

Stevens, B. (2022). World of EdCraft: Teaching statistics at the University of Tennessee Knoxville. Harvard Data Science Review, 4(2). https://doi.org/10.1162/99608f92.e303e4d3

Willems, S. P. (2022). Word of EdCraft: Teaching supply chain management at the MIT Sloan School of Management. Harvard Data Science Review, 4(2). https://doi.org/10.1162/99608f92.2b7d0bc7

Xu, Q. (2020, August 28). 15.482 breakout room assignment. Retrieved November 24, 2021, from https://drive.google.com/file/d/1bO7rUGJ7gXDZZMQiPtkgTGZjlAvSS0L2/view?usp=sharing


Appendices

Appendix A: Sample Zoom Chat Window Transcript From 15.482

The following is a portion of the complete transcript of the chat window for lecture 12 of 15.482 Fall 2020. The topic of the lecture was “Pricing, Insurance, and Ethics.” The blue boxes provide groupings of the chats according to the topics being covered in the lecture at the corresponding times.

Appendix B: 15.482 Fall 2020 After-Class Evaluation (Implemented in Qualtrics)

  1. Please rate the performance of all members of your breakout session (and make sure to include yourself so as to preserve the anonymity of this survey):

Rating:

1

2

3

Definition:

Effective

 Ineffective

Absent

Student #1

 

 

 

Student #2

 

 

 

Student #3

 

 

 

Student #4

 

 

 

Student #5

 

 

 

Student #6

 

 

 

  1. Please rate the performance of the instructor:

Rating:

1

2

3

4

5

Definition:

Ineffective

Somewhat Ineffective

Neutral

Somewhat Effective

Effective

  1. Please rate the performance of the speaker(s):

Rating:

1

2

3

4

5

Definition:

Ineffective

Somewhat Ineffective

Neutral

Somewhat Effective

Effective

Speaker #1

 

 

 

 

 

Speaker #2

 

 

 

 

 

Speaker #3

 

 

 

 

 

 

 

 

 

 

  1. Do you have any comments or suggestions for improving the class? Every suggestion will be reviewed and considered, though we may not be able to implement them all.


©2022 Andrew W. Lo. This article is licensed under a Creative Commons Attribution (CC BY 4.0) International license, except where otherwise indicated with respect to particular material included in the article.

Comments
0
comment
No comments here
Why not start the discussion?