Table 6 also shows selected activity statistics for each course. CS50X is excluded, as CS50X course activity was logged on a separate platform. The number of “clicks” is operationalized as the number of events (e.g. video plays, e-text page accesses, problem attempts, forum posts) in the server log files. This serves as a rough indicator of the total discrete actions that a user takes in a course. This number is unsurprisingly much higher for registrants who explored or were certified. The “active days” metric is a similar effort at describing a user’s activity in a particular course. It is simply the number of discrete days, demarcated in UTC time, that a user engages in some type of course activity.

For all registrants who viewed a course, the typical registrant accessed course content over two to five separate days, depending on the course. For registrants who explored or were certified, the typical registrant accessed course content over 24 to 63 separate days, depending on the course. Variation across courses is related to the length of the course and the amount of content, among other factors.

Discussion forum usage rates range across courses. Table 6 shows numbers of registrants who post one or more times in the discussion forums. Figure 11 displays the percentages of course “viewers” who post at least once in the forum, which range from 6.5% of viewers in Circuits- 3:6.002x to 25.7% in JusticeX and 33.3% in Biology:7.00x. Among certified registrants, discussion forum usage is more prevalent, with percentages ranging from 37% in Circuits- 3:6.002x to 62% in HeroesX and 70% in Biology:7.00x. Distributions of forum usage show extreme positive skew, with small but substantial numbers of registrants posting frequently. Excluding instructors and staff, dozens of registrants had post counts in the hundreds, and a few had post counts in the thousands.

Figure 11. Forum participation: percentages of “viewed” registrants and certified registrants with 1 or more forum post, by course.

Figure 11

Figure 12 shows percentages of registrants whose last actions in a course are in a given week, referenced by the beginning of the course or the registration enrollment date, whichever is sooner. This is known in some fields as a plot of “hazard probabilities” that describe the rate of attrition in a given time period, for individuals who have persisted to that time period. In this context, the plot describes the week-to-week percentages of registrants whose last action in a course is in that particular week. The plot shows largely similar patterns across courses, where, on average, half of registrants in the first week have their last activity in that week. In subsequent weeks, the percentage of students who remain are unlikely to have their last activity in any remaining week. This pattern continues until the ends of courses (not shown due to variability in course durations), where percentages rise again as courses end, and viewers and explorers are active for the last time. The plot indicates that registrants who are active after the first week have a fairly low chance of leaving in subsequent weeks.

Figure 12. Average percentage of active registrants whose last action in a course is in a particular week. For example, among registrants still active by Week 5 of a course, around 10% will have their last action in that week. Only includes registrants enrolling before the third week of the course. CS50X is not included. Selected individual course estimates are shown in gray to illustrate course-to-course variation. The course-level average percentage is shown in black.

Figure 12

Finally, Figure 13 begins to demonstrate how course activity metrics can tell a more complete story about opportunities to learn than assessments can alone. Figure 13 plots distributions of the percent of course chapters accessed by students along the horizontal axis. On the vertical axis is the course grade, adjusted in a piecewise linear fashion so that 0% and 100% retain their meaning but 60% is the passing cutoff for all courses. Grades of 0% and 1% are excluded in the histogram to avoid distorting the scale but are included in the scatterplot. Every registrant is represented as a dot on this plot, with “only registered” registrants at the origin, “only viewed” registrants in the lower left quadrant, certified students above the horizontal line, and “explorers” to the right of the vertical line. The registrants in the lower right quadrant are of particular interest and generally overlooked, we argue, by short-sighted certification rates. However, the overall story concerns the vast numbers of students who exist in all regions of this scatterplot, representing the immense diversity of learning approaches that registrants take in these open online courses.

Figure 13. Distributions of course activity (in terms of the percentage of chapters accessed) and course grades (for grades above 1%, linearly adjusted across courses to a common certification cutoff of 60%). CS50X is not included.

Figure 13