The buzz around massive open online courses, or MOOCs, hasn’t dissipated in the years since the trend has picked up.

That’s a good thing for associations, which have looked with interest at this online education model, driven by free or low-cost offerings, in recent years.

The problem is that this model is so new that we’ve had little idea of how to gauge its effectiveness for both the student and the facility—not a lot of research has been released to the world concerning the education offerings that work the best.

Fortunately, a new report from Harvard and the Massachusetts Institute of Technology (MIT), each of which offers classes on a popular MOOC platform called edX, could help clear the air.

The study, covering the first four years of using edX for this purpose, is one of the largest studies on the phenomenon yet, and highlights both the successes and pitfalls around these platforms. Among them:

MOOC courses offer a great way to capture diverse audiences: One notable thing about the courses offered by MIT and Harvard is that they tended to bring in a somewhat diverse mix of course-takers. While two-thirds of course-takers were male, the courses tended to be made up of more than 70 percent international students, and roughly 42 percent of participants were over the age of 30. (The median age was 29.) About a third came from a teaching background or taught the same subject matter themselves, and roughly three-quarters had a bachelor’s degree. International students tend to most often be from high-GDP countries, though there have been success stores from countries such as Rwanda, which had a relatively large number of its residents receive certifications.

Drop-off numbers tend to be large: Of the 4.45 million people that accessed content in a course, just 16.6 percent made it halfway through the course, while just 5.5 percent received a certificate. But at the scale with which Harvard and MIT work, that 5.5 percent rate is still pretty significant: That means 244,000 people received a certificate, whether or not they paid.

Technology-related courses are generally the most popular: Despite the fact that both schools offer a wide array of courses covering different kinds of material, the most popular MOOC courses were the computer-science courses. For participants that took more than one course, the computer-science courses had the effect of being hubs for numerous other subjects, covering science, government, and the humanities. This is despite the fact that there are far fewer comp sci courses than for any other topic.

Courses tend to become less popular when repeated: Over the four years in which Harvard and MIT have offered edX courses, those courses, on average, have seen enrollment declines around 25 percent per iteration—though it’s not that way across the board. “However,” the report states, “these declines vary widely by course. Some courses increase in enrollment and others decline by 50% or more.”

Business model shifts can have negative effects: At the beginning of 2016, edX changed its model so that free students who were taking audited courses could no longer receive cumulative certifications for finishing the course material. This, the report notes, had a negative effect on the number of students who had completed certifications but also, as TechCrunch contributor Devin Coldewey noted, may have also dampened enrollment numbers despite an expansion in the total number of courses.

Learning About Business Models

Clearly, these numbers offer a bit of a mixed bag: Free students represent the largest percentage of the schools’ MOOC audience, and they appear to have been turned off by the loss of certifications, despite the fact that most students never get far enough to get those certifications anyway.

And education offerings require a reboot every once in a while, meaning that you can’t keep running the same course over and over again.

Ultimately, the most interesting shift may be why free certification was limited in edX. A recent piece in The Economist implies that this element might be part of a broader trend in the MOOC space, which also includes players like the for-profits Coursera and Udacity and the nonprofit Khan Academy, toward shoring up their financial pictures.

“In their search for a business model, some platforms are now focusing much more squarely on employment,” the magazine states, citing the example of Udacity’s nanodegrees on applied topics like self-driving cars, some of which have been developed with the help of big-name employers.

Google’s Android course through Udacity, for example, is said to be good enough for employment by some tech firms, the magazine notes.

Is Now the Time For Your MOOC?

This dovetails into what could be a positive trend for associations, albeit one with a double-edged sword: Education outside of the university system could gain momentum through MOOCs, especially with the growth of certifications. That’s good for associations, which tend to offer a lot in the way of education.

But those certifications need to have the right level of weight, or they could get lost on the shuffle. A paragraph from the Economist piece lays down the point astutely:

The trouble with digital badges is that they tend to proliferate. Illinois State University alone created 110 badges when it launched a [program] with Credly in 2016. Add in MOOC certificates, LinkedIn Learning courses, competency-based education, General Assembly and the like, and the idea of creating new currencies of knowledge starts to look more like a recipe for hyperinflation.

The market for MOOCs is starting to mature and show some decent directions for the future, but your association still has to do its homework—and you still need to know how to properly consider how all this learning is going to look on either a resume or its modern equivalent.

With specialized education taking a lot of interesting directions in the future, now’s a good time to dip your toes into MOOCs.

Just be aware that it’s going to be a learning experience—for both you and your online students.

(iStock/Thinkstock)