Follow up to USDOE Report

In June, I wrote a post about the Department of Education’s Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies (2009). In that post, I reported the main findings listed in the press release, but had not fully evaluated the report. I took this opportunity to do so. The report has a lot to offer, so bear with me here!

The report outlines the following four purposes (U.S. Department of, 2009, p. xi):

1. How does the effectiveness of online learning compare with that of face-to-face instruction?
2. Does supplementing face-to-face instruction with online instruction enhance learning?
3. What practices are associated with more effective online learning?
4. What conditions influence the effectiveness of online learning?

It is a quantitative meta-analysis of existing empirical studies, contrasting the effectiveness of online, blended, and face-to-face learning. A process of research followed by thorough screening, coding, and full-text analysis yielded 51 studies for evaluation. The screening process looked for reports that (a) compared the modalities in question, (b) focused on learning outcomes, (c) demonstrated strong research methods and design, and (d) gave enough data to evaluate impact. Additionally, to control for inappropriate influence, they weighted studies based on their sample size before combining results during the data analysis. They also tested for homogeneity, moderator variables, practice variables, condition variables, and methods variables.

The results demonstrated the following:

1. Blended (online and face-to-face) learning maintained the strongest learning outcome advantage, followed by online only, and then face-to-face. However, this is not directly due to the delivery medium, but has more to do with increased time on task, greater accessibility to resources, and overall pedagogical differences evident in blended and online learning.
2. There is no significant difference in outcomes when comparing blended and online only mediums.
3. Overall, studies found no significant difference between instructional methodologies that included multiple media (video, audio, PowerPoint, etc…), and those that did not.
4. However, some advantage is present when students have some control over how they interact with media. For example, can students pause or chapter-jump a video or narrated PowerPoint, or are they required to watch them completely and linearly.
5. Research regarding the effectiveness of including quizzing in online learning is inconclusive.
6. Including simulations slightly increases students’ outcomes.
7. Including tools or assignments requiring students to reflect on their learning is the most effective method for improving student learning outcomes.
8. Online learning is an equally effective choice for undergraduates, graduates, and professionals throughout a large variety of studies. However, it is not as effective for K-12. (Although available studies in this area were limited.)
9. Learning platforms combining asynchronous and synchronous methods seemed more effective than platforms with only one of these.

For online higher education, these results are encouraging. And, I am happy the effectiveness of online learning is unconstrained by the medium, and relies more on how it us utilized by educators. This is reminiscent of Gardner’s (2003) plea to ensure technology serves education, instead of the reverse.

Most significant to the development of our global society, is the overwhelming effectiveness of self-reflection, self-monitoring, and the concepts of transformational learning (Palloff & Pratt, 2007). This seems to fall in line with Goleman’s emotional intelligence (O’Neil, 1996), and Pink’s (2005) (video)arguments for the coming of a conceptual age. However, it does seem to decry connectivist (Siemens, 2008) theories in favor of constructivism or cognitivism.

Combining the above, with the reports’ call to “redesign instruction to incorporate additional learning opportunities” (U.S. Department of, 2009, p. 51), should lead to:

1. An increased focus on transformational and reflective learning
2. A stronger awareness of emotional intelligence
3. Organic course designs including (a) more learner controlled media, (b) student choice of assessment methods, (c) less traditional quizzing, and (d) combined synchronous and asynchronous methods.
4. An increase in hybrid or blended courses
5. Additional research studies focused on filling the “[lack] a coherent body of linked studies that systematically test theory-based approaches in different contexts” (U.S. Department of, 2009, p. 49).

References
Gardner, H. (2003, April 21). Multiple intelligences after 20 years. Paper presented to the American Educational Research Association, Chicago, IL. Retrieved July 29, 2009 from [ http://www.pz.harvard.edu/PIs/HG_MI_after_20_years.pdf ]http://www.pz.harvard.edu/PIs/HG_MI_after_20_years.pdf

O'Neil, J. (1996). On emotional intelligence: A conversation with Daniel Goleman. Educational Leadership, 54(1), 6 - 11.

Pink, D. (2005, February). Wired 13.02: Revenge of the Right Brain. Wired. Retrieved August 1, 2009, from [ http://www.wired.com/wired/archive/13.02/brain.html?pg=2&topic=brain&topic_set= ]http://www.wired.com/wired/archive/13.02/brain.html?pg=2&topic=brain&topic_set=.

Palloff, R., & Pratt, K. (2007). Building online learning communities: Effective strategies for the virtual classroom. San Francisco: Jossey-Bass.

Siemens, G. (2008, January 27). Learning and knowing in networks: Changing roles for educator and designers [Paper presented to ITFORUM]. Retrieved June 3, 2009, from [ http://it.coe.uga.edu/itforum/Paper105/Siemens.pdf ]http://it.coe.uga.edu/itforum/Paper105/Siemens.pdf

U.S. Department of education. (2009, May). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. In B. Means, Y. Toyama, R. Murphy, M. Bakia, & K. Jones (Eds.), Office of planning, evaluation, and policy development (No. ED-04-CO-0040). Retrieved June 26, 2009, from U.S. Department of Education Web site: [ http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf ]http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf