The value of multimedia in learning

The value of multimedia in learning

by Patti Shank

Summary

As broadband becomes ubiquitous, interactive designers are increasingly called upon to incorporate multiple media and dynamic graphics into their work. Presenting instruction in multiple media can be more effective than doing it through a single medium (such as text), but what is important is combining media effectively, not merely adding media.

Effective multimedia for learning requires carefully combining media in well-reasoned ways that take advantage of each medium's unique characteristics. The most effective multimedia provides learning experiences that mirror real-world experiences and let learners apply the content in various contexts.

From computer-based training to multimedia

In a previous career, as head of training for a clinical medicine organization, I bought a computer-based learning package that taught medical terminology to medical assistants, technicians, and transcriptionists. Completely text based, the program was rather revolutionary for the time. I remember that Bonnie Newton, my training buddy, and I were happy to be able to provide an alternative for those who couldn't attend her immensely popular medical terminology classes. This training was critically important for those who had to know the meaning of terms like “macrovascular” and “macroglossia,” and the existence of multiple training options gave them the flexibility to get their training as they needed it.

Jump ahead 15 years to current medical terminology e-learning. Now graphics illustrate each term, audio demonstrates the correct pronunciation, animations allow visualization of different parts of each whole, and video shows everyday use. Learners can make use of electronic flashcards and download print resources for help with studying. The classroom-based course allowed for live interaction. The old computer-based training provided flexibility for training. Multimedia offers the potential to augment learning with a vibrancy that the old computer-based training couldn't easily achieve. For example, multimedia can add clarity through multiple views, as in process guidelines alongside an animation. It can provide depth through additional information channels and resources. It can also add richness and meaning, through video, to show as well as tell. And, if it is not done thoughtfully and well, it can add needless complexity and provoke frustration.

Welcome to multimedia learning. What is multimedia?

Definitions of multimedia vary. Richard Mayer, professor of psychology at the University of California, Santa Barbara, defines multimedia as presentation of content that relies on both text and graphics. This definition, in my opinion, is a good start, but it doesn't provide deep enough insights about the essential factors that can make multimedia effective (or less effective) for learning.

Mao Neo and Ken T. K. Neo, faculty at Multimedia University in Malaysia, extend this definition. They say that multimedia is “the combination of various digital media types, such as text, images, sound, and video, into an integrated multisensory interactive application or presentation to convey a message or information to an audience.” This definition appeals to me because it implies that the combination adds up to more than the elements by themselves, which I believe is the key quality of multimedia when it comes to learning. In any combinations or permutations of common media formats, the whole should be greater than the sum of the parts.

Multimedia certainly has the potential to extend the amount and type of information available to learners. Multimedia can offer layers of beneficial resources, provide gratuitous information leading to frustration and overload, or anything in between. For example, online encyclopedias can provide links to videos and additional articles on specific topics of interest. News stories can reference links to audio commentaries, replays of video footage, and links to websites with additional resources. Online instruction can include explanations, links to resources, simulations, illustrations and photographs, and myriad types of activities that can also include multiple media. Too many resources and media, however, and the benefits get crowded out by the need to figure out what's what.

Figure 1: Multiple media in online applications training

Figure 1 shows a screen from a complex application training developed by Learning Peaks. The table of contents (which disappears when not needed, to save screen space) hints at a variety of multimedia devices, including rollovers, animations, simulations, and job aids.

Mayer explains how we process information through two basic channels, verbal and visual. Many people assume that multimedia is obviously better because it uses both channels. Researchers have found that multimedia helps people learn more easily because it appeals more readily to diverse learning preferences. Multiple media can be used to take advantage of the fact that our brains access information in nonlinear ways. Although multimedia can provide opportunities for improved learning, it can also be ineffective, even detrimental, when implemented poorly.

Learning and multimedia

In the next few sections I'll describe how contemporary learning theory explains the way people learn, how multimedia can augment or detract from learning, research that explains how multimedia impacts learning, design practices that augment learning, and why lack of attention to good design can lead to inferior learning environments.

Before reviewing the research on how multimedia can augment or detract from learning, I'll explore some common notions about how people learn, the complexity of the learning process, and the need to view multimedia research with an eye toward this complexity.

Views of learning and instruction

Learning is often viewed as information transfer from one person's head (an instructor or expert) into another's (the learner). Learners are thought to obtain information from an expert and add it to their own memory.

Figure 2: Learning as information transfer

Although this view of learning is widely held, it is too simplistic: it conceives of learners as passive receivers of information and doesn't provide guidance for designing effective learning environments. In fact, designers who hold this view of learning often design learning environments that may not include elements critical to effective learning, such as meaningful interaction, feedback, and the ability to learn over time.

A contrasting view is that learning requires people to personally integrate and make sense of new information while they are applying it in their daily lives. In this view, learning requires struggling to understand how new information meshes with existing knowledge and how to integrate into complex skills and abilities — not just remembering isolated facts or procedures.

Figure 3: Learning as a complex integrative process transfer

Consider the world of difference between merely being able to restate information and the ability to apply the information in the course of living and working. A great deal of instruction is aimed at rote memorization or superficial learning, but that approach doesn't go far enough. Complex skills and abilities that can be used in real life are the true goal of learning, not simply the ability to recall information.

Declarative knowledge is knowing about (the ability to state, list, match, describe, and so on). Procedural knowledge is knowing how (the ability to accomplish complex real-world skills). Copier technicians who can list the parts of the copier have declarative knowledge. Those who know how the parts work together and can use that understanding to troubleshoot a malfunction have procedural knowledge. Declarative knowledge is commonly part of procedural knowledge, but it isn't enough. Too often, instruction is developed at the declarative level, while actual tasks require people to work at a procedural level.

The purpose of effective instruction is to provide formal opportunities for complex skills and abilities — procedural knowledge — to develop. In the transmission model of learning, the point of designing instruction is to present information and then assess whether learners remember it. This model is appropriate when providing information , as opposed to instruction, where no specific skills requirement has been established but not appropriate for instruction. In the construction model of learning, the point of designing instruction is to create opportunities for learners to gain increasingly more complex skills and abilities and then assess whether they apply use the knowledge in real situations.

Contemporary learning theorists such as Spiro, Bereiter, and Brown believe that a key goal of instruction is to provide opportunities for learners to develop mastery in the areas of life they are each involved in. One important step that learners take in developing that mastery is building effective mental models. A mental model is an internal representation of reality. So instruction on how a copier works must help learners internalize how the parts work together so they can operate or fix it, not just match pictures of parts to part names.

Cognitive scientist and consultant Donald Norman describes how accurate mental models help us operate more efficiently and effectively in the world. Helping people form effective mental models has become a primary emphasis in the fields of human-computer interaction and computer usability. Accurate or complete mental models are important in instructional design too, because they are a cornerstone of effective performance.

Benefits of multimedia in learning

Well-designed multimedia helps learners build more accurate and effective mental models than they do from text alone. Shephard synthesized studies showing potential benefits of well-designed multimedia, including:

1. Alternative perspectives
2. Active participation
3. Accelerated learning
4. Retention and application of knowledge
5. Problem-solving and decision-making skills
6. System understanding
7. Higher-order thinking
8. Autonomy and focus
9. Control over pacing and sequencing of information
10. Access to support information

Mayer also describes potential benefits of multimedia. Given that humans possess visual and auditory information processing capabilities, multimedia, he explains, takes advantage of both capabilities at once. In addition, these two channels process information quite differently, so the combination of multiple media is useful in calling on the capabilities of both systems. Meaningful connections between text and graphics potentially allow for deeper understanding and better mental models than from either alone.

Figure 4: Text and video used together in online sales skills training

Figure 4 shows a screen from sales skills training developed by Learning Peaks. In this example, the text on the right briefly describes what the viewer will see in the video. Mayer's spatial contiguity principle (described below, in Table 1) explains that corresponding text and images should be placed next to each other to improve learning.

Hede and Hede offer a model of the myriad factors that affect the potential for learning from multimedia (Figure 5).

Figure 5: Hede and Hede's model of multimedia effects on learning

The model helps designers consider what factors are likely to make multimedia more or less effective for learning. Liao's meta-analysis shows inconsistent learning outcomes from multimedia, but those inconsistencies are likely due to the multiple factors in the model: Liao's findings show how important overall instructional design is to the effectiveness of multimedia because each factor in Hede and Hede's model can affect learning. My take on the inconsistency of research is that many projects do not take enough of these factors into account, so the outcomes are bound to be inconsistent. Again, multimedia can make a positive impact on learning, but it needs to be designed with a great deal of consideration.

In addition to these primarily cognitive effects, Astleitner and Wiesner and others describe how multimedia can affect emotions and motivation. For example, video has emotional components that can affect how people view the content. If the people in the video come across as aloof in a message about customer service, for example, learners may feel the content is suspect. Motivational and emotional aspects should therefore be considered when designing multimedia and likely correspond to Hede and Hede's cognitive engagement, motivation, and learner style factors.

How multimedia works in learning

Alessi and Trollip describe how effectively designed learning environments (including multimedia learning environments) include these four elements:

1. Presentation of information
2. Guidance about how to proceed
3. Practice for fluency and retention
4. Assessment to determine need for remediation and next steps

Figures 6-8 show examples of these four elements.

Figure 6: Text and graphics used together to present information

Figure 7: Text and graphics used to provide guidance

Figure 8: Simulation used in practice and assessment

These four elements of a learning environment can be embedded in e-learning or used in a combination of technology-based and non-technology-based instruction, but the environment must include all four elements to be effective. Although most of the elements can be implemented without multimedia, multimedia can make them even more effective and meaningful. Consider my earlier example, medical terminology drill-and-practice based on text versus graphics and simulations. Or pages of text hyperlinks versus hyperlinks annotated with pictures and descriptions which help learners determine which links will be most relevant to them. The point is that if the right elements are chosen and combined, they are potentially more compelling and effective.

Research by Mayer is commonly cited to show retention and transfer effects resulting from multimedia when the principles in below are adhered to. These principles stem from cognitive science's understanding of the limitations of working memory and methods for encoding into long-term memory.

Table 1: Principles that influence the effectiveness of multimedia (Mayer)

Multimedia learning is also of interest to people working outside traditional educational fields. Human factors researcher Lawrence Najjar looked at existing research on how multimedia affects learning and found that these practices could be beneficial for learning effectiveness:
- Select media with the best characteristics for communicating the particular type of information - for example, graphics help people retain spatial information better than text
- Use multimedia specifically to support, relate to, or extend learning, not just as embellishment
- Present media elements together so that they support each other
- Use multimedia that effectively employs verbal and visual processing channels to help learners integrate content with prior knowledge (this is called elaborative processing)
- Allow learners to control, manipulate, and explore positively impacts learning and elaborative processing
- Use familiar metaphors and analogies, feedback, and personalization to augment motivation
- Encourage learners to actively process and integrate rather than receive passively
- Match assessments media to presentation of information media

Designing multimedia applications

In instructional design, the purpose of multimedia isn't just to incorporate multiple media, insert cool effects, or add complexity (which can detract from learning). Use each medium to its advantage and to combine media so that the potential learning is greater and more effective than using single elements alone.

The following table shows how different types of media can support different purposes.

Table 2: Example media types and tools for various instructional purposes

Many of these media types and tools appeared in earlier figures. For example, Figures 1, 4 and 6-8 depict navigation and the use of text and graphics. Figures 6-8 show representations of a real application. Figure 8 shows changes over time and direct practice. Figures 9 and 10, shown next, indicate a process and hidden concepts.

Figure 9: Process model using text and graphics

Figure 10: Rollover used to reveal hidden concepts (source: Helen Macfarlane, Medical Illustrator, www.uchsc.edu/ltc/Fertilization.html)

According to theorists like Van Merrienboer, learning environments that are more directed appear to be best for novices, whereas more expert learners tend to prefer less directed approaches. Allowing learners to select the approach that best suits their expertise and learning style (and then change their mind) presents some design (and sometimes resource) challenges but is often needed for a mixed audience.

Well-designed multimedia can enhance motivation, learning, and transfer. The most effective multimedia provides learning experiences that mirror real-world experiences and allow learners to apply what they've learned in various contexts.

Multimedia design: A team sport

Designing complex multimedia generally requires a team effort because a good result requires many different design skills. For example, on the two projects shown in Figures 1, 4 and 6-9, our team called on many different people and skills. We needed instructional design skills to determine the goal of instruction and select instructional strategies and multimedia elements, writing skills to write content, information architecture skills to structure the content so it was easily to follow and access, graphic design skills to develop clear and attractive navigation and explanatory graphics, multimedia skills to work with instructional designers to create interactive elements, usability research skills to make sure that the whole worked well and wouldn't hopelessly frustrate learners, and infrastructure skills to make sure it would work on the client's systems. Not all projects require one or more people for each of these functions, but most require some elements of all of them.

Some concerns about multimedia and learning

It would be easy to conclude that multimedia is always the way to go, but that's a stretch. The first consideration is whether you have the resources and skill sets needed to do it well. Designers need to select media only when it supports a learning need. It is far more effective to build well-executed single-media instructional materials than poorly executed multimedia.

Multimedia adds complexity both to the screen and to the tasks that learners need to perform. By definition, a learning environment is already an unfamiliar and perhaps complex space. Use multimedia to help learners find their way around it. If multiple types of media and content are required, use position and prominence to direct learners' attention to the most important information. Consistent-looking and consistently placed navigation elements clarify what to do next. Help learners determine how to proceed but don't restrict their choices unless it's absolutely necessary (and it's almost never necessary). Lockstep or highly constrained navigation, for example, can make learners feel as if they are being coerced rather than led, with no control over their own learning.

Learners should always be able to tell where they are and how to get to where they want to go. Test screen layout with learners to gauge ease of use, and be willing to make changes. And always provide ways for learners to get help with the technology and with questions about the content.

Consider whether the combination of media adds to the learning experience. Better yet, ask learners. Is it confusing? Frustrating? Too much? In my own design work, I make sure that the learner can determine when to use additional media (for example, start an animation or video, or listen to a narration or see the text of it) rather than making them turn it off if they don't want it.

One more fundamental implication of using multimedia is that experiencing it requires multiple senses. That means you must consider the implications of each medium you want to use on people who have visual, auditory, or other disabilities.

Summary

Multimedia that's effective in learning doesn't simply consist of using multiple media together, but combining media mindfully in ways that capitalize on the characteristics of each individual medium and extend and augment the learning experience.

Research shows how multimedia can extend and augment learning. Uses of multimedia vary, from practice to games to discovery learning. Designers should first determine what outcomes they are trying to achieve and then select elements well suited for these outcomes (for example, select audio to allow learner to hear differences in tone). Then they need to make sure that the multimedia elements are designed well and work well together.

A designer with an information transfer view of learning is likely to limit the potential benefits of multimedia learning environments by continuously building drill-and-practice types of environments even when these are not called for instructionally. On the other hand, someone with a knowledge construction view of learning may have such an expansive view of what is possible that the resulting multimedia environment may be frustrating or overwhelming for learners. In all cases, determining when to use multimedia and designing good multimedia require real consideration and benefits from a team of people with instructional design, graphic arts, information architecture, and usability skills.

Mayer and Najjar provide us with principles for designing multimedia environments, and these can be extremely helpful. One caution: Don't be held captive to these (or any other design) principles without consideration of the idiosyncrasies of the specific situation. Being a good designer means learning and knowing when to follow the rules and when the situation calls for a unique response.

Although multimedia offers designers enormous opportunities for making learning environments meaningful and effective, multimedia by itself does not assure a good learning environment. Hede and Hede provide us with a list of critical factors that need to be considered. Research in this field is in its infancy (or at least toddlerhood) and will help us design the most effective multimedia environments for learning.

References

• Astleitner, H., and C. Wiesner (2004). An integrated model of multimedia learning and motivation. Journal of Educational Multimedia and Hypermedia, 13(1), 3-21.
• Alessi, S. M., and S. R. Trollip (2001). Multimedia for learning: Methods and development, 3e. Boston: Allyn & Bacon.
• Beichner, R. (1994). Multimedia editing to promote science learning. Journal of Educational Multimedia and Hypermedia, 3, 55-70.
• Bereiter, C., and M. Scardamalia (1996). Rethinking learning. In D. Olson and N. Torrance (eds.), Handbook of education and human development: New models of learning, teaching and schooling (485-513). Cambridge, Mass.: Basil Blackwell.
• Bill, D. T. (1999). Popular theory supporting the use of computer simulation for experiential learning.
• Broughton, A. (n.d.). Developing multimedia learning environments. University of Central Lancashire, Learning Development Unit.
• Brown, J. S., and P. Duguid (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.
• Craig, S. D., B. Gholson, and D. M. Driscoll (2002). Animated pedagogical agents in multimedia educational environments: Journal of Educational Psychology, 94(2), 428-434.
• Greeno, J. G. (1997). On claims that answer the wrong questions. Educational Researcher, 26(1), 5-17.
• Hede, T., and A. Hede (2002). Multimedia effects on learning: Design implications of an integrated model. In S. McNamara and E. Stacey (eds.), Untangling the Web: Establishing Learning Links. Proceedings ASET Conference 2002. Melbourne, 7-10 July.
• Herrington, J., and P. Standen (2000). Moving from an instructivist to a constructivist multimedia learning environment. Journal of Educational Multimedia and Hypermedia, 9(3), 195-205.
• Kafai, Y., C. Ching, and S. Marshall (1997). Children as designers of educational multimedia software. Computers Education, 29, (2/3), 117-126.
• Liao, Y. (1999). Effects of hypermedia on students' achievement: A meta-analysis. Journal of Educational Multimedia and Hypermedia, 8(3), 255-277.
• Mayer, R. E. (2001). Multimedia learning. Cambridge, United Kingdom: Cambridge University Press.
• Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13, 125-139./li>
• Najjar, L. J. (1998). Principles of educational multimedia user interface design. Human Factors 41(2), 311-323.
• Neo, M., and K. Neo (2001). Innovative teaching: Using multimedia in a problem-based learning environment. Educational Technology & Society Education 4(4).
• Norman, D. A. (1988). The design of everyday things. New York: Doubleday.
• Norman, D. A. (1993). Things that make us smart. Reading, Mass.: Addison-Wesley Publishing Group.
• Reeves, T. (February 12, 1998). The impact of media and technology in schools: A research report prepared for the Bertelsmann foundation.
• Shephard, A. (n.d.). Case for computer-based multimedia in adult literacy classrooms. Encyclopedia of Educational Technology.
• Spiro, R. J., M. J. Feltovich, and R. J. Coulson (1991). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Educational Technology, May, 24-33.
• Van Merrienboer, J. J. G., R. E. Clark, and M. B. M. De Croock (2002). Blueprints for complex learning: The 4C/ID-model. Educational Technology, Research and Development, 50(2), 39-64.