Skip to content
Home » Prominent Multimodal Discourse Analysis Case Studies & Research

Prominent Multimodal Discourse Analysis Case Studies & Research

Are you ready to enhance your learning by asking the assistant?

Log In to Your Account

Alternatively, if you don't have an account yet

Register Now!

Multimodal discourse analysis has advanced significantly due to key research by Kress and van Leeuwen, Jewitt, Machin and Mayr, Norris, and Scollon and Scollon. Their case studies examine various dimensions of multimodal communication, from media and classroom interaction to public signs and everyday social encounters.

1. Kress and van Leeuwen (2006): Reading Images

Gunther Kress and Theo van Leeuwen’s influential work, “Reading Images: The Grammar of Visual Design,” first published in 1996 and updated in 2006, is a foundational text in the field of multimodal discourse analysis. It provides a comprehensive framework for analyzing visual communication, particularly in media, by treating visuals as a language with its own grammar.

1) Key Contributions of “Reading Images”:

  1. Visual Grammar:
    • Kress and van Leeuwen proposed that images have a structured grammar similar to verbal language, consisting of specific elements that contribute to meaning.
    • They identified key components like composition, modality, color, and perspective, which influence how visuals communicate messages.
  2. Composition and Layout:
    • The book explains how the arrangement of elements within an image contributes to its meaning.
    • They discuss principles like “information value” (placement of elements conveys importance), “salience” (visual prominence), and “framing” (how elements are separated or connected).
  3. Representational, Interactive, and Compositional Meanings:
    • Representational Meaning: Refers to the content of the image, such as people, objects, and settings.
    • Interactive Meaning: Involves the relationship between the viewer and the image, shaped by gaze, perspective, and distance.
    • Compositional Meaning: Concerns the overall arrangement and organization of the image, affecting how elements relate to each other and to the whole.
  4. Modality and Realism:
    • Modality in visual communication refers to the perceived reality of an image.
    • Kress and van Leeuwen explored how aspects like color saturation, depth, and detail contribute to an image’s modality, influencing its believability.

2) Applications of Their Framework:

  1. Advertising: Analyzing how advertisements use composition, color, and framing to convey messages and persuade consumers.
  2. News Media: Understanding how news images are designed to influence viewers’ perceptions of events through composition and modality.
  3. Educational Materials: Studying how textbooks and other instructional visuals are designed to facilitate understanding through the structured use of visual elements.
  4. Political Campaigns: Investigating how political advertisements and campaign materials use visual grammar to shape public opinion.

3) Impact of “Reading Images”:

  • Multimodal Analysis: The book laid the foundation for multimodal analysis by providing a systematic framework for analyzing visual elements alongside textual ones.
  • Interdisciplinary Approach: It has been widely influential in fields like media studies, education, and advertising, where understanding visual communication is crucial.
  • New Theoretical Directions: Their work inspired further research into how different modes interact to create meaning in various contexts.

“Reading Images” by Kress and van Leeuwen remains a seminal work in multimodal discourse analysis, offering a detailed framework for understanding the grammar of visual design and its role in communication.

2. Jewitt (2009): Multimodal Aspects of Classroom Interaction and Digital Media

Carey Jewitt is a prominent scholar in the field of multimodal discourse analysis, and her work in 2009 made significant contributions to the understanding of multimodality in education and digital media.

1) Key Contributions of Jewitt (2009):

  1. Multimodal Classroom Interaction:
    • Jewitt’s research explores how different modes (e.g., speech, writing, gestures, visual aids) are used together in classroom settings.
    • She analyzes how teachers and students use multimodal resources to facilitate teaching and learning, such as how visual aids and gestures can reinforce verbal instruction.
  2. Digital Media in Education:
    • Examines how digital tools like interactive whiteboards, tablets, and educational software introduce new multimodal elements into classroom communication.
    • Explores the impact of digital media on how students interact with content and how teachers adapt their teaching strategies.
  3. Multimodal Literacy:
    • Investigates the importance of multimodal literacy in education, emphasizing the need for students to develop skills in interpreting and creating multimodal texts.
    • Highlights the increasing role of digital media in shaping how students understand and produce communication.
  4. Changing Modes of Representation:
    • Analyzes how new digital tools and media are changing traditional modes of representation in education.
    • Examines how technology enables new ways of organizing and presenting information, such as hypertext and multimedia presentations.
  5. Teacher and Student Roles:
    • Studies the changing roles of teachers and students in a digital, multimodal environment.
    • Explores how teachers can leverage multimodal resources to engage students and how students navigate these resources to support their learning.

2) Applications of Jewitt’s Work:

  1. Educational Policy and Curriculum Design: Insights from Jewitt’s research inform policies and curricula that emphasize multimodal literacy and the integration of digital tools in education.
  2. Teacher Training: Provides frameworks for teacher training programs to prepare educators to effectively use multimodal resources in their teaching.
  3. Instructional Design: Informs the design of educational materials that incorporate multimodal elements to enhance learning.
  4. Educational Technology: Guides the development of educational technology that aligns with best practices in multimodal teaching and learning.

3) Impact of Jewitt’s Research:

  • Multimodal Pedagogy: Jewitt’s work has influenced how educators understand and incorporate multimodality in teaching practices.
  • Digital Literacy: Emphasizes the importance of digital literacy as an integral part of modern education, shaping how educators and policymakers approach digital tools.
  • Comprehensive Framework: Offers a detailed framework for analyzing the use of multimodal resources in educational settings, providing valuable insights for educators and researchers alike.

Carey Jewitt’s 2009 research has significantly advanced the understanding of multimodal discourse in education, highlighting the impact of digital media on teaching and learning.

3. Machin and Mayr (2012): Investigating Visual and Textual Modes in News Media

David Machin and Andrea Mayr’s book “How to Do Critical Discourse Analysis: A Multimodal Introduction,” published in 2012, is a notable work in the field of multimodal discourse analysis, particularly for its focus on the interplay between visual and textual modes in news media. Their research integrates critical discourse analysis with multimodal analysis to uncover how media constructs meaning.

1) Key Contributions of Machin and Mayr (2012):

  1. Multimodal Critical Discourse Analysis (MCDA):
    • Introduces a framework that combines critical discourse analysis and multimodal analysis.
    • Focuses on the power dynamics and ideologies conveyed through the interplay of text, images, and other modes.
  2. Analyzing Visual and Textual Interaction:
    • Provides tools for analyzing how visual and textual elements interact in news media to construct narratives.
    • Explores how visuals can complement or contradict textual information, shaping the viewer’s interpretation.
  3. Representation of Social Actors:
    • Investigates how media visually and textually represents different social actors, such as politicians, protestors, and marginalized groups.
    • Analyzes how visual elements like body language, facial expressions, and dress contribute to framing these actors.
  4. Language and Power:
    • Examines the language used in news media to shape public opinion and reinforce or challenge power structures.
    • Analyzes linguistic choices like vocabulary, metaphors, and framing to understand their ideological implications.
  5. Framing of Issues:
    • Investigates how news media frames different social and political issues through the interaction of text and visuals.
    • Highlights how the framing of issues can influence public perception and discourse.

2) Applications of Machin and Mayr’s Work:

  1. Media Literacy: Provides insights for developing media literacy education, helping individuals critically analyze media content.
  2. Journalism and Media Studies: Informs journalism and media studies by providing a framework to analyze the construction of narratives in news media.
  3. Political Communication: Helps understand how political messages are conveyed and shaped through media, informing strategies for political communication.
  4. Advertising and Public Relations: Offers a framework for analyzing how visual and textual elements in advertising shape consumer perceptions.

3) Impact of Machin and Mayr’s Research:

  • Comprehensive Analytical Framework: Their work provides a robust framework for analyzing the multimodal construction of meaning in media.
  • Critical Analysis of Media: Highlights the importance of critical analysis in understanding the power dynamics and ideologies embedded in media communication.
  • Intersection of Text and Visuals: Offers valuable insights into how visual and textual modes interact to influence public perception.

Machin and Mayr’s 2012 research provides a comprehensive guide for analyzing the multimodal construction of meaning in news media, emphasizing the importance of understanding visual and textual interactions in shaping public discourse.

4. Norris (2004): Analyzing Multimodal Interactions in Everyday Social Encounters

Sigrid Norris’s work, particularly her 2004 book “Analyzing Multimodal Interaction: A Methodological Framework,” is a foundational contribution to the field of multimodal discourse analysis. Her research provides detailed frameworks and methodologies for understanding the multimodal nature of everyday interactions.

1) Key Contributions of Norris (2004):

  1. Multimodal Interaction Analysis:
    • Norris introduces Multimodal Interaction Analysis (MIA), a framework for analyzing the interplay of different modes in everyday communication.
    • Focuses on how modes like gesture, gaze, posture, and spatial arrangement interact with spoken language to create meaning.
  2. Embodied Communication:
    • Emphasizes the importance of embodiment in communication, exploring how body movements and gestures contribute to meaning-making.
    • Analyzes how nonverbal modes interact with speech to enhance or modify verbal messages.
  3. Multimodal Density:
    • Introduces the concept of multimodal density, which refers to the concentration of different modes in communication.
    • High multimodal density implies a rich interaction of modes, while low density indicates less modal interaction.
  4. Interaction Order:
    • Norris discusses how the interaction order, or the sequence and structure of interactions, influences the interpretation of meaning.
    • Analyzes how modes shift dynamically in their roles and importance during interactions.
  5. Social Actors and Higher-Level Actions:
    • Analyzes how social actors use a series of actions (e.g., gaze, posture, speech) to perform higher-level actions that convey intentions or achieve goals.
    • Examines how these actions work together in face-to-face communication.

2) Applications of Norris’s Work:

  1. Social Interaction Studies: Provides frameworks for understanding how people use multiple modes in everyday interactions, from casual conversations to formal meetings.
  2. Educational Contexts: Offers insights into how teachers and students use multimodal communication in classrooms, including gestures and spatial arrangements.
  3. Healthcare Communication: Helps analyze doctor-patient interactions to understand how gestures, facial expressions, and spatial arrangements affect communication.
  4. Business Communication: Provides tools for analyzing how body language, gaze, and spoken language interact in business meetings and negotiations.

3) Impact of Norris’s Research:

  • Detailed Methodological Framework: Norris’s book provides a step-by-step guide for conducting multimodal interaction analysis, making it accessible for researchers.
  • Emphasis on Embodied Interaction: Highlights the importance of understanding how nonverbal modes like gestures and gaze contribute to communication.
  • Insight into Face-to-Face Communication: Offers valuable insights into the complexity of face-to-face communication, revealing the interplay of different modes.

Sigrid Norris’s 2004 work has been influential in shaping how researchers understand and analyze the multimodal nature of everyday social interactions. Her methodologies continue to provide a robust foundation for studying the intricacies of multimodal communication.

5. Scollon and Scollon (2003): Introducing ‘Geosemiotics’ for Analyzing Multimodal Signs in Public Spaces

Ron Scollon and Suzanne Wong Scollon’s 2003 book, “Discourses in Place: Language in the Material World,” introduced the concept of geosemiotics, which provides a framework for analyzing multimodal signs in public spaces. Geosemiotics emphasizes the interaction between language, visual design, and the physical environment, revealing how meaning is created in specific locations.

1) Key Contributions of Scollon and Scollon (2003):

  1. Geosemiotics Framework:
    • Introduces geosemiotics, a framework for understanding how meaning is constructed through the interaction of language, visual design, and spatial positioning.
    • Focuses on how signs and texts in public spaces interact with their physical surroundings to convey meaning.
  2. Indexicality:
    • Emphasizes the importance of indexicality, or the relationship between a sign and its physical context.
    • Analyzes how signs point to specific places, ideas, or entities through their location and design.
  3. Interaction Order:
    • Describes how the interaction order, or the social norms governing behavior in specific spaces, affects the interpretation of signs.
    • Explores how signs in different settings convey meaning based on the social interactions they are part of.
  4. Visual Semiotics:
    • Analyzes the visual elements of signs, such as typography, color, and layout, to understand how they contribute to meaning-making.
    • Explores how visual design choices align with or challenge cultural conventions.
  5. Place Semiotics:
    • Examines the role of place in shaping the meaning of signs, considering how physical location influences interpretation.
    • Investigates how the environment (e.g., streets, buildings) affects the way signs are read and understood.

2) Applications of Scollon and Scollon’s Work:

  1. Linguistic Landscape Analysis: Provides a framework for analyzing the linguistic landscapes of cities, understanding how multilingual signs reflect social dynamics.
  2. Urban Planning and Architecture: Informs urban planning and architectural design by analyzing how signs and symbols interact with urban spaces.
  3. Tourism Studies: Examines how signs in tourist areas are designed to appeal to international visitors and convey specific messages.
  4. Marketing and Advertising: Analyzes how outdoor advertising uses geosemiotics to target specific audiences through strategic placement and design.

3) Impact of Scollon and Scollon’s Research:

  • Novel Analytical Framework: Their work introduces a new analytical framework that integrates the study of language, visual design, and spatial context.
  • Emphasis on Contextual Meaning: Highlights the importance of physical and social context in shaping the interpretation of signs.
  • Interdisciplinary Approach: Combines insights from linguistics, semiotics, urban studies, and visual communication.

Scollon and Scollon’s 2003 work, “Discourses in Place,” provides a comprehensive approach to understanding the multimodal dimensions of public signage and how it interacts with its environment to create meaning.


These studies have shaped our understanding of multimodal discourse, offering frameworks to analyze visual communication, classroom interactions, media, and public signs. Each contributes to a nuanced understanding of how different modes interplay to shape meaning.

Frequently Asked Questions.

What did Kress and van Leeuwen contribute to the field of multimodal discourse analysis with their work “Reading Images”?

In “Reading Images” (2006), Kress and van Leeuwen developed a framework for analyzing visual communication, focusing on how images in media convey meaning. They introduced concepts like “visual grammar,” which help understand how images structure meaning similarly to language.

How did Jewitt’s research in 2009 expand the understanding of multimodal communication?

Carey Jewitt explored the multimodal aspects of classroom interaction and digital media. Her research highlighted how different modes interact in educational contexts, providing insights into how teachers and students use various semiotic resources for communication and learning.

What are the key findings from Machin and Mayr’s (2012) study on multimodality in news media?

Machin and Mayr investigated how visual and textual modes interact in news media, showing that images and text often work together to convey specific ideological messages. Their work reveals how news media uses multimodal communication to shape public perception and reinforce narratives.

What insights does Norris’s (2004) research provide into everyday multimodal interactions?

Sigrid Norris analyzed everyday social encounters to understand how multiple modes like gestures, gaze, and spatial arrangements contribute to communication. Her work offers insights into the complexity of everyday interactions and how meaning is created through the interplay of different modes.

What is ‘geosemiotics,’ and how do Scollon and Scollon (2003) use it in their research?

‘Geosemiotics’ is a term introduced by Scollon and Scollon that refers to the study of the semiotic aspects of the physical world, particularly signs and symbols in public spaces. Their work analyzes how these signs contribute to communication in the context of their physical location, revealing the role of space in meaning-making.

How does “Reading Images” contribute to understanding visual communication?

“Reading Images” provides a structured approach to analyzing the visual aspects of communication. Kress and van Leeuwen’s framework helps decode how images create meaning, similar to how sentences do in language, using elements like composition, color, and framing.

What methodologies did Jewitt use to analyze multimodal classroom interactions?

Jewitt used video recordings of classroom interactions and digital media to analyze the use of different modes in teaching and learning. She applied multimodal transcription to capture how gestures, speech, and digital tools interact in educational contexts.

How do Machin and Mayr explain the interaction between text and images in news media?

Machin and Mayr explain that text and images in news media often complement each other to create a cohesive message. They show how this interplay can reinforce specific narratives or ideologies, influencing how news stories are perceived.

What distinguishes Norris’s approach to studying multimodal interactions?

Norris’s approach is distinguished by its focus on real-life social encounters and its detailed analysis of non-verbal modes like gestures and spatial arrangements. She emphasizes the simultaneous use of multiple modes in everyday communication.

Why is ‘geosemiotics’ significant in multimodal discourse analysis?

‘Geosemiotics’ is significant because it incorporates the spatial dimension into the analysis of multimodal communication. It shows how signs and symbols interact with their physical surroundings to create meaning, which is essential for understanding public communication.

How did Kress and van Leeuwen’s work change the way visual communication is analyzed?

Their work introduced a systematic approach to analyzing visual communication that treats images as structured similarly to language. This approach helped formalize the study of visual grammar, enabling deeper insights into how visuals convey meaning.

What implications do Jewitt’s findings have for digital literacy in education?

Jewitt’s findings emphasize the importance of understanding digital literacy, as she shows how digital tools and media shape classroom interactions. Her work suggests that educators need to be aware of the multimodal nature of communication to effectively engage students.

How can Machin and Mayr’s analysis be applied to contemporary news media?

Their analysis can be applied to contemporary news media by examining how current news outlets use images and text to create specific narratives. It can reveal how news stories are framed to influence public perception and understanding.

What are some practical applications of Norris’s research on multimodal interactions?

Practical applications include enhancing communication strategies in interpersonal and professional settings by understanding how non-verbal cues contribute to effective communication. It also provides insights into improving teaching methods by utilizing various modes of interaction.

How do Scollon and Scollon link geosemiotics to cultural context?

Scollon and Scollon link geosemiotics to cultural context by analyzing how signs and symbols in public spaces reflect cultural norms and values. They show how these signs are interpreted differently depending on their cultural and spatial contexts.

Leave a Reply

Your email address will not be published. Required fields are marked *