Article Citation: Nneka Rachel Igbeneghu (2024). The Communicative Functions of the Image Act in Advertisements. DEGEL: The Journal of the Faculty of Arts and Islamic Studies, Vol. 21, No. 1. ISSN 0794-9316
THE
COMMUNICATIVE FUNCTIONS OF THE IMAGE ACT IN ADVERTISEMENTS
By
Nneka Rachel Igbeneghu
The unwritten
interaction established when people look at one another, either directly or
indirectly, in real-life situations is also present in the viewer/image
relationship in visual communication. Interpersonal interaction in visual
communication involves a represented participant (image or human) and an
interactive participant (viewer). The interactions among people, places, and
objects represented in images have been studied by many scholars from diverse
perspectives, including semiotics, cognitive approaches, visual literacy,
text-image relations, multimodal discourse analysis, art history, and aesthetic
composition. However, little attention has been given to the concept of the
“image act,” which refers to the interpersonal interaction that occurs when
individuals engage visually with one another or with images in visual
communication. This study adopts Kress and van Leeuwen's (2006) social semiotic
framework, The Grammar of Visual Design, which is grounded in Hallidayan
metafunctions within Systemic Functional Linguistics, to examine the meanings
that emerge from the viewer/image relationship in selected German magazine
advertisements. The aim of the study is to uncover the types of meanings
generated through viewer/image interpersonal interaction in specific social
contexts and to identify the strategies advertisers in the selected German
magazines employ in the placement of their advertisements. The findings reveal
that visual contact with images establishes a form of interaction that
influences the viewer. The study further demonstrates that images perform two
communicative functions through gaze direction: ‘demand’ and ‘offer’. A direct
gaze at the viewer constitutes a ‘demand’, while the absence of such gaze is
categorized as an ‘offer’. Notably, the ‘offer’ function is more prevalent in
the selected German magazine advertisements. The study concludes that exploring
the image act fosters critical thinking and supports diverse meaning-making
processes in the teaching and learning of foreign languages.
Keywords: image,
viewer, visual, gaze, interaction, multimodality, advertisement.
Abstrakt
Die
ungeschriebene Interaktion, die entsteht, wenn Menschen sich im wirklichen
Leben direkt oder indirekt ansehen, ist auch in der Betrachter-Bild-Beziehung
in der visuellen Kommunikation erreichbar. Die interpersonelle Interaktion in
der visuellen Kommunikation umfasst einen dargestellten Teilnehmer (Bild oder
Mensch) und einen interaktiven Teilnehmer (Betrachter). Die Wechselwirkungen
zwischen Menschen, Orten und Dingen, die in Bildern dargestellt werden, wurden
von vielen Wissenschaftlern aus verschiedenen Perspektiven untersucht, nämlich
(Semiotik und Kognition, visuelle Kompetenz, Text-Bild-Beziehungen, multimodale
Diskursanalyse, Kunstgeschichte, ästhetische Komposition usw.). Dem ‘Bildakt’,
d. h. der zwischenmenschlichen Interaktion, die stattfindet, wenn Menschen sich
gegenseitig oder Bilder in der visuellen Kommunikation betrachten, wurde jedoch
wenig Aufmerksamkeit geschenkt. Diese Studie verwendet das sozialsemiotische
Rahmenwerk von Kress & Van Leuween, die Grammatik des visuellen Designs (2006), basierend auf
hallidayanischen Metafunktionen in der systemischen funktionalen Linguistik, um
die Bedeutungen zu untersuchen, die von der Betrachter-Bild-Beziehung in
ausgewählten deutschen Zeitschriftenanzeigen ausgehen. Das Ziel dieser Studie ist
es, die Art von Bedeutungen aufzudecken, die in der interpersonellen
Interaktion zwischen Betrachter und Bild in einem gegebenen sozialen Kontext
produziert werden. Und zeigen auch Strategien, die Werbetreibende in
ausgewählten deutschen Zeitschriften bei der Platzierung ihrer Anzeigen
anwenden. Die Studie zeigt, dass beim Betrachten von Bildern ein Kontakt
hergestellt wird, der den Betrachter beeinflusst, sie zeigt auch, dass Bilder
durch ihre Blickrichtung zwei Funktionen erfüllen können, nämlich: ‘Nachfrage‘
und ‘Angebot‘. Ein direkter Blick auf den Betrachter (Nachfrage) oder nicht
(Angebot). Das Angebot ist jedoch das am häufigsten verwendete Muster in den
ausgewählten deutschen Zeitschriftenanzeigen. Die Studie kommt zu dem Schluss,
dass die Auseinandersetzung mit dem Bildakt kritisches Denken anregt und
unterschiedliche Sinnbildungsprozesse im Fremdsprachenunterricht und -lernen
fördert.
Schlüsselwörter: Bild, Betrachter, visuell,
Blick, Interaktion, Multimodalität, Werbung.
Introduction
Visual resources such
as images, gestures, and written texts are considered from a functionalist
perspective in visual communication, similar to any other semiotic resource,
due to their ability to perform multiple metafunctions simultaneously in the
process of meaning-making. Images are not merely tools for aesthetic or
expressive purposes; they also embody structured social, political, and
communicative dimensions. The growing body of evidence on the significance of
visual communication, coupled with the problematic absence of adequate
frameworks for discussing and conceptualizing what is actually communicated
through images and visual design, has necessitated a more rigorous
investigation of visual forms of communication alongside linguistic forms. This
reflects a shift away from earlier misconceptions that meaning can only be
constructed through linguistic means. This position aligns with Roland Barthes’
discussion in his essay Rhetoric of the Image (1977: 32–51). Barthes
(1967a) argues that the meaning of images, and other semiotic codes such as
dress and food, is always related to, and in some sense dependent on, verbal
texts (Kress & van Leeuwen, 2006: 18). He further claims that images are inherently
polysemous, open to multiple interpretations, and therefore require the
anchoring function of language to achieve a fixed meaning. He describes visual
meaning as a "floating chain of signifieds" that can only be
stabilized through the linguistic message (ibid.). However, Barthes’
proposition overlooks a crucial point: the visual component of a text
constitutes an independently organized and structured message. While it may be
connected to the verbal component, it is by no means dependent on it, and the
same holds true in reverse (ibid.). Traditionally, linguistic grammar has been
studied in isolation from meaning. However, Halliday (1985: 101) posits that
grammar goes beyond formal rules of correctness; it is a means of representing
patterns of experience. It enables individuals to construct a mental
representation of reality and to make sense of their experiences both in the
external world and within themselves. Grammatical structures thus serve as
resources for encoding interpretations of experience and patterns of social
interaction. The same principle applies to visual communication. Just as
linguistic structures influence interpretation, visual structures also guide
the viewer toward specific meanings and social relations. For instance, while
language may convey distinctions through the choice of word classes and clause
structures, visual communication may express similar distinctions through the
use of colour, framing, or compositional arrangements, all of which affect the
meaning constructed by the viewer (Kress & van Leeuwen, 2006: 18). This study employs the Kress
& Van Leuween Grammar of Visual Design (2006) to examines the viewer/image relation and the meanings that are established
as a result of their interaction in a specific social context. Exploring
meanings in images exposes foreign language learners to different ways of
meaning-making other than linguistic means. It affords learners of foreign
languages the opportunity of decoding meaning through multiple means. This
enhances the cognitive ability of a learner. Hence, the cognitive ability is
ignited through critical thinking. In line with this, this study will help in
teaching and learning of German and other foreign languages.
Multimodality
and Multimodal Texts
Multimodality is a
concept that has been introduced and developed over the past two decades to
account for the diverse resources employed in communication to convey meaning.
The term serves both as a descriptor of a communicative phenomenon and as the
designation for a diversified and expanding field of research. As a
communicative phenomenon, multimodality refers to the integration of different
semiotic resources, or modes, within texts and communicative events, including
still and moving images, speech, writing, layout, gesture, and proxemics
(Adami, 2016: 3). Historically, the dominant role assigned to verbal language,
particularly written language, has overshadowed the multiplicity of semiotic
resources that societies have socially developed for communication. This
dominance not only reflects the advanced level of articulation achieved in
speech and writing, but also underscores the disproportionate focus of research
and education on describing, prescribing, and standardizing language use.
Consequently, the study of other modes has largely remained confined to
specialized disciplines such as musicology and the arts (Adami, 2016: 3).
However, in recent years, the social impact of
digital technologies for text production, among other factors, has made more
visible the fact that texts are multimodal and hence that language alone cannot
be sufficient to explain meaning made through them. Digital technologies have
reduced costs for the production of printed images and the use of colour. Their
‘(market-led) widespread use has made available to an unprecedented number of
sign-makers forms of text production that afford modes other than speech and
writing’ (4). Online environments have provided sign-makers with platforms and
easy-to-use interfaces for publishing their multimodal texts and distributing
them to diversified audiences, thus making the phenomenon of multimodality
visible to an unprecedented extent. The digital texts we daily engage with make
meaning through the combined use of colour, writing, sound, images, and layout,
at least. It is not only the case of texts that we encounter on the web, but
also of ‘texts’ that we interact with daily, to fulfil ordinary tasks in our
offline environments, such as the ‘interfaces displayed on the screens of ATM
machines or those for purchasing a train ticket, for instance’ (5). This holds
also for the texts that we produce; everyday communication in digital
environments confronts sign-makers with a wide range of modal options. The
multimodal character of digital texts is also redefining the use of the
resources of language (5); writing itself is changing its functions, as lexis
integrated in visual ensembles/syntagms (6), or as something to be acted upon
rather than read, as in the case of URLs used as hyperlinks (Adami, 2015);
writing is also increasingly developing resources for meaning-making, like
those of font (7), which are generally disregarded in linguistic studies.
Multimodality
in Advertising Discourse
Advertising is a
communicative act aimed at capturing the audience’s attention, whether towards
a product, idea, or service. To achieve this, advertisers and marketers employ
various tools to enhance persuasive appeal and sustain the audience's engagement.
Print advertising, in particular, exemplifies the diverse roles that discourse
can play in society. It demonstrates how people communicate and interact within
the various spheres of their daily lives (Simoes & Freitas, 2012: 427).
While advertising has existed since the nineteenth century, its advancement has
paralleled broader social and technological developments. Contemporary
advertising now employs multiple modes that go beyond language, incorporating
visual elements such as photographs, graphics, cartoons, and other imagery.
The emergence of a
trend toward visual representation of information, previously conveyed
primarily through language, and the increased accessibility of technologies
enabling new methodological approaches to multimodal communication have
facilitated the growing use of multimodal strategies in advertising discourse
(O'Halloran, 2011: 122). Multimodality refers to the various ways distinct
semiotic modes, such as language, image, music, sound, and animation, are
deployed together and contextually integrated within a communicative event to
produce meaning. Advertising discourse represents a significant domain for the
application of multimodal discourse, as it is fundamentally geared towards
human interaction. The rise of new discourse forms and the increasingly
multimodal nature of contemporary communication have enabled the integration of
textual and visual elements within structured layouts. A mode is defined as “a
socially shaped and culturally given resource for making meaning” (Kress, 2009:
54). The combination of multiple modes, such as language, image, music, and
video, in a single communicative act is referred to as multimodal
communication. This relationship concerns "all modes used in communication
and the way they are represented" (El-Sayid, 2018: 544). In advertising
discourse, multimodality refers to how advertisers employ various communicative
modes to convey messages and persuade consumers. The meaning-making potential
and affordances of different modes necessitate their integration to enhance
communicative effectiveness.
Literature Review (Conceptual
Review)
Visual communication conveys information and ideas
visually by visual means. Along with its various types, it has been studied
from various perspectives, such as: visual literacy (Felten 2008); social
semiotics dimensions (Stoian, 2015, Jewitt & Oyama, 2001); semiotics and
cognitive perspective (Wong 2018); visual as a thinking tool (Lazo 2012).
Stoian (2015) explores the theories and models of visual social semiotics from
Kress & van Leeuwen’s Reading Images: The Grammar of Visual
Design (1996, 2006). He notes that Halliday’s metafunctions of language
have been extended to visual communication and are categorize as
representational, interactive and compositional. Stoian (2005:29) highlights
some domains where visual analysis can be applied such as ‘promotion, websites,
teaching material, printings, fairy tales etc’. Wong (2018) examines visual
image analysis from semiotics and cognitive perspectives which he considers as
new ways of analyzing and interpreting images. The study notes that ‘visual-analytic
tools suggested by the social semiotic approach perfectly complements the
inward cognition of an image-viewer, a synergy which has rarely been envisaged
by scholars from both disciplines’ (39). In other words, visual analysis is
seen as both semiotically and cognitively relevant. He suggests that knowing
the cognitive processes required to fully comprehend visual structures is
necessary in visual communication. The social semiotic perspective of visual
communication involves ‘the description of semiotic resources, what can be said
and done with images (and other visual means of communication) and how the
things people say and do with images can be interpreted’ (Jewitt & Oyama
2001: 134). Visual resources are, thus, considered from a functionalist
approach; like any semiotic resource, they perform several metafunctions
simultaneously in order to convey meaning. Felten (2008) states in his work
titled ‘Visual Literacy’ that the new ‘pictorial turn’ means that images no
longer exist primarily to entertain and illustrate (1). Rather they are
becoming central to communication and meaning-making. In other words, visual
literacy involves the ability to understand, produce, and use culturally
significant images, objects, and visible actions. This ends the centuries-long
domination of texts and words in culture observations, because they always
portray meaning (Leavy, 2009; Sturken & Cartwright, 2009 in Lazo 2012:8).
The importance of images ‘lies in their influencing power and their high
resonance on audience’s emotions, which enables their potential to generate an
extraordinary impact on population’ (Lazo, 2012:7). The way an image is
interpreted is never fixed. If the creator does not take into account the
reference frame of the spectator it is very possible that communication will
not occur. The designer makes the image with a special intention. If the
spectator makes the right interpretation that depends on the correct use of
visual elements. Leavy (2009 cf Lazo, 2012:8) explains that the persuasive
power of images, via art works, advertisements, photographs, and digital
manipulations, relies on their descriptive and evocative power which has been
used to promote beliefs. This author argues that the evocative power of images
lies in the fact that they are profoundly rooted in an individual’s memory;
that images can remain and recall emotions, thus are used as vehicles to
influence.
El- Sayed (2018) investigates Text-Image Relations
in Print Advertisement within the framework of Kress and Van Leeuwen’s
multimodal discourse analysis, drawing upon Hallidayan Systemic Functional
Linguistics, particularly Martinec & Salway's system, two analytical
approaches were applied to the study, namely, Martinec & Salway's system of
text-image relations and Cheong's Generic Structure Potential (GSP). It focuses
on the semantic relations that hold between the text and image in
advertisement. The study examined the integration of the textual and visual
modes and the ways they contribute to the meaning-making process in the genre.
It was demonstrated that imagery and text are integrated and interrelated and
both serve the advertisers' overall intended meaning. Likewise, both facilitate
orientation and processing of the advertisement. It concludes that the visual
and linguistic elements of the advertisements support and complement each other
to get across the intended idea to the viewer.
Hai Ly & Kwan Jung (2015) carried out a
Multimodal Discourse analysis of the Visual Design of Two Advertising Images
within Kress & Van Leeuwen (2006) social semiotic principles. The Grammar
of Visual design of two digital images were examined focusing on their
representational and interactive dimensions in relation to the magazine
advertisement genre, with the aim of revealing the meaning making potentials
embedded in the images. The results show that the framework identified
narrative and conceptual processes, relations between participants and viewers,
and symbolic attributes of the images, which all contribute to the sociological
interpretations of the images. The identities and relationships between viewers
and participants suggested in the images signify desirable qualities that may
be associated to the product of the advertiser. The study highlighted the
potential of images to convey multi-layered meanings. This paper argues that
like language, images are rich in meaning potential and are governed by visual
grammar structures which can be utilized to decode the meanings of images. It
concludes that further development in visual semiotic theory, images, as a
semiotic resource, may have the potential to take a more prominent role in
communication.
The Interactions in Visual
Communication
The visual
communication on its own has resources that enhance interaction between the
image-producer and the viewer. The visual communication involves two
participants which Kress and Van Leeuwen (2006:114) termed ‘the represented and
the interactive participants’. The represented participants constitute ‘the
people, the places and the things depicted in images and the interactive
participants are the people who communicate with each other through images, the
producers and viewers of images. Kress and Van Leeuwen note that three kinds of
relations exist between these two participants (the represented and
interactive). The first relation exists between the ‘represented participants’.
The second relation exists between the ‘interactive and represented
participants’. This expresses the attitude of the interactive participants
towards the represented participants. The third relation is between the
‘interactive participants’, this expresses ‘the things the interactive
participants do to or for each other through images’ (114). The interactive
participants are the actual people who ‘produce and make sense of images in the
context of social institutions which, to different degrees and in different
ways regulate what may be ‘said’ with images, how it should be said, and how it
should be interpreted’ (114). Most times, the interaction is often ‘direct and
immediate; producer and viewer know each other and are involved in face-to-face
interaction’ (ibid). This face-to-face interaction could be seen in the
processes of taking photograph of each other, drawing maps to give directions
and engaging in explaining ideas through diagrams.
However,
indirect interaction also occurs where there is no immediate engagement of the
producer and the viewer. For instance, the photographs that often appear on the
pages of magazines, newspapers, books etc. The only thing we see is the
photographs and not all the people that are involved in the production like the
producers, the photographers that took the shot, the assistant who processed
and printed them, the agency that selected and distributed them, the picture
editor who chooses it, the layout artist who cropped it and determine its size
and position on the page. The way most viewers will not meet the people that
contributed in the production process but may have a limited idea of the
production process of the image. The same manner the producers can never
actually know their audiences, rather they create a mental image of ‘the
viewers’ and ‘the way’ viewers make sense of their pictures (114).
The
interactive participants and represented participants are always recognizable
in every direct communication since there is always a producer and a viewer.
But when there is a disconnection between the context of production and the
context of reception, when the producer is not physically present and the
viewer is alone with the image. This could be seen in writing. ‘Writers too are
not usually physically present when their words are read and must address their
readers in the guise of represented participants. Readers too are alone with
the written word, and cannot usually become writers in turn’ (Kress & Van
Leeuwen, 2006:114-115). In other words, despite the disconnection between the
context of production and the context of reception, there still exist common
features between the two, which are the ’image itself, and knowledge of the
communicative resources’ (115) which enhances its expression and clarity; and
the manner social interactions and relations can be represented in images. It
is a known fact that the producer and the viewer possess different forms of
knowledge. The producer is considered to be more active by engaging in the
‘sending’ as well as the ‘receiving’ of ‘messages’ (115). That is, the
producers are able to ‘write’ as the viewers are able only to ‘read’ (115).
However,
Kress and Van Leeuwen insist that attempts to make the knowledge explicit will
show that the interactive meanings are visually encoded in ways that rest on
competencies shared by producers and viewers (2006:114). This stresses the fact
that understanding and decoding of meanings that are embedded in visual
resources (e.g. Images) rest greatly on the acquisition and exhibition of
knowledge of the ways and manners social interactions and relations are encoded
in any given visual resources’ (114). Hence, our ability to explain and
understand the embedded meanings in images is enhanced by our knowledge of the
way social interactions and relations are represented through images.
Theoretical Framework and
Methodology
Halliday’s theory of metafunctions (1994; 2004;
Halliday & Matthiessen, 2004), which postulates that language fulfils three
metafunctions simultaneously, namely ideational, interpersonal and textual
metafunctions have been extended to visual communication by Kress and van
Leeuwen (2006) grammar of visual design which presents the theory and
vocabulary needed to talk about and read images. They assert that ‘the visual,
like all semiotic modes, has to serve several communicational (and
representational) requirements, in order to function as a full system of
communication’ (2006: 41). The metafunctions are renamed (i) representational
which has to do with the patterns of representation, in other words, the way
experience is encoded visually. (ii) interactive is represented by the patterns
of interactions between participants. Several types of participants are
involved in visual communication, namely represented (depicted) and interactive
(real). They can lead to various relations within the same types and between
the different types (Kress & van Leeuwen 2006: 114). And (iii)
compositional functions being related to ‘the way in which representations and
communicative acts cohere into the kind of meaningful whole we call ‘text’
(Kress & van Leeuwen 1996: 181). Meaning is built by three interrelated
systems: information value, salience and framing. In addition, these systems
are not restricted to single pictures, but they apply also to composite visuals,
i.e. visuals, which combine text, image and/or other graphic elements, and
their layouts. The reality of the metafunctions is embedded visually in images
as ‘types of structure, process, participant and circumstance; the types of
image act, social distance, perspective, point of view, modality; and the types
of information value, salience and framing’ (29). The interaction and relation
established between participants; and the meaning composed are decoded
visually. These meaning-making metafunctions and visual resources cannot only
be described and observed, but also interpreted. The complexity created by the
description and interpretation of images and/or any other type of visuals
raises interest in various fields where visual communication is thoroughly used.
This model proposed by Kress and van Leeuwen is applicable to domains, such as
‘promotion, websites, teaching material, paintings, fairy tales, spatial
arrangements of buildings, tourism and/or cultural aspects of non-verbal
communication’ (Stoian, 2015:29).
The
Interpersonal Metafunction (Interactive Metafunction)
Base on Halliday’s interpersonal Metafunction, the
study examines image/viewer interaction towards communicative acts in a
multimodal text. In verbal communication interpersonal Metafunction manifests
when a speaker in the act of speaking adopts for himself a particular speech
role, and in so doing assigns to the listener a complementary role which he
wishes him to adopt in his turn. For example, in asking a question, a speaker
is taking on the role of seeking information and requiring the listener to take
on the role of supplying the information demanded. The interpersonal meaning in
language is extended to interactive meaning in image which establishes a kind
of (imaginary) relationship between the producer, image and the viewer of the
image. The interactive metafunctions is represented by the patterns of
interactions between participants. Several types of participants are involved
in visual communication, namely represented (depicted) and interactive (real).
They can lead to various relations within the same types and between the
different types (Kress & van Leeuwen 2006: 114).
The interactive meaning of images has three
dimensions: image act, social distance and point of view. The image act is
related to the gaze direction of the represented participants, which can be
directed at the viewer (demand) or not (offer). Demands are considered to
establish an imaginary relation with the viewer since they address him/her
directly. The act can also be emphasized by facial expressions and gestures
(Kress & van Leeuwen 2006: 116-118). It could also be a smile on the face
of the participant or a kind of dressing or position of part of the body
(Olateju et.al (2014:56). Machin (2007 in Olateju et.al. 2014) notes that ‘this
is imaginary relationship that communicates something. Unlike real life where
there will be immediate consequences, there will not be any immediate
consequences if we do not react or respond to an image act’ (56). In a situation
where a viewer wants to practically respond to the demand, if it is an
advertisement the viewer goes for the product being advertised. In images where
represented participants do not look directly at the viewer, Kress and Van
Leeuwen (2006:119) call them ‘offer’ images because they offer ‘the represented
participants to the viewer as items of information, objects of contemplation,
impersonally, as though they were specimens in a display case’. Based on
investigation of interpersonal meaning by Halliday (1994), Kress and Van
Leeuwen (1997) go into more and more detail to exploit interpersonal meaning
from a multimodal perspective ranging from image, movement/action to sound,
music and speech and the like (Baldry and P. Thibault, 2006, Van Leeuwen,
1999). A specific aspect of the interpersonal meaning in advertising discourse
is represented and conveyed through gaze. This study adopts the interactive
metafunctions of images as propose by Kress and Van Leeuwen (2006) to examine
the interaction between the represented participants and the interactive
participants in adverts from selected German magazine adverts., in view of
revealing the imaginary relationship
that exists between the participants.
Table
1: Types of Demand Relations
|
Facial Expression/Gestures |
Relationship between Represented Participant and Viewer |
|
Smile |
Asks viewers to enter relation of social affinity |
|
Cold Stare |
Asks viewer to relate to participant as an inferior |
|
Seductive Pout |
Asks viewer to desire the participant |
|
Looking up leading |
Asks viewer for pity |
|
Finger pointed at viewer
|
Grabs viewer’s attention, invite viewer to come closer |
|
Defensive gesture |
Asks viewer to stay away |
The Concept of ‘Image Act’/Gaze:
‘Demand’ and ‘Offer’ in Relation to the Concept of ‘Speech Act’
It is possible for represented participants to
involve the viewer in a certain (imaginary) action: This is called an
‘image act’ (Kress and van Leeuwen, 2006:117). An Image act can be realised by
a certain action, with which a presented participant is depicted. For example,
an image of a woman who is holding a glass of wine towards the viewer, smiling,
can be said to engage in an image act, in that she is represented to be
inviting the viewer to take part in the action: accepting the glass of wine. As
one of the formal categories which provides the medium through which
represented participants can interact with the viewer, there is eye contact
between the two parties. This is Contact and it has two kinds of
realisations: Demand and Offer. The former interactive meaning is
realised when a represented participant has eye contact with the viewer; such
as a represented participant looking at or smiling at the viewer. In line with
Halliday (1985), the participant’s gaze (and the gesture, if present) demands
something from the viewer, demands that the viewer enters into some kind of
imaginary relation with him or her. They may smile showing demand for a social
affinity relationship with a viewer. Also, a stare at the viewer with cold
disdain indicating demand perhaps as an inferior relates to a superior. They
may seductively look at the viewer in which case the viewer is asked to desire
them (Kress & Van Leuween, 2006:118). The later type of interaction is
realised when a represented participant has no direct eye contact with the
viewer and the represented participant presents him/her/itself to be ‘looked
at’ by the viewer, that is, ‘it offers the represented participants to the
viewer as items of information, objects of contemplation, impersonally, as
though they were specimens in a display case’ (Kress & Van Leuween,
2006:119).
The ‘demand’
and ‘offer’ concepts can be related to another concept in linguistics,
that is the ‘speech act’. This Halliday terms ‘speech functions’ in his
introduction to Grammar, (1985). According to Halliday (1985) each of these
speech acts is part of an interactional dyad, and has its ‘expected’ and its
‘discretionary’ (alternative) social response (Kress and Van Leeuwen,
2006:122). Kress and Van Leeuwen (1996:122) believe that images can equally
perform some ‘image acts’. In image act ‘a demand’ can be expressed through a smile
on the face of the participant, open hands for embrace, a serious unsmiling
outlook, a pointed finger, a kind of dressing, body position that could express
a visual confidence, alertness or security consciousness etc. In other words,
the ‘image acts’ and the ‘speech acts’ do not work in the same way, even though
they express some similarities. The ‘speech act’ concept is of immediate effect
and mostly applicable in face-to-face communication. It is of immediate
enactment and response, while the ‘image act’ concept does not work in the same
manner. For instance, in image act, when images express ‘offer’ they only offer
information (Kress & Van Leuween, 2006:123). In relation to
advertisement Kress and Van Leeuwen (2006:123) explicate:
In
advertising, image may show someone offering something to the viewer and this
offer may in fact be a real offer, which can be obtained by writing to an
address specified in the advertisement. But if there is such an ‘offer’ of
goods and services in images, it must take the form of an ‘offer of
information’. It must be represented. It cannot be enacted directly’
The
‘speech act’ concept is carried out directly while the ‘image act’ must be a
representation and with no direct response.
Methodology
The data
base of this study consists of one hundred and sixty-five advert posters of
visual images and their linguistic captions purposively selected from four
different German magazines in relation to the objectives and focus of the
study. These numbers were selected for a wider range capture of adverts, which
will enhance a dynamic application of the approach upon which the analysis will
be based. The data for the study covered a period of twelve months (1st
January, 2017 to 31st December, 2017). The magazines considered were: Der Stern,
Der Spiegel, Die Bunte and Bild der
Frau. They were
selected based on their on-line rating. The data were grouped into two major
categories which are human image and non-human image represented participants.
The human image represented participant is of two categories (a) Renowned
Personality Image Representation (RPIR) and Non-Renowned Personality Image
Representation (NRPIR) while the non-human represented participant is made up
of (a) Animal Image Representation (AIR), (b) Hybrid Image Representation
(HIR), (c) Abstract Image Representation (ABIR), and (d) Inanimate Image
Representation (INIR).
The study adopts a descriptive-qualitative analysis
method to explore the communicative functions of the image act in multimodal
text. The general principle of the study is drawn from researches on visual
communication, multimodal discourse analysis, interactive Metafunction
especially on image/viewer interpersonal relation in social contexts. The aim
is to reveal the kind of interpersonal meanings that could be achieved in
image/viewer interaction in multimodal text. It identifies the represented
(image, human) and interactive (viewer) participants in the selected texts and
the kinds of relationship that are suggested to exist between the viewer and
represented participants? The interpersonal meanings that are realised from
their interaction in relation to the purposes of the magazine advertisement
genre? These are based on Halliday’s interpersonal metafiction, paying
attention to the contribution of the interpersonal Metafunction to the
communicative effect and purpose of the image/viewer interaction in a given
social context. Kress
& Van Leeuwen ’s Grammar of Visual Design (2006) was adopted to
investigate the viewer/image interaction in selected German magazine adverts.
Discussion and Result
This
qualitative analysis examines the interpersonal functions in visual
communication using Kress and van Leeuwen’s social semiotic framework, which is
grounded in the interpersonal metafunction of Systemic Functional Linguistics
as developed by Halliday. The interaction between the image producer and the
viewer, referred to as interactive participants, constitutes the second
dimension of analysis proposed by Kress and van Leeuwen (2006), based on
Halliday’s (1978) interpersonal function. According to Kress and van Leeuwen,
producers visually encode social meanings into images through elements such as
the gaze of the represented participant, the perceived distance between the
participant and the viewer, and the angle from which the participant is
presented. In this study, the selected advertising images will be analysed
specifically from the perspective of the represented participant’s gaze.
The Image Act (Demand)
Just as
the speech act emphasises the use of language to carry out one function or the
other, the ‘image act’ focuses on the use of image in performing one act or the
other. In visual communication, messages are transmitted through images,
therefore the manner they are positioned (standing, seated, side by side, face
to face) with each other also passes one message or the other to the viewer.
These messages are expected to be decoded by the viewer, in order to assign
meaning to a particular image that is presented.
A direct
look by a represented participant at the viewer creates a visual form of direct
address, which Kress and Van Leeuwen (2006:118) in line with Halliday (1985)
term ‘demand’. The participant’s gaze (and gesture, if present) demands
something from the viewer, demands that the viewer enters into some kind of
imaginary relation with him or her. It may be a ‘cold disdain’ stare which may
express demand for a relation as ‘an inferior relates to superior’ (118). The
same applies to gestures, a hand can point at the viewer, in a visual ‘Hey, you
there, I mean you’ or invite the viewer to come closer, or hold the viewer at
bay with a defensive gesture, as if to say, stay away from me’. In each case
the image expects something from the viewers or wants them to do something
either to come closer, to stay at distance, or form a ‘pseudo-social’ bond of a
particular kind with the represented participant. And in doing this, images
reveal to a certain degree the identity of a particular viewer (e.g. male,
inferior to the represented participant, etc.), and in this way exclude other
viewers (118). Kress and Van Leeuwen further point out that: ‘represented
participants who look at the viewer are usually human (or animal), but not
always. The headlights of a car can be drawn as eyes looking at the viewer’
(118). For instance, a company or an organization may use an image of a
creature whose head and body has the shape of the product being advertised.
This image may be presented smiling, waving or doing any other thing to attract
customers’ attention, thus ‘demanding’ a friendly recognition with the intended
customers. Kress and Van Leeuwen (118) explain ‘whether they are human or not,
by being represented as looking at the viewer, they are represented as human,
anthropomorphized to some degree’
In other
words, whatever is being represented be it human or not, as long as it gazes at
the viewer, it serves as a human representation. The identity of the viewer
could be revealed through the use of imperative, which expresses command. For
instance, an image with a bold ‘X’ sign on a cigarette addresses smokers.
Again, images of males and females on certain kind of attire or some attire
marked with a bold ‘X’ address viewers that are indecent in their dressing.
This indicates that any form of indecent dressing is not allowed in that place.
Hence, the unconcerned viewers are exempted.
NRPIR23
The portrait displayed above shows the picture of a man, a woman, a boy
and a girl. The man is of blond hair and is putting on a white long-sleeved
shirt. The woman has a long blond hair, and she is wearing a sky-blue
long-sleeved shirt. The boy is of blond long hair and a blue shirt. The girl
also has a long blond hair and a brown dress. The background of the displayed
image is white and grey in colour. They are all smiling and looking directly at
the viewer’s angle. Judging from their appearance and position, one tends to
suspect that they are a family. They appear joyful, relaxed and fulfilled; this
could be seen in their outlook
The kind of
relation the represented participants in the picture presented above demand is
determined by their facial expression. They look directly at the viewer,
creating, thus, a visual form of direct address, thereby, establishing an
imaginary relation with the viewer using a demand act. Their gaze and smile indicate
that they seek recognition and demand the viewer to enter into a relation of social affinity and
identification with them.
Image Act
(Offer)
An indirect address by images, that is when pictures address us
indirectly depicting the viewer as the subject of the look while the
represented participant is indicated as the object of the viewer. Kress and Van
Leeuwen (2006:119) note that ‘the viewer’s role is that of an invisible
onlooker’. They categorize all images which do not contain human or quasi-human
participants looking directly at the viewer into this group. These kinds of
images they call an ‘offer’ according to Halliday (1985). ‘It ‘offers’ the
represented participants to the viewers as items of information, objects of
contemplation, impersonally, as though they were specimen in a display case’
(119). That is when there is no eye line contact between the represented
participants and the interactive participants. Diagrams, maps and charts are
most often found in the ‘offer’ context as a result of a kind of knowledge they
offer, which are ‘objective, dispassionate, ostensibly free of emotive
involvement and subjectivity that are highly valued’ (118). Those kinds of
images serve as something to be studied, a sort of case study. Serving as
‘items of information’ entails that they have facts embedded in them which the
viewers are expected to decode in order to fulfil a given communicative goal.
One of the aims of ‘offer’ image is allowing the viewer to engage in critical
thinking, extensive and intensive examination of the represented participant to
deduce the facts and ideology behind such images. However, in the process of
decoding the message embedded in a given represented participant, the
producer’s ideology and the viewer’s ideology may not go in the same direction,
because the viewer is guided by his own cultural perspective which may differ
from the producer’s cultural and ideological background and influence. Hence,
the critical thinking could be in line with the viewer’s perspective. The
choice between ‘offer’ and ‘demand’ differs and each is suitable and
appropriate to a specific context.
In some contexts, like television, news reading and the posed magazine
photograph, the ‘demand’ picture is preferred. Because these contexts require a
sense of connection between the viewers and the authority figures, celebrities
and role models they depict. In other context like film and television drama
and scientific illustration the ‘offer’ is preferred as a result of a real or
imaginary barrier that is erected between the represented participants and the
viewers. A sense of disengagement, in which the viewer must have the illusion
that the represented participants do not know they are being looked at and in
which the represented participants must pretend that they are not being watched
(Kress and Van Leeuwen, 2006:120).
In any
case, in the context of ‘demand’ there is the need for connection between the
viewer and what the represented participant depicts while in the context of
‘offer’ the reverse is the case. This means that what is acceptable in one
context may not be acceptable in the other context.
The represented participant in the displayed picture has a white phone on
his hands and smokes are seen coming out from both ears. He has all his
attention focused on the phone. Beside the human represented participant are
two phones placed on a white background with the linguistic captions NEU IN DIGITAL placed above and NEUES SPITZENMODEL below the two phones.
Relating the linguistic caption Mein
Hass auf Whatsapp: Warum Ich immer noch lieber telefoniere with image
of the represented participant. One guesses that the represented participant is
facing some challenges with the Whatsapp.
As a result of this he prefers communicating through phone call than chatting
on whatsapp. This is evident in his statement Warum
Ich immer noch lieber telefoniere. However,
the above statement and the linguistic captions on the displayed phones
contradict each other because the actual problem is not expressed. In other
words, one does not know if the problem has to do with the phone or the
WhatsApp application the represented participant is using.
The eyes of
the represented participant in Fig. 2 are directed not at the viewer but at the
phone. The represented participant becomes the object of what Kress and van
Leeuwen (2006:116) call ‘the viewer’s dispassionate scrutiny’ and the viewer
becomes an ‘invisible onlooker’ (116). As a result, Fig. 1 can be categorized
as an ‘offer’ image where the represented participant offers himself to the
viewer as an object for contemplation. Without eye contact, the viewer may see
the participant in an impersonal and detached way, perhaps as someone who is
preoccupied in his own world, which the viewer is excluded from.
The summary and individual distributions are presented in the tables
below:
Table 2 The
Summary Distribution of Image Act
|
Image Act |
Number |
Percentage |
|
Demand (Gaze) |
38 |
23 |
|
Offer |
71 |
43 |
|
Misc |
54 |
34 |
|
Total |
165 |
100 |
The above table of distribution for image act shows that the demand (Gaze) could be seen in 38
adverts which amounts to twenty-three percent (23%) of the data, while offer could be found in 71 adverts
resulting to forty-three percent (43%). Then those adverts that are neither
gaze nor offer are categorized under miscellaneous and they are 54 which
comprise of thirty four percent (34%) of the whole data. It could be observed
that the offer is the most applied pattern followed by the miscellaneous, then
the gaze pattern.
Conclusion
This study examines how interpersonal
meaning is represented and communicated through the image act by
exploring the types of meaning realised in image/viewer interactions in
advertising. According to Hai Ly and Kwan Jung (2015:55), the primary function
of advertising is to introduce a wide range of consumer goods and services to
the public. In many cases, advertisers utilise social images and constructs to
promote specific products. Therefore, it can be inferred that one of the core
functions of advertisements is to establish social relationships between the
represented participants and the viewer. The development of advertising has
progressed alongside social and technological evolution, resulting in
high-quality layout designs and the deployment of multiple modes, including
language and visual elements such as images, graphics, and other visual
resources for meaning-making. Kress and van Leeuwen (1996; 2006) have developed
a grammar of visual design, providing the theoretical framework and vocabulary
necessary for interpreting and discussing images. They extend Halliday’s
metafunctions to visual communication and classify them as representational,
interactive, and compositional. The current era of digital literacy presents
multiple opportunities that enhance the teaching and learning of foreign
languages, as it facilitates engagement through various modes of communication
and promotes the development of diverse skills. The integration of multimodal
resources and activities into the learning process fosters cognitive
development and critical thinking. It encourages learners to engage with
classroom activities through transitions between visual and verbal modes,
thereby increasing motivation, improving comprehension, and supporting the
interpretation of complex concepts. Consequently, it exposes learners to, and
familiarises them with, diverse approaches to learning.
References
Adami, E. (2016). ‘Multimodality,’ in O. Garcìa, N.
Flores and M. Spotti, Oxford Handbook of Language and
Society (1-36). Oxford: Oxford University Press.
Barthes, R. (1967). Elements of semiology. London:
Cape.
Barthes, R. (1977). Image-Music-Text. London:
Fontana.
El-Sayed, Y. (2018). Text-Image Relations in Print
Advertisement: A Multimodal Discourse Analysis. ELLS, 9(1), 544-605
Kress, G. (2009). What is mode? In C. Jewitt (Ed.), The
Routledge handbook of multimodal analysis. Abingdon: Routledge. 54-68.
Halliday, M. A. K. (1978). Language as social
semiotic: The social interpretation of language and meaning, London: Edward
Arnold.
Halliday, M. A. K. (1994, 1985) An Introduction
to Functional Grammar. London: Hodder Arnold.
Halliday,
M.A.K., & C. Matthiessen (2004) An Introduction to Functional Grammar (3rd
ed), London: Hodder Arnold.
Hai Ly& Kwan Jung (2015). Multimodal Discourse:
A Visual Design Analysis of Two Advertising Images. International Journal of
Contents, 11(2), 50-56.
Jewitt, C., & R. Oyama. (2001). Visual Meaning:
A Social Semiotic Approach’, in van Leeuwen T., and C. Jewitt (eds), Handbook
of Visual Analysis, London: Sage Publications, 134-156.
Kress, G., & T. van Leeuwen (1996, 2006). Reading
Images: The Grammar of Visual Design. London: Routledge.
Stoian, C. (2015). The Discourse of Tourism and
National Heritage: A Contrastive Study from a Cultural Perspective,
Newcastle upon Tyne: Cambridge Scholars Publishing.
Lazo, G. (2012). The Visual as a Thinking Tool.
Unpublished MA Thesis. University of Auckland New Zealand.
O'Halloran, K. L. (2011). Multimodal Discourse
Analysis. In K. Hayland and B. Paltridge (Eds.), Bloomsbury companion to
discourse analysis. London: Bloomsbury. 120-137.
Olateju, M.; Oyebode, O.; & Ademilokun, M.
(2014). Doing Multimodal Discourse Analysis. College Press and
Publishers Limited. Ibadan.
Peter, F. (2008). Visual Literacy. Change: The
Magazine of Higher Learning, 40 (6), 60-64.
Simoes, E. & Freitas, L. (2012). Advertising and
Discourse Analysis. In J. P. Gee & M. Handford (Eds.), The Routledge
handbook of discourse analysis. London and New York: Routledge. 427-440.
Stoian, E. (2015). Analysing Images: A Social
Semiotic Perspective. Scientific Bulletin of the Politehnica University of
Timişoara Transactions on Modern Languages. Vol. 14, (1), 23-30.
Wong, M. (2018) Analysing Visual Imagery: Connecting
Semiotic and Cognitive Perspectives. cognitive semantics 4 (2018),
39-75.
Internet Sources
www.frugalfamily.co.uk Accessed: 24-02-2018.
www.wordpress.com. Accessed: 24-02-2018.
www.blogger.com. Accessed: 24-02-2018.
www.stern.de. Accessed: 15-12.2017.
www.spiegel.de. Accessed: 06-12-2017.
www.bunte.de.accessed:14-04-2018.
www.bild der frau.de. accessed:19-03-2018.
0 Comments