Polysemiotic communication

Language is a particular kind of semiotic system, but there are also others like gesture, depiction and music. When the combine in communication, as they commonly do, we have polysemiotic communication. Examples are when we speak and gesture, as well as use facial expressions. Or when we combine these semiotic systems with depiction, either online while drawing (on the sand, or on a whiteboard), or offline in various media genres like advertisements. This theme studies how these semiotic systems combine and interact with one another, across different cultures, and media.

Metaphor has been studied in cognitive linguistics in terms of unconscious mental processes such as  “cross-domain mappings” and “simulations” (Gibbs, 2006; Lakoff & Johnson, 1999). We accept some ideas from this work, but maintain that it confuses motivations for using metaphor, from metaphors themselves, as a particular kind of iconic (i.e. resemblance-based) sign. We have developed a theoretical model that distinguishes between the Embodied, Sedimented and Situated levels of meaning making, called the Motivation and Sedimentation Model, and apply it to metaphors in different semiotic systems.

Stampoulidis, G., Bolognesi, M., & Zlatev, J. (2019). A cognitive semiotic exploration of metaphors in Greek street art. Cognitive Semiotics, 12(1).

The term “intersemiotic translation” introduced by Jakobson’s (1959) has been influential, at the same time as it has outgrown Jakobson’s initial narrow definition as “an interpretation of verbal signs