Our former colleague, and now researcher at the university in Tromsö, Simon Devylder (on the zoom link), and I will present some of the theory behind, and more importantly - the coding scheme and some first results from the recently concluded POLYSEM project. We look forward to your feedback! NOTE: DUE TO THE FLU, WE WILL HAVE THE SEMINAR ON ZOOM ONLY

Human communication is by default polysemiotic: it involves the spontaneous combination of two or more semiotic systems, the most important ones being language, gesture and depiction. We formulate an original cognitive-semiotic framework for the analysis of polysemiosis, contrasting this with more familiar (though ambiguous) systems based on “multimodality”. To be fully explicit, we developed a coding system for the analysis of polysemiotic utterances containing speech, gesture and drawing, and implemented this in the ELAN video annotation software.  Methodologically we used the conceptual-empirical loop of cognitive semiotics. This implies that our theoretical framework guided general considerations, such as distinguishing between the “tiers” of gesture and depiction, and the three kinds of semiotic grounds (iconic, indexical, symbolic). On the other hand, the precise decisions on how to operationalize these were made only after extensive work with the material and discussions among the authors. The material involved systematic analysis of 23 video-recordings of sand drawing performances on Paama, Vanuatu and 20 sand stories of the Pitjantjatjara culture in Central Australia. We describe the coding system and provide illustrative examples from the Paamese and Pitjantjatjara data, remarking on both similarities and differences in the polysemiosis of the two cultural practices. We conclude by summarizing the contributions of the study and point to some directions for future research.