Program
This is the detailed program including all talk and poster contributions for MMSYM24 September 25 – 27.
Please be aware that this is a tentative program and minor changes could be made in the next few weeks.
UPDATE: Unfortunately, Silva Ladewig’s talk had to be cancelled due to illness. On Friday, 27.09. we will start the first session at 09:20.
Download the pdf versions here:
A highlight of the Welcome Reception will be a string quartet performing a selection of classical music. We are delighted to welcome these artists:
Carolin Grün (Akademie des hr-Sinfonieorchesters) – Violin 1
Mixia Kang (Akademie des hr-Sinfonieorchesters) – Violin 2
Franziska Hügel (Akademie des hr-Sinfonieorchesters) – Viola
Simon Napp (Hochschule für Musik und Darstellende Kunst Frankfurt am Main) – Violoncello
We are looking forward to their performance!
Keynotes:
| Julie Hunter LinaGora Labs, Toulouse | “Situated Conversation and Conversational Cobots” |
| Judith Holler Donders Institute for Brain, Cognition & Behaviour, Nijmegen | “Producing and comprehending multimodal utterances in face-to-face conversation” |
| Petra Wagner Bielefeld University | “The multimodal expression of (non-)understanding in dyadic explanations – some lessons learned” |
Oral Presentations:
Session 1 (25.09. 11:00 – 12:20): Prosody-Gesture Interaction
| Patrick Louis Rohrer, Ronny Bujok, Lieke van Maastricht & Hans Rutger Bosker Donders Centre for Cognition, Radboud University, Nijmegen; Max Planck Institute for Psycholinguistics, Nijmegen; Centre for Language Studies, Radboud University, Nijmegen | “The timing of non-referential beat gestures affects lexical stress perception in Spanish regardless of individuals’ working memory capacity” |
| Massimo Moneglia & Giorgina Cantalini University of Florence; Civica Scuola Interpreti e Traduttori ‘Altiero Spinelli’, Milan | “Prosodic Synchrony and the Semantic Anchors of Referential Gestures” |
| Paula G. Sánchez-Ramón, Frank Kügler & Pilar Prieto Universitat Pompeu Fabra; Goethe University Frankfurt; Institució Catalana de Recerca i Estudis Avançats | “The influence of gesture presence in the prosodic realization of focus types in the Catalan language” |
| Florence Baills & Stefan Baumann Universitat de Lleida; University of Cologne | “Gesture, prosody and syntax as markers of information structure in French” |
Session 2 (25.09. 13:40 – 15:00): Processing of multimodal data
| Walter Philip Dych, Karee Garvin & Kathryn Franich Binghamton University; Harvard University | “A toolkit for automating co-speech gesture data annotation and analysis” |
| Lena Pagel, Simon Roessig & Doris Mücke University of Cologne; University of York | “Introducing DiCE: A novel approach to elicit and capture multimodal accommodation via 3D electromagnetic articulography, audio, and video” |
| Romain Pastureau & Nicola Molinaro Basque Center on Cognition, Brain and Language (BCBL), San Sebastiá; Universidad del País Vasco/Euskal Herriko Unibertsitatea, San Sebastián; Ikerbasque, Basque Foundation for Science | “Krajjat: A Python Toolbox for Analysing Body Movement and Investigating its Relationship with Speech” |
| Davide Ahmar, Šárka Kadavá & Wim Pouw Donders Institute for Brain, Cognition and Behavior; Leibniz Centre for General Linguistics | “MOBILE MULTIMODAL LAB: An Open-Source, Low-Cost and Portable Laboratory for the study of Multimodal Human Behavior” |
Session 3 (25.09. 16:50 – 18:10): Multimodality and Development
| Sara Coego, Núria Estve-Gibert & Pilar Prieto Universitat Pompeu Fabra; Universitat Oberta de Catalunya; Institució Catalana de Recerca i Estudis Avançats (ICREA) | “Preschoolers’ use of prosody and gesture in marking focus types” |
| Anita Slonimska, Alessia Giulimondi, Emanuela Campisi & Asli Ozyurek Max Planck Institute for Psycholinguistics, Nijmegen; Utrecht University; Catania University; Donders Institute for Brain, Cognition and Behavior, Nijmegen | “Simultaneity in iconic two-handed gestures: a communicative strategy for children” |
| Joel Espejo-Álvarez, Júlia Florit-Pons, Claire Lien Luong, Mireia Gómez i Martínez, Alfonso Igualada & Pilar Prieto Universitat Pompeu Fabra; University of Cork; Universitat Oberta de Catalunya; Institució Catalana de Recerca i Estudis Avançats | “The impact of a multimodal oral narrative intervention on boosting the frequency of use and the quality of children’s non-dominant language” |
| Mariia Pronina, Júlia Florit-Pons, Sara Coego & Pilar Prieto The University of the Balearic Islands; Universitat Pompeu Fabra; Institució Catalana de Recerca i Estudis Avançats (ICREA) | “Different developmental paths of multimodal imitation in typically and non-typically developing preschool and primary school children” |
Session 4 (26.09. 10:00 – 11:00): Embodiment and Arts
| Lara Pearson, Thomas Nuttall & Wim Pouw Max Planck Institute for Empirical Aesthetics, Frankfurt; Universitat Pompeu Fabra; Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen | “Motif-Gesture Contiguity in Karnatak Vocal Performance: A Multimodal Computational Analysis” |
| Nasim Mahdinazhad Sardhaei, Marzena Zygis & Hamid Sharifzadeh Leibniz Center for General Linguistics | “Orofacial signals beyond sight: A study of expressive faces and whispered voices in German” |
| Elena Nicoladis University of British Columbia | “The effects of familiarity on children’s pantomimes” |
Session 5 (26.09. 11:30 – 12:50): Sign languages
| Sonja Gipper, Anastasia Bauer, Jana Hosemann & Tobias-Alexander Herrmann University of Cologne | “Multimodal feedback in signed and spoken languages: Evidence for a shared infrastructure of conversation” |
| Marisa Cruz & Sónia Frota University of Lisbon | “Four seasons in one head: The prosodic phrasing of enumerations in Portuguese Sign Language” |
| Clara Lombart University of Namur, NaLTT, LSFB-Lab | “How visual cues make information units more prominent in spoken and signed languages: A case study on French and French Belgian Sign Language (LSFB)” |
| Anastasia Bauer, Anna Kuder, Marc Schulder & Job Schepens University of Cologne; University of Hamburg | “The phonetics of addressee’s head nods in signed and spoken interaction using a computer vision solution” |
Session 6 (26.09. 14:20 – 15:20): Methodological Perspectives
| Geert Brône, Bert Oben & Julie Janssens University of Leuven | “Looking together. An eye-tracking corpus of museum visitors’ shared experience and joint attention” |
| Sam O’Connor Russell & Naomi Harte Trinity College Dublin, Ireland | “Towards Multimodal Turn-taking for Naturalistic Human-Robot Interaction” |
| Mojenn Schubert Leibniz-Institute for the German Language, Mannheim | “Navigating the topical landscape: Pointing at others as an embodied backlinking device in multi-party interaction” |
Session 7 (26.09. 17:10 – 18:30): Phonetic aspects of gestures
| Gilbert Ambrazaitis, Margaret Zellers & David House Linnaeus University, Växjö; Kiel University; KTH Royal Institute of Technology, Stockholm | “Pitch accent realization as a function of accompanying manual or eyebrow gestures in spontaneous Swedish dialogue” |
| Martine Grice, Alexandra Vella, Maria Lialiou, Florence Baills, Aviad Albert, Petra B. Schumacher, Nadia Pelageina & Solveigh Janzen University of Cologne; University of Malta; Universitat de Lleida | “Gesture apex coordination with prosodic structure and tonal events in Maltese English” |
| Kathryn Franich & Vincent Nwosu Harvard University; University of Calgary | “Timing of Co-Speech Gesture in Igbo: Influence of Metrical Prominence and Tonal Melody” |
| Helene Springer, Henrik Garde, Frida Splendido & Marianne Gullberg Lund University; Lund University Humanities Lab | “Quantifying the visual salience of Swedish vowels: A computer vision approach” |
Session 8 (27.09. 09:00 – 10:20): Semantics/Pragmatics of gestures
| Andy Lücking, Alexander Mehler & Alexander Henlein Goethe University Frankfurt, Text Technology Lab | “The Gesture–Prosody Link in Multimodal Grammar” |
University of Göttingen | |
| Daniel K. E. Reisinger & Marianne Huijsmans University of British Columbia; University of Alberta | “On the Role of Co-speech Gesture with ʔayʔaǰuθəm D Elements” |
| Cornelia Loos & Sophie Repp University of Hamburg; University of Cologne | “The many ways to mark agreement & rejection: Multimodal polar responses in German” |
Postersessions:
Postersession 1 (25.09. 15:00 – 16:20)
| D1-01 Alina Naomi Riechmann & Hendrik Buschmeier Bielefeld University | “Automatic Reconstruction of Dialogue Participants’ Coordinating Gaze Behaviour from Multiple Camera Perspectives” |
| D1-02 Luca Béres, Ádám Boncz, Péter Nagy & István Winkler HUN-REN Research Centre for Natural Sciences, Budapest; Budapest University of Technology and Economics, Budapest | “The role of synchronization in face-to-face communication: A dual eye-tracking and motion capture study” |
| D1-03 Sharice Clough, Beyza Sümer, Kristel de Laat, Annick Tanguay, Sarah Brown-Schmidt, Melissa C. Duff & Aslı Özyürek MPI for Psycholinguistics; Vanderbilt University Medical Center; University of Amsterdam | “Spatial Narratives from Remote and Recent Memory in Individuals with Alzheimer’s Disease and Healthy Older Adults: A Multimodal and Kinematic Perspective” |
| D1-04 Stefanie Shattuck-Hufnagel & Ada Ren-Mitchell MIT RLE Speech Communications Group; MIT Media Lab | “Kinematic gestural evidence for higher-level prosodic constituents in speech” |
| D1-05 Aleksandra Ćwiek, Šárka Kadavá, Wim Pouw & Susanne Fuchs Leibniz-Centre General Linguistics; Donders Institute for Brain, Cognition, and Behaviour; University of Göttingen | “The Communicative Consequences of Multimodal Coordination” |
| D1-06 Schuyler Laparle & Merel Scholman Tilburg University; Utrecht University; Saarland University | “Signaling discourse relations in multimodal communication” |
| D1-07 Marion Schulte Rostock University | “Social meaning and multimodality: The performance of scientific authority” |
| D1-08 Vera Wolfrum, Carina Lüke & Simone Schaeffner Julius-Maximilians University Würzburg | “The influence of linguistic input on the multimodal language processing of primary school children” |
| D1-09 Stefan Lazarov & Angela Grimminger Paderborn University | “Verbal signals of understanding do not predict a decrease of gesture deixis” |
| D1-10 Elena Nicoladis, Anahita Shokrkon & Shiva Zarezadehkheibari University of British Columbia; University of Alberta | “Farsi-English bilinguals’ gesture production while telling a story” |
| D1-11 Kazuki Sekine & Ikuko Nonaka Waseda University | “Effects of Bowing During Japanese Telephone Conversation on Acoustic Properties” |
Postersession 2 (26.09. 15:20 – 16:40)
| D2-01 Alexander Henlein, Alexander Mehler & Andy Lücking Goethe University Frankfurt, Text Technology Lab | “Virtually Restricting Modalities in Interactions: Va.Si.Li-Lab for Experimental Multimodal Research” |
| D2-02 Han Zhou Heidelberg University | “A Theoretical Model for Analyzing Metaphors in Multimodal Communication: Exemplified by Pictorial and Verbo-Pictorial Metaphors in Editorial Cartoons” |
| D2-03 Patrizia Paggio, Manex Agirrezabal & Bart Jongejan University of Copenhagen; University of Malta | “Movement entrainment in online meetings” |
| D2-04 Alina Gregori & Susanne Fuchs Goethe University Frankfurt; Leibniz-Center for General Linguistics; ILCB & IMéRA at Aix-Marseille University | “Moving Meetings by Moving Prosody and Gesture” |
| D2-05 Marion Bonnet, Cornelia Ebert, Kurt Erbach & Markus Steinbach Göttingen University; Goethe University Frankfurt | “Show me the choice” |
| D2-06 Christoph Rühlemann & James Trujillo University of Freiburg; University of Amsterdam | “The effect of gesture expressivity on emotional resonance in storytelling interaction” |
| D2-07 Himmet Sarıtaş & Şeyda Özçalışkan Georgia State University | “Does gesture play a similar role in the communication of second language learners in face-to-face and online interactions?” |
| D2-08 Emanuel Schütt, Merle Weicker & Carolin Dudschig University of Tübingen; Goethe University Frankfurt; University of Cologne | “Human language comprehenders appear to integrate rapidly gestural and verbal expressions of “yes” and “no”: Evidence from a two-choice response time task” |
| D2-09 Ingrid Vilà-Giménez, Mariia Pronina & Pilar Prieto Universitat de Girona; Universitat de les Illes Balears; Institució Catalana de Recerca i Estudis Avançats; Universitat Pompeu Fabra | “Exploring children’s storytelling: The link between narrative abilities, receptive vocabulary and gesture rate in 7- to 9-year-olds” |
| D2-10 Nathalie Frey & Carina Lüke University of Würzburg | “Multimodal insights into the lexical development of mono- and multilingual children with SLCN” |
| D2-11 Júlia Florit-Pons, Pilar Prieto, Alfonso Igualada Universitat Pompeu Fabra; Institució Catalana de Recerca i Estudis Avançats; Universitat Oberta de Catalunya; Institut Guttmann | “A multimodal narrative intervention for boosting NDD children’s oral narrative skills” |
Postersession 3 (27.09. 10:20 – 11:40)
| D3-01 Arianna Colombani, Varghese Peter, Quian Yin Mai, Outi Tuomainen, Natalie Boll-Avetisyan, Amanda Saksida & Mridula Sharma International Doctorate for Experimental Approaches to Language and Brain (IDEALAB); Macquarie University; University of Potsdam; University of the Sunshine Coast, Brisbane; Institute for Maternal and Child Health-IRCCS “Burlo Garofolo”, Trieste | “Cross-situational learning of word-pseudosign pairs in children and adults: a behavioral and event-related potential study” |
| D3-02 Stéphanie Caët, Loulou Kosmala, Carla Ferran & Marine Laval Université de Lille; UMR 8163 Savoirs, Textes, Langage; Université Paris Nanterre; EA 370 CREA | “Participation of deaf children with a cochlear implant in family dinner interactions: the role of gesture” |
| D3-03 Katharina J. Rohlfing, Nils Tolksdorf, Angela Grimminger, Koki Honda & Kazuki Sekine Paderborn University; Waseda University | “Using social robots for cross-cultural gesture elicitation in children: Psycholinguistic considerations on dialogue design” |
| D3-04 Maria Graziano, Joost van de Weijer & Marianne Gullberg Lund University Humanities Lab; Centre for Languages and Literature, Lund University | “Exploring gesture distribution over disfluency markers in competent speakers and language learners” |
| D3-05 Joanna Wójcicka, Anna Kuder & Justyna Kotowicz University of Warsaw; University of Cologne; University of Silesia Katowice | “Language Control and Multimodal Behavior in Native Hearing PJM-Polish Bilinguals Using Spoken Polish” |
| D3-06 Fien Andries, Katharina Meissl & Clarissa de Vries KU Leuven | “Mocking enactments: a case-study of multimodal stance-stacking” |
| D3-07 Margaret Zellers, Jan Gorisch & David House Kiel University; Leibniz-Institut für Deutsche Sprache; Kungliga Tekniska Högskolan | “Referential gestures and the management of turn-taking in conversation” |
| D3-08 Johannes Heim, Rebecca Woods, Franziska Busche & Sophie Repp University of Aberdeen; Newcastle University; University of Cologne | “Multimodal profiles of different (negative) question types” |
| D3-09 Federica Raschellà, Frida Splendido, Nadja Althaus, Marieke Hoetjes & Gilbert Ambrazaitis Linnaeus University, Växjö; Lund University; University of East Anglia, Norwich; Radboud University, Nijmegen | “Embodied pronunciation training for the Swedish complementary length contrast” |
| D3-10 Anna Inbar & Yael Maschler The Academic College Levinsky-Wingate; University of Haifa | “Pointing at the addressee in Hebrew face-to-face interaction” |
| D3-11 Vivien Lohmer & Friederike Kern Bielefeld University | “The role of interactive gestures in explanatory interactions” |
