EVENT-MODELLING FRAMEWORK: HOW DOES VERBAL AUTOMATICITY AFFECT TEXT AND IMAGE CONSTRUAL IN GAZE AND SPEECH?*

EVENT-MODELLING FRAMEWORK: HOW DOES VERBAL AUTOMATICITY AFFECT TEXT AND IMAGE CONSTRUAL IN GAZE AND SPEECH?*

Author:  M.I. Kiose, A.I. Izmalkova

Affiliation:  Moscow State Linguistic University

Abstract: 
The study develops Event-modelling framework in scaling the impact of verbal automaticity in the gaze costs and the post hoc reports of the participants exposed to Text and Image stimuli.
We reveal the construal operations in Referent, Event Frame and Perspective which serve as good predictors for the cases when higher verbal automaticity allows to decrease and increase the gaze costs. Gaze costs are lower in Perspective construal (Multidirectional but distinct path in Text and Image, Outer observer path in Text), Event construal (Creation / destruction in Image, Event with cause and effect stated in Text, Interpersonal relations in Image, among others) and Referent construal (Agentive participant in Text, Head of participant in Image).
At the same time, gaze costs become higher in multiple Referent construal operations (Person-participant in Text, Two participants / objects in Text, Facial movements of participant in Image, Ab-stract object in Image, among others), presumably, it allows to alleviate the gaze costs in more compli-cated operations of Event construal and Perspective construal. The impact of verbal automaticity is although less observed in post hoc reports.

Keywords:  Event-modelling framework, construal, oculographic experiment, verbal automaticity, gaze costs, post hoc report.

References:  Altmann G.T.M., Garnham A. & Dennis Y. Avoiding the garden path: Eye movements in con-text // Journal of memory and language, 31, 1992. P. 685-712.
Antipov V. & Zhegallo A. Three-dimensional perception of flat images in computerized environment // Experimental Psychology, 7(3), 2014. P. 97-111.
Ariel M. The development of person agreement markers: from pronoun to higher agreement markers / M. Barlow, S. Kemmer (eds.) // Usage-based models of language, CSIL Publications, 2000. P. 197-260.
Arutunova N.D. Syntax // General Linguistics. The inner structure of language. M.: Nauka, 1972. P. 259-343.
Barabanshhikov V. Facial expressions and their perception. M.: RAS Institute of Psychology, 2012.
Bergen B. Embodiment / E. Dabrowska, D. Divjak (eds.) // Handbook of Cognitive Linguistics. Berlin: Walter de Gruyter, 2015. P. 10-30.
Chen S. Cognitive load measurement from eye activity: Acquisition, efficacy, and real-time system design. PhD Thesis. Sydney: The University of New South Wales, 2014.
Chen S. & Epps J. Task Load Estimation from Multimodal Head-Worn Sensors using Event Sequence Features // IEEE Transactions on Affective Computing 1, 2019. P. 1-14.
Cienki A.J. & Iriskhanova O.K. (eds.) Aspectuality across Languages: Event Construal in Speech and Gesture. John Benjamins, Amsterdam, 2018.
Demyankov V.Z. Event in semantics, pragmatics and interpretation coordinates // Izvestiya АN USSR. Literature and language series, 42(4), 1983. P. 320-329.
Divjak D., Milin P. & Medimorec S. Construal in language: A visual-world approach to the effects of linguistic alternations on event perception and conception // Cognitive Linguistics 19, 2020. P. 37-72.
Dixon M., Laurence J.R. Hypnotic susceptibility and verbal automaticity: automatic and strategic processing differences in the Stroop color-naming task // Journal of Abnormal Psychology, 101(2), 1992. P. 344-347.
Flecken M., Gerwien J., Carroll M. & von Stutterheim C. Analyzing gaze allocation during language planning; A cross-linguistic study on dynamic events // Language and Cognition, 7, 2015. P. 138-166.
Forceville Ch., Urios-Aparisi E. (eds.). Multimodal Metaphor. Berlin: Mouton de Gruyter, 2005.
Gibbs R.W.Jr. & Perlman M. The contested impact of cognitive linguistic research on the psycholinguistics of metaphor understanding / G. Kristiansen, M. Achard, R. Dirven & F.J. Ruiz Mendoza Ibáñez (eds.) // Cognitive linguistics: Current applications and future perspectives. Berlin: Mouton de Gruyter, 2006. P. 211-228.
Givoni S., Giora R. Salience and Defaultness / F. Liedtke & Tuchen A. (eds.) // Handbuch Pragmatik. Stuttgart: J.B. Metzler, 2018. P. 207-213.
Graumann C.F. & Kallmeyer W. (eds.) Per-spective and perspectivation in discourse. Amster-dam: John Benjamins Publishing Company, 2002.
Hart C. & Queralto J.M. What can cognitive linguistics tell us about language-image relations? A multidimensional approach to intersemiotic convergence in multimodal texts // Cognitive Linguistics, 32(4), 2021. P. 529-562.
Iriskhanova O. On perspectivization in cognitive linguistics // Cognitive Studies of Language, 15, 2013. P. 43-58.
Iriskhanova O. (ed.) Multimodal dimensions of discourse. M.: R-Valent, 2021.
Jackendoff R. Foundations of language: Brain, meaning, grammar, evolution. Oxford: Oxford University Press, 2002.
Kholodnaya M. Cognitive styles: on the nature of individual mind. St. Petersburg: Piter, 2004.
Kiose M. Orchestrations of speech and gesture in cinematic shots. Discourse of argumentation and description // Cognitive Studies of Language, 4(47), 2021. P. 255-270.
Kiose M., Rzheshevskaya A. Interdiscourse switching in drama text. Gaze patterns and comprehension check results // Issues in Cognitive Linguistics, 4, 2021. P. 55-66.
Kleijn S., Mak W.M., Sanders T. 2021. Causality, subjectivity and mental spaces: Insights from on-line discourse processing // Cognitive Linguistics, 32(1), 2021. P. 35-65.
Langacker R.W. Foundations of cognitive grammar. Vol. I: Theoretical prerequisites. Stanford, CA: Stanford University Press, 1987.
Logan G. Automaticity and reading: Perspectives from the instance theory of automatization // Reading & Writing Quarterly: Overcoming Learning Difficulties, 13, 1997. P. 123-146.
Papafragou A., Hulbert J. & Trueswell J. Does language guide event perception? Evidence from eye movements // Cognition, 108, 2008. P. 155-184.
Pustejovsky J. The Syntax of Event Structure // Cognition, 41, 1991. P. 47-81.
Pustejovsky J. The Generative Lexicon. Cambridge, Massachusetts: MIT Press, 1995.
Schneider W. Automaticity / R.A. Wilson & F.C. Keil (eds.) // The MIT Encyclopedia of the Cognitive Sciences. Cambridge, Massachusetts: MIT Press, 1999. P. 63.
Siewerska A. On the discourse basis of person agreement / T. Virtanen (ed.) // Approaches to cognition through text and discourse. Berlin, N.Y.: Mouton de Gruyter, 2004. P. 33-48.
Stroop J.R. Studies of Interference in Serial Verbal Reactions // Journal of Experimental Psychology, 18(6), 1935. P. 643-662.
Talmy L. The relation of grammar to cognition / B. Rudzka-Ostyn (ed.) // Topics in Cognitive Linguistics. Amsterdam: John Benjamins Publishing Company, 1988. P. 165-205.
Talmy L. Fictive motion in language and caption / P. Bloom, M.A. Peterson, L. Nadel & M.F. Garrett (eds.) // Language and Space. Cambridge, MA: MIT Press, 1996. P. 211-276.
Talmy L. Toward a Cognitive Semantics. Two volumes. Cambridge: MIT Press, 2000.
Thurlow R. & van den Broek P. Automaticity and inference generation during reading comprehension // Reading & Writing Quarterly, 13, 1997. P. 165-181.
Vendler Z. Verbs and times // The Philosophical Review, 66(2), 1957. P. 143-160.
Verhagen A. Construal and perspectivization / D. Geeraerts & Cuyckens H. (eds.) // The Oxford Handbook of Cognitive Linguistics. Oxford: Oxford University Press, 2007. P. 48-81.
Zykova I. (ed.) Linguistic creativity in different discourses. Limits and allowances. M.: R-Valent, 2021.

For citation:  Kiose, M. I.,  Izmalkova, A. I. (2022). Event-modelling framework: how does verbal automaticity affect text and image construal in gaze and speech? Voprosy Kognitivnoy Lingvistiki, 3, 17-30.

Pages:  17-30