COMMUNICATIVE PRACTICES IN USING RUSSIAN AS A FOREIGN LANGUAGE: THE STUDY OF GAZE BEHAVIOR, SPEECH AND GESTURE IN DESCRIPTIVE DISCOURSE

COMMUNICATIVE PRACTICES IN USING RUSSIAN AS A FOREIGN LANGUAGE: THE STUDY OF GAZE BEHAVIOR, SPEECH AND GESTURE IN DESCRIPTIVE DISCOURSE

Author:  M.I. Kiose, A.A. Rzheshevskaya

Affiliation:  Moscow State Linguistic University

Abstract:  The research addresses the problem of exploring communicative practices of information comprehension and transfer in the multimodal behavior of Russian language learners. To proceed, the study appeals to multimodal discourse construal theories which view discourse in its perception (in gaze behavior) and generation (in speech and gesture behavior). Discourse construal requires both verbal and non-verbal performance, which necessitates the research on the distribution of speech, gaze and gestures to be further integrated into the teaching process. To comply with the problem, this study reports the results of the multimodal experiment where two participant groups with language proficiency levels B1/B2 and C1/C2, first read and then reported the contents of descriptive discourse fragments.
The study aims to identify the characteristics of gaze perception of the fragments of descriptive discourse as well as speech and gesture patterns in the participants’ accounts mediated by language proficiency level. We hypothesize that the multimodal behavior of readers and speakers is determined by the local discourse structure regarding its communicative, narrative, rhetorical relations and the relations describing speech disfluencies; additionally, the level of language proficiency affects the relations distribution and their further multimodal realization.
The research data are 40 records of gaze behavior registered while reading and 40 records of speech and gesture behavior in the spoken accounts of participants rendering the obtained information to an interested listener. At the stage of discourse perception, we observed the differences in total fixation time and number of saccades in the areas of interest concomitant with the rhetorical relation of Joint; these metrics were higher in the C1/C2 learners’ group. At the stage of discourse generation, we revealed significant differences in all types of relations in the speech of two groups, with the highest differences found in the distribution of communicative relations Agreement and Metadiscursive, narrative relation Setting, rhetorical relations Interpretation and Joint, speech disfluencies relations Correction and Repetition. The gesture distribution showed specificity within two groups which ranked higher with the relations Repetition and Hesitation.
The results evidence of a growing necessity to form the communicative practices of learners as mediated by local discourse structure, as well as reveal the ontology of these relations in the multimodal discourse of foreign language learners.

Keywords:  communicative practices, descriptive discourse, local discourse structure, multimodal behaviour,
Russian as a foreign language, gaze, speech, gesture.

References:  Alexeeva S., Chernova D., Norkina M., Kharchevnik M. Predictability effect in L2 reading: eye-tracking data // AMLaP 29 Conference. Architectures and Mechanisms for Language Processing. 2023. P. 62.
Ardhani R.R. The effectiveness of bottom-up and top-down approaches. The reading comprehension skill for junior high school students // Proceedings of JEE, Journal of English and Education. 2016. Vol. 5. P. 80-90.
Babashamsi P., Bolandifar S., Shakib N. Various models for reading comprehension process // Proceedings of International Journal of Applied Linguistics and English Literature. 2023. Vol. 2. P. 150-154.
Blinnikova I.V., Rabeson M.D., Blinnikov G.B., Izmalkova A.I. Complexity of visual semantic search in first and second languages: eye-movement analysis // Research Result. Theoretical and Applied Linguistics. 2023. Vol. 9(1). P. 34-49.
Cienki A. Gesture, space, grammar, and cognition / ed. by P. Auer, M. Hilpert, A. Stukenbrock, B. Szmrecsanyi // Space in language and linguistics: Geographical, interactional, and cognitive perspectives. Berlin: Walter de Gruyter. 2013. P. 667-686.
Fehlmann B., Coynel D., Schicktanz N., Milnik A, Gschwind L., Hofmann P., Papassotiropoulos A., de Quervain D.J-F. Visual exploration at higher fixation frequency increases subsequent memory recall // Cerebral Cortex Communications. 2020. Vol. 1(1). Article number: tgaa032.
Goldin-Meadow S. Hearing gesture: How our hands help us think. Cambridge, MA: The Belknap Press, 2003.
Grishina E.A. Russian gesticulation as a linguistic phenomenon: Corpus studies. M.: YaSK, 2017.
Gullberg M. Gestures and second language acquisition // Handbook of cognitive linguistics and second language acquisition / ed. by P. Robinson, N.C. Ellis. N.Y.: Routledge, 2008. P. 276-305.
Hadar U., Dar R., Teitelman A. Gesture during speech in first and second language: Implications for lexical retrieval // Gesture. 2001. Vol. 1(2). P. 151-165.
Hsu H-C, Brône G and Feyaerts K. When gesture “takes over”: Speech-embedded nonverbal depictions in multimodal interaction // Frontiers in Psychology. 2021. Article number: 11:552533.
Inhoff A., Kim A., Radach R. Regressions during reading // Vision. 2019. Vol. 3(3). Article number: 35.
Iriskhanova O.K., Cienki A. The semiotics of gestures in cognitive linguistics: Contribution and challenges // Вопросы когнитивной лингвистики. 2018. № 4. С. 25-36. [Iriskhanova O.K., Cienki A. The semiotics of gestures in cognitive linguistics: Contribution and challenges // Voprosy kognitivnoj lingvistiki. 2018. № 4. S. 25-36.]
Iriskhanova O.K., Kiose M.I., Leonteva A.V., Agafonova O.V. Vague reference in expository discourse: multimodal regularities of speech and gesture // Computational Linguistics and Intellectual Technologies. 2023. Iss. 22. P. 172-180.
Kendon A. Pragmatic functions of gestures: Some observations on the history of their study and their nature // Gesture. 2017. Vol. 16(2). P. 157-175.
Kibrik A.A., Podlesskaya V.I. Night Dream Stories: A Corpus Study of spoken Russian discourse. Moscow: Languages of Slavonic Culture, 2009.
Konovalova A., Zubov V. Exploring the role of L1 reading ability when reading in L2 // ExLing 2022: Proceedings of 13th International Conference of Experimental Linguistics. 2022. P. 101-104.
Lin Y.L. Speech-accompanying gestures in L1 and L2 conversational interaction by speakers of different proficiency levels // International Review of Applied Linguistics in Language Teaching. 2019. Vol. 60. P. 123-142.
Litvinenko A.O., Kibrik A.A., Fedorova O.V., Nikolaeva Ju.V. Annotating hand movements in multichannel discourse: Gestures, adaptors and manual postures // The Russian Journal of Cognitive Science. 2018. Vol. 5(2). P. 4-17.
Loginova E.G. Meaning construal through multimodal clusters in the theatrical discourse // Languages and Modalities. 2022. Vol. 2. P. 27-36.
Mann W.C, Thomspon S.A. Rhetorical structure theory: Towards a functional theory of text organization // Text – Interdisciplinary Journal for the Study of Discourse. 1988. Vol. 8. P. 243-281.
Müller C. Gestural modes of representation as techniques of depiction // Body—Language—Communication / Eds. C. Müller, A. Cienki, E. Fricke, S. Ladewig, D. McNeill, and J. Bressem. Berlin: De Gruyter Mouton. 2014. Vol. 2. P. 1687-1702.
Nagpal J., Nicoladis E., Marentette P. Predicting individual differences in L2 speakers’ gestures // International Journal of Bilingualism. 2011. Vol. 15(2). P. 205-214.
Rayner K. Eye movements and attention in reading, scene perception, and visual search // Quarterly Journal of Experimental Psychology. 2009. Vol. 62. P. 1457-1506.
Shi J. A Corpus-based study of contrastive / concessive linking adverbials in spoken English of Chinese EFL learners // Studies in Literature and Language. 2017. Vol. 14. P. 17-25.
Tapper M. Connectives in advanced Swedish EFL learners’ written English–preliminary results // Working Papers in Linguistics. 2005. Vol. 5. P. 116-144.
Toldova S.Yu., Davydova T., Kobozeva M.V., Pisarevskaya D.B. Discourse features of blogs in subcorpus of Russian Ru RSTree Bank // Computational Linguistics and Intellectual Technologies. 2020. Vol. 19(26). P. 747-761.
Velichkovsky B.M., Joos M., Helmert J.R., Pannasch S. Two visual systems and their eye movements: evidence from static and dynamic scene perception / eds. B.G. Bara, L. Barsalou, M. Bucciarelli // Proceedings of the XXVII Annual Conference of the Cognitive Science Society. Mahwah: Lawrence Erlbaum. 2005. P. 2283-2288.
Wetzel M., Zufferey S., Gygax P. Second language acquisition and the mastery of discourse connectives: Assessing the factors that hinder L2-learners from mastering French connectives // Languages. 2020. Vol. 5. Article number: 35.
Whitford V, Titone D. Second-language experience modulates eye movements during first- and second-language sentence reading: evidence from a gaze-contingent moving window paradigm // Journal of Experimental Psychology. Learning, Memory and Cognition. 2015. Vol. 41(4). P. 1118-11129.
Zufferey S., Willem M., Liesbeth D., Sanders T. Advanced learners’ comprehension of discourse connectives: The role of L1 transfer across on-line and o-line tasks // Second Language Research. 2015. Vol. 31. P. 389-411.
Болдырев Н.Н., Дубровская О.Г. О формировании социокультурной специфики дискурса // Вопросы когнитивной лингвистики. 2015. № 3. С. 14-25. [Boldyrev N.N., Dubrovskaya O.G. O formirovanii sociokul'turnoj specifiki diskursa // Voprosy kognitivnoj lingvistiki. 2015. № 3. S. 14-25.]
Ирисханова О.К. (ред.) Полимодальные измерения дискурса. М.: Изд. дом ЯСК, 2021. [Iriskhanova O.K. (red.) Polimodal'nye izmereniya diskursa. M.: Izd. dom YaSK, 2021.]
Киосе М.И., Ржешевская А.А., Петров А.А. Локальная структура дискурса описания в речи взрослых, изучающих русский язык как иностранный // Компьютерная лингвистика и интеллектуальные технологии. 2025. Вып. 23. С. 158-168. [Kiose M.I., Rzheshevskaya A.A., Petrov A.A. Lokal'naya struktura diskursa opisaniya v rechi vzroslyh, izuchayushchih russkij yazyk kak inostrannyj // Komp'yuternaya lingvistika i intellektual'nye tekhnologii. 2025. Vyp. 23. S. 158-168.]
Литвиненко А.Л. Описание структуры дискурса в рамках Теории Риторической структуры: применение на русском материале // Труды Международного семинара Диалог 2001 по компьютерной лингвистике и ее приложениям. Теоретические проблемы. 2001. Т. 1. С. 159-168. [Litvinenko A.L. Opisanie struktury diskursa v ramkah Teorii Ritoricheskoj struktury: primenenie na russkom materiale // Trudy Mezhdunarodnogo seminara Dialog 2001 po komp'yuternoj lingvistike i ee prilozheniyam. Teoreticheskie problemy. 2001. T. 1. S. 159-168.]
Подлесская В.И., Кибрик А.А. Самоисправления говорящего и другие типы речевых сбоев как объект аннотирования в корпусах устной речи // Научно-техническая информация. Серия 2: Информационные процессы и системы. 2007. С. 2-23. [Podlesskaya V.I., Kibrik A.A. Samoispravleniya govoryashchego i drugie tipy rechevyh sboev kak ob"ekt annotirovaniya v korpusah ustnoj rechi // Nauchno-tekhnicheskaya informaciya. Seriya 2: Informacionnye processy i sistemy. 2007. S. 2-23.]

For citation:  Kiose, M. I., & Rzheshevskaya, A. A. (2025). Communicative practices in using Russian as a foreign language: the study of gaze behavior, speech and gesture in descriptive discourse. Voprosy Kognitivnoy Lingvistiki, 3, 19-31.

Pages:  19-31