Effectiveness of AI Chatbots in Promoting Informal Speaking Proficiency and Social Pragmatic Skills in Uzbekistan: A Multi-analysis Study

Authors

DOI:

https://doi.org/10.54855/callej.261234567

Keywords:

AI chatbots, informal speaking proficiency, social pragmatic skills, EFL learners, Uzbekistan

Abstract

This research investigates the role of artificial intelligence (AI) chatbots in improving informal speaking proficiency and social pragmatic abilities in upper-intermediate male EFL learners located in Tashkent, Uzbekistan. A quasi-experimental design was utilized, involving 50 learners aged 18–24 from an English Language Institute. These participants were randomly assigned to either an experimental group (EG), which interacted with a ChatGPT-based AI chatbot, or a Control Group (CG), which engaged in teacher-led speaking practice. Pre- and post-tests, which included a speaking proficiency rubric and a social pragmatics assessment, evaluated changes in fluency, accuracy, lexical complexity, and pragmatic competence (such as turn-taking and politeness strategies). Quantitative data were analyzed using paired and independent t-tests, whereas qualitative survey responses regarding learner attitudes were explored through thematic analysis. Results indicated notable enhancements in both groups, with the EG showing more substantial progress (mean improvement = 6.39 vs. 1.59 points for speaking proficiency; 0.9 vs. 0.3 for social pragmatics; p < 0.05). Participants indicated a boost in confidence, heightened engagement, and a decrease in anxiety when interacting with chatbots. These findings highlight the potential of AI chatbots to enhance traditional instruction by offering flexible, low-anxiety practice environments. They provide valuable insights for teachers, curriculum designers, and material developers on how to incorporate AI tools to improve EFL communicative competence.

Author Biographies

  • Azam Kholikov, Head of the Department of Mother Language and Teaching Methodology in Primary Education, National Pedagogical University of Uzbekistan, Tashkent, Uzbekistan

    Professor Dr. Azam Abdusalomovich Khalikov was born on March 10, 1958, in Zharkurgan District, Surkhandarya Region, Uzbekistan. He graduated with honors from the Faculty of Russian Philology at the Termez State Pedagogical Institute named after M.T. Oybek in 1984. After teaching Russian language and literature, he completed postgraduate studies at the Nizami State Pedagogical Institute, where he defended his Candidate’s dissertation in 1994 on “The Use of the Idealization Method in the Study of Humanitarian Educational Subjects.” In 2018, he earned his Doctor of Pedagogical Sciences degree with a dissertation on “Development of Pedagogical Skills of Future Teachers in Higher Educational Institutions.” Prof. Khalikov has authored about 20 textbooks and teaching manuals and over 250 scientific papers, many of which have been translated into other languages. A dedicated scholar and mentor, he has supervised numerous doctoral and master’s students. He currently serves as Acting Head of the Department of Theory and Methodology of Primary Education at the Nizami State Pedagogical University. Prof. Khalikov is a laureate of the “People’s Education” award, and recipient of the “Hero of Labor” and “Amir Temur” medals. 

  • Irodakhon Abdullayeva, Department of Art Studies, Faculty of Exact and Applied Sciences, Urganch State Pedagogical Institute, Urganch, Uzbekistan

    Irodakhon Abdullayeva was born on October 2, 1997, in Tashkent, Uzbekistan, into a family of intellectuals. She graduated with distinction from Secondary School No. 312 in Uchtepa District (2012) and from Almazar Medical College (2015). Motivated by a growing passion for education, she pursued a Bachelor’s degree in Music Education at the Tashkent State Pedagogical University named after Nizami (2017–2021). During her undergraduate studies, she distinguished herself as an active student and young researcher, publishing around 10 scientific papers and participating in national conferences. In 2019, she was awarded the Islam Karimov State Scholarship, one of Uzbekistan’s most prestigious academic honors. From 2021 to 2023, she completed her Master’s degree in Music Education at the same university, further engaging in international conferences, including those held in Bukhara (Uzbekistan) and Orenburg (Russian Federation). Currently, Ms. Abdullayeva is a researcher at the Tashkent State Pedagogical University, working on the dissertation topic “Development of Creative Competence of Students of Higher Educational Institutions” (specialty 13.00.01 – Theory of Pedagogy). Her work focuses on fostering creativity, patriotism, and intellectual growth through modern pedagogical and innovative teaching approaches. 

  • Zokir Mamadiyarov, Alfraganus University, Tashkent, Uzbekistan

    Irodakhon Abdullayeva was born on October 2, 1997, in Tashkent, Uzbekistan, into a family of intellectuals. She graduated with distinction from Secondary School No. 312 in Uchtepa District (2012) and from Almazar Medical College (2015). Motivated by a growing passion for education, she pursued a Bachelor’s degree in Music Education at the Tashkent State Pedagogical University named after Nizami (2017–2021). During her undergraduate studies, she distinguished herself as an active student and young researcher, publishing around 10 scientific papers and participating in national conferences. In 2019, she was awarded the Islam Karimov State Scholarship, one of Uzbekistan’s most prestigious academic honors. From 2021 to 2023, she completed her Master’s degree in Music Education at the same university, further engaging in international conferences, including those held in Bukhara (Uzbekistan) and Orenburg (Russian Federation). Currently, Ms. Abdullayeva is a researcher at the Tashkent State Pedagogical University, working on the dissertation topic “Development of Creative Competence of Students of Higher Educational Institutions” (specialty 13.00.01 – Theory of Pedagogy). Her work focuses on fostering creativity, patriotism, and intellectual growth through modern pedagogical and innovative teaching approaches. 

  • Amina Ruzikulova, Navoi State University, Navoi, Uzbekistan

    Dr. Amina Ruzikulova (born 1976) is an Uzbek scholar and Associate Professor at Navoi State Pedagogical Institute. She earned her Bachelor’s and Master’s degrees in Psychology from the same institution (1999–2007). Beginning her academic career in 2001 as a laboratory assistant, she has since served as lecturer, senior lecturer, and academic mentor in the Department of General Pedagogy and Psychology. In 2022, Dr. Ro‘ziqulova defended her PhD dissertation titled “Organizational and Psychological Aspects of Strengthening Marital and Family Relations in Uzbek Families” under the supervision of Professor Nurmat Soginov and was awarded the title of Associate Professor in 2024. She is the author of over 110 scholarly works, including textbooks, teaching manuals, methodological guides, and Scopus-indexed articles. Her teaching integrates innovative and interactive pedagogical technologies to foster student engagement and professional competence. Actively involved in academic and community life, Dr. Ro‘ziqulova serves on the Institute’s Public Council, mentors young faculty, and contributes to the moral and civic education of students.

References

Aboulghazi, M., Amiri, E. M., & El Karfa, A. (2024). Teachers' perceptions toward pragmatics and pragmatic teaching. Arab World English Journal, 15(3), 31-47. https://dx.doi.org/10.24093/awej/vol15no3.3

Adamopoulou, E., & Moussiades, L. (2020). Chatbots: History, technology, and applications. Machine Learning with Applications, 2, Article 100006. https://doi.org/10.1016/j.mlwa.2020.100006

Afrouz, M.; Alkawaz, A.; Nejad Ansari, D. & Dabaghi, A. (2023). The Effect of Explicit Pragmatic Instruction on Iraqi EFL Students' Production of Speech Acts: Pragmalinguistic vs. Sociopragmatic Errors in Focus. Journal of English Language Teaching and Learning, 15(31), 1-17. https://dx.doi.org/10.22034/elt.2023.54597.2521

Ahmad, N. A., Che, M. H., & Zainal, A. (2018). Review of chatbots design techniques. International Journal of Computer Applications, 18(8), 7-10. https://doi.org/10.5120/ijca2018917606

Annamalai, N., Ab Rashid, R., Hashmi, U. M., Mohamed, M., Alqaryouti, M. H., & Sadeq, A. E. (2023). Using chatbots for English language learning in higher education. Computers and Education: Artificial Intelligence, 5, 100153. https://doi.org/10.1016/j.caeai.2023.100153

Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32(8), 827–877. https://doi.org/10.1080/09588221.2018.1535508

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Brown, H. D., & Lee, H. (1994). Teaching by principles: An interactive approach to language pedagogy (Vol. 1, p. 994). Prentice Hall Regents.

Çakmak, F. (2022). Chatbot-human interaction and its effects on EFL students’ L2 speaking performance and anxiety. Novitas-ROYAL (Research on Youth and Language), 16(2), 113-131.

Chapelle, C. (2003). English language learning and technology: Lectures on applied linguistics in the age of information and communication technology. John Benjamins.

Chuah, K. M., & Kabilan, M. (2021). Teachers’ views on the use of chatbots to support English language teaching in a mobile environment. International Journal of Emerging Technologies in Learning (IJET), 16(20), 223–237.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications.

Demir, A. (2024). Artificial Intelligence and English Language Teaching: Review of the Research Articles from 2017 to 2022. International Journal of Social and Humanities Sciences Research (JSHSR), 11(113), 2225-2231. https://doi.org/10.5281/zenodo.14257198

Du, J., & Daniel, B. K. (2024). Transforming language education: A systematic review of AI-powered chatbots for English as a foreign language speaking practice. Computers and Education: Artificial Intelligence, 6, 100230. https://doi.org/10.1016/j.caeai.2024.100230

Duong, T., & Suppasetseree, S. (2024). The effects of an artificial intelligence voice chatbot on improving Vietnamese undergraduate students’ English speaking skills. International Journal of Learning, Teaching and Educational Research, 23(3), 293–321. https://doi.org/10.26803/ijlter.23.3.15

Guo, K., Zhong, Y., Li, D., & Chu, S. K. W. (2023). Effects of chatbot-assisted in-class debates on students’ argumentation skills and task motivation. Computers & Education, 201, 104862. https://doi.org/10.1016/j.compedu.2023.104862

Haristiani, N. (2019). Artificial intelligence (AI) chatbot as language learning medium: An inquiry. International Conference on Education, Science and Technology, 1387(1), 012020. https://doi.org/10.1088/1742-6596/1387/1/012020

Hasanova, D. (2007). Teaching and learning English in Uzbekistan. English Today, 23(1), 3-9. https://doi.org/10.1017/S0266078407001022

Hsu, H. L., Chen, H. H. J., & Todd, A. G. (2023). Investigating the impact of the Amazon Alexa on the development of L2 listening and speaking skills. Interactive Learning Environments, 31(9), 5732–5745. https://doi.org/10.1080/10494820.2021.2016864

Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12518

Hymes, D. (1972). On communicative competence. In J. B. Pride & J. Holmes (Eds.), Sociolinguistics: Selected readings (pp. 269–293). Penguin.

Ji, H., Han, I., & Ko, Y. (2023). A systematic review of conversational AI in language education: Focusing on the collaboration with human teachers. Journal of Research on Technology in Education, 55(1), 48–63. https://doi.org/10.1080/15391523.2022.2142873

Kasper, G., & Rose, K. (2002). Pragmatic development in a second language. Blackwell.

Kasper, G., & Rose, K. R. (2002). Pragmatic development in a second language. Language Learning, 52(Suppl1), 1–352.

Khasawneh, M. A. S., (2023). Factors affecting the improvement of speaking skills among Jordanian EFL learners. Journal of Language Teaching and Research, 14(6), 1559-1568. https://doi.org/10.17507/jltr.1406.14

Kim, H.-S., Cha, Y., & Kim, N. Y. (2021). Effects of AI chatbots on EFL students’ communication skills. Korean Journal of English Language and Linguistics, 21, 712–734. https://doi.org/10.15738/kjell.21..202108.712

Koç, F. Ş., & Savaş, P. (2025). The use of artificially intelligent chatbots in English language learning: A systematic meta-synthesis study of articles published between 2010 and 2024. ReCALL, 37(1), 4–21. https://doi.org/10.1017/S0958344024000168

Kozhevnikova, E. (2014). Exposing students to authentic materials as a way to increase students' language proficiency and cultural awareness. Procedia - Social and Behavioral Sciences, 116, 2124-2129. https://doi.org/10.1016/j.sbspro.2014.01.967

Krashen, S. (1982). Principles and practice in second language acquisition. Pergamon.

Krashen, S. D. (1985). The input hypothesis: Issues and implications. Longman.

Leech, G. (1983). Principles of pragmatics. Longman.

Liu, Z., Zhang, W., & Yang, P. (2025). Can AI chatbots effectively improve EFL learners’ learning effects?—A meta-analysis of empirical research from 2022–2024. Computer-Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2025.2456512

Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. Ritchie & T. Bhatia (Eds.), Handbook of second language acquisition (pp. 413–468). Academic Press.

Mokoro, E. (2024). Pragmatic competence in second language learners. European Journal of Linguistics, 3(2), 15-28. https://doi.org/10.47941/ejl.2044

Munir, E., & Yavuz, M. A. (2024). Study of pragmatic analysis of literature on the development of ELT students’ receptive and productive skills: A case of North Cyprus. Heliyon, 10(21), e39558. https://doi.org/10.1016/j.heliyon.2024.e39558

Nazara, S. (2011). Students’ perception of EFL speaking skill development. Journal of English Teaching, 1(1), 28-43. https://doi.org/10.33541/jet.v1i1.50

Qiao, H., & Zhao, A. (2023). Artificial intelligence-based language learning: illuminating the impact on speaking skills and self-regulation in Chinese EFL context. Frontiers in psychology, 14, 1255594. https://doi.org/10.3389/fpsyg.2023.1255594

Schmidt, R. (1990). The role of consciousness in second language learning. Applied Linguistics, 11(2), 129–158.

Swain, M. (1985). Communicative competence: Some roles of comprehensible input and comprehensible output in its development. In S. Gass & C. Madden (Eds.), Input in second language acquisition (pp. 235–253). Newbury House.

Tai, T. Y., & Chen, H. H. J. (2024). Improving elementary EFL speaking skills with generative AI chatbots: Exploring individual and paired interactions. Computers & Education, 220, 105112. https://doi.org/10.1016/j.compedu.2024.105112

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.

Walker, A., & White, G. (2013). Technology enhanced language learning: Connecting theory and practice. Oxford University Press.

Wang, C., Zou, B., Du, Y., & Wang, Z. (2024). The impact of different conversational generative AI chatbots on EFL learners: An analysis of willingness to communicate, foreign language speaking anxiety, and self-perceived communicative competence. System, 127, 103533. https://doi.org/10.1016/j.system.2024.103533

Wiboolyasarin, W., Wiboolyasarin, K., Tiranant, P., Jinowat, N., & Boonyakitanont, P. (2025). AI-driven chatbots in second language education: A systematic review of their efficacy and pedagogical implications. Ampersand, 14, 100224. https://doi.org/10.1016/j.amper.2025.100224

Wu, X., & Li, R. (2024). Unraveling effects of AI chatbots on EFL learners’ language skill development: A meta-analysis. The Asia-Pacific Education Researcher, 1-12. https://doi.org/10.1007/s40299-024-00853-2

Zhai, C., & Wibowo, S. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, 100134. https://doi.org/10.1016/j.caeai.2023.100134

Zhang, R., Zou, D., & Cheng, G. (2024). A review of chatbot-assisted learning: Pedagogical approaches, implementations, factors leading to effectiveness, theories, and future directions. Interactive Learning Environments, 1–29. https://doi.org/10.1080/10494820.2023.2202704

Zhang, Z., & Huang, X. (2024). The impact of chatbots based on large language models on second language vocabulary acquisition. Heliyon, 10, e25370. https://doi.org/10.1016/j.heliyon.2024.e25370

Zheng, S. (2024). The effects of chatbot use on foreign language reading anxiety and reading performance among Chinese secondary school students. Computers and Education: Artificial Intelligence, 7, 100271. https://doi.org/10.1016/j.caeai.2024.100271

Downloads

Published

2025-12-31

Issue

Vol. 26 No. 7 (2025): Special Issue: AI-mediated Informal Digital Learning of English (AIIDLE): Insights, Challenges, and Practice

Section

Research Articles

License

Copyright (c) 2025 Author and CALL-EJ

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright of articles is retained by authors and CALL-EJ. As CALL-EJ is an open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings. Sources must be acknowledged appropriately.

How to Cite

Effectiveness of AI Chatbots in Promoting Informal Speaking Proficiency and Social Pragmatic Skills in Uzbekistan: A Multi-analysis Study. (2025). Computer-Assisted Language Learning Electronic Journal, 26(7), 137-161. https://doi.org/10.54855/callej.261234567