Developing and Validating a Productive Emotion Vocabulary Test for Adult L2 Speakers of English

Abstract

The ability to recognize and express different emotions is intricately intertwined with language proficiency. For instance, lower proficiency learners tend to produce more general (e.g., happy) than specific words (e.g., thrilled) (Yeh et al., 2007). Research also indicates that emotion words are stored, processed, and retrieved differently (Pavlenko, 2008) from generic and academic words. 

Existing measures of emotion vocabulary knowledge, such as label generation tasks (e.g., Bazhydai et al., 2018) and the Children’s Emotion-Specific Vocabulary Vignettes Test (CEVVT; Streubel et al., 2020) suffer from limitations, however. These include low number of items and item formats (Dylman et al., 2020), decontextualized items (Barrett, 2017), and a lack of validity evidence in EFL/ESL contexts.

To address this, our study explored how the adapted vignettes from the CEVVT and newly designed ones function with adult L1 and L2 speakers of English. Since the original vignettes were validated with German-speaking children using an interview format, we investigated if the vignettes elicit the target responses, how these differ from dominant responses, and whether this is affected by language proficiency. 

107 participants with diverse language backgrounds provided three emotion words for each of the 22 vignettes presented in random order and completed the LHQ3 (Li et al., 2020) and LexTALE (Lemhöfer & Broersma, 2012) to control for language background and proficiency. The by-item analyses indicate a moderate naming agreement, but prompt reconsidering the target responses as only 50% match the dominant responses. Regression analyses reveal that language proficiency explains only a small variance in the dominant responses, indicating the need for more nuanced scoring methods. 

The study provides preliminary validity evidence for the PEVT as a measure of productive emotion vocabulary and offers future directions to improve its sensitivity in capturing knowledge of more specific emotion words. The presentation highlights implications for language education and assessment.

Bio

Dr Csaba Z Szabo joined the Institute for Language Education at Moray House, University of Edinburgh in October 2023. He specialises in TESOL, Language Education, and Applied Linguistics with specific interest in vocabulary learning, teaching and assessment in bi- and multilingual contexts, emotion vocabulary testing, language assessment, academic achievement in EMI contexts, and cross-linguistic influences at the level of lexis. He is a keen advocate of promoting research with a practical impact and has a commitment to social justice and evidence-informed teaching for high-quality student experience.

Previously, he worked as an assistant professor of TESOL at the University of Nottingham Malaysia, where he served as the Director for Research and Knowledge Exchange and Postgraduate Research in the School of Education. He was also founding and organising chair of the Alternative Research Lounge, a bi-weekly research forum for students and faculty. As such, he has significantly contributed to developing quantitative literacy skills in Education and Language students, both in Malaysia and the UK. Within this role, he has been nominated for the Lord Dearing Award (2021, 2022), Tri-Campus Award (2023), and Vice-Chancellor’s Medal (2023). He holds an MA in Teaching English as Foreign Language from Swansea University, Wales and PhD in Second Language Acquisition from The Open University, UK. He became Fellow of Higher Education Academy (Advanced HE, UK) in 2020.

He has published and regularly reviews articles for leading journals in the area of Language Education (e.g., System, International Journal of Bilingual Education and Bilingualism, Language Testing Journal, Applied Linguistics Review, AsiaTEFL). He presents regularly at international conferences (e.g., BAAL, AAAL, and EuroSLA). He collaborates with experts across disciplines and beyond borders. For example, he is member of the Applied Linguistics, Language, and Literacy Lab at the University of Macau and the Mind and NeuroDevelopment Research Group at the University of Nottingham, Malaysia.