Evaluating EFL Learner Agency and Critical Literacy during AI-assisted Comprehension and Production Tasks

Abstract

This mixed-methods pilot study investigates the interdependence of learner agency and critical AI literacy among 109 university EFL students across comprehension and production tasks. It evaluates whether learners actively use generative AI as a "Constitutive Digital Other" to scaffold learning, or passively default to a "Subordinate Surrogate". Behavioural observations and performance data reveal a severe decline in cognitive agency during AI interaction; notably, 62.4% of participants outsourced written production entirely to automated generation. Only a small fraction demonstrated the critical literacy required to actively modify prompts or detect errors. This uncritical, passive reliance engenders high levels of deceptive proficiency among students, resulting in strong formative scores that fail to yield summative academic performance. Consequently, the findings highlight the importance of self-regulated learning for learners to progress from automation to agency.