*Workshop co-located with EACL, 24-29 March 2026, Rabat, Morocco* LLMs in so many languages? When can such a claim be substantiated, and how should it be evaluated? This workshop brings together the community to answer these questions through three goals: - Establish a dedicated venue for multilingual evaluation, including resources, metrics, and methodologies; - Advance and standardize evaluation practices to enhance accuracy, scalability, fairness, and cross-system comparability; - Integrate cultural and social dimensions into multilingual evaluation.
*Call for Papers* We invite archival (ACL Anthology) or non-archival submissions, in ACL's short or long format. We accept both direct and ARR-reviewed submissions. Topics include, but are not limited to: - Evaluation resources beyond English or Western-centric perspectives and materials; - Annotation methodology and procedures; - Standardised reporting, and scientific comparison of multilingual performance; - Evaluation protocols: ranking vs direct assessment, rubric-based vs reference-based vs reference-free, prompt variations, etc; - Metrics, LLM judges, and reward models; - Complex tasks: multimodality, fairness, long I/O, tool using, code-switching, literary, etc; - Sociocultural and cognitive variation affecting the use and evaluation across languages; - Scalable evaluation of cultural and factual knowledge; - Efficient evaluation of a massive number of languages and tasks; - AI-assisted evaluation: data, methods, metrics, and standards; - Other position, application-, or theory-focused contributions. *Dates* (tentative, all dates are 23:59 AoE) - Direct submission deadline: 19 Dec 2025 - Pre-reviewed ARR submission deadline: 02 Jan 2026 - Notification of acceptance: 23 Jan 2026 - Camera-ready version due: 03 Feb 2026 - Workshop date: 28 or 29 Mar 2026 --- Website: https://multilingual-multicultural-evaluation.github.io/ Email: [email protected] Submission: OpenReview, link TBD Best regards, Pinzhen Chen on behalf of MME organizers _______________________________________________ Corpora mailing list -- [email protected] https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/ To unsubscribe send an email to [email protected]
