RetroEval 2026: Symposium on Natural Language Generation Evaluations
============================================
1-2 June 2026, Aberdeen, Scotland, United Kingdom
https://retroeval.github.io/
============================================
Evaluation in the field of Natural Language Generation (NLG) has changed
considerably over the past several decades. This special symposium in honour of
Prof. Ehud Reiter’s retirement provides a forum for academic and industry
researchers to look back on the topic of how evaluations in the field of NLG
have changed and to explore unaddressed challenges. The two day symposium will
be held in-person at the Sir Duncan Rice Library in the historic University of
Aberdeen, June 1-2, 2026. For this symposium, we welcome submissions of long
papers, short papers, and extended abstracts.
*** Workshop Theme ***
Ehud Reiter has been a leading light in Natural Language Generation (NLG)
research throughout the four decades he worked in this area, in both academia
(Aberdeen) and industry (CoGenTech; Arria NLG). Ehud’s influence extends to all
aspects of NLG, but the areas in which it has arguably been the strongest is
evaluation of NLG systems. On the occasion of his retirement, this workshop,
which is held in his honour, will therefore focus on evaluation of NLG systems,
highlighting in particular some of the topics that Ehud has tended to emphasize
(see below) such as the importance of reproducibility and the risks of data
contamination.
*** Topics of Interest ***
Topics of interest include (but are not limited to) the following:
-- The aims of NLG and NLG evaluation
-- Intrinsic versus extrinsic NLG evaluation
-- Evaluation of NLG systems in the real world
-- Impact assessment of NLG systems and LLMs
-- New evaluation challenges arising from the use of LLMs
-- Hallucination annotation and its role in NLG evaluation
-- Statistical analysis for NLG evaluations
-- Data contamination in NLG evaluation
-- LLMs as evaluators: opportunities and pitfalls
-- The role of LLMs in the development of evaluation metrics
-- Reproduction and reproducibility of human evaluation experiments
-- Publication bias: What to do with negative results?
-- Pre-publication of research hypotheses in NLG evaluation
-- NLG evaluation versus psycholinguistic experimentation: what can we learn
from each other?
-- Disciplinary cultures and evaluation methods
-- Evaluating NLG systems/LLMs for assistive technology
*** Submission Types ***
The workshop accepts the following submission types:
• Long Papers (archival)
• Short Papers (archival)
• Extended Abstracts (non-archival)
Accepted contributions will be presented as oral or poster presentations.
*** Archival Submissions ***
• Long papers:
• Up to 8 pages (excluding references)
• Unlimited references
• Up to 2 appendix pages
• 1 additional page in the final version to address reviewer comments
• Short papers:
• Up to 4 pages (excluding references)
• Unlimited references
• Up to 1 appendix page
• 1 additional page in the final version for reviewer comments
*** Non-Archival Submissions ***
• Extended abstracts:
• Up to 2 pages including references
• 1 additional appendix page for tables/figures
• Selection based on the symposium fit
*** Submission Format ***
• Two-column ACL 2026 format
• LaTeX template only
• PDF submissions only
• Submissions via OpenReview
*** Important Dates ***
Note: All deadlines are 23:59 UTC-12.
• ARR commitment deadline (archival): 16 March, 2026
• Direct paper submission deadline (archival): 24 April, 2026
• Direct paper submission deadline (non-archival): 1 May, 2026
• Notification of acceptance: 8 May, 2026
• Camera-ready deadline: 22 May, 2026
• Symposium dates: 1-2 June, 2026
*** Review Policy ***
Long and short papers will follow ACL double-blind review policies. Submissions
must be anonymized, including self-references and links. Papers violating
anonymity requirements will be rejected without review.
Contact and Information
• Website: https://retroeval.github.io/
• Email: [email protected] <mailto:[email protected]>
Workshop Organisers: Saad Mahamood (Shopware), David Howcroft (University of
Aberdeen), Kees van Deemter (Utrecht University), Albert Gatt (Utrecht
University), Simone Balloccu (TU Darmstadt), Margaret Mitchell (Hugging Face),
Alberto Bugarín Diz (CiTIUS & University of Santiago de Compostela), Jose María
Alonso-Moral (CiTIUS & University of Santiago de Compostela), Adarsa
Sivaprasad (University of Aberdeen), Chenghua Lin (Manchester University), and
Alexandra Johnstone (University of Aberdeen). _______________________________________________
Corpora mailing list -- [email protected]
https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/
To unsubscribe send an email to [email protected]
