SCAI Eval 2024 Shared Task on Conversational Search Evaluation
==============================================================
Conversational search has many faces, and we want so see them all.
Rather than evaluating systems, this shared task calls on the community
to gather comprehensive evaluation resources to provide a clearer
picture of evaluation:
1. Example data of diverse search-oriented conversations;
2. Definitions and annotation guidelines for aspects to evaluate (e.g.,
correctness, coherence, readability); and
3. Implementations of evaluation metrics.
Both novel and already published contributions are welcome. Contributors
will be invited to present at the next SCAI workshop (venue to be
determined) and co-author a SIGIR Forum article. So join our mailing
list at https://groups.google.com/g/scai-info to stay up-to-date with
our task and get into contact.
Web: https://scai.info/scai-eval-2024/ (details, instructions, examples)
Mailing list: https://groups.google.com/g/scai-info
Submission deadlines:
- January 15, 2024: deadline for aspect definitions
- January 29, 2024: deadline for data and metrics
Task Committee:
- Johannes Kiesel, Bauhaus-Universität Weimar
- Maik Fröbe, Friedrich-Schiller-Universität Jena
- Andrea Papenmeier, University of Twente
- Alexander Frummet, Universität Regensburg
- Paul Owoicho, University of Glasgow
--
Johannes Kiesel
Bauhaus-Universität Weimar
Bauhausstr. 9a, Room 106
99423 Weimar, Germany
Phone: +49 (0)3643 - 58 3720
_______________________________________________
Corpora mailing list -- corpora@list.elra.info
https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/
To unsubscribe send an email to corpora-le...@list.elra.info
