Il 16/07/25 16:19, Daniele Gambetta ha scritto:
non so se era già passato, ma in questi giorni tra colleghi se ne
parla molto..
https://aaai.org/aaai-launches-ai-powered-peer-review-assessment-system/
lì si legge
The pilot program will thoughtfully integrate LLM technology at two
specific points in the established review process:
1. Supplementary First-Stage Reviews: LLM-generated reviews will be
included as one component of the
initial review stage, providing an additional perspective
alongside traditional human expert evaluations.
2. Discussion Summary Assistance: LLMs will assist the Senior Program
Committee (SPC) members by
summarizing reviewer discussions, helping to highlight key points
of consensus and disagreement among
human reviewers.
riflessioni su questi temi erano già presenti nel 2019 in
Beta Writer. /Lithium-ion batteries. A machine-generated summary of
current research/. New York, NY: Springer, 2019.
https://doi.org/10.1007/978-3-030-16800-1.
nell'introduzione, scritta da Henning Schoenenberger:
machine-generated content makes it even more necessary to re-emphasize
the crucial role of peer-review itself. Though peer-review is also in
the course of being continuously re-defined (and in the future we
expect to see substantial progress in machine-support also in this
regard) we still think that for the foreseeable future we will need a
robust human review process for machine-generated text.
Especially in the area of Deep Learning it becomes increasingly
difficult to understand how a result has been actually derived. While
concepts such as Explainable Artificial Intelligence (XAI) become more
and more crucial, also the review process on machine-generated
research content needs refinement, if not a complete re-definition.
The term peer itself indicates a certain inadequacy for
machine-generated research content. Who are the peers in this context?
Would you as a human reader consider yourself as peer to a machine?
And should an expert in a specific research field become an expert of
neural networks and Natural Language Processing as well in order to be
able to evaluate the quality of a text and the related research? In
the field of machine-summarization of texts this might not be an issue
yet, especially since the underlying sources are peer-reviewed.
However, soon enough we will see machine-generated texts from
unstructured knowledge bases that will lead to more complex evaluation
processes. Also in this area, we have to work together to find answers
and define common standards related to machine-generated content.
come Shönenberger credo che l'attività di review prodotta da un LLM non
sia una peer review (l'LLM non è un peer dello studioso). ma credo che
un sistema di IA possa compiere su input del reviewer delle operazioni
di analisi testuale di supporto
M
------------------------------------------------------------------------
non credo a nessuna liberazione né individuale né collettiva
che si ottenga senza il costo di un’autodisciplina,
di un’autocostruzione, di uno sforzo
Italo Calvino
------------------------------------------------------------------------
Maurizio Lana
Università del Piemonte Orientale
Dipartimento di Studi Umanistici
Piazza Roma 36 - 13100 Vercelli
