[ The Types Forum (announcements only), http://lists.seas.upenn.edu/mailman/listinfo/types-announce ]
Dear all, We are looking for motivated people to be members of the ICFP 2021 Artifact Evaluation Committee (AEC); students, researchers and people from the industry or the free-software community are all welcome. The artifact evaluation process aims to improve the quality and reproducibility of research artifacts for ICFP papers. You can either nominate yourself or nominate someone else (students, colleagues, etc; we will of course check later that this person is willing to be a committee member) by filling the nomination form. Nomination form: https://framaforms.org/icfp21-aec-nomination-form-1613482593 More information is available on the AEC webpage: https://icfp21.sigplan.org/track/icfp-2021-artifact-evaluation#advice-for-reviewers The primary responsibility of committee members is to review the artifacts submitted corresponding to the already accepted papers in the main research track. In particular, run the associated tool, check whether the results in the main paper can be reproduced, and inspect the tool and the data. We expect evaluation of one artifact to take about a full day, and each committee member to receive 2 to 3 artifacts to review. All of the AEC work will be done remotely/online. The ICFP AEC committee will start work in late spring of 2021, with the review work happening between May 26th and June 22nd. (The full schedule is on the AEC webpage.) Here are some reasons why we think nominee may like it: 1. Reviewing artifacts helps our community improve the quality of its artifacts, such as research software and mechanized proofs. Participating to this effort feels valuable and useful. 2. Nominees may learn some nice tricks by studying how other researchers implement, evaluate and document their ideas. 3. There is some evidence that members of the artifact-evaluation committee like the job: each year, more than half of them volunteer to be part of the committee again for the next year. (The cons: unpaid work, currently less prestigious than reviewing papers, the evaluation process is less clearly-defined and well-understood than for papers.) Come join us in improving the quality of research in our field! Best, the Artifact Evaluation Chairs: Gabriel Scherer and Brent Yorgey.