On 14/04/2021 19.02, Chris Lamb wrote: > A quick update: as permitted by IEEE, the paper is now available in an > open access / preprint capacity: > > https://ieeexplore.ieee.org/document/9403390 > https://arxiv.org/abs/2104.06020
I reviewed the latter and found some issues: > doing so is inefficient when source code is available for audit was very confusing to read. I read it multiple times and understood it as "source code makes audit inefficient" until some time later re-reading with more context. Should be something about "auditing source-code is more efficient than auditing binaries" > The mechanics of reproducibility testing suggest that this issue would not > have been readily discovered another way. not sure if mechanics are people here or mechanisms - and not sure how either would suggest something. Why not "We believe that..." or "Our experience (in rb) leads us to think..." ? > However, this has not yet been achieved, partly because time and effort are > not inexhaustible or fungible resources in volunteer communities This is hard to parse, not only because of the double-negation ("not in-"). Does it mean: Engineers have limited time and volunteers even more so? And 'fungible' means you can not just put a noob's hour in and achieve as much as an expert-hour? For the list of common issues: code compiling with -march=native is a common occurrence that also is a bug found easily by rb. I often find that in our HPC and science package sections. In the debugging section, you only mentioned looking at diffoscope output. Did you consider adding some of the other useful ways mentioned in section 2 of https://github.com/bmwiedemann/reproducibleopensuse/blob/devel/howtodebug ? and some grammar fixes: -a extremely mature +an extremely mature -tool that recursively unpacks a large number of archive formats and translate tens of binary formats +tool that recursively unpacks a large number of archive formats and translates tens of binary formats Ciao Bernhard M.
OpenPGP_signature
Description: OpenPGP digital signature