New submission from STINNER Victor <vstin...@python.org>:
When building the Fedora package of Python 3.10, *sometimes* the "make
regen-all" fail to build with errors like:
------------
/builddir/build/BUILD/Python-3.9.0/Modules/_weakref.c:131:5: error:
'_WEAKREF_GETWEAKREFCOUNT_METHODDEF' undeclared here (not in a function)
131 | _WEAKREF_GETWEAKREFCOUNT_METHODDEF
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/builddir/build/BUILD/Python-3.9.0/Modules/_weakref.c:132:5: error: expected
'}' before '_WEAKREF__REMOVE_DEAD_WEAKREF_METHODDEF'
132 | _WEAKREF__REMOVE_DEAD_WEAKREF_METHODDEF
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------
Example: https://src.fedoraproject.org/rpms/python3.9/pull-request/40
The _WEAKREF_GETWEAKREFCOUNT_METHODDEF macro is defined by
Modules/clinic/_weakref.c.h which is included by Modules/_weakref.c (at line
13).
It seems like while Modules/_weakref.o was being built,
Modules/clinic/_weakref.c.h was being generated "at the same time".
The "make regen-all" command runs "make clinic" and "make regen-importlib"
targets:
* "make regen-importlib" builds Modules/_weakref.o from Modules/_weakref.c and
Modules/clinic/_weakref.c.h
* "make clinic" always rewrites "Modules/clinic/_weakref.c.h" file
On Fedora, packages are built using MAKEFLAGS="-j<N>" where <N> is a number,
like "-j4". Since there is no dependency between "clinic" and "regen-importlib"
targets, these two targets *can* be run in parallel.
It seems like "make clinic" always rewrites *all* generated files and the half
of writes are *not* atomic.
Attached PR makes all writes of clinic.py atomic *and* also avoids modifying
the file modification file if the content does not change, as done in other
"make regen-all" targets (using Tools/scripts/update_file.py).
----------
components: Build
messages: 381325
nosy: vstinner
priority: normal
severity: normal
status: open
title: Race condition in "make regen-all" when running jobs in parallel
versions: Python 3.10, Python 3.8, Python 3.9
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue42398>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
