[issue24421] Race condition compiling Modules/_math.c

[Martin Panter]

Changes by Martin Panter :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Roundup Robot]

Roundup Robot added the comment:

New changeset d4190ed586a4 by Martin Panter in branch '2.7':
Issue #24421: Compile _math.c separately to avoid race condition
https://hg.python.org/cpython/rev/d4190ed586a4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Roundup Robot]

Roundup Robot added the comment:

New changeset a99580157968 by Martin Panter in branch '3.5':
Issue #24421: Compile _math.c separately to avoid race condition
https://hg.python.org/cpython/rev/a99580157968

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Roundup Robot]

Roundup Robot added the comment:

New changeset 76624d47ee99 by Martin Panter in branch 'default':
Issue #24421: Compile _math.c separately to avoid race condition
https://hg.python.org/cpython/rev/76624d47ee99

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Martin Panter]

Martin Panter added the comment:

Here is an alternative patch to compile _math.c once from the main makefile, 
before the extension modules are built. I have only tested this on Linux, 
building in the source directory, and from a separate directory.

--
versions:  -Python 3.4
Added file: http://bugs.python.org/file41270/math-once.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Martin Panter]

Martin Panter added the comment:

Ideally, I think the solution would be to compile _math.c once, as a dependency 
shared between both extension modules (perhaps like a static library if 
necessary). Does anyone know if the build system supports this, or how to do it?

Another semi-favourable solution might be to compile _math.c twice, but send 
the output to separate directories, e.g. . . ./math-build/_math.o and . . 
./cmath-build/_math.o. Again, I don’t know if this is practical.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Mike Gilbert]

Changes by Mike Gilbert :


--
nosy: +floppymaster

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Tal Einat]

Tal Einat added the comment:

Adding Thomas Wouters to the nosy list, since he's listed on the Experts Index 
under autoconf/makefiles as an interest area. Hopefully he can help move this 
forward.

--
nosy: +twouters

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24421
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Martin Panter]

Martin Panter added the comment:

Here is a hacky patch that adds #include _math.c so that it is not compiled 
as a separate object file. This was suggested by Mark in Issue 7518. It is far 
from a perfect solution, but I cannot suggest anything better without knowing 
more about Python’s build system.

--
keywords: +patch
stage:  - patch review
Added file: http://bugs.python.org/file39724/include-math.c.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24421
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Martin Panter]

Martin Panter added the comment:

I think this may have been introduced when _math.c was added as a source file 
of the “cmath” module in r76978 (Issue 7518). In /setup.py:585 there are two 
distutils.core.Extension objects, both mentioning _math.c. In the 
build_ext._build_extensions_parallel() method at 
/Lib/distutils/command/build_ext.py:449, it appears to be invoking 
build_extension() in two concurrent threads, once for each module, including 
math and cmath.

I don’t know how this code is meant to work, but it looks like either the 
“build_ext” class should not be creating the same object file in two different 
threads, or /setup.py should not be giving it the same _math.c file for two 
different modules.

--
components: +Distutils
nosy: +dstufft, eric.araujo
versions: +Python 2.7, Python 3.4, Python 3.5, Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24421
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Tal Einat]

Tal Einat added the comment:

Confirmed on OSX 10.10. Here's my output:

running build
running build_ext
building 'cmath' extension
./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g 
-fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement 
-I./Include -I. -IInclude -I/usr/local/include 
-I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c 
/Users/taleinat/dev/cpython/Modules/cmathmodule.c -o 
build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/cmathmodule.o
building 'math' extension
./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g 
-fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement 
-I./Include -I. -IInclude -I/usr/local/include 
-I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c 
/Users/taleinat/dev/cpython/Modules/mathmodule.c -o 
build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/mathmodule.o
./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g 
-fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement 
-I./Include -I. -IInclude -I/usr/local/include 
-I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c 
/Users/taleinat/dev/cpython/Modules/_math.c -o 
build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o
./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g 
-fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement 
-I./Include -I. -IInclude -I/usr/local/include 
-I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c 
/Users/taleinat/dev/cpython/Modules/_math.c -o 
build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o
INITIAL COMPILE
RECOMPILE
RECOMPIILE TRUNCATING
INITIAL COMPILE FINISHED
./slow-cc.py -bundle -undefined dynamic_lookup 
build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/mathmodule.o
 build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o 
-L/usr/local/lib -o build/lib.macosx-10.10-x86_64-3.6/math.cpython-36m-darwin.so
ld: file too small (length=0) file 
'build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o' 
for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
RECOMPILE FINISHED
./slow-cc.py -bundle -undefined dynamic_lookup 
build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/cmathmodule.o
 build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o 
-L/usr/local/lib -o 
build/lib.macosx-10.10-x86_64-3.6/cmath.cpython-36m-darwin.so

--
nosy: +taleinat

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24421
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Martin Panter]

New submission from Martin Panter:

Once or twice I have had the math or cmath modules fail to build. There seems 
to be a race condition, because the _math.c file seems to be compiled twice, 
once for each module. Attached is a GCC wrapper that should help reproduce this 
fairly consistently. First, build Python normally:

$ ./configure
$ make -j2

Then force Modules/_math.c to be recompiled using my wrapper script to amplify 
the race condition:

$ touch Modules/_math.c
$ rm 
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o
$ make -j2 CC=./slow-cc.py
running build
running build_ext
building 'cmath' extension
compiling for  cmath
./slow-cc.py [. . .]/Modules/cmathmodule.o
building 'math' extension
./slow-cc.py [. . .]/Modules/mathmodule.o
./slow-cc.py -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code 
-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes 
-Werror=declaration-after-statement -I./Include -I. -IInclude 
-I/usr/local/include -I/media/disk/home/proj/python/cpython/Include 
-I/media/disk/home/proj/python/cpython -c 
/media/disk/home/proj/python/cpython/Modules/_math.c -o 
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o
INITIAL COMPILE
./slow-cc.py -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code 
-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes 
-Werror=declaration-after-statement -I./Include -I. -IInclude 
-I/usr/local/include -I/media/disk/home/proj/python/cpython/Include 
-I/media/disk/home/proj/python/cpython -c 
/media/disk/home/proj/python/cpython/Modules/_math.c -o 
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o
RECOMPILE
RECOMPIILE TRUNCATING
INITIAL COMPILE FINISHED
./slow-cc.py -shared 
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/mathmodule.o
 
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o 
-L/usr/local/lib -lm -o 
build/lib.linux-x86_64-3.6/math.cpython-36m-x86_64-linux-gnu.so
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o:
 file not recognized: File truncated
collect2: error: ld returned 1 exit status
RECOMPILE FINISHED
./slow-cc.py -shared 
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/cmathmodule.o
 
build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o 
-L/usr/local/lib -lm -o 
build/lib.linux-x86_64-3.6/cmath.cpython-36m-x86_64-linux-gnu.so

Failed to build these modules:
math

--
components: Build
files: slow-cc.py
messages: 245121
nosy: vadmium
priority: normal
severity: normal
status: open
title: Race condition compiling Modules/_math.c
type: behavior
Added file: http://bugs.python.org/file39672/slow-cc.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24421
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com