[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

POWER6 AIX 3.x and PPC64 AIX 3.x buildbots are back to green.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

Fantastic! Many thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:


New changeset c846ef004d79ee8e9645d3e5e8b3b0cb97b5013f by Victor Stinner in 
branch 'master':
bpo-39936: _aix_support uses _bootsubprocess (GH-18970)
https://github.com/python/cpython/commit/c846ef004d79ee8e9645d3e5e8b3b0cb97b5013f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

I converted attached _aix_support.py into PR 18970 (with minor changes).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18318
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18970

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

The good news! Your patch, better rewrite, of _aix_platform.py is working!

Many thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

re: Michael: this issue is about bootstraping Python. If you want to test a 
patch or test something else, you must restart from a clean copy of the source 
code. Either use "make distclean", "git clean -fdx", or recreate the source 
directory (git clone, decompress an archive, etc.). Then restart from scratch: 
./configure && make.

Understood: between these two timestamps: Date: 2020-03-11 18:55 and Date: 
2020-03-11 21:36 - I was doing just that on 4 different VM's: AIX 5.3, 6.1, 7.1 
and 7.2 - just in case there may be anything specific re: AIX versions (the 
bots run on AIX 7.1 and 7.2). I also switched away from any NFS based builds 
because test behaves differently when the build area is an NFS area compared to 
jfs2.

FYI: I have decided to focus on AIX 7.1 and AIX 7.2 - and am only using gcc (I 
was also testing runs with both xlc and gcc before I started posting).

Further, I like your suggestion to try: subprocess except _bootsubprocess and 
shall make a PR to that end. Much more clear about what is going on.

What I am going to do bewtween now and my next post is again - clean slate - 
update master to latest version - on two servers.

On one server I'll apply your suggestion to _aix_platform.py; on the other I'll 
patch Lib/distutils/util.py and Lib/sysconfig.py to not even call 
_aix_platform.py. Ideally, there will be a difference that leads to an 
understanding of the root cause.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

Actually, I had already done that:

diff --git a/Lib/_aix_support.py b/Lib/_aix_support.py
index 2c5cd3297d..c7f4491633 100644
--- a/Lib/_aix_support.py
+++ b/Lib/_aix_support.py
@@ -12,7 +12,8 @@ try:
 _tmp_bd = get_config_var("AIX_BUILDDATE")
 _bgt = get_config_var("BUILD_GNU_TYPE")
 except ImportError:  # pragma: no cover
-_have_subprocess = False
+import _bootsubprocess as subprocess
+_have_subprocess = True
 _tmp_bd = None
 _bgt = "powerpc-ibm-aix6.1.7.0"

And after that test, I tried with no calls to _aix_support.py

diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py
index 4b002ecef1..513af52ecd 100644
--- a/Lib/distutils/util.py
+++ b/Lib/distutils/util.py
@@ -79,8 +79,9 @@ def get_host_platform():
 machine += ".%s" % bitness[sys.maxsize]
 # fall through to standard osname-release-machine representation
 elif osname[:3] == "aix":
-from _aix_support import aix_platform
-return aix_platform()
+return "%s-%s.%s" % (osname, version, release)
+# from _aix_support import aix_platform
+# return aix_platform()
 elif osname[:6] == "cygwin":
 osname = "cygwin"
 rel_re = re.compile (r'[\d.]+', re.ASCII)
diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
index 4003726dc9..b116e96007 100644
--- a/Lib/sysconfig.py
+++ b/Lib/sysconfig.py
@@ -666,8 +666,9 @@ def get_platform():
 machine += ".%s" % bitness[sys.maxsize]
 # fall through to standard osname-release-machine representation
 elif osname[:3] == "aix":
-from _aix_support import aix_platform
-return aix_platform()
+return "%s-%s.%s" % (osname, version, release)
+# from _aix_support import aix_platform
+# return aix_platform()
 elif osname[:6] == "cygwin":
 osname = "cygwin"
 import re



So, even when _aix_support.py is not involved at all, _socket.so is being 
ignored - even though it was built.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

Try to replace Lib/_aix_support.py with attached _aix_support.py: in short, it 
uses _bootsubprocess if subprocess is not available.

--
Added file: https://bugs.python.org/file48972/_aix_support.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

Michael: this issue is about bootstraping Python. If you want to test a patch 
or test something else, you must restart from a clean copy of the source code. 
Either use "make distclean", "git clean -fdx", or recreate the source directory 
(git clone, decompress an archive, etc.). Then restart from scratch: 
./configure && make.

> NameError: name 'sys' is not defined

Right, that's a stupid bug in my patch. You must import sys before.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

OK. I removed the _aix_support.py from the formula.

After make see the new (rather) old build paths for "socket"

aixtools@x064:[/home/aixtools/python-3.9]find . | grep socket
./Doc/howto/sockets.rst
./Doc/library/socket.rst
./Doc/library/socketserver.rst
./Lib/__pycache__/socket.cpython-39.opt-1.pyc
./Lib/__pycache__/socket.cpython-39.opt-2.pyc
./Lib/__pycache__/socket.cpython-39.opt-4.pyc
./Lib/__pycache__/socket.cpython-39.pyc
./Lib/__pycache__/socketserver.cpython-39.pyc
./Lib/socket.py
./Lib/socketserver.py
./Lib/test/__pycache__/mock_socket.cpython-39.pyc
./Lib/test/__pycache__/test_socket.cpython-39.pyc
./Lib/test/__pycache__/test_socketserver.cpython-39.pyc
./Lib/test/mock_socket.py
./Lib/test/test_socket.py
./Lib/test/test_socketserver.py
./Modules/socketmodule.c
./Modules/socketmodule.h
./PCbuild/_socket.vcxproj
./PCbuild/_socket.vcxproj.filters
./build/lib.aix-7.1-3.9/_socket.so
./build/temp.aix-7.1-3.9/home/aixtools/python-3.9/Modules/socketmodule.o

I remove these two files again and run make:

aixtools@x064:[/home/aixtools/python-3.9]make
 CC='xlc_r' LDSHARED='Modules/ld_so_aix xlc_r -bI:Modules/python.exp' 
OPT='-DNDEBUG -O'  _TCLTK_INCLUDES='' _TCLTK_LIBS=''  ./python -E ./setup.py  
build
running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
building '_asyncio' extension
xlc_r -DNDEBUG -O -I./Include/internal -I./Include -I. 
-I/home/aixtools/python-3.9/Include -I/home/aixtools/python-3.9 -c 
/home/aixtools/python-3.9/Modules/_asynciomodule.c -o 
build/temp.aix-7.1-3.9/home/aixtools/python-3.9/Modules/_asynciomodule.o
Modules/ld_so_aix xlc_r -bI:Modules/python.exp 
build/temp.aix-7.1-3.9/home/aixtools/python-3.9/Modules/_asynciomodule.o -o 
build/lib.aix-7.1-3.9/_asyncio.so
building '_socket' extension
xlc_r -DNDEBUG -O -I./Include/internal -I./Include -I. 
-I/home/aixtools/python-3.9/Include -I/home/aixtools/python-3.9 -c 
/home/aixtools/python-3.9/Modules/socketmodule.c -o 
build/temp.aix-7.1-3.9/home/aixtools/python-3.9/Modules/socketmodule.o
1500-030: (I) INFORMATION: PyInit__socket: Additional optimization may be 
attained by recompiling and specifying MAXMEM option with a value greater than 
8192.
Modules/ld_so_aix xlc_r -bI:Modules/python.exp 
build/temp.aix-7.1-3.9/home/aixtools/python-3.9/Modules/socketmodule.o -o 
build/lib.aix-7.1-3.9/_socket.so
*** WARNING: renaming "_asyncio" since importing it failed: No module named 
'_socket'

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_gdbm _lzma _sqlite3
_tkinter  readline
To find the necessary bits, look in setup.py in detect_modules() for the 
module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc  atexitpwd
time


Following modules built successfully but were removed because they could not be 
imported:
_asyncio

The diff!

diff --git a/Lib/_aix_support.py b/Lib/_aix_support.py
index 2c5cd3297d..c7f4491633 100644
--- a/Lib/_aix_support.py
+++ b/Lib/_aix_support.py
@@ -12,7 +12,8 @@ try:
 _tmp_bd = get_config_var("AIX_BUILDDATE")
 _bgt = get_config_var("BUILD_GNU_TYPE")
 except ImportError:  # pragma: no cover
-_have_subprocess = False
+import _bootsubprocess as subprocess
+_have_subprocess = True
 _tmp_bd = None
 _bgt = "powerpc-ibm-aix6.1.7.0"

diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py
index 4b002ecef1..513af52ecd 100644
--- a/Lib/distutils/util.py
+++ b/Lib/distutils/util.py
@@ -79,8 +79,9 @@ def get_host_platform():
 machine += ".%s" % bitness[sys.maxsize]
 # fall through to standard osname-release-machine representation
 elif osname[:3] == "aix":
-from _aix_support import aix_platform
-return aix_platform()
+return "%s-%s.%s" % (osname, version, release)
+# from _aix_support import aix_platform
+# return aix_platform()
 elif osname[:6] == "cygwin":
 osname = "cygwin"
 rel_re = re.compile (r'[\d.]+', re.ASCII)
diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
index 4003726dc9..b116e96007 100644
--- a/Lib/sysconfig.py
+++ b/Lib/sysconfig.py
@@ -666,8 +666,9 @@ def get_platform():
 machine += ".%s" % bitness[sys.maxsize]
 # fall through to standard osname-release-machine representation
 elif osname[:3] == "aix":
-from _aix_support import aix_platform
-return aix_platform()
+return "%s-%s.%s" % (osname, version, release)
+# from _aix_support import aix_platform
+# return aix_platform()
 elif osname[:6] == "cygwin":
 osname = "cygwin"
 import re

IMHO - this problem has nothing to do with _aix_support.py - That issue was 
fixed earlier by the new _bootsubprocess.

--

___
Python 

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Change by Michael Felt :


Added file: https://bugs.python.org/file48971/make.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

So, this is what I have on screen. Will add the log in a moment.

[1] +  Donemake 2>&1|tee make.log &
aixtools@x064:[/home/aixtools/python-3.9]find . | grep socket
./Doc/howto/sockets.rst
./Doc/library/socket.rst
./Doc/library/socketserver.rst
./Lib/__pycache__/socket.cpython-39.opt-1.pyc
./Lib/__pycache__/socket.cpython-39.opt-2.pyc
./Lib/__pycache__/socket.cpython-39.opt-4.pyc
./Lib/__pycache__/socket.cpython-39.pyc
./Lib/__pycache__/socketserver.cpython-39.pyc
./Lib/socket.py
./Lib/socketserver.py
./Lib/test/__pycache__/mock_socket.cpython-39.pyc
./Lib/test/__pycache__/test_socket.cpython-39.pyc
./Lib/test/__pycache__/test_socketserver.cpython-39.pyc
./Lib/test/mock_socket.py
./Lib/test/test_socket.py
./Lib/test/test_socketserver.py
./Modules/socketmodule.c
./Modules/socketmodule.h
./PCbuild/_socket.vcxproj
./PCbuild/_socket.vcxproj.filters
./build/lib.aix-7104-1806-32-3.9/_socket.so
./build/temp.aix-7104-1806-32-3.9/home/aixtools/python-3.9/Modules/socketmodule.o
aixtools@x064:[/home/aixtools/python-3.9]rm 
./build/lib.aix-7104-1806-32-3.9/_socket.so 
./build/temp.aix-7104-1806-32-3.9/home/aixtools/python-3.9/Modules/socketmodule.o

aixtools@x064:[/home/aixtools/python-3.9]make V=1
 CC='xlc_r' LDSHARED='Modules/ld_so_aix xlc_r -bI:Modules/python.exp' 
OPT='-DNDEBUG -O'  _TCLTK_INCLUDES='' _TCLTK_LIBS=''  ./python -E ./setup.py  
build
running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
building '_asyncio' extension
xlc_r -DNDEBUG -O -I./Include/internal -I./Include -I. 
-I/home/aixtools/python-3.9/Include -I/home/aixtools/python-3.9 -c 
/home/aixtools/python-3.9/Modules/_asynciomodule.c -o 
build/temp.aix-7104-1806-32-3.9/home/aixtools/python-3.9/Modules/_asynciomodule.o
Modules/ld_so_aix xlc_r -bI:Modules/python.exp 
build/temp.aix-7104-1806-32-3.9/home/aixtools/python-3.9/Modules/_asynciomodule.o
 -o build/lib.aix-7104-1806-32-3.9/_asyncio.so
building '_socket' extension
xlc_r -DNDEBUG -O -I./Include/internal -I./Include -I. 
-I/home/aixtools/python-3.9/Include -I/home/aixtools/python-3.9 -c 
/home/aixtools/python-3.9/Modules/socketmodule.c -o 
build/temp.aix-7104-1806-32-3.9/home/aixtools/python-3.9/Modules/socketmodule.o
1500-030: (I) INFORMATION: PyInit__socket: Additional optimization may be 
attained by recompiling and specifying MAXMEM option with a value greater than 
8192.
Modules/ld_so_aix xlc_r -bI:Modules/python.exp 
build/temp.aix-7104-1806-32-3.9/home/aixtools/python-3.9/Modules/socketmodule.o 
-o build/lib.aix-7104-1806-32-3.9/_socket.so
*** WARNING: renaming "_asyncio" since importing it failed: No module named 
'_socket'

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_gdbm _lzma _sqlite3
_tkinter  readline
To find the necessary bits, look in setup.py in detect_modules() for the 
module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc  atexitpwd
time


Following modules built successfully but were removed because they could not be 
imported:
_asyncio

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

So, with the patch - the process stops at:

aixtools@x064:[/home/aixtools/python-3.9]make
 CC='xlc_r' LDSHARED='Modules/ld_so_aix xlc_r -bI:Modules/python.exp' 
OPT='-DNDEBUG -O'  _TCLTK_INCLUDES='' _TCLTK_LIBS=''  ./python -E ./setup.py  
build
Traceback (most recent call last):
  File "/home/aixtools/python-3.9/Lib/subprocess.py", line 72, in 
import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aixtools/python-3.9/./setup.py", line 4, in 
import subprocess
  File "/home/aixtools/python-3.9/Lib/subprocess.py", line 77, in 
import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aixtools/python-3.9/./setup.py", line 16, in 
sys.modules['subprocess'] = _bootsubprocess
NameError: name 'sys' is not defined
make: 1254-004 The error code from the last command is 1.


Stop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

I'll take a look at what you are suggesting.

The starting point (before the rm command) is the make command that I run again.

What I notice - read am thinking - is that _socket.so is being created by the 
"setup.py build" - so, if you can help me make that bit more verbose, I'll work 
on that too.

+++
Script command is started on Wed Mar 11 15:29:44 CDT 2020.
$
$
$
$ rm ./build/lib.aix-7200-1543-32-3.9/_socket.so 
./build/temp.aix-7200-1543-32-3.9/home/aixtools/python/cpython-master/Modules/socketmodule.o
$
$
$ make V=1
 CC='gcc -pthread' LDSHARED='Modules/ld_so_aix gcc -pthread 
-bI:Modules/python.exp' OPT='-DNDEBUG -g -fwrapv -O3 -Wall'  
_TCLTK_INCLUDES='' _TCLTK_LIBS=''  ./python -E ./setup.py  build
running build
running build_ext
ldd: /lib/libreadline.a: File is an archive.
building '_asyncio' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall 
-std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter 
-Wno-missing-field-initializers -Werror=implicit-function-declaration 
-fvisibility=hidden -I./Include/internal -I./Include
 -I. -I/home/aixtools/python/cpython-master/Include 
-I/home/aixtools/python/cpython-master -c 
/home/aixtools/python/cpython-master/Modules/_asynciomodule.c -o  
build/temp.aix-7200-1543-32-3.9/home/aixtools/python/cpython-master/Modules/_asynciomodule.o
Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp 
build/temp.aix-7200-1543-32-3.9/home/aixtools/python/cpython-master/Modules/_asynciomodule.o
 -o build/lib.aix-7200-1543-32-3.9/_asyncio.so
building '_socket' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall 
-std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter 
-Wno-missing-field-initializers -Werror=implicit-function-declaration 
-fvisibility=hidden -I./Include/internal -I./Include
 -I. -I/home/aixtools/python/cpython-master/Include 
-I/home/aixtools/python/cpython-master -c 
/home/aixtools/python/cpython-master/Modules/socketmodule.c -o 
build/temp.aix-7200-1543-32-3.9/home/aixtools/python/cpython-master/Modules/socketmodule.o
Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp 
build/temp.aix-7200-1543-32-3.9/home/aixtools/python/cpython-master/Modules/socketmodule.o
 -o build/lib.aix-7200-1543-32-3.9/_socket.so
*** WARNING: renaming "_asyncio" since importing it failed: No module named 
'_socket'

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc  atexitpwd
time


Following modules built successfully but were removed because they could not be 
imported:
_asyncio

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

Yeah, sys must be imported first. Did you try to fix the patch? Does it work?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

I tried your patch on AIX 7.2, looks like you need to shift sys import to the 
top


Traceback (most recent call last):
  File "/home/isidentical/cpython/./setup.py", line 16, in 
sys.modules['subprocess'] = _bootsubprocess
NameError: name 'sys' is not defined
make: The error code from the last command is 1.


Stop.

--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

Another approach to patch _aix_support.py, replace:

try:
import subprocess
_have_subprocess = True
_tmp_bd = 
_bgt = get_config_var("BUILD_GNU_TYPE")
except ImportError:  # pragma: no cover
_have_subprocess = False
_tmp_bd = None
_bgt = "powerpc-ibm-aix6.1.7.0"

with:

try:
import subprocess
_have_subprocess = True
_tmp_bd = get_config_var("AIX_BUILDDATE")
_bgt = get_config_var("BUILD_GNU_TYPE")
except ImportError:  # pragma: no cover
import _bootsubprocess as subprocess
_have_subprocess = True
_tmp_bd = None
_bgt = "powerpc-ibm-aix6.1.7.0"

Use _bootsubprocess when subprocess is not available. If this approach works, 
_have_subprocess=False code path can be removed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

Michael.Felt: Can you please try to build the master branch of Python with the 
attached setup-aix.patch applied?

Something like:

git checkout master
git apply setup-aix.patch
./configure CFLAGS="-O0"
make 2>&1|tee make.log

Then attach make.log.

--
keywords: +patch
Added file: https://bugs.python.org/file48970/setup-aix.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[Michael Felt]

Michael Felt  added the comment:

While looking through the history of bot builds - I consistently see 420 tests 
being done on one bot - but test_socket does not always show up in the list.

I'll look at this as I can, but "free-time" is limited. Delay is not a lack of 
interest.

--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

STINNER Victor  added the comment:

AIX_BUILDDATE variable is written in pyconfig.h by configure:

> https://buildbot.python.org/all/#/builders/119/builds/383

configure: "checking for the system builddate... 1806"

> https://buildbot.python.org/all/#/builders/119/builds/451

configure: "checking for the system builddate... 1806"

So AIX_BUILDDATE seems to be set in both case.

But on build 451, the "build/lib.aix-7100-9898-32-3.9-pydebug" directory name 
contains 9898: this number comes from _aix_support._MISSING_BD. It's used when 
_have_subprocess is false or when get_config_var("AIX_BUILDDATE") cannot be 
converted into an integer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39936] Python fails to build _asyncio on module on AIX

[STINNER Victor]

New submission from STINNER Victor :

The commit 1ec63b62035e73111e204a0e03b83503e1c58f2e of bpo-39763 broke the 
Python compilation on AIX:
https://bugs.python.org/issue39763#msg363749

--

The last successful build was before the commit 
1ec63b62035e73111e204a0e03b83503e1c58f2e:
https://buildbot.python.org/all/#/builders/119/builds/383

_socket compilation:
(...) -o build/lib.aix-7100-9898-32-3.9-pydebug/_socket.so

pythoninfo:

sys.path: [
  '/home/buildbot/buildarea/3.x.aixtools-aix-power6/build',
  
'/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/target/lib/python39.zip',
  '/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib',
  
'/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/build/lib.aix-7100-9898-32-3.9-pydebug',
  '/home/buildbot/.local/lib/python3.9/site-packages']

Both steps use "build/lib.aix-7100-9898-32-3.9-pydebug/" directory.

--

Recent failure:
https://buildbot.python.org/all/#/builders/119/builds/451

_socket compilation:

(...) -o build/lib.aix-7104-1806-32-3.9-pydebug/_socket.so

pythoninfo:

sys.path: [
  '/home/buildbot/buildarea/3.x.aixtools-aix-power6/build',
  
'/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/target/lib/python39.zip',
  '/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib',
  
'/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/build/lib.aix-7100-9898-32-3.9-pydebug',
  '/home/buildbot/.local/lib/python3.9/site-packages']

So the compilation uses "build/lib.aix-7104-1806-32-3.9-pydebug/" directory, 
whereas pythoninfo uses "build/lib.aix-7100-9898-32-3.9-pydebug/" directory.

It can explain why setup.py fails to import _socket later: it was built in a 
different directory.

--

I see that _aix_support._aix_bosmp64() has two code paths depending on the 
subprocess module can be imported:

# subprocess is not necessarily available early in the build process
# if not available, the config_vars are also definitely not available
# supply substitutes to bootstrap the build
try:
import subprocess
_have_subprocess = True
_tmp_bd = get_config_var("AIX_BUILDDATE")
_bgt = get_config_var("BUILD_GNU_TYPE")
except ImportError:  # pragma: no cover
_have_subprocess = False
_tmp_bd = None
_bgt = "powerpc-ibm-aix6.1.7.0"

def _aix_bosmp64():
# type: () -> Tuple[str, int]
"""
Return a Tuple[str, int] e.g., ['7.1.4.34', 1806]
The fileset bos.mp64 is the AIX kernel. It's VRMF and builddate
reflect the current ABI levels of the runtime environment.
"""
if _have_subprocess:
# We expect all AIX systems to have lslpp installed in this location
out = subprocess.check_output(["/usr/bin/lslpp", "-Lqc", "bos.mp64"])
out = out.decode("utf-8").strip().split(":")  # type: ignore
# Use str() and int() to help mypy see types
return str(out[2]), int(out[-1])
else:
from os import uname

osname, host, release, version, machine = uname()
return "{}.{}.0.0".format(version, release), _MISSING_BD

--

_aix_support._aix_bosmp64() is called by _aix_support.aix_platform() which is 
called by get_host_platform() of distutils.util.

Currently, setup.py does:

* Inject _bootsubprocess into sys.modules['subprocess'] so "import subprocess" 
works
* Build all C extensions
* Remove sys.modules['subprocess'], so the next "import subprocess" may or may 
not load Lib/subprocess.py which uses the newly built C extensions like 
_posixsubprocess and select
* Attempt to load C extensions: if an import fails, rename the file: it happens 
for _asyncio on AIX, that's the bug

--
components: Build
messages: 363946
nosy: vstinner
priority: normal
severity: normal
status: open
title: Python fails to build _asyncio on module on AIX
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com