[issue41916] cxx pthread check is not overrideable
New submission from Dustin Spicuzza :
I'm cross-compiling python to ARM, following instructions from the crossenv
project @
https://crossenv.readthedocs.io/en/latest/quickstart.html#build-or-obtain-host-python
I was getting pthread related errors when using cross-built extension modules.
I realized the issue was that distutils wasn't passing -pthread to gcc.
On my native python (Fedora 32 Python 3.8):
>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC',
>>> 'CXX', 'LDSHARED')})
{'CC': 'gcc -pthread',
'CXX': 'g++ -pthread',
'LDSHARED': 'gcc -pthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now '
'-g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g'}
In my cross-compiled python environment:
>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC',
>>> 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc',
'CXX': 'arm-frc2020-linux-gnueabi-c++',
'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -shared'}
As you can see, no -pthread flag. I went digging in the configure script and
found this comment in configure.ac:2110:
# Some compilers won't report that they do not support -pthread,
# so we need to run a program to see whether it really made the
# function available.
... well, that doesn't work when cross-compiling. I looked at some other
variables set to override things for cross compiling, and tried adding
"ac_cv_pthread_is_default=no ac_cv_pthread=yes ac_cv_cxx_thread=yes" to my
cross-compile ./configure, and it's closer:
>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC',
>>> 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc -pthread',
'CXX': 'arm-frc2020-linux-gnueabi-c++',
'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -pthread -shared'}
Closer, but not there yet! It appears that the $ac_cv_cxx_thread is not
externally settable (unlike ac_cv_pthread), which I guess is because it's not
wrapped in a AC_CACHE_VAL macro?
It seems that the right fix would be for someone to use AC_CACHE_VAL to allow
$ac_cv_cxx_thread to be set externally? I'm happy to cargo cult some code from
other parts of configure.ac and make a PR for it, but it's not immediately
clear (a) what bad side effects this could cause and (b) what the best
mechanism to use to regenerate configure et al is?
--
components: Cross-Build
messages: 377858
nosy: Alex.Willmer, virtuald
priority: normal
severity: normal
status: open
title: cxx pthread check is not overrideable
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue41916>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41916] cxx pthread check is not overrideable
Change by Dustin Spicuzza : -- keywords: +patch pull_requests: +21530 stage: -> patch review pull_request: https://github.com/python/cpython/pull/22525 ___ Python tracker <https://bugs.python.org/issue41916> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41916] cxx pthread check is not overrideable
Dustin Spicuzza added the comment: FWIW, I've been using the patch in my PR and it seems to work. -- ___ Python tracker <https://bugs.python.org/issue41916> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29249] Pathlib glob ** bug
Dustin Spicuzza added the comment: I just ran into this also. It seems like a very strange omission that match and glob don't support the same patterns (and I'm surprised that they don't share more code). -- nosy: +virtuald ___ Python tracker <https://bugs.python.org/issue29249> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29249] Pathlib glob ** bug
Dustin Spicuzza added the comment: Because of backwards compatibility (despite a statement saying it's not guaranteed for pathlib), I think the best approach would be to create a 'globmatch' function for PurePath instead of modifying the match function, and document that the match function does a different kind of matching. This isn't a patch for cpython per se (ironically, don't have time for that this month...), but here's a MIT-licensed gist that patches pathlib2 and adds a globmatch function to it, plus associated tests extracted from pathlib2 and my own ** related tests. Works for me, feel free to do with it as you wish. https://gist.github.com/virtuald/dd0373bf3f26ec0730adf1da0fb929bb -- ___ Python tracker <https://bugs.python.org/issue29249> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29877] compileall hangs when accessing urandom even if number of workers is 1
Change by Dustin Spicuzza : -- keywords: +patch pull_requests: +4745 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue29877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29877] compileall hangs when accessing urandom even if number of workers is 1
Dustin Spicuzza added the comment: I'm sure that the platform (a RT linux customized by National Instruments) has issues related to urandom, as this has reared it's ugly head with other PEP 525 related issues also: https://bugs.python.org/issue29208 I'll have to try those suggestions to investigate the os.urandom issue. -- ___ Python tracker <https://bugs.python.org/issue29877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29877] compileall hangs when accessing urandom even if number of workers is 1
Dustin Spicuzza added the comment: Finally got around to looking at this. # cat /proc/sys/kernel/random/entropy_avail 183 .. ran python3 -c 'import os; os.getrandom(1, flags=os.GRND_NONBLOCK)' a few times, but didn't run into a hang. Seems like the entropy_avail is staying at about the same level? It's worth noting that I ran into this error with last year's firmware image, and I haven't actually tested this year's firmware image for this particular issue... just that I had it on my TODO list to fix for this year. :) I think it's still a good change to make regardless. -- ___ Python tracker <https://bugs.python.org/issue29877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29208] BlockingIOError during system startup
New submission from Dustin Spicuzza: I haven't dug into this very deeply yet, so I'm seeking some clarity on this issue before doing so. According to my reading of PEP 522 (https://www.python.org/dev/peps/pep-0522/), BlockingIOError will be raised in security sensitive APIs, but not when just importing the random module. However, it seems that this is not actually the case: BlockingIOError: [Errno 11] Resource temporarily unavailable The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/lvuser/py/robot.py", line 15, in import wpilib File "/usr/local/lib/python3.6/site-packages/wpilib/__init__.py", line 72, in from ._impl.main import run File "/usr/local/lib/python3.6/site-packages/wpilib/_impl/main.py", line 7, in from pkg_resources import iter_entry_points File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 36, in import email.parser File "/usr/local/lib/python3.6/email/parser.py", line 12, in from email.feedparser import FeedParser, BytesFeedParser File "/usr/local/lib/python3.6/email/feedparser.py", line 27, in from email._policybase import compat32 File "/usr/local/lib/python3.6/email/_policybase.py", line 9, in from email.utils import _has_surrogates File "/usr/local/lib/python3.6/email/utils.py", line 28, in import random File "/usr/local/lib/python3.6/random.py", line 742, in _inst = Random() SystemError: returned a result with an error set Background: I'm the maintainer of the RobotPy project, which allows students to use Python to control their robots in the FIRST Robotics competition. The robot controller we use is an NI roboRIO, which is powered by an ARM cortex-a9. This error was encountered when the robot is booting up with a python program. Presumably the program is started early enough in the boot process where the urandom blocking issue mentioned in the PEP occurs. I'd love some help figuring out what's going on here, and the best way to fix it. Thanks! I'm also tracking this issue at https://github.com/robotpy/robotpy-wpilib/issues/243 -- components: Library (Lib) messages: 284996 nosy: virtuald priority: normal severity: normal status: open title: BlockingIOError during system startup versions: Python 3.6 ___ Python tracker <http://bugs.python.org/issue29208> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29208] BlockingIOError during system startup
Dustin Spicuzza added the comment: Crap, after reading more carefully, I see that PEP was rejected in favor of PEP 524. According to that PEP, then I must implement the wait_for_system_rng() function? It's a bit weird, because I'm not explicitly using the random module and don't care to use it, I'm using pkg_resources... which is importing the email parser which imports random which creates a random instance which causes this bug. It seems to me then that the stdlib should be modified to avoid importing random unless they actually need it, otherwise this gets triggered. Or am I missing something here? -- ___ Python tracker <http://bugs.python.org/issue29208> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29208] BlockingIOError during system startup
Dustin Spicuzza added the comment: I'll try applying that fix tomorrow and see if the issue is addressed. -- ___ Python tracker <http://bugs.python.org/issue29208> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29208] BlockingIOError during system startup
Dustin Spicuzza added the comment: I'm able to confirm that the patch does indeed fix the problem. Thanks everyone! -- ___ Python tracker <http://bugs.python.org/issue29208> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29877] compileall fails with urandom error even if number of workers is 1
New submission from Dustin Spicuzza:
Found on Python 3.6 on a low-resource platform (NI RoboRIO), it seems that this
occurs only because the ProcessPoolExecutor is being imported. A proposed fix
would only import ProcessPoolExecutor if -j > 1. Stacktrace follows:
/usr/local/bin/python3 -m compileall -j 1 /home/lvuser/py
^CTraceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/compileall.py", line 20, in
from concurrent.futures import ProcessPoolExecutor
File "/usr/local/lib/python3.6/concurrent/futures/__init__.py", line 17, in
from concurrent.futures.process import ProcessPoolExecutor
File "/usr/local/lib/python3.6/concurrent/futures/process.py", line 53, in
import multiprocessing
File "/usr/local/lib/python3.6/multiprocessing/__init__.py", line 16, in
from . import context
File "/usr/local/lib/python3.6/multiprocessing/context.py", line 5, in
from . import process
File "/usr/local/lib/python3.6/multiprocessing/process.py", line 311, in
_current_process = _MainProcess()
File "/usr/local/lib/python3.6/multiprocessing/process.py", line 298, in
__init__
self._config = {'authkey': AuthenticationString(os.urandom(32)),
--
components: Library (Lib)
messages: 289973
nosy: virtuald
priority: normal
severity: normal
status: open
title: compileall fails with urandom error even if number of workers is 1
versions: Python 3.6
___
Python tracker
<http://bugs.python.org/issue29877>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29877] compileall hangs when accessing urandom even if number of workers is 1
Changes by Dustin Spicuzza : -- title: compileall fails with urandom error even if number of workers is 1 -> compileall hangs when accessing urandom even if number of workers is 1 ___ Python tracker <http://bugs.python.org/issue29877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29877] compileall hangs when accessing urandom even if number of workers is 1
Changes by Dustin Spicuzza : -- type: -> behavior ___ Python tracker <http://bugs.python.org/issue29877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
