[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Ronald Oussoren added the comment: > My understanding is that this is specifically a problem with the Objective-C > runtime that _scproxy.c accesses. The runtime is not thread safe and > whereas in earlier versions of macOS, it silently failed, now macOS is > explicitly aborting the

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Barry A. Warsaw added the comment: My understanding is that this is specifically a problem with the Objective-C runtime that _scproxy.c accesses. The runtime is not thread safe and whereas in earlier versions of macOS, it silently failed, now macOS is explicitly aborting the process.

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Ronald Oussoren added the comment: I'm in favour of keeping integration with system proxy settings, that's more user friendly. Note that _scproxy is an example of triggering this using the standard library, the same may happen when using 3th-party libraries on PyPI if those happen to use

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Ned Deily added the comment: I wish there was a better solution but, as long as we want to continue to integrate with the system-provided proxy configuration, I guess we're kind of stuck. I'll try to expand Ronald's wording into a doc PR. -- ___

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Barry A. Warsaw added the comment: I agree that we should close this issue, possibly with a documentation fix. I wrote a blog posting about my investigations: https://wefearchange.org/2018/11/forkmacos.rst.html I don't think there's really much Python itself can do, and developers will

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Ronald Oussoren added the comment: I'm in favour of closing this issue. If anything needs to be done its adding a warning to the documentation of os.fork() to the effect that doing substantial work in the child can be problematic on macOS. Maybe something like: .. warning:: On macOS the

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Change by Barry A. Warsaw : -- versions: +Python 3.6, Python 3.7, Python 3.8 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Change by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

STINNER Victor added the comment: > 3) Find a workaround (such as calling the APIs used by _scproxy in a clean > supprocess). I dislike the idea of *always* spawning a child process, I prefer to leave it as it is, but add a recipe in the doc, or add a new helper

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Ronald Oussoren added the comment: Calling get_proxies() in a subprocess would also work, although I'd then prefer to use a small daemon proces to avoid the startup cost of a new process for every call to _scproxy functions. There is a conflict between two goals

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

STINNER Victor added the comment: Another workaround is to call get_proxies() in a fresh subprocess, and use a pipe to retrieve the result. -- ___ Python tracker

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

Ned Deily added the comment: Ronald is the expert on this but, from what I understand, I don't think there is any reason to spend time on trying to further analyze this. This issue has been around since day one of _scproxy and affects all versions of Python on macOS. There

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

STINNER Victor added the comment: @Mirko: You can please try to get the Python traceback of the Ansible crash? You may want to try faulthandler: enable it and write its output into a file. https://faulthandler.readthedocs.io/ --

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

STINNER Victor added the comment: Confirmation from Apple: https://developer.apple.com/library/content/technotes/tn2083/_index.html#//apple_ref/doc/uid/DTS10003794-CH1-SUBSECTION52 """ Many Mac OS X frameworks do not work reliably if you call fork but do not call

[issue31818] [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?

STINNER Victor added the comment: Hi, can you please explain how to reproduce your issue? According to the crash report, it seems like you are running Ansible on macOS and that the Python function _scproxy.get_proxies() was called. get_proxies() calls