[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The installers on Python.org no longer use the system version of libsqlite, 
sidestepping that issue. 

That doesn't fix the issue with _scproxy, but that's something we cannot easily 
fix in Python itself.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Eitan Adler]

Change by Eitan Adler :


--
nosy: +eadler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Chris Jerdonek]

Chris Jerdonek added the comment:

> We can't solve that problem; only Apple can;

> So, if we don't change _scproxy or urllib*'s use of it, only Apple can fix 
> the problem.

In the Django ticket I mentioned in my comment above, one of the commenters 
said, "Just ran the tests at the mentioned commit on my macOS Sierra public 
beta with a fresh 3.5.2 python environment. No problems there."

(https://code.djangoproject.com/ticket/27086#comment:4 )

In other words, the issue that affected me on Mac OS X El Capitan (whose root 
cause is this issue I believe) wasn't present in Sierra.

Do you think this means Apple has addressed the issue in the next version of 
its OS?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Chris Jerdonek]

Chris Jerdonek added the comment:

FWIW, I just came across an issue in Django's test suite that I believe is 
caused by the issue reported here. Some of Django's unit tests were hanging for 
me when run in "parallel" mode (which uses multiprocessing). Here is the ticket 
I filed there:
https://code.djangoproject.com/ticket/27086

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Chris Jerdonek]

Changes by Chris Jerdonek :


--
nosy: +chris.jerdonek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Evan Jones]

Evan Jones added the comment:

I have a crazy idea, but I'm not 100% sure how to implement it: If Python was 
able to detect and report this error in a friendly way, it would allow people 
to easily understand what is happening and to work around it. How can we do it?

First idea: In the implementation of os.fork(), detect if libdispatch has been 
used. If it has, throw an exception. I think this is probably possible using 
the libdispatch public APIs, but I'll need to figure out the details. In 
general, this could apply on Linux as well: throw an exception if the process 
has more than one thread running?

Second idea: On Mac OS X only, libdispatch is intentionally crashing the 
process. We could install a signal handler that attempts to detect *this 
specific crash* in order to throw a friendlier exception, or at worst crash 
with a useful message.

Third idea: Add documentation to the multiprocessing module and os.fork that 
they are very unsafe on Mac OS X?

Maybe there is a better way of making this crash "friendlier"?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Ronald Oussoren]

Ronald Oussoren added the comment:

There is a clear gap between two use cases for Python on OSX:

1) Integrate nicely with Apple stuff

2) Be useful as a development platform for software that will be deployed on 
Linux machines.

Both use cases are valid, but have slightly different trade-offs. I have both 
use-cases, but tend not to do things that cause problems here.  In any case I 
tend to favour the first use case, it is easy enough to develop locally and 
test on Linux (VMs, docker, ...) for the second use case.

A particular problem is using os.fork, as Evan notes there are problems with 
using Apples frameworks in programs that use fork without exec-ing a new 
program. Libdispatch is one problem, but higher-level frameworks are also 
problematic (the _scproxy problem is not only caused the use of libdispatch).

Spawning off a small executable for the _scproxy case could be worthwhile, 
although I haven't fully thought about the implications of that. I wouldn't 
like removing the use of _scproxy, it uses the documented programmatic way to 
access system-wide proxy configuration. Using something like scutil(1) might 
work as well, but is hackish.

Changing the default mode for multiprocessing could also be useful, but could 
result in calls about code that works on Linux but doesn't on OSX due to 
differences in fork mode.

BTW. Using fork without exec is unsafe in any program using threads, people 
tend to run into this more on OSX because Apple's libraries use background 
threads while other Unix-y platforms don't tend to do that.

As to shipping the sqlite with the CPython source: that's not something we do 
in general. It would be better to document the issue in the build instructions 
for CPython, and/or tweak the CPython build process to issue a warning when it 
detects that is using the system version of sqlite on OSX.

BTW2. Another problematic issue on OSX is accessing the system trust store for 
SSL certificates.  AFAIK the current binaries still use Apple's build of 
OpenSSL that uses a private API to access the system trust store, using a 
standalone OpenSSL would require some way to make OpenSSL use the trust store; 
either by shipping a script to dump the trust store in a format that OpenSSL 
can use or by using Apple's APIs to access the trust store. IIRC there is an 
issue about that, but I cannot find it at the moment. Using the public APIs 
might result in similar problems as mentioned in this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Evan Jones]

Evan Jones added the comment:

To be clear: My reproduction scripts crash both Python 2.7.10 and Python 3.5.1 
when you:

1. Download the source bundle from python.org.
2. Run ./configure; make
3. Use the built binary (because ./configure picks up the system version of 
libsqlite.dylib)

I did some more digging: The underlying root cause is Mac OS X's 
libdispatch.dylib. A ton of system APIs (like this proxy one, or GUI libraries, 
etc), use it. It seems the proxy settings API use it to manage inter-process 
communication. libdispatch has code that explicitly "poisons" the process if it 
forks. I think this is because it internally spawns threads, so the forked 
child state is unreliable, and they figure it is better for it to crash than to 
fail randomly. This is the classic "don't mix threads and fork" issue, its just 
that the threads are hidden inside a bunch of system APIs.

One fix for this particular bug would be for _scproxy to fork and use IPC to 
read the settings, which I think was mentioned in Issue13829. I think it would 
not also be crazy to ship the amalgamated sqlite3 with Python, to avoid an 
accidental dependency on sqlite3.

Finally: it might make sense to have 'forkserver' be the default mode for 
multiprocessing on Mac OS X, since there are other things that cause this same 
problem (Tkinter is reported on the internet).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Ned Deily]

Ned Deily added the comment:

Thanks for your analysis!  I'm sure that you are correct about Issue20353.  It 
could also be the root cause of other crashes reported when internet proxies 
are used with urllib* invoking the _scproxy helper extension module, as 
reported in Issue13829.

The question is what can be done.  There seem to be two separate issues here.  
The first is a crash when using Python's sqlite3 module, the problem your test 
case files demonstrate.  Note this *isn't* a problem when using the sqlite3 
module from any of the current python.org OS X binary installer Pythons (2.7 or 
3.x) because those Pythons do not use the Apple-supplied system libsqlite3; 
they already link with their own newer private copy.  Your py2 test, of course, 
does fail when using the Apple-supplied system Pythons.  We can't solve that 
problem; only Apple can; you could open a bug report with them.  If other 
distributors of Python on OS X rely on the system libsqlite3, they could avoid 
such crashes by also supplying their own copy.  For example, MacPorts already 
does, so your test cases don't fail with their Pythons.  I don't know what 
other distributors, like Homebrew or Anaconda, do.  We could add a note to 
Mac/README.

Second, the stickier (and totally separate) problem is what to do about 
_scproxy.  If, under the covers, the calls that _scproxy make to the 
Apple-supplied System Configuration framework use the un-forksafe Apple 
libsqlite3, there is nothing we can do about that; supplying a private copy of 
libsqlite3 isn't going to change what the framework uses and, in any case, it 
would be a really bad idea to even try to hack that.  So, if we don't change 
_scproxy or urllib*'s use of it, only Apple can fix the problem.  Since we 
can't expect that that is going to happen, the question becomes what 
alternatives are there.  One would be to find a way to eliminate _scproxy or 
its use of the unsafe SC framework calls.  Another approach would be to simply 
document ths restriction that urllib calls invoking ProxyHandler must be made 
in a main process (or whatever the precise restriction is) and leave it at that 
(https://docs.python.org/dev/library/urllib.request.html#urllib.request.ProxyHandler).

Ronald, what do you think?

--
title: Mac system sqlite3 not fork safe: Bundle a version? -> Apple-supplied 
libsqlite3 on OS X is not fork safe;  can cause crashes
type:  -> crash
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com