[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Gilles Duboscq]

Gilles Duboscq  added the comment:

Thanks Victor we'll look into moving away from this pattern.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

> For us it happens when using multiprocessing: the main process forks 2 
> processes and both use urlopen at roughly the same time. We are seeing this 
> on Python 3.7.2 on macOS 10.14.3.

>From what I understood, calling fork() (and then continue to execute regular 
>Python code) is no longer safe in macOS 10.14 and must no longer be used. Only 
>fork+exec is safe (spawn a new child process), but posix_spawn() is preferred 
>on macOS. But I'm not a macOS expert.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

Oh wait, I removed that lock:

commit 0de437de6210c2b32b09d6c47a805b23d023bd59
Author: Victor Stinner 
Date:   Thu May 28 17:23:39 2020 +0200

bpo-25920: Remove socket.getaddrinfo() lock on macOS (GH-20177)

On macOS, socket.getaddrinfo() no longer uses an internal lock to
prevent race conditions when calling getaddrinfo(). getaddrinfo is
thread-safe is macOS 10.5, whereas Python 3.9 requires macOS 10.6 or
newer.

The lock was also used on FreeBSD older than 5.3, OpenBSD older than
201311 and NetBSD older than 4.

Please open a new issue specific to macOS: specify your macOS version, your 
Python version and how you installed or built Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Gilles Duboscq]

Gilles Duboscq  added the comment:

I get '0' so it was not built with HAVE_GETHOSTBYNAME_R.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

"Segmentation fault with (...) threads (...) getaddrinfo"

Aha, another victim on a getaddrinfo() implementation which is not thread safe.

See this code in Modules/socketmodule.c:

/* Lock to allow python interpreter to continue, but only allow one
   thread to be in gethostbyname or getaddrinfo */
#if defined(USE_GETHOSTBYNAME_LOCK)
static PyThread_type_lock netdb_lock;
#endif

Can you please check if your Python was built with HAVE_GETHOSTBYNAME_R?

$ python3
Python 3.9.1 (default, Jan 20 2021, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
>>> import sysconfig; repr(sysconfig.get_config_var('HAVE_GETHOSTBYNAME_R'))
'1'


Modules/socketmodule.c is full of #ifdef involving macOS...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Gilles Duboscq]

Gilles Duboscq  added the comment:

I'm not sure it's the same but we have seen stack traces looking like the one 
there: https://github.com/astropy/astropy/issues/9699

Current thread 0x7fffa857e3c0 (most recent call first):
  File "/sw/lib/python3.7/socket.py", line 748 in getaddrinfo
  File "/sw/lib/python3.7/socket.py", line 707 in create_connection
  File "/sw/lib/python3.7/http/client.py", line 938 in connect
  File "/sw/lib/python3.7/http/client.py", line 966 in send
  File "/sw/lib/python3.7/http/client.py", line 1026 in _send_output
  File "/sw/lib/python3.7/http/client.py", line 1247 in endheaders
  File "/sw/lib/python3.7/http/client.py", line 1298 in _send_request
  File "/sw/lib/python3.7/http/client.py", line 1252 in request

For us it happens when using multiprocessing: the main process forks 2 
processes and both use urlopen at roughly the same time. We are seeing this on 
Python 3.7.2 on macOS 10.14.3.

--
nosy: +gilles-duboscq

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

If someone manages to reproduce the bug, please provide *at least* the Python 
traceback. You can use faulthandler to get it. It would also be very useful to 
get the C call stack using gdb (where command in gdb).

In the meanwhile, I close the issue since only the reporter reproduced the 
crashed.

Please provide more information about your OS, OS version, how you installed 
Python, Python package version, etc. if you want to reopen the issue.

--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Manjusaka]

Manjusaka  added the comment:

Hello Victor

I have tried on both MacOS and Ubuntu 18.04 from 3.8.2 to the newest code in 
master, and can't reproduce this problem

macOS-10.15.4-x86_64-i386-64bit
Python 3.9.0a4+ (heads/master:9a81ab107a, Mar 24 2020, 02:06:30)
[Clang 11.0.3 (clang-1103.0.32.26)]

Linux-4.15.0-1060-aws-x86_64-with-glibc2.27
Python 3.8.2 (default, Mar 23 2020, 19:21:16)
[GCC 8.3.0]

Linux-4.15.0-1060-aws-x86_64-with-glibc2.27
Python 3.9.0a4 (default, Mar 23 2020, 19:25:43)
[GCC 8.3.0]

I think it may be caused by the kernel version and the glibc version, I don't 
have enough evidence to prove it.

I will try the Python 3.7.6 to the newest version in Ubuntu 19.10 with Kernel 
5.3.0-29 and GCC 8.3.0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

Try to get the gdb traceback (C call stack).

You may also try: python3 -X faulthandler urllib_segfault.py.

But I expect a crash in Python finalization, where there is no Python frame and 
so empty traceback.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

Python 3.7.3 is outdated. Can you please try on newer Python version?


I failed to reproduce the crash on Fedora 31. I tested:

vstinner@apu$ python3.7 -VV
Python 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

vstinner@apu$ python3.8 -VV
Python 3.8.2 (default, Feb 26 2020, 00:00:00) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

vstinner@apu$ python3.9 -VV
Python 3.9.0a4 (default, Feb 27 2020, 00:00:00) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

vstinner@apu$ python3-debug -VV
Python 3.7.6 (default, Jan 30 2020, 09:04:19) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Manjusaka]

Manjusaka  added the comment:

use the same GCC 8.3.0 to recompile the Python 3.7.3 still no fault

maybe we need the core dump to figure it out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Manjusaka]

Manjusaka  added the comment:

I have tried it on Python 3.7.3 Ubuntu 18.04 

Linux-4.15.0-1060-aws-x86_64-with-debian-buster-sid
Python 3.7.3 (default, Mar 23 2020, 18:15:26)
[GCC 7.5.0]

there is no segmentation fault 

would you mind sharing the core dump to help us find more detail about the 
crash?

--
nosy: +Manjusaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Anne Archibald]

New submission from Anne Archibald :

This was discovered in the astropy test suite, where ThreadPoolExecutor is used 
to concurrently launch a lot of urllib.request.urlopen. This occurs when the 
URLs are local files; I'm not sure about other URL schemes.

The problem appears to occur in python 3.7 but not python 3.8 or python 3.6 (on 
a different machine).

$ python urllib_segfault.py
Linux-5.3.0-29-generic-x86_64-with-Ubuntu-19.10-eoan
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0]
Segmentation fault (core dumped)
$ python3.8 urllib_segfault.py
Linux-5.3.0-29-generic-x86_64-with-glibc2.29
Python 3.8.0 (default, Oct 28 2019, 16:14:01) 
[GCC 9.2.1 20191008]
$ python3 urllib_segfault.py 
Linux-4.15.0-88-generic-x86_64-with-Ubuntu-18.04-bionic
Python 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]
$

The Astropy bug report: https://github.com/astropy/astropy/issues/10008

--
components: Library (Lib)
files: urllib_segfault.py
messages: 363374
nosy: Anne Archibald
priority: normal
severity: normal
status: open
title: Segmentation fault with urllib.request.urlopen and threads
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48950/urllib_segfault.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com