Re: python 3 subprocess performance

2019-04-14 Thread Jan Beich
Kubilay Kocak writes: > On 12/04/2019 8:41 pm, Dima Pasechnik wrote: > >> On Fri, Apr 12, 2019 at 9:46 AM Alexander Zagrebin wrote: >>> >>> В Fri, 12 Apr 2019 09:36:13 +0200 >>> Dima Pasechnik пишет: >>> On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin wrote: > > В Thu, 11

Re: python 3 subprocess performance

2019-04-13 Thread Kubilay Kocak
On 12/04/2019 8:41 pm, Dima Pasechnik wrote: On Fri, Apr 12, 2019 at 9:46 AM Alexander Zagrebin wrote: В Fri, 12 Apr 2019 09:36:13 +0200 Dima Pasechnik пишет: On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin wrote: В Thu, 11 Apr 2019 17:32:42 +0200 Jan Bramkamp пишет: The reason is

Re: python 3 subprocess performance

2019-04-12 Thread Alexander Zagrebin
В Fri, 12 Apr 2019 12:41:53 +0200 Dima Pasechnik пишет: > > > > В Thu, 11 Apr 2019 17:32:42 +0200 > > > > Jan Bramkamp пишет: > > > > > > > > > The reason is that that python does something stupid (tm). It > > > > > tries to close all file descriptors (except a few whitelisted > > > > > ones)

Re: python 3 subprocess performance

2019-04-12 Thread Dima Pasechnik
On Fri, Apr 12, 2019 at 9:46 AM Alexander Zagrebin wrote: > > В Fri, 12 Apr 2019 09:36:13 +0200 > Dima Pasechnik пишет: > > > On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin > > wrote: > > > > > > В Thu, 11 Apr 2019 17:32:42 +0200 > > > Jan Bramkamp пишет: > > > > > > > The reason is that

Re: python 3 subprocess performance

2019-04-12 Thread Alexander Zagrebin
В Fri, 12 Apr 2019 09:36:13 +0200 Dima Pasechnik пишет: > On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin > wrote: > > > > В Thu, 11 Apr 2019 17:32:42 +0200 > > Jan Bramkamp пишет: > > > > > The reason is that that python does something stupid (tm). It > > > tries to close all file

Re: python 3 subprocess performance

2019-04-12 Thread Dima Pasechnik
On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin wrote: > > В Thu, 11 Apr 2019 17:32:42 +0200 > Jan Bramkamp пишет: > > > The reason is that that python does something stupid (tm). It tries to > > close all file descriptors (except a few whitelisted ones) up to the > > maximum file descriptor

Re: python 3 subprocess performance

2019-04-12 Thread Alexander Zagrebin
В Thu, 11 Apr 2019 17:32:42 +0200 Jan Bramkamp пишет: > The reason is that that python does something stupid (tm). It tries to > close all file descriptors (except a few whitelisted ones) up to the > maximum file descriptor number. It does this by asking the kernel for > the maximum possible

Re: python 3 subprocess performance

2019-04-11 Thread Jan Bramkamp
The reason is that that python does something stupid (tm). It tries to close all file descriptors (except a few whitelisted ones) up to the maximum file descriptor number. It does this by asking the kernel for the maximum possible number and closing everything it doesn't want to keep. Some time

Re: python 3 subprocess performance

2019-04-11 Thread Konstantin Schukraft
Please run python under truss -f. Does it try to close(2) all possible file descriptors? Does the runtime decrease if fdescfs is mounted at /dev/fd? truss -f -p 17407 17407: read(5,0x1265d2e9120,5) = 0 (0x0) 17407: close(5) = 0 (0x0) 17407:

Re: python 3 subprocess performance

2019-04-11 Thread Jan Bramkamp
Please run python under truss -f. Does it try to close(2) all possible file descriptors? Does the runtime decrease if fdescfs is mounted at /dev/fd? On 11.04.19 15:16, Alexander Zagrebin wrote: > Hi! > > I've noticed the subprocess performance issue with python 3. > For example, this simple

Re: python 3 subprocess performance

2019-04-11 Thread Konstantin Schukraft
Hi, I've noticed the subprocess performance issue with python 3. For example, this simple script takes just 0,15 second to complete with python 2.7, but more than 5 sec with python 3.6: import subprocess for i in range(100): p = subprocess.Popen(['uname', '-a'], stdout=subprocess.PIPE,

python 3 subprocess performance

2019-04-11 Thread Alexander Zagrebin
Hi! I've noticed the subprocess performance issue with python 3. For example, this simple script takes just 0,15 second to complete with python 2.7, but more than 5 sec with python 3.6: import subprocess for i in range(100): p = subprocess.Popen(['uname', '-a'], stdout=subprocess.PIPE,