Re: python 3 subprocess performance

2019-04-14 Thread Jan Beich
Kubilay Kocak writes: > On 12/04/2019 8:41 pm, Dima Pasechnik wrote: > >> On Fri, Apr 12, 2019 at 9:46 AM Alexander Zagrebin wrote: >>> >>> В Fri, 12 Apr 2019 09:36:13 +0200 >>> Dima Pasechnik пишет: >>> On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin wrote: > > В Thu, 11

Re: python 3 subprocess performance

2019-04-13 Thread Kubilay Kocak
On 12/04/2019 8:41 pm, Dima Pasechnik wrote: On Fri, Apr 12, 2019 at 9:46 AM Alexander Zagrebin wrote: В Fri, 12 Apr 2019 09:36:13 +0200 Dima Pasechnik пишет: On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin wrote: В Thu, 11 Apr 2019 17:32:42 +0200 Jan Bramkamp пишет: The reason is

Re: python 3 subprocess performance

2019-04-12 Thread Alexander Zagrebin
В Fri, 12 Apr 2019 12:41:53 +0200 Dima Pasechnik пишет: > > > > В Thu, 11 Apr 2019 17:32:42 +0200 > > > > Jan Bramkamp пишет: > > > > > > > > > The reason is that that python does something stupid (tm). It > > > > > tries to close all file descriptors (except a few whitelisted > > > > > ones)

Re: python 3 subprocess performance

2019-04-12 Thread Dima Pasechnik
On Fri, Apr 12, 2019 at 9:46 AM Alexander Zagrebin wrote: > > В Fri, 12 Apr 2019 09:36:13 +0200 > Dima Pasechnik пишет: > > > On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin > > wrote: > > > > > > В Thu, 11 Apr 2019 17:32:42 +0200 > > > Jan Bramkamp пишет: > > > > > > > The reason is that th

Re: python 3 subprocess performance

2019-04-12 Thread Alexander Zagrebin
В Fri, 12 Apr 2019 09:36:13 +0200 Dima Pasechnik пишет: > On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin > wrote: > > > > В Thu, 11 Apr 2019 17:32:42 +0200 > > Jan Bramkamp пишет: > > > > > The reason is that that python does something stupid (tm). It > > > tries to close all file descrip

Re: python 3 subprocess performance

2019-04-12 Thread Dima Pasechnik
On Fri, Apr 12, 2019 at 9:11 AM Alexander Zagrebin wrote: > > В Thu, 11 Apr 2019 17:32:42 +0200 > Jan Bramkamp пишет: > > > The reason is that that python does something stupid (tm). It tries to > > close all file descriptors (except a few whitelisted ones) up to the > > maximum file descriptor n

Re: python 3 subprocess performance

2019-04-12 Thread Alexander Zagrebin
В Thu, 11 Apr 2019 17:32:42 +0200 Jan Bramkamp пишет: > The reason is that that python does something stupid (tm). It tries to > close all file descriptors (except a few whitelisted ones) up to the > maximum file descriptor number. It does this by asking the kernel for > the maximum possible numb

Re: python 3 subprocess performance

2019-04-11 Thread Jan Bramkamp
The reason is that that python does something stupid (tm). It tries to close all file descriptors (except a few whitelisted ones) up to the maximum file descriptor number. It does this by asking the kernel for the maximum possible number and closing everything it doesn't want to keep. Some time lat

Re: python 3 subprocess performance

2019-04-11 Thread Konstantin Schukraft
Please run python under truss -f. Does it try to close(2) all possible file descriptors? Does the runtime decrease if fdescfs is mounted at /dev/fd? truss -f -p 17407 17407: read(5,0x1265d2e9120,5) = 0 (0x0) 17407: close(5) = 0 (0x0) 17407: lsee

Re: python 3 subprocess performance

2019-04-11 Thread Jan Bramkamp
Please run python under truss -f. Does it try to close(2) all possible file descriptors? Does the runtime decrease if fdescfs is mounted at /dev/fd? On 11.04.19 15:16, Alexander Zagrebin wrote: > Hi! > > I've noticed the subprocess performance issue with python 3. > For example, this simple scrip

Re: python 3 subprocess performance

2019-04-11 Thread Konstantin Schukraft
Hi, I've noticed the subprocess performance issue with python 3. For example, this simple script takes just 0,15 second to complete with python 2.7, but more than 5 sec with python 3.6: import subprocess for i in range(100): p = subprocess.Popen(['uname', '-a'], stdout=subprocess.PIPE, std

python 3 subprocess performance

2019-04-11 Thread Alexander Zagrebin
Hi! I've noticed the subprocess performance issue with python 3. For example, this simple script takes just 0,15 second to complete with python 2.7, but more than 5 sec with python 3.6: import subprocess for i in range(100): p = subprocess.Popen(['uname', '-a'], stdout=subprocess.PIPE, stde