[issue20104] expose posix_spawn(p)
John Jones added the comment: I agree with everything you're saying Gregory, however I don't think the significance of the memory doubling is as inconsequential as you might first think. For example, i have on my 64bit Linux system 128Gb of RAM, and a numpy table that's around 70Gb. Spawning a subprocess, even though memory is doubled for a very short period of time, is enough to raise a MemoryError, despite the subprocess i'm spawning using only 2 or 3Mb after the exec(). I do appreciate that for most Python users however, they will not see much benefit from what I imagine is quite a lot of development work. FWIW, I did try the posix_spawn module, but i couldn't figure out how to write data to the stdin of a posix_spawn subprocess, and gave up in place of the commonly recommended solution to this problem (via StackExchange) of spawning lots of subprocesses before you put stuff in memory. Fortunately, for my problem, this was a possible solution. For others I think they're going to have to use posix_spawn, or an entirely different programming language if that doesn't work. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue20104> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20104] expose posix_spawn(p)
John Jones added the comment: To prevent subprocess/os.fork() doubling my memory after loading a large numpy array into memory, I now have to start my script with 650 calls to subprocess.Popen(), which just sit their waiting for some stdin to start doing something. This doesn't happen until after the numpy array is loaded, else I quickly run out of memory. I think this sort of situation results in more unnecessary complexity compared to using posix_spawn, in spite of the concerns about using posix_spawn. Perhaps subprocess.Popen just needs a "posix_spawn=True" parameter? -- nosy: +John Jones ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue20104> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24617] os.makedirs()'s [mode] not correct
New submission from John Jones:
os.makedirs() gives the optional variable mode to set the permissions on the
directories it creates.
While it seems to work for all triplet octal values (777,755,etc) it doesn't
seem to work on values with the sticky bit (1777,1755,etc)
I know that to set the value as octal, you need, for some reason, to prepend a
'0' to the number, such that the final value is '01755' - but even if you do
int('1755',8) the error is there. Below I make a directory and then chmod it to
the right value:
os.makedirs('/Users/Carolin/Desktop/demo',01703)
drwx-x 2 Carolin staff 68 12 Jul 18:53 demo
os.chmod('/Users/Carolin/Desktop/demo',01703)
drwxwt 2 Carolin staff 68 12 Jul 18:53 demo
--
messages: 246655
nosy: John Jones
priority: normal
severity: normal
status: open
title: os.makedirs()'s [mode] not correct
type: behavior
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24617
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14046] argparse: assertion failure if optional argument has square/round brackets in metavar
John Jones added the comment: Im kind of surprised this bug has lasted for so many years :) setting part_regexp to: r'\(.*?\(.*?\).*?\)+|\[.*?\[.*?\].*?\]+|\S+' fixes the issue for me, although its not very elegant -- nosy: +John Jones ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14046 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
