[issue11369] Add caching for the isEnabledFor() computation
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Bill, Thanks for the suggestion and the patch. It's a good idea, though I wonder whether you found that the bulk of the time spent in isEnabledFor() was actually spent in getEffectiveLevel()? That's the one which loops through a logger's ancestors looking for a level which is actually set, so intuitively it would take a while - especially for deeper levels in the logging hierarchy. If so (which I suspect to be the case, but it would be good to have you confirm it), a better solution may be to cache the effective level. Roughly how deep are/were your logger hierarchies in the situation where you experienced performance problems? I'm happy to look at caching effective level for Python 3.3: The 2.X branches are now closed for additions other than bugs and security issues. -- assignee: - vinay.sajip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11369 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8841] GetoptError strings should be localized
Filip Gruszczyński grusz...@gmail.com added the comment: Here is a small patch. Could you advise me, how can I test it now? Or is it an issue, that doesn't require writing new tests? -- keywords: +patch Added file: http://bugs.python.org/file20970/8841.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8841 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11270] logging: RotatingFileHandler crash when opening the Logfile in an Texteditor
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Closing, as I said I would, since no feedback has been received for a week. -- components: +Library (Lib) -Extension Modules status: pending - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11270 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11370] Fix distutils to carry configure's LIBS through to extension modules.
New submission from John Szakmeister j...@szakmeister.net: This is a port of r301 from unladen swallow to the py3k-jit branch. On the Mac, there was an extra '-framework CoreFoundation' that was being passed in python3 (maybe it wasn't there in Python 2.x?). At any rate, I changed the logic to ignore anything that didn't start with a -l or end with a .a. Not sure if that's the best answer, or if we should allow -framework and friends to pass through. I put this under distutils, but it's really targeted towards the py3k-jit branch... which hasn't been updated since June. :-( -- assignee: tarek components: Distutils files: libs-to-extension.patch keywords: patch messages: 129863 nosy: eric.araujo, jszakmeister, tarek priority: normal severity: normal status: open title: Fix distutils to carry configure's LIBS through to extension modules. type: feature request Added file: http://bugs.python.org/file20971/libs-to-extension.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11370 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8841] getopt errors should be specialized
Éric Araujo mer...@netwok.org added the comment: I suggested subclassing to solve the more general problem of the caller being able to tell one getopt error from another, for which it is a pretty common solution. Now I see what you meant and understand your request. optparse has five different error classes and argparse has two, but they don’t bubble up to the caller, they’re caught and turned into error messages. getopt is different, since it require its users to do the gruntwork, so having different exceptions for different user errors would be helpful. Compatibility would not be an issue, we’d just make all classes inherit from GetoptError so that except clauses would still work. I think you should take this to the python-ideas mailing list: Do other getopt users need this functionality? What subclasses should there be? Filip: Localization of error messages in getopt is another enhancement. Would you mind opening another report and moving your patch there? Thanks. -- title: GetoptError strings should be localized - getopt errors should be specialized ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8841 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11371] Localization of error messages in getopt
New submission from Filip Gruszczyński grusz...@gmail.com: In http://bugs.python.org/issue8841 Eric asked to open a new ticket about localisation of getopt error messages. I attach a patch. -- files: 8841.patch keywords: patch messages: 129865 nosy: eric.araujo, gruszczy priority: normal severity: normal status: open title: Localization of error messages in getopt type: feature request Added file: http://bugs.python.org/file20972/8841.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11371 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8841] getopt errors should be specialized
Filip Gruszczyński grusz...@gmail.com added the comment: Here it is: http://bugs.python.org/issue11371 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8841 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9276] pickle should support methods
Ram Rachum cool...@cool-rr.com added the comment: Amaury: Your example succeeds on Linux but fails on Windows: $ python3.2 main.py CALLED Traceback (most recent call last): File C:\Python32\Lib\pickle.py, line 679, in save_global klass = getattr(mod, name) AttributeError: 'module' object has no attribute 'foo' During handling of the above exception, another exception occurred: Traceback (most recent call last): File main.py, line 7, in module p.start(); p.join() File C:\Python32\Lib\multiprocessing\process.py, line 130, in start self._popen = Popen(self) File C:\Python32\Lib\multiprocessing\forking.py, line 267, in __init__ dump(process_obj, to_child, HIGHEST_PROTOCOL) File C:\Python32\Lib\multiprocessing\forking.py, line 190, in dump ForkingPickler(file, protocol).dump(obj) File C:\Python32\Lib\pickle.py, line 237, in dump self.save(obj) File C:\Python32\Lib\pickle.py, line 344, in save self.save_reduce(obj=obj, *rv) File C:\Python32\Lib\pickle.py, line 432, in save_reduce save(state) File C:\Python32\Lib\pickle.py, line 299, in save f(self, obj) # Call unbound method with explicit self File C:\Python32\Lib\pickle.py, line 623, in save_dict self._batch_setitems(obj.items()) File C:\Python32\Lib\pickle.py, line 656, in _batch_setitems save(v) File C:\Python32\Lib\pickle.py, line 299, in save f(self, obj) # Call unbound method with explicit self File C:\Python32\Lib\pickle.py, line 683, in save_global (obj, module, name)) _pickle.PicklingError: Can't pickle function foo at 0x00C4EBB8: it's not found as mod.foo User@TURING ~/Desktop/temp $ Traceback (most recent call last): File string, line 1, in module File C:\Python32\Lib\multiprocessing\forking.py, line 370, in main self = load(from_parent) EOFError -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9276 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9276] pickle should support methods
Antoine Pitrou pit...@free.fr added the comment: Amaury: Your example succeeds on Linux but fails on Windows: $ python3.2 main.py I think the difference has to do with Python 3 vs. Python 2. In Python 3 unbound methods are not wrapped in a specific object, and so ForkingPickler does not find their global name. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9276 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10367] python setup.py sdist upload --show-response can fail with UnboundLocalError: local variable 'result' referenced before assignment
Éric Araujo mer...@netwok.org added the comment: Sorry for overlooking this. The test is good, I could trigger the bug with it and then fix it with the patch. Would you mind adding the same test for upload_docs? The code was originally copied from upload, so we should test it too. Philip: I understand your motives but respectfully disagree with them. The urgency was only that I didn't want the other contributors to this issue to feel as though the bar on their contributions were being raised higher every time they jumped the previous bar. I try not to put off people by demanding a test, but asking if they want to write one and providing guidance. I think it’s worked quite well with a lot of our recent contributors from Montréal and other places, but I’m open to critiques about my tone or phrasing, I don’t want to sound like I require a huge amount of work from people who report bugs. I did it to make them feel like somebody was doing *something*. In hindsight, that was not necessarily the best tactic. ;-) Well, I don’t really care about appearances or feelings; the truth of the situation is that there are a lot of easy and hard bugs and a few people to look at them, I don’t think we should mask it. Prior to December, I was reactive in this report. (Given the nature of the bugfix and the bugfix-only status of the 2.7 line, I don't think the lack of an automated test is a big deal; the odds that anything other than trivial bugfixes will be applied to this module in the future seem negligible to me. You know more about distutils than me, but from the short time I’ve worked with this codebase, read bug reports and mailing list archives, I’ve found that it’s too optimistic to change something without an automated test. I have committed a bug in upload.py once :) That’s why I decided to draw a line and stop guessing and hoping things would be right and rely on regression tests. This adds a bit of work for our contributors, but my intention is not to put off people, but to make things more robust. We owe that to our users and to future distutils contributors. I do hope this explains why seemingly trivial changes require a test too. I’d also like to thank you for your help in this report. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10367 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10999] os.chflags refers to stat constants, but the constants are not documented in the stat module
Michal Nowikowski godf...@gmail.com added the comment: Improved the patch: - moved code example up, just below S_* functions list -- Added file: http://bugs.python.org/file20973/documented-os_chflags-flags-v2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10999 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9100] test_sysconfig fails (test_user_similar)
Éric Araujo mer...@netwok.org added the comment: Thanks for the report. What ./configure options did you use? I wonder if this is a duplicate of #11171 or #10086. -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9100 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11371] Localization of error messages in getopt
Filip Gruszczyński grusz...@gmail.com added the comment: Yes, I have run ./python -m test -v test_getopt and it was ok. Running xgettext --from-code utf-8 getopt.py produces no output. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11371 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11371] Localization of error messages in getopt
Éric Araujo mer...@netwok.org added the comment: I will wait for any negative feedback from another core dev and then commit this. Thanks. -- assignee: - eric.araujo resolution: - accepted status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11371 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11370] Fix distutils to carry configure's LIBS through to extension modules.
John Szakmeister j...@szakmeister.net added the comment: Looks like this should've been done against the py3k branch instead. Looking at py3k, it's changed quite a bit since the last merge into py3k-jit. Perhaps there is a better way to do this under py3k? Or it's handled already? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11370 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11372] Remove xrange from argparse docs
New submission from Ross Lagerwall rosslagerw...@gmail.com: As the title says, this patch replaces xrange with range for the 3.3 docs. -- assignee: docs@python components: Documentation files: doc.patch keywords: patch messages: 129876 nosy: docs@python, rosslagerwall priority: normal severity: normal status: open title: Remove xrange from argparse docs versions: Python 3.3 Added file: http://bugs.python.org/file20974/doc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11372 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11371] Localization of error messages in getopt
Filip Gruszczyński grusz...@gmail.com added the comment: Cool, thanks for all your advice. -- status: pending - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11371 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
STINNER Victor victor.stin...@haypocalc.com added the comment: subprocess_errpipe_buffer.patch: Patch for 2.7 which improve by 30% the dummy benchmark (run /bin/false 10,000 times). It avoids the creation of the 1 MB buffer: start with a buffer of a single byte (just to check if there is data or if the pipe was closed). -- keywords: +patch Added file: http://bugs.python.org/file20975/subprocess_errpipe_buffer.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Antoine Pitrou pit...@free.fr added the comment: subprocess_errpipe_buffer.patch: Patch for 2.7 which improve by 30% the dummy benchmark (run /bin/false 10,000 times). It avoids the creation of the 1 MB buffer: start with a buffer of a single byte (just to check if there is data or if the pipe was closed). Does it improve on non-dummy benchmarks? Otherwise I don't think it's worth the trouble. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
STINNER Victor victor.stin...@haypocalc.com added the comment: Quick benchmark: - fork(), execv(), waitpid(): 19081.1 ms - os.popen(): 25769.8 ms - subprocess.Popen(): 40025.8 ms - subprocess.Popen() patched: 26155.2 ms I tested Python 2.7 (debug build) on Linux (Debian Sid) with a Pentium 4 (Hyperthreading, 2 fake cores), 2 GB of memory. Without the patch, subprocess is 55% slower than popen. With the patch, subprocess is 1% slower than popen. It is not a good news: subprocess should be faster before my benchmark tries Popen() without stdout=PIPE ;-) pitrou I think your analysis is wrong. These mmap() calls are for pitrou anonymous memory, most likely they are emitted by the libc's pitrou malloc() to get some memory from the kernel. In other words pitrou they will be blazingly fast. Are you sure? :-) Note that popen() is 35% slower than fork()+execv()+waitpid(). Patched subprocess is 37% slower than fork()+execv()+waitpid(). Try attached bench_subprocess.py script to get your own results. -- Added file: http://bugs.python.org/file20976/bench_subprocess.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Antoine Pitrou pit...@free.fr added the comment: Victor, your patch doesn't even apply on 3.x. That code doesn't exist anymore... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9276] pickle should support methods
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: OK, let's go back to the __namespace__ idea, then. A long time ago I had the idea that the ast compiler could remember the list of named blocks (classes, functions) with their line numbers; for example, the statement class C spans from line 20 to 40, def foo() spans from line 26 to 30. Then, it's easy to locate the function C.foo if you only have its co_firstlineno. This would also be a replacement for the awful hacks in Lib/trace.py::file_module_function_of() (please read the code and say aaargh!). This information would be stored in the module itself, and probably compressed with techniques similar to co_lnotab. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9276 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Antoine Pitrou pit...@free.fr added the comment: Benchmark under 3.x (without obsolete patch): - fork + execv + waitpid: 4794.4 ms - os.popen: 19792.9 ms - subprocess.popen: 10152.1 ms Benchmark under 2.x (without patch:) - fork + execv + waitpid: 4292.7 ms - os.popen: 12697.6 ms - subprocess.popen: 5496.3 ms Looks like there's a regression on both os.popen and subprocess.popen. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Antoine Pitrou pit...@free.fr added the comment:
Here is a quick profile under 3.x:
Time: 10178.0 ms
320812 function calls (320809 primitive calls) in 10.182 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
15.9370.0015.9370.001 {built-in method read}
11.4620.0001.4620.000 {built-in method waitpid}
11.0420.0001.0420.000 {built-in method fork_exec}
10.7520.0008.1310.001 subprocess.py:1119(_execute_child)
10.2280.228 10.182 10.182 bench_subprocess.py:1(module)
10.1770.0008.3450.001 subprocess.py:631(__init__)
20.0790.0000.0790.000 {built-in method close}
10.0610.0001.5840.000 subprocess.py:1368(wait)
So it looks like our parent Python process spends its time (read(), waitpid())
waiting for its child...
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11314
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Antoine Pitrou pit...@free.fr added the comment: Interestingly, reducing from the max open file descriptors from 8192 (my default) to 512 halves the runtime and solves the regression: $ ulimit -n 512 $ ./python bench_subprocess.py pid: 31631 Time: 4903.8 ms So, even though implemented in C, the file descriptor closing logic is still quite costly! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11373] Fix 2 new typos in the docs
New submission from Boštjan Mejak bostjan.me...@gmail.com: Try to fix this in every Python documentation that you can. The typo was found here: http://docs.python.org/library/stdtypes.html#memoryview Create a memoryview that references obj. obj must support the buffer protocol. Builtin objects that /.../. Fix this to Create a memoryview that references obj, which must support the buffer protocol. Built-in objects that /.../. Thank you. -- assignee: docs@python components: Documentation messages: 129886 nosy: Retro, docs@python priority: normal severity: normal status: open title: Fix 2 new typos in the docs versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11373 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
STINNER Victor victor.stin...@haypocalc.com added the comment: pitrou Victor, your patch doesn't even apply on 3.x. pitrou That code doesn't exist anymore... subprocess.Popen() does still read errpipe_read, but using a buffer of 50,000 bytes instead of 1 MB (the traceback is not more send to the parent process). Benchmark on Python 3.2 (debug build, same computer than msg129880): - fork + execv + waitpid: 20052.0 ms - os.popen: 40241.7 ms - subprocess.Popen (C): 28467.2 ms - subprocess.Popen (C, close_fds=False): 22145.4 ms - subprocess.Popen (Python): 40351.5 ms Bad: - os.popen is 41% is slower than subprocess: I suppose that it is the usage of stdout=PIPE (creation of the pipe) which make it slower. But 41% is huge just to create a pipe (without writing into it)! - subprocess(close_fds=True) (default) is 22% slower than subprocess(close_fds=False) - os.popen of Python 3 is 56% slower than os.popen of Python 2 Good: - subprocess of Python 3 is 29% faster than subprocess of Python 2. Other results: - subprocess of Python 3 is 9% slower than patched subprocess of Python 2. - subprocess (default options) is 42% slower than fork+execv+waitpid (this is close to the Python 2 overhead). - subprocess implemented in Python is 42% slower than the C implementation of subprocess. pitrou Looks like there's a regression on both os.popen and subprocess.popen. os.popen() uses subprocess in Python 3. The worst regression is os.popen of Python 3 is 56% slower than os.popen of Python 2. I don't think that it is related to Unicode because my benchmark doesn't write or read any data. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
STINNER Victor victor.stin...@haypocalc.com added the comment: os.popen is 41% is slower than subprocess: I suppose that it is the usage of stdout=PIPE (creation of the pipe) which make it slower Oh no, it's because os.popen() calls subprocess.Popen() with shell=True: the overhead is the shell. Nothing wrong here. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Charles-Francois Natali neolo...@free.fr added the comment: pitrou I think your analysis is wrong. These mmap() calls are for pitrou anonymous memory, most likely they are emitted by the libc's pitrou malloc() to get some memory from the kernel. In other words pitrou they will be blazingly fast. Are you sure? :-) Well, it is fast. It's true that mmap is slower than brk, since the kernel zero-fills the pages, but the overhead probably doesn't come from this, but more likely from pymalloc or malloc, and also from the call to _PyBytes_resize in posix_read when the number of bytes read is smaller than what has been requested. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Charles-Francois Natali neolo...@free.fr added the comment: So, even though implemented in C, the file descriptor closing logic is still quite costly! Yes, see this recent issue: http://bugs.python.org/issue11284 In the reporter's case, it's much worse, because FreeBSD (at least the version he's using) has a SC_OPEN_MAX of 655000, so that passing close_fds=True bumps the Popen runtime to 3 seconds! Some Unix offer a closefrom() or similar function to close all open file descriptors (and only open ones), which should be much faster. I'm not aware of anything equivalent under Linux, but it might be worth looking into. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
STINNER Victor victor.stin...@haypocalc.com added the comment: As written by Charles-Francois Natali (msg129890), we can use closefrom(). Pseudo-code: find the biggest fd than that be kept open, call closefrom(highest+1), and then use close() (os.closerange) for fd in 0..highest that have to be closed. closefrom() is available on OpenBSD, Solaris and NetBSD. I don't know for FreeBSD: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2007-07/msg00035.html On Linux, we can use os.listdir(/proc/self/fd) to get the list of open files. -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Antoine Pitrou pit...@free.fr added the comment: The patch is wrong, as explained by Charles-François. If someone wants subprocess to use closefrom(), a patch is required :) But I think at least a mention in the documentation is warranted. -- nosy: +gregory.p.smith, pitrou priority: normal - high stage: patch review - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
STINNER Victor victor.stin...@haypocalc.com added the comment: Benchmark on subprocess with a less trivial example. Run 100x python -c pass: 8.63 sec without my patch, 8.53 sec with my patch = only 1% faster, so the patch is just useless on a real world example. Finally, I think that there is just nothing to do on Python 2: the overhead between fork(), os.popen and subprocess doesn't impact real world programs. On Python 3: the most critical issue is that close_fds=True is much slower than close_fds=False, but there is already an issue for that (#11284). So let's close this issue. You can still comment it if you disagree. But if you would like to reopen it: please come with numbers of a benchmark on real programs (not on exit 0 with a shell or /bin/false). -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
STINNER Victor victor.stin...@haypocalc.com added the comment: Benchmark extracted from #11314, Python 3.2 on Linux: subprocess(/bin/false, close_fds=True) is 22% slower than subprocess(/bin/false, close_fds=False). -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11368] xml.etree.ElementTree.Element should have a reference to parent
anatoly techtonik techto...@gmail.com added the comment: That's not a good reason to close report. Such limitations of xml.etree.ElementTree should be at least documented properlt. And even documented it is still a valid feature request for standard library. Not for current implementation, but perhaps for Python4. -- status: closed - open type: - feature request ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11374] pkgutil.extend_path do not recognize py{c,o} file
New submission from Alexandre Badez alexandre.ba...@gmail.com:
extend_path only test if init.py files exist, but it should also test
init.pyc and/or init.pyo.
--
components: Library (Lib)
messages: 129896
nosy: Alexandre.Badez
priority: normal
severity: normal
status: open
title: pkgutil.extend_path do not recognize py{c,o} file
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11374
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11374] pkgutil.extend_path do not recognize py{c,o} file
Changes by Alexandre Badez alexandre.ba...@gmail.com: -- type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11374 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11368] xml.etree.ElementTree.Element should have a reference to parent
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Anatoly, do you find the suggestions in http://effbot.org/zone/element.htm#accessing-parents useful to you? If so, this paragraph could be copied into the documentation. -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11374] pkgutil.extend_path do not recognize py{c,o} file
Alexandre Badez alexandre.ba...@gmail.com added the comment: I've made a simple patch. -- keywords: +patch Added file: http://bugs.python.org/file20977/patch.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11374 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11375] urllib2 over SOCKS doesn't use proxy for DNS
New submission from OJW owh...@yahoo.com: #!/usr/bin/python import socks import socket socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4, 127.0.0.1, 9050, True) socket.socket = socks.socksocket import urllib2 print urllib2.urlopen(http://example.com/;).read() Expected result: all network communications go through SOCKS4A proxy Actual result: DNS lookup of example.com goes through default gateway, followed by HTTP connection via the proxy -- components: Library (Lib) messages: 129899 nosy: OJW priority: normal severity: normal status: open title: urllib2 over SOCKS doesn't use proxy for DNS type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11375 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11374] pkgutil.extend_path do not recognize py{c,o} file
Changes by Alexandre Badez alexandre.ba...@gmail.com: Removed file: http://bugs.python.org/file20977/patch.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11374 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11374] pkgutil.extend_path do not recognize py{c,o} file
Alexandre Badez alexandre.ba...@gmail.com added the comment: A little change in the patch -- Added file: http://bugs.python.org/file20978/patch.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11374 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11375] urllib2 over SOCKS doesn't use proxy for DNS
Changes by Jérôme Radix jerome.ra...@gmail.com: -- nosy: +jerome.radix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11375 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9313] distutils error on MSVC older than 8
Martin v. Löwis mar...@v.loewis.de added the comment: Éric, what forward porting exactly needs to be done? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9313 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Charles-Francois Natali neolo...@free.fr added the comment: Attached is a patch adding os.closefrom. If closefrom(2) is available, it's used. Otherwise, two options: - if sysconf and _SC_OPEN_MAX are defined, we close each file descriptor up to _SC_OPEN_MAX - if not, we choose a default value (256), and close every FD up to this value subprocess has been converted to use it, and a test has been added in test_os Unfortunately, I only have Linux boxes, so I can't really test it. Remarks: - is it OK to provide posix_closefrom even though the underlying platform doesn't support it ? - no error code is returned (since when closing every FD manually this wouldn't make much sense), even though closefrom(2) does return one - for the test, I only close FDs 7 to avoid closing stdin/stdout/stder, but you might have a better idea - this won't fix the problem for Linux, which doesn't have closefrom(2). Is it worth using /proc/self/fd interface ? -- Added file: http://bugs.python.org/file20979/py3k_closefrom.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11375] urllib2 over SOCKS doesn't use proxy for DNS
Changes by Senthil Kumaran orsent...@gmail.com: -- assignee: - orsenthil nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11375 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11376] Solaris/GCC/shared lib problem
New submission from Dave Abrahams d...@boostpro.com: http://twistedmatrix.com/trac/ticket/4916#comment:2 suggests that maybe there's a bug in distutils. Something in the build process for twisted is deciding that I have Sun CC installed instead of letting pycc, which can handle the job of picking the correct PIC flag, do it properly: /usr/lib/python2.6/pycc -DNDEBUG -KPIC -I/usr/include/python2.6 -c conftest.c -o conftest.o gcc: unrecognized option `-KPIC' -- assignee: tarek components: Distutils messages: 129903 nosy: dabrahams, eric.araujo, tarek priority: normal severity: normal status: open title: Solaris/GCC/shared lib problem type: compile error versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11376 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1559549] ImportError needs attributes for module and file name
Changes by Filip Gruszczyński grusz...@gmail.com: -- nosy: +gruszczy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1559549 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Changes by Charles-Francois Natali neolo...@free.fr: Removed file: http://bugs.python.org/file20979/py3k_closefrom.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Changes by Charles-Francois Natali neolo...@free.fr: Added file: http://bugs.python.org/file20980/py3k_closefrom.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Charles-Francois Natali neolo...@free.fr added the comment: Attached is a new version falling back to /proc/self/fd when closefrom(2) is not available (on Unix), working on Linux. It's indeed much faster than the current approach. Note that it's only used if _posixsubprocess is not available, because in that case the FD are closed from _posixsubprocess.c:child_exec. To make it available to _posixsubprocess, I was thinking of putting the closefrom code in a helper function, which would then be called from posix_closefrom and _posixsubprocess. Is this the proper way to do ? If yes, what file would be a good recipient for that ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Changes by s7v7nislands s7v7nisla...@gmail.com: Removed file: http://bugs.python.org/file20834/py3k.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Changes by s7v7nislands s7v7nisla...@gmail.com: Removed file: http://bugs.python.org/file20835/python27.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11377] Deprecate (remove?) platform.popen()
New submission from STINNER Victor victor.stin...@haypocalc.com: Extract of the documentation: 15.14.3.1. Win95/98 specific platform.popen(cmd, mode='r', bufsize=None) Portable popen() interface. Find a working popen implementation preferring win32pipe.popen(). On Windows NT, win32pipe.popen() should work; on Windows 9x it hangs due to bugs in the MS C library. Python 3 doesn't support Windows 9x/Me anymore: we should deprecate it, or maybe directly remove it. subprocess.Popen() is a better alternative: it supports Unicode, it handles EINTR, etc. -- components: Library (Lib) messages: 129905 nosy: haypo, lemburg priority: normal severity: normal status: open title: Deprecate (remove?) platform.popen() versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11377 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11378] os.path.join when second starts with '/' (linux/unix)
New submission from Alessandro Forghieri a...@orion.it:
Observed behavior:
os.path.join('/foo/bar', '/baz')
'/baz'
Expected behavior:
should return '/foo/bar/baz' or (suboptimally) raise an exception.
Why:
Observed beahvior violates the minimal surprise principle and does not appear
to respond to any particular rationale (by the same token it could return
/foo/bar as well)
--
components: Library (Lib)
messages: 129906
nosy: al_form2000
priority: normal
severity: normal
status: open
title: os.path.join when second starts with '/' (linux/unix)
type: behavior
versions: Python 2.5, Python 2.6, Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11378
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11377] Deprecate (remove?) platform.popen()
Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: New submission from STINNER Victor victor.stin...@haypocalc.com: Extract of the documentation: 15.14.3.1. Win95/98 specific platform.popen(cmd, mode='r', bufsize=None) Portable popen() interface. Find a working popen implementation preferring win32pipe.popen(). On Windows NT, win32pipe.popen() should work; on Windows 9x it hangs due to bugs in the MS C library. Python 3 doesn't support Windows 9x/Me anymore: we should deprecate it, or maybe directly remove it. subprocess.Popen() is a better alternative: it supports Unicode, it handles EINTR, etc. Does it prevent a shell window from opening on Windows ? Does subprocess.Popen() use the system's PATH for finding the executable ? Since it's a documented API, we could replace it with an implementation that uses subprocess.Popen(), but not remove it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11377 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11378] os.path.join when second argument starts with '/' (linux/unix)
Changes by Alessandro Forghieri a...@orion.it: -- title: os.path.join when second starts with '/' (linux/unix) - os.path.join when second argument starts with '/' (linux/unix) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11378 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11378] os.path.join when second argument starts with '/' (linux/unix)
Daniel Urban urban.dani...@gmail.com added the comment: It does exactly what is documented: If any component is an absolute path, all previous components (on Windows, including the previous drive letter, if there was one) are thrown away, and joining continues. (http://docs.python.org/dev/py3k/library/os.path#os.path.join) -- nosy: +durban ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11378 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11378] os.path.join when second argument starts with '/' (linux/unix)
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: And the current behaviour is useful; for example, the absolute path of a file name can be computed with os.path.join(os.getcwd(), filename). This works with both absolute and relative paths. os.path.join() won't change; you probably should consider another function, like os.path.normpath(a + '/' + b) -- nosy: +amaury.forgeotdarc resolution: - invalid stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11378 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10654] test_datetime fails on Python3.2 windows binary
Max m...@alleged.net added the comment: This is still occurring with the release version of Python 3.2, installed from the 32-bit MSI, on Windows XP. -- nosy: +max-alleged ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10654 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11378] os.path.join when second argument starts with '/' (linux/unix)
Alessandro Forghieri a...@orion.it added the comment: Documented it is, and change it may not, but I am still unconvinced. Anything that joins should (IMHO) preserve the leading stuff - this one throws it away - sometimes. Also I do believe a number of other languages take a different stance (I am no pitonista, but took a double take when I noticed this). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11378 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Changes by Charles-Francois Natali neolo...@free.fr: Removed file: http://bugs.python.org/file20980/py3k_closefrom.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Gregory P. Smith g...@krypto.org added the comment: There is no OS API call to provide a *safe* way to get a list of all open file descriptors as part of POSIX in general that can be called after the fork() and before the exec(). It must be async signal safe. The closefrom() call available in Solaris and recent BSDs looks useful, is it async signal safe? I still want to find a way to do this nicely on Linux (even if it means me going and implementing a closefrom syscall to be added to 2.6.39). Your posix_closefrom() implementation as written today is not safe to call between fork() and exec() due to the opendir/readdir implementation. It can and will hang processes at unexpected times. Another thought that I've seen done in other subprocess implementations as a compromise: Offer a way to use the hacky *not guaranteed to close everything if the process is multithreaded* version that has the parent get the list of fds before the fork() and the child closing those before exec(). That leaves the race condition of new fds being opened between the creation of that list and the fork() but would be fast. If we can detect if any other threads exist in the program (they may have been created by C extension modules or by a C program that is embedding Python within it) we could conditionally use that approach vs the close-everything-possible approach so that people using this with generic programs that don't involve threads are not so heavily impacted. -- assignee: - gregory.p.smith ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Gregory P. Smith g...@krypto.org added the comment: According to http://www.unix.com/man-page/All/3c/closefrom/ closefrom() is not async-signal-safe. :( -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11379] Remove lightweight from minidom description
New submission from Stefan Behnel sco...@users.sourceforge.net: http://docs.python.org/library/xml.dom.minidom.html presents MiniDOM as a Lightweight DOM implementation. The word lightweight is easily misunderstood as meaning efficient or memory friendly. MiniDOM is well known to be neither of the two. The first paragraph then continues: xml.dom.minidom is a light-weight implementation of the Document Object Model interface. It is intended to be simpler than the full DOM and also significantly smaller. Again, smaller can be misread as low memory footprint, whereas it is actually supposed to refer to an incomplete DOM API implementation. And simpler is also clearly exaggerated when compared to the alternative ElementTree package. I would like to see this changed and combined with a clear and visible comment that MiniDOM has very high resource profile, e.g. 19.7. xml.dom.minidom — Pure Python DOM implementation xml.dom.minidom is a pure Python implementation of the Document Object Model interface, as known from other programming languages. It is intended to provide a smaller API than the full DOM. Note, however, that MiniDOM has a very large memory footprint compared to other Python XML libraries. If you need a fast and memory friendly XML tree implementation with a vastly simpler API, use the xml.etree package instead. -- assignee: docs@python components: Documentation messages: 129914 nosy: docs@python, scoder priority: normal severity: normal status: open title: Remove lightweight from minidom description versions: Python 2.7, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11379 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11378] os.path.join when second argument starts with '/' (linux/unix)
Eric Smith e...@trueblade.com added the comment: I appreciate that you're unconvinced of its usefulness, but you can be assured that some of us do find it a useful and desirable behavior. And Python isn't the only system that works this way, emacs does the same thing when you enter filenames. -- nosy: +eric.smith ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11378 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11284] slow close file descriptors in subprocess, popen2, os.popen*
Charles-Francois Natali neolo...@free.fr added the comment: Your posix_closefrom() implementation as written today is not safe to call between fork() and exec() due to the opendir/readdir implementation. It can and will hang processes at unexpected times. Yeah, I remove the patch when I realized that. According to http://www.unix.com/man-page/All/3c/closefrom/ closefrom() is not async-signal-safe. :( Strange. I was sure closefrom was implemented with fcntl. I still want to find a way to do this nicely on Linux (even if it means me going and implementing a closefrom syscall to be added to 2.6.39). Well, arguably, CLOEXEC is designed to cope with this kind of situation. closefrom is more like a hack (and mostly useless if it's really not async-safe). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11284 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11354] argparse: nargs could accept range of options count
Changes by Wojciech Muła wojciech_m...@poczta.onet.pl: Removed file: http://bugs.python.org/file20948/test.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11354 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11354] argparse: nargs could accept range of options count
Wojciech Muła wojciech_m...@poczta.onet.pl added the comment: Daniel, thanks for note, fixed. -- Added file: http://bugs.python.org/file20981/test.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11354 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11354] argparse: nargs could accept range of options count
Changes by SilentGhost ghost@gmail.com: -- nosy: +bethard ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11354 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11379] Remove lightweight from minidom description
Martin v. Löwis mar...@v.loewis.de added the comment: -1. The description is factually correct - minidom *does* have a lower footprint than other Python DOM implementations (such as 4DOM). -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11379 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Aaron Sherman a...@ajs.com added the comment: I think it's still safe to say that high performance applications which need to create many hundreds or thousands of children (e.g. large monitoring systems) will still need another solution that isn't subprocess. That being said, you're right that no one is going to care about the extra overhead of subprocess in a vacuum, and most applications fork one or a very small number of processes at a time and typically end up waiting for orders of magnitude more time for their output than they spend creating the process in the first place. As I said when I opened this issue, I wasn't terribly concerned with most applications. That being said, I can't really submit a full-blown monitoring system against this bug, so the best I could do would something that did lots of os.popens or subprocess.Popens in parallel in a contrived way and see how the performance scales as I tune lots to different values. Sadly, the time I have for doing that needs to be spent writing other code, so I'll leave this closed and let someone else raise the issue in the future if they run into it. I can always build a dispatcher in C and communicate with it via IPC to get around the immediate concern of scalability. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Antoine Pitrou pit...@free.fr added the comment: That being said, I can't really submit a full-blown monitoring system against this bug, Perhaps you can take a look at Shinken, it is a Nagios-lookalike written in Python. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11377] Deprecate (remove?) platform.popen()
STINNER Victor victor.stin...@haypocalc.com added the comment: os.popen() is deprecated since Python 2.6 (Use the subprocess module.) and it is no more documented in Python 3. The following documentation explains how to replace os.popen() by subprocess: http://docs.python.org/py3k/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3 Does it prevent a shell window from opening on Windows ? Yes, but only if shell=True. Does subprocess.Popen() use the system's PATH for finding the executable ? Yes. The subprocess module calls os.get_exec_path() which reads PATH environment variable. Since it's a documented API, we could replace it with an implementation that uses subprocess.Popen(), platform_popen.patch patchs platform.popen() to reuse os.popen() (which uses subprocess). It adds also tests (there was no test for platform.popen) but not remove it. Can't we at deprecate platform.popen in favor of subprocess, with a documentation to explain how to port code to subprocess? My patch doesn't touch platform documentation. -- os.popen() does still exist in Python 3.2, but it is no more documented. Its documentation should be fixed or the function should be removed. -- keywords: +patch Added file: http://bugs.python.org/file20982/platform_popen.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11377 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11377] Deprecate (remove?) platform.popen()
Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: STINNER Victor victor.stin...@haypocalc.com added the comment: os.popen() is deprecated since Python 2.6 (Use the subprocess module.) and it is no more documented in Python 3. The following documentation explains how to replace os.popen() by subprocess: http://docs.python.org/py3k/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3 Does it prevent a shell window from opening on Windows ? Yes, but only if shell=True. Does subprocess.Popen() use the system's PATH for finding the executable ? Yes. The subprocess module calls os.get_exec_path() which reads PATH environment variable. One more question: Doe subprocess work from inside Windows GUI applications ? (I added the _popen class to work around issues with those a long while back) Since it's a documented API, we could replace it with an implementation that uses subprocess.Popen(), platform_popen.patch patchs platform.popen() to reuse os.popen() (which uses subprocess). It adds also tests (there was no test for platform.popen) Hmm, but if os.popen() is no longer supported in Python 3, how can we still use it in platform ? The os module still provides an popen() function which uses subprocess. The documentation for the os module says: http://docs.python.org/py3k/library/os.html?highlight=os#os.plock os.popen(...) Run child processes, returning opened pipes for communications. These functions are described in section File Object Creation. If you then go to the suggested section, there's no mention of os.popen(). popen() is also referenced in a few other places in the os module documentation. The documentation should either be removed completely, or restored to match the existing os module function popen(). but not remove it. Can't we at deprecate platform.popen in favor of subprocess, with a documentation to explain how to port code to subprocess? Sure, just not remove it immediately. My patch doesn't touch platform documentation. -- os.popen() does still exist in Python 3.2, but it is no more documented. Its documentation should be fixed or the function should be removed. Indeed. See above. Thanks, -- Marc-Andre Lemburg eGenix.com ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11377 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11377] Deprecate (remove?) platform.popen()
Marc-Andre Lemburg m...@egenix.com added the comment: BTW: The _popen class can be removed as well, if the function is replaced with os.popen() or the subprocess module Popen(). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11377 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9373] pulldom has low code coverage
Benjamin Peterson benja...@python.org added the comment: r88712 Thanks -- nosy: +benjamin.peterson resolution: - accepted status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9373 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: Is there a way to test this? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Antoine Pitrou pit...@free.fr added the comment: Valgrind perhaps? If the keys container was a list or tuple it would simply take a refleak run of the test suite. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: I'm not sure. What infrastructure do we have to leaked memory that was allocated with PyMem_MALLOC? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: I meant to *detect* leaked memory, of course. :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: I played around with this a little. That code path doesn't appear to be exercised during the existing unit tests. I'll add a test so the leak at least shows up when the tests are run under valgrind. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11380] close failed in file object destructor when Broken pipe happens on stdout
New submission from ulidtko ulid...@gmail.com: The following script fails: from time import sleep try: while True: sleep(0.1) print blah except KeyboardInterrupt: pass when being launched with redirections like this: $ python test.py | tee /dev/null and the tee process gots killed *before* the python process (by e.g. hitting ^C in most shells). The output is: close failed in file object destructor: sys.excepthook is missing lost sys.stderr When the python process is SIGINTed prior to the tee process, everything goes as expected: buffered output to stdout gots flushed and seen on the console, the script finishes successfully, with zero exit status, tee finishes too. When the tee receives SIGINT first, and then python, script loses the output and prints the mentioned message. strace shows that python is receiving (but probably not handling properly) Broken pipe error in response to write(2): [pid 17874] write(1, blah\nblah\nblah\nblah\nblah\nblah\nbl..., 680) = -1 EPIPE (Broken pipe) -- components: IO messages: 129930 nosy: ulidtko priority: normal severity: normal status: open title: close failed in file object destructor when Broken pipe happens on stdout type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11380 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10829] PyUnicode_FromFormatV() bugs with % and %% format strings
Ray.Allen ysj@gmail.com added the comment: Hi, haypo, would you mind modify your newly added parse_format_flags() function so that it can diff the precision value of '%.0s' and '%s'(Currently both of them return precision as 0)? Because if used with string formatters(%s, %R, %S, %A, ...), they should be very different. And so I can work on issue7330 with it. Just return precision as -1 to indicate no precision is designated could work. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10829 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11372] Remove xrange from argparse docs
Fred L. Drake, Jr. fdr...@acm.org added the comment: Committed for Python 3.3.0: r88717 Committed for Python 3.2.1: r88718 -- assignee: docs@python - fdrake nosy: +fdrake resolution: - accepted stage: - committed/rejected status: open - closed versions: +Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11372 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11381] pretending the not operator is a function behaves surprisingly
New submission from Abafei aba...@gmail.com: I'm not sure if this is a bug per se, since I don't think pretending operators are callable is in the docs, but: pretending an operator (at least the not operator) is callable, like so: not(True) can be surprising: (not 1) == 9 False not(1) == 9 True Now, I know this is valid because Python is very lenient about whitespace (and the parenthenses are really just eval '1' first) , but, this is still confusing behavior to someone who does not know about that. I think the same problem may be possible in the case of statements. A possible solution is to make sure there is at least some white-space between alphabetical operators and statements. -- messages: 129933 nosy: abafei priority: normal severity: normal status: open title: pretending the not operator is a function behaves surprisingly type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11381 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11379] Remove lightweight from minidom description
Stefan Behnel sco...@users.sourceforge.net added the comment: Well, I'm not aware of many people who use 4DOM these days, and if that's what it's meant to refer to, maybe that should be made more obvious, because it currently is not at all. Even cDomlette uses only half of the memory according to http://effbot.org/zone/celementtree.htm When you say that the description is factually correct, that does by no means imply that the average reader will understand how it's meant. My point is that almost everyone who reads this will draw the wrong conclusions. Also, when you say lower footprint, that does not yet make it light weight in any way. It still uses something like ten times as much memory as cElementTree or lxml in Python 2 (and likely much more than even that in Python 3), and still something like 4-5 times as much as plain Python ElementTree. That's a huge difference. What about this phrasing then: MiniDOM has a smaller memory footprint than some of the other DOM compliant implementations for Python (such as 4DOM), but uses about 10x more memory than the faster and simpler xml.etree.cElementTree module. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11379 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11381] pretending the not operator is a function behaves surprisingly
Georg Brandl ge...@python.org added the comment: I don't think this is wise, or necessary. I'm not sure why you would treat an operator as callable; the tutorial hopefully doesn't tell you such things. -- nosy: +georg.brandl resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11381 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
