[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory

Evgeny Kapun added the comment: Simple test: while True: try: listdir(0) except NotADirectoryError: pass -- ___ Python tracker ___ __

[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory

Evgeny Kapun added the comment: To make fdopendir fail, just pass any valid FD which points to a non-directory, such as a file or a pipe. -- ___ Python tracker ___ _

[issue17862] itertools.chunks(iterable, size, fill=None)

Changes by Raymond Hettinger : -- assignee: -> rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http

[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory

Larry Hastings added the comment: Patch attached. I don't know how to make fdopendir fail, so I had to do it by inspection. While I was in there, I ifdef'd out the variable that should only be used if fdopendir is available. -- keywords: +patch stage: needs patch -> patch review Adde

[issue17810] Implement PEP 3154 (pickle protocol 4)

Alexandre Vassalotti added the comment: Antoine, can you share the code for your benchmarks which show performance improvements when framing is enabled? I am seeing the same 10-15% slowdown even when pickling stuff to pure Python objects: ### Without the patch ./python -m timeit -r 50 -s "impo

[issue17890] argparse: mutually exclusive groups full of suppressed args can cause AssertionErrors

Terry J. Reedy added the comment: Garrett and Yogesh, please submit contributor license agreements http://www.python.org/psf/contrib/contrib-form/ if you have not yet. When one is properly recorded, a * appears after your name. -- nosy: +terry.reedy versions: +Python 3.3 ___

[issue17897] Optimize unpickle prefetching

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue17887] docs: summary page - generator vs iterator vs iterable

Terry J. Reedy added the comment: To make the subset relationships clear in the iterable entry, perhaps replace See also iterator, sequence, and generator. with Some iterables are iterators and some iterators are generators. Some iterables are sequences. For details, see iterator, sequence,

[issue17897] Optimize unpickle prefetching

Alexandre Vassalotti added the comment: Do you have benchmark results to show the code with the patch is faster? -- ___ Python tracker ___ ___

[issue17887] docs: summary page - generator vs iterator vs iterable

Terry J. Reedy added the comment: Briefly, An iterable is an object that can be iterated with an iterator. An iterator is an iterable that responds to next() calls, including the implicit calls in a for statement. A generator is an iterator created by a generator function, which is a function w

[issue17902] Document that _elementtree C API cannot use custom TreeBuilder for iterparse or IncrementalParser

Changes by Antoine Pitrou : -- nosy: +eli.bendersky stage: -> patch review versions: +Python 3.3 ___ Python tracker ___ ___ Python-bu

[issue17862] itertools.chunks(iterable, size, fill=None)

Terry J. Reedy added the comment: [Anatoly, 'Versions 3.5' is for changes that should *not* happen in 3.4, such as a planned removal for something deprecated in 3.3.] -- nosy: +terry.reedy versions: -Python 3.5 ___ Python tracker

[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory

Changes by Antoine Pitrou : -- nosy: +larry stage: -> needs patch ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue17859] improve error message for saving ints to file

Terry J. Reedy added the comment: I think the point is that the error message does not make much sense unless one knows a) that the file is open in binary mode (and that could have happened in a different file) and b) that 'does not support the buffer interface' means 'is not a bytes-like obje

[issue17810] Implement PEP 3154 (pickle protocol 4)

Antoine Pitrou added the comment: > The framing patch seems to have a significant negative effect on > performance. I wouldn't call it significant. Any speedup or slowdown less than 50% is unlikely to be noticeable in real-world applications. Mitigating the regression is probably a matter of tw

[issue17810] Implement PEP 3154 (pickle protocol 4)

Alexandre Vassalotti added the comment: The framing patch seems to have a significant negative effect on performance. Report on Linux avassalotti 3.2.5-gg1130 #1 SMP Mon Feb 4 02:25:47 PST 2013 x86_64 x86_64 Total CPU cores: 12 ### fastpickle ### Min: 0.447194 -> 0.505841: 1.13x slower Avg: 0.

[issue17902] Document that _elementtree C API cannot use custom TreeBuilder for iterparse or IncrementalParser

New submission from Aaron Oakley: It would really help to document that the C API can only use the default xml.etree.ElementTree.TreeBuilder for targets with iterparse (and by extension, IncrementalParser). I got a nice surprise about that when I went from 3.2 to 3.3 and started getting "Type

[issue17854] symmetric difference operation applicable to more than two sets

Terry J. Reedy added the comment: Sets have methods that do not have operators (such as len, isdisjoint), operators that do not have non-special methods (such as in, <), and method-operator pairs that do the same thing (such as (union, |), (symmetric_difference, ^)). For the pairs, it gives the

[issue17810] Implement PEP 3154 (pickle protocol 4)

Antoine Pitrou added the comment: Here is an updated framing patch which fixes the issue reported by Alexandre. There are also a couple added tests. -- Added file: http://bugs.python.org/file30118/framing3.patch ___ Python tracker

[issue17849] Missing size argument in readline() method for httplib's class LineAndFileWrapper

Changes by Serhiy Storchaka : -- nosy: +orsenthil, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue17901] _elementtree.TreeBuilder raises IndexError on end if constructed with element_factory=None

New submission from Aaron Oakley: When the _elementtree module is in use, the TreeBuilder class will raise an IndexError in treebuilder_handle_end if __init__ was passed "None". I discovered this while writing a subclass of TreeBuilder with a modified __init__ method that delegated to TreeBuil

[issue17900] Recursive OrderedDict pickling

New submission from Serhiy Storchaka: Here is a patch which allows pickling of recursive OrderedDicts. Actually it simplifies __reduce__() code. -- components: Library (Lib) files: OrderedDict_pickle_recursive.patch keywords: patch messages: 188325 nosy: alexandre.vassalotti, pitrou, rh

[issue17898] gettext bug while parsing plural-forms metadata

Steve Strassmann added the comment: There seem to be several bugs involving this particular inner loop in gettext._parse(), but I don't think they're equivalent. The present bug (issue17898) is that parsing a plural header breaks the following header when it happens to be a comment. issue1475

[issue17898] gettext bug while parsing plural-forms metadata

R. David Murray added the comment: Does this bear any relationship to issue 1475523? (And yes, I know it is...sad...that that issue hasn't been fixed yet.) -- nosy: +r.david.murray ___ Python tracker

[issue17899] os.listdir() leaks FDs if invoked on FD pointing to a non-directory

New submission from Evgeny Kapun: When called with a file descriptor as an argument, os.listdir() duplicates it to pass to fdopendir(3). If this call fails, the new file descriptor is not closed, which leads to file descriptor leak. -- components: Library (Lib) messages: 188322 nosy: a

[issue11016] Add S_ISDOOR to the stat module

Christian Heimes added the comment: I have done some research. The POSIX 1.b extensions aren't used on any supported system (neither Linux nor BSD, Solaris or AIX). random...@fastmail.us has compiled a list of even more file types: Heirloom toolchest "ls" supports: http://heirloom.cvs.sourcef

[issue11406] There is no os.listdir() equivalent returning generator instead of list

Changes by Evgeny Kapun : -- nosy: +abacabadabacaba ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue12077] Harmonizing descriptor protocol documentation

Changes by Ned Batchelder : -- nosy: +nedbat ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue17898] gettext bug while parsing plural-forms metadata

Changes by Ned Batchelder : -- nosy: +nedbat ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue17810] Implement PEP 3154 (pickle protocol 4)

Alexandre Vassalotti added the comment: Those methods wouldn't be much more a maintenance burden than the special cases already present in the implementation of __reduce__. These methods would only need to be provided by classes that wishes to support efficient in-place pickling provided by pr

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: I have uploaded a patch with failing tests that should work after this is all said and done. Philip, please make sure I covered your tests as expected (I tweaked one because it already was working the way I did it). This way we at least know what we are aiming f

[issue17898] gettext bug while parsing plural-forms metadata

New submission from Steve Strassmann: The gettext.py parser used by django (lib/python2.7/gettext.py), GNUTranslations._parse(), around line 313 does not use clean values for k,v on each iteration ("for item in tmsg.splitlines():") To reproduce the problem (see traceback, below), try parsing a

[issue17897] Optimize unpickle prefetching

New submission from Serhiy Storchaka: When C implementation of the unpickle reads a data, it also prefetches some data using peek() (if available) and then concatenates read and peeked chunks in the one input buffer. This causes an additional copying when a large data is read. The proposed pat

[issue17895] TemporaryFile name returns an integer in python3

Serhiy Storchaka added the comment: http://docs.python.org/3/library/io.html#io.FileIO.name The file name. This is the file descriptor of the file when no name is given in the constructor. -- nosy: +serhiy.storchaka ___ Python tracker

[issue17896] Move Windows external libs from \..\ to \externals

Changes by Antoine Pitrou : -- nosy: +loewis ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue17810] Implement PEP 3154 (pickle protocol 4)

Antoine Pitrou added the comment: > I think it could be worthwhile to investigate a generic API for > pickling collections in-place. For example, a such API would helpful > for pickling set subclasses in-place. Is the use case important enough? Otherwise, this is more __special_method__ complica

[issue17895] TemporaryFile name returns an integer in python3

Antoine Pitrou added the comment: Given the following property of TemporaryFile: "Under Unix, the directory entry for the file is removed immediately after the file is created. Other platforms do not support this; your code should not rely on a temporary file created using this function having

[issue17810] Implement PEP 3154 (pickle protocol 4)

Alexandre Vassalotti added the comment: I am currently fleshing out an improved implementation for the reduce protocol version 4. One thing I am curious about is whether we should keep the special cases we currently have there for dict and list subclasses. I recall Raymond expressed disagreeme

[issue1159051] Handle corrupted gzip files with unexpected EOF

Serhiy Storchaka added the comment: In both cases broken applications use the undocumented implementation details. But such changes are extremely strong for bugfix release and they should not be done without a special need. I propose to revert these changes in 2.7, 3.2 and 3.3 (possibly leavin

[issue17895] TemporaryFile name returns an integer in python3

R. David Murray added the comment: The 'name' attribute of TemporaryFile is not part of the API. It happens to exist only because the underlying file object has a 'name' attribute. On posix platforms the value is not really useful for anything. In other words, that numpy code was buggy to s

[issue15984] Wrong documentation for PyUnicode_FromObject() and PyUnicode_FromEncodedObject()

Serhiy Storchaka added the comment: In other languages usually the world "coercion" is used for implicit conversion, i.e. int->long, int->float, float->complex. str->unicode in Python 2 (that's what PyUnicode_FromObject() does). But the last conversion is not supported in Python 3. The term "c

[issue17896] Move Windows external libs from \..\ to \externals

New submission from Zachary Ware: PCbuild\readme.txt has a comment from Trent Nelson dated 2-April-2008 suggesting moving the external library location for Windows from "\..\" to "\external\" to make switching between versions easier. I've gotten rather annoyed with having all of the external

[issue15902] imp.load_module won't accept None for the file argument for a C extension

Changes by Brett Cannon : ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-

[issue15902] imp.load_module won't accept None for the file argument for a C extension

Brett Cannon added the comment: Leaving it up to the release manager to decide if this is worth cherrypicking or just waiting for the next bugfix release. -- nosy: +larry priority: normal -> release blocker status: closed -> open ___ Python tracker

[issue15834] 2to3 benchmark not working under Python 3

Changes by Brett Cannon : -- resolution: -> fixed stage: needs patch -> committed/rejected status: open -> closed ___ Python tracker ___

[issue15834] 2to3 benchmark not working under Python 3

Roundup Robot added the comment: New changeset 66f168b468c3 by Brett Cannon in branch 'default': #15834: Make 2to3 so it can actually be proper translated. http://hg.python.org/benchmarks/rev/66f168b468c3 -- nosy: +python-dev ___ Python tracker

[issue17890] argparse: mutually exclusive groups full of suppressed args can cause AssertionErrors

paul j3 added the comment: I see three solutions - 1) gholms' patch which removes '() ' and [] ' 2) Yogesh's patch which removes all duplicated spaces. 3) remove the 2 asserts. The first 2 do the same thing most of the time, but may differ if the user somehow inserts spaces into names. Th

[issue17895] TemporaryFile name returns an integer in python3

New submission from Julian Taylor: sys.version_info(major=3, minor=3, micro=1, releaselevel='final', serial=0) In [3]: type(tempfile.TemporaryFile().name) Out[3]: builtins.int in python2 it returned a string, this is a somewhat pointless api change which breaks some third party code, e.g. numpy

[issue17888] docs: more information on documentation team

Changes by Tshepang Lekhonkhobe : -- nosy: +tshepang ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue17860] subprocess docs lack info how to use output result

Changes by Tshepang Lekhonkhobe : -- nosy: +tshepang ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue17859] improve error message for saving ints to file

Éric Araujo added the comment: Ah, right, I missed the part about the file being opened in binary mode. -- ___ Python tracker ___ ___

[issue15984] Wrong documentation for PyUnicode_FromObject() and PyUnicode_FromEncodedObject()

R. David Murray added the comment: Well, while 'coercion' does refer to changing from one type to another, and technically we are doing that here, in OO we generally think of subclasses as more-or-less being of the same type as the superclass. So I think it would be clearer to spell out that

[issue15902] imp.load_module won't accept None for the file argument for a C extension

Changes by Brett Cannon : -- resolution: -> fixed stage: -> committed/rejected status: open -> closed versions: +Python 3.4 ___ Python tracker ___ _

[issue15902] imp.load_module won't accept None for the file argument for a C extension

Roundup Robot added the comment: New changeset c0a21617dbee by Brett Cannon in branch '3.3': Issue #15902: Fix imp.load_module() to accept None as a file when http://hg.python.org/cpython/rev/c0a21617dbee New changeset 322c556260d5 by Brett Cannon in branch 'default': #15902: merge w/ 3.3 http:/

[issue17838] Can't assign a different value for sys.stdin in IDLE

Roger Serwy added the comment: 3.2 still has the problem fixed in issue17585 for 3.3 and 3.4. This only applies if issue17585 patch gets applied to 3.2. -- ___ Python tracker __

[issue11016] Add S_ISDOOR to the stat module

Christian Heimes added the comment: The stat module (or whatever it is going to be in Python 3.4) is missing more checks than S_ISDOOR: # Solaris S_ISDOOR() S_ISPORT() # POSIX 1.b real-time extension S_ISMSG() S_ISSEM() S_ISSHM() # whiteout, translucent file systems S_ISWHT -- nosy:

[issue15535] Fix pickling efficiency of named tuples in 2.7.3

Raymond Hettinger added the comment: > why the different fix for 3.3 I reverted the 2.7.4 addition of __dict__ rather than introduce more differences between point releases with possible unintended effects. In 3.3, the __dict__ attribute was there from the outset and was advertised in the do

[issue11406] There is no os.listdir() equivalent returning generator instead of list

Ben Hoyt added the comment: Thanks. I thought about that -- but I think I *want* to benchmark it when they're cached, so that we're comparing apples with apples, cached system calls with cached systems calls. The benchmark would almost certainly be a lot "better" (BetterWalk would be even faster)

[issue15535] Fix pickling efficiency of named tuples in 2.7.3

Ben Hoyt added the comment: 2.7 fix works for me, thanks! Just curious -- why the different fix for 3.3 (addition of __getstate__ instead of removal of __dict__)? -- ___ Python tracker

[issue12077] Harmonizing descriptor protocol documentation

Changes by Raymond Hettinger : -- assignee: -> rhettinger priority: normal -> low ___ Python tracker ___ ___ Python-bugs-list mailing

[issue17894] Edits to descriptor howto

Changes by Raymond Hettinger : -- assignee: docs@python -> rhettinger nosy: +rhettinger priority: normal -> low ___ Python tracker ___ ___

[issue15535] Fix pickling efficiency of named tuples in 2.7.3

Changes by Raymond Hettinger : -- assignee: amaury.forgeotdarc -> rhettinger resolution: -> fixed status: open -> closed ___ Python tracker ___ _

[issue15535] Fix pickling efficiency of named tuples in 2.7.3

Changes by Raymond Hettinger : -- Removed message: http://bugs.python.org/msg188064 ___ Python tracker ___ ___ Python-bugs-list mailin

[issue15535] Fix pickling efficiency of named tuples in 2.7.3

Roundup Robot added the comment: New changeset 65cd71abebc8 by Raymond Hettinger in branch '3.3': Issue #15535: Fix pickling of named tuples. http://hg.python.org/cpython/rev/65cd71abebc8 -- ___ Python tracker ___

[issue17890] argparse: mutually exclusive groups full of suppressed args can cause AssertionErrors

Yogesh Chaudhari added the comment: A user generated line with multiple spaces may be essential if we are getting the usage data (particularly from) a csv file (that may have random spaces in certain fields) or some other form of stored or auto-generated data. -- _

[issue15535] Fix pickling efficiency of named tuples in 2.7.3

Roundup Robot added the comment: New changeset 18303391b981 by Raymond Hettinger in branch '2.7': Issue #15535: Fix regression in pickling of named tuples. http://hg.python.org/cpython/rev/18303391b981 -- nosy: +python-dev ___ Python tracker

[issue11406] There is no os.listdir() equivalent returning generator instead of list

Charles-François Natali added the comment: > However, the reason I'm keen on iterdir_stat() is that I'm seeing it speed up > os.walk() by a factor of 10 in my recent tests (note that I've made local > mods, so these results aren't reproducible for others yet). This is doing a > walk on a dir t