[issue9859] Add tests to verify API match of modules with 2 implementations
Daniel Stutzbach added the comment: With regret, I have not had time to work on patches and am unlikely to have time in the near future. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9859 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23565] local_clear walks the list of threads without holding head_lock.
New submission from Daniel Stutzbach: local_clear in _threadmodule.c walks the list of threads without holding head_mutex. Since the list of threads can change when holding only head_mutex and not on the GIL, the list can change while local_clear is executing, which may cause Bad Things to happen. -- components: Library (Lib) messages: 237078 nosy: stutzbach priority: normal severity: normal status: open title: local_clear walks the list of threads without holding head_lock. type: behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23565 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19081] zipimport behaves badly when the zip file changes while the process is running
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19081 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18606] Add statistics module to standard library
Daniel Stutzbach added the comment: As the person originally trying to take the mean of timedelta objects, I'm personally fine with the workaround of: py m = statistics.mean([x.total_seconds() for x in data]) py td(seconds=m) datetime.timedelta(2, 43200) At the time I was trying to take the mean of timedelta objects, even the total_seconds() method did not exist in the version of Python I was using. On the flip side, wouldn't sum() work on timedelta objects if you simply removed the isinstance(start, numbers.Number) check? -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18606 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2226] Small _abcoll Bugs / Oddities
Changes by Daniel Stutzbach stutzb...@google.com: -- assignee: stutzbach - rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2226 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9141] Allow objects to decide if they can be collected by GC
Daniel Stutzbach stutzb...@google.com added the comment: On Tue, Apr 17, 2012 at 4:08 AM, Antoine Pitrou rep...@bugs.python.org wrote: has_finalizer() in gcmodule.c doesn't check for weakref callbacks, so a weakref callback can still be invoked from tp_dealloc. Unless I'm mistaken, weakrefs will be handled and removed by handle_weakrefs() before delete_garbage() is called. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9141 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14534] Add method to mark unittest.TestCases as do not run.
Daniel Stutzbach stutzb...@google.com added the comment: Wouldn't the subclass inherit the False value? Then the user would need to remember to override the value again in the subclass, which is error prone. Antoine: I've used the pattern you describe on a couple of occasions, and it routinely confuses my code reviewers. -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14534 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9141] Allow objects to decide if they can be collected by GC
Daniel Stutzbach stutzb...@google.com added the comment: On Fri, Apr 6, 2012 at 12:51 PM, Jim Jewett rep...@bugs.python.org wrote: __del__ methods do run, even if an object was collected by the cycle detector. And they can't do any harm that couldn't also be done by a C finalizer. No, if an object with a __del__ method is part of a cycle, it is not collected. The objects get appended to gc.garbage instead. See: http://docs.python.org/py3k/library/gc.html#gc.garbage -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9141 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9141] Allow objects to decide if they can be collected by GC
Daniel Stutzbach stutzb...@google.com added the comment: Are __del__ and tp_del getting conflated in this conversation? I don't see a __del__ method on generator objects: filfre:$ python3.1 Python 3.1.2 (r312:79147, Dec 9 2011, 20:47:34) [GCC 4.4.3] on linux2 (i for i in range(5)).__del__ Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'generator' object has no attribute '__del__' but I may be missing something. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9141 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13804] Python library structure creates hard to read code when using higher order functions
Daniel Stutzbach stutzb...@google.com added the comment: Ah - in your first example (the one with 3 lines) did you mean to use .extend instead of .append? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13804 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13804] Python library structure creates hard to read code when using higher order functions
Daniel Stutzbach stutzb...@google.com added the comment: If I'm understanding Martin Häcker's code correctly, the list comprehension equivalent is: self.questions = [topic.questions for topic in self.topics] The reduce() variants are not only much harder to read, but they will take O(n**2) operations because they create an O(n) list O(n) times. -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13804 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13742] Add a key parameter (like sorted) to heapq.merge
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13742 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds
Daniel Stutzbach stutzb...@google.com added the comment: Gladly. :-) -- resolution: - out of date stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8654 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12682] Meaning of 'accepted' resolution as documented in devguide
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12682 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11640] Shelve references globals in its __del__ method
Daniel Stutzbach stutzb...@google.com added the comment: Could you add a comment above the lines defining self._BytesIO, describing why they're being set? Someone else might see them as unnecessary and rip them out if there's no explanation. Can a test launch Python in a subprocess, set up the appropriate data structure, let the subprocess Python shutdown, then check the subprocess's stderr for the error? It's a little convoluted, but perhaps it could be built into a general utility function that could be used to test __del__ methods in other modules, too. Not sure that it's worth the effort though. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11640 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12612] Valgrind suppressions
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12612 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12434] Strengthen 2.7 io types warning
Daniel Stutzbach stutzb...@google.com added the comment: On Sat, Jul 16, 2011 at 2:04 AM, Eli Bendersky rep...@bugs.python.orgwrote: Therefore, I propose to change this error message to: unicode argument expected, got '%s' as Terry suggested. Sounds good to me. -- Added file: http://bugs.python.org/file22687/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12434 ___div class=gmail_quoteOn Sat, Jul 16, 2011 at 2:04 AM, Eli Bendersky span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/span wrote:brblockquote class=gmail_quote style=margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex; Therefore, I propose to change this error message to:br  quot;unicode argument expected, got #39;%s#39;quot;br  as Terry suggested.br/blockquotedivbr/divdivSounds good to me. /div/divbr-- brDaniel Stutzbachbr ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12345] Add math.tau
Daniel Stutzbach stutzb...@google.com added the comment: On Wed, Jun 15, 2011 at 8:10 PM, Nick Coghlan rep...@bugs.python.orgwrote: I'd like to add a new constant to the math module: tau = 2*math.pi Rather than repeating all the reasons for why tau makes more sense than pi as the fundamental circle constant, I'll just refer interested readers to http://tauday.com/ (The following is best if read aloud using your best pirate impression) I'd rather name the new constant matey. Then we have: The circumference of a circle is: r matey The area of a circle is: one-quarter r r matey etc. -- nosy: +stutzbach Added file: http://bugs.python.org/file22397/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12345 ___div class=gmail_quoteOn Wed, Jun 15, 2011 at 8:10 PM, Nick Coghlan span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/span wrote:brblockquote class=gmail_quote style=margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex; I#39;d like to add a new constant to the math module:br br  tau = 2*math.pibr br Rather than repeating all the reasons for why tau makes more sense than pi as the fundamental circle constant, I#39;ll just refer interested readers to a href=http://tauday.com/; target=_blankhttp://tauday.com//a/blockquote divbr/divdiv(The following is best if read aloud using your best pirate impression)/divdivbr/divdivI#39;d rather name the new constant quot;mateyquot;.  Then we have:/divdivbr/divdivThe circumference of a circle is: r matey/div divThe area of a circle is: one-quarter r r matey/divdivetc./div/divbr-- brDaniel Stutzbachbr ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12192] Doc that collection mutation methods return item or None
Daniel Stutzbach stutzb...@google.com added the comment: Comment: This rule applies to special methods like __getitem__ and __setitem__. 'lis.append(item)' is equivalent to lis.__setitem__(len(lis):len(lis), item), so it should not be so surprising that it has the same return. It's not true for all special methods. __iadd__ and __imul__ return the list itself. That's a minor nitpick though. +1 on adding and return None to the docstrings of methods that often trip people up. Not everyone will read them, but some people will. -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12192 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8777] Add threading.Barrier
Changes by Daniel Stutzbach stutzb...@google.com: -- stage: - committed/rejected ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8777 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12128] Allow `abc.abstractproperty` to be overridden by a data attribute
Daniel Stutzbach stutzb...@google.com added the comment: SGTM. I've written code where this would have been useful. Could you write a patch? -- nosy: +stutzbach stage: - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12128 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12127] Inconsistent leading zero treatment
Daniel Stutzbach stutzb...@google.com added the comment: -0 on making it valid ever, due to the different meaning in many other languages. -1 on making it valid while many still use a version of python where it's valid with a different meaning. Maybe for Python 4. ;-) -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12127 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12128] Allow `abc.abstractproperty` to be overridden by a data attribute
Daniel Stutzbach stutzb...@google.com added the comment: I misread the original request. I'm +1 on making the following work, if it doesn't work already: class MySubClass(MyAbstractClass): SOME_LIMIT = 5 # Implements abstract property with fixed value We should be able to check that at instance creation time. I think abstract methods and properties are to force the class to define methods/properties. I don't think we should be checking if the instance adds things that are not defined at the class level (i.e., don't try to detect self.SOME_LIMT = 5). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12128 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12128] Allow `abc.abstractproperty` to be overridden by a data attribute
Daniel Stutzbach stutzb...@google.com added the comment: Daniel, the behavior you describe is already present in Python 3.2. Awesome. :) Do you have a compelling use-case for making self.x = 5 satisfy an abstractproperty requirement? One of the problems with that approach is that the instance does not satisfy the requirements until the line in __init__ is called that sets self.x. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12128 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11994] [2.7/gcc-4.4.3] Segfault under valgrind in string.split()
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11994 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Changes by Daniel Stutzbach stutzb...@google.com: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9971] Optimize BufferedReader.readinto
Daniel Stutzbach stutzb...@google.com added the comment: Looking at this again, I agree with John. For BufferedIOBase, read() is abstract while readinto() is concrete. That seems backward, and, indeed, it's the opposite of RawIOBase, where readinto() is abstract and read() is concrete. Unfortunately, this code has shipped, so changing which methods are abstract may not be practical. On the other hand, the documentation doesn't mention which methods are abstract versus concrete. All of that said, we can freely change the C implementation of BufferedReader which is a concrete class. That would allow us to cut out the extra allocoations/deallocations, even if we can't clean up the abstract vs concrete method issue. Basically, this would require greatly expanding buffered_readinto() in bufferedio.c to use _bufferedreader_raw_read() and related functions. As I think about this more... I'm not sure how much performance there is to gain here in practice. It seems like any time I'd want to use readinto(), it's because I want to do my own buffering, in which case why would I use a BufferedReader? I'm thinking that BufferedIOBase only provides a readinto() method for completeness, so it can be used as a drop-in replacement for an unbuffered file object. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9971 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: I checked in a fix to 3.3 just before the transition to hg. I confess that I've been procrastinating on getting hg set up properly, which is why I haven't gotten to checking this in to 3.2. Georg, if I get this in by Wednesday, will that be soon enough for your schedule for 3.2.1? (it sounds like it will be) -- nosy: +georg.brandl priority: normal - release blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11854] __or__ et al instantiate subclass of set without calling __init__
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11854 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6634] sys.exit() called from threads other than the main one: undocumented behaviour
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6634 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11845] Refcounting error in compute_slice_indices in rangeobject.c
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11845 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11707] Create C version of functools.cmp_to_key()
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11707 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7995] On Mac / BSD sockets returned by accept inherit the parent's FD flags
Daniel Stutzbach stutzb...@google.com added the comment: I'm confused by the patch (ed0259230611). The patch comment and the NEWS item state the returned socket is now always non-blocking but the code change adds sock.setblocking(True). -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7995 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11674] list(obj), tuple(obj) swallow TypeError (in _PyObject_LengthHint)
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11674 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11635] concurrent.futures uses polling
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11635 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11640] Shelve references globals in its __del__ method
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11640 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11628] cmp_to_key generated class should use __slots__
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11628 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11608] GzipFile cannot be used for streaming
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11608 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11583] os.path.isdir() is slow on windows
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11583 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11562] += on list inside a tuple raises TypeError but succeds anyway
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach stage: - test needed versions: +Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11562 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11480] Cannot copy a class with a metaclass other than type
Changes by Daniel Stutzbach stutzb...@google.com: -- keywords: +needs review nosy: +alexandre.vassalotti, stutzbach stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11480 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11455] issue a warning when populating a CPython type dict with non-string keys
Daniel Stutzbach stutzb...@google.com added the comment: For what it's worth, I believe this could be implemented easily by calling _PyDict_HasOnlyStringKeys at the end of the class creation process. -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11455 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553375] Add traceback.print_full_exception()
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach versions: +Python 3.3 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1553375 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9427] logging.error('...', exc_info=True) should display upper frames, too
Changes by Daniel Stutzbach stutzb...@google.com: -- dependencies: +Add traceback.print_full_exception() nosy: +stutzbach versions: +Python 3.3 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9427 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11402] _PyUnicode_Init leaks a little memory once
New submission from Daniel Stutzbach stutzb...@google.com: By the time _PyUnicode_Init is called and does the following: /* Init the implementation */ free_list = NULL; numfree = 0; free_list is already in use. _PyUnicode_Init clobbers whatever was stored there, leaking memory. In Python 2, this was not an issue because no unicode functions were called prior to initialization. In Python 3, Unicode objects are created when initializing other types. I found this using valgrind to search for definitely lost memory. I'd like to fix all of those cases, to make it easier to run valgrind on a regular basis. I'll upload a patch momentarily. -- assignee: stutzbach components: Interpreter Core messages: 130090 nosy: stutzbach priority: normal severity: normal stage: patch review status: open title: _PyUnicode_Init leaks a little memory once type: resource usage versions: Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11402 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11402] _PyUnicode_Init leaks a little memory once
Changes by Daniel Stutzbach stutzb...@google.com: -- keywords: +needs review, patch Added file: http://bugs.python.org/file20999/unicode-leak.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11402 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: I'm not sure. What infrastructure do we have to leaked memory that was allocated with PyMem_MALLOC? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: I meant to *detect* leaked memory, of course. :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: I played around with this a little. That code path doesn't appear to be exercised during the existing unit tests. I'll add a test so the leak at least shows up when the tests are run under valgrind. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4600] __class__ assignment: new-style? heap? == confusing
Changes by Daniel Stutzbach stutzb...@google.com: -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4600 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11333] Add empty __slots__ to collections.abc abstract base classes
Daniel Stutzbach stutzb...@google.com added the comment: +1. I've bumped into exactly this problem (https://github.com/DanielStutzbach/blist/issues/closed#issue/29) I'm not intimately familiar with how __slots__ works. Are there any drawbacks to adding an empty __slots__ to the ABCs? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11333 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10868] ABCMeta.register() should work as a decorator
Daniel Stutzbach stutzb...@google.com added the comment: Sounds good to me. Thanks for the clarifications and satisfying my curiosity! :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10868 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
New submission from Daniel Stutzbach stutzb...@google.com: In list.sort, if a key function throws an exception, the memory to store the keys is never freed. I introduced the bug in r86937. I'll upload a patch for review shortly. -- assignee: stutzbach components: Interpreter Core messages: 129574 nosy: pitrou, rhettinger, stutzbach priority: normal severity: normal stage: patch review status: open title: Memory leak after key function failure in sort type: resource usage versions: Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach stutzb...@google.com added the comment: Below is a link to the code where leak occurs and the patch goes: http://svn.python.org/view/python/branches/py3k/Objects/listobject.c?view=markuppathrev=88554#l1944 -- keywords: +3.2regression, needs review, patch Added file: http://bugs.python.org/file20918/plug-leak.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11335 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10868] ABCMeta.register() should work as a decorator
Daniel Stutzbach stutzb...@google.com added the comment: In what use-cases would you want to call MyABC.register() when defining a class instead of inheriting from MyABC? I always thought of the register() as hack to make it possible to support types written in C, which can't inherit from the ABC. -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10868 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7111] abort when stderr is closed
Daniel Stutzbach stutzb...@google.com added the comment: That's an interesting point. Do you know of places where we use fd 2 instead of sys.stderr? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7111 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7111] abort when stderr is closed
Daniel Stutzbach stutzb...@google.com added the comment: On Thu, Feb 3, 2011 at 11:56 AM, Alexander Belopolsky rep...@bugs.python.org wrote: 3rd party extensions. What is the use case for python -? Is it important enough to justify the risk of accidental data loss? I don't think closing stderr via the command line is an important use case, but pythonw.exe and Unix daemon processes are important use cases. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7111 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7111] abort when stderr is closed
Daniel Stutzbach stutzb...@google.com added the comment: On Thu, Feb 3, 2011 at 12:18 PM, Antoine Pitrou rep...@bugs.python.org wrote: I just took a look at http://pypi.python.org/pypi/python-daemon/, and it uses dup2() to redirect standard streams, which is far nicer. I'm more worried about the case where a daemon launches python. At startup, could we check that 2 and 3 are valid file descriptors, and, if not, open /dev/null? That way, they cannot later be inadvertently assigned to some other file? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7111 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach stutzb...@google.com added the comment: Committed in r87339. Thanks everyone for the feedback! -- assignee: eric.araujo - stutzbach keywords: -needs review resolution: - accepted stage: patch review - committed/rejected status: open - closed versions: -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2226] Small _abcoll Bugs / Oddities
Daniel Stutzbach stutzb...@google.com added the comment: Raymond, Do you have around 10 minutes today to look at the patch I submitted in Issue 8743? It appears to solve both this issue and that one, which have priorities critical and high, respectively. It's a reasonably small patch: around 20 lines changed plus 20 lines added, all Python code. Here's a direct link: http://bugs.python.org/file20045/set-with-Set.patch If it looks good to you, I'd like to commit it in time for 3.2b2 so that it gets as much exercise as possible before 3.2-final. Earlier you had expressed that adding the __rand__ methods to collections.Set would be tricky issue, so I'm concerned that I have overlooked something important and obvious. (Feedback from others is, of course, welcome as well) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2226 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach stutzb...@google.com added the comment: Doc change committed to py3k in r87346. Thanks, SilentGhost! I also committed r87349 to reverse r87162 (which was in the wrong branch). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2690 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach stutzb...@google.com added the comment: On Fri, Dec 17, 2010 at 2:17 PM, Amaury Forgeot d'Arc wrote wrote: Err... in r87339 there is a typo in all occurrences of 'Py_ReprEntr': Py_ReprEnter of course! Well, that's embarrassing. Fixed in r87354. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Changes by Daniel Stutzbach stutzb...@google.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8040] It would be nice if documentation pages linked to other versions of the same document
Daniel Stutzbach stutzb...@google.com added the comment: On Tue, Dec 14, 2010 at 7:22 PM, Ezio Melotti rep...@bugs.python.org wrote: If you want to work on this you should also consider that the URLs used on docs.python.org don't necessary match the directory structure present when building the doc locally. For example here it would be: .../py3k/library/fractions.html .../release31-maint/library/fractions.html .../release27-maint/library/fractions.html .../release26-maint/library/fractions.html Will this directory structure change as part of the hg migration? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8040 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
New submission from Daniel Stutzbach stutzb...@google.com: Attached is a patch to add documentation for Py_ReprEnter and Py_ReprLeave. Assigning to d...@python for review. -- assignee: stutzbach - d...@python keywords: +needs review, patch nosy: +d...@python stage: needs patch - patch review Added file: http://bugs.python.org/file20054/repr.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10711] Rip out HTTP 0.9 support
Daniel Stutzbach stutzb...@google.com added the comment: Would it be worth keeping (but modifying) test_http_0_9 to verify that the server complains in the expected way? -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10711] Rip out HTTP 0.9 support
Daniel Stutzbach stutzb...@google.com added the comment: At minimum, I think we should apply this part of Antoine's patch: -# Most web servers default to HTTP 0.9, i.e. don't send a status line. -default_request_version = HTTP/0.9 +default_request_version = HTTP/1.0 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach stutzb...@google.com added the comment: Good point. My perspective is skewed by all of the time that I spend working on container types. :-) How about if I change the first sentence to the following? Properly implementing :attr:`tp_repr` for container types requires special recursion handling. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Changes by Daniel Stutzbach stutzb...@google.com: Added file: http://bugs.python.org/file20071/repr-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach stutzb...@google.com added the comment: Thanks. New patch attached with a cross-reference to reprlib.recursive_repr. -- Added file: http://bugs.python.org/file20072/repr-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8753 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8040] It would be nice if documentation pages linked to other versions of the same document
Changes by Daniel Stutzbach stutzb...@google.com: -- versions: -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8040 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2226] Small _abcoll Bugs / Oddities
Daniel Stutzbach stutzb...@google.com added the comment: Minor point of clarity: you mean __rand__ not __radd__, right? Set objects do not support addition at all. Adding the __rand__ methods to collections.Set in and of itself is straightforward: def __rsub__(self, other): return self._from_iterable(other) - self __ror__ = __or__ __rand__ = __and__ __rxor__ = __xor__ I'm not sure I understand the can of worms. While replacing concrete tests with abstract tests may be worthwhile goal in its own right, why is it necessary to solve the particular shortcoming of missing __r* methods? Probably I'm missing something. With just the minimal change above, what kinds of things do you expect to fail? (assuming Issue8743 is also fixed) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2226 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8743] set() operators don't work with collections.Set instances
Daniel Stutzbach stutzb...@google.com added the comment: Would it be sufficient to: 1) Restrict collections.Set()'s operators to accept collection.Set but not arbitrary iterables, and 2) Fix Issue2226 and let set() | MySimpleSet() work via collections.Set.__ror__ Attached is a patch that implements this approach, nominally fixing both this and Issue2226. This solutions seems much too simple in light of how long I've been thinking about these bugs. I suspect there are code hobgoblins waiting to ambush me. ;) -- Added file: http://bugs.python.org/file20045/set-with-Set.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8743 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10542] Py_UNICODE_NEXT and other macros for surrogates
Daniel Stutzbach stutzb...@google.com added the comment: In bltinmodule.c, it looks like some of the indentation doesn't line up? Bikeshedding aside, it looks good to me. I agree with Eric Smith that the first part macro name usually refers to the type of the first argument (or the type the first argument points to). Examples: - Py_UNICODE_ISSPACE : Py_UNICODE - int - Py_UNICODE_TOLOWER : Py_UNICODE - Py_UNICODE - Py_UNICODE_strlen: Py_UNICODE * - size_t This is true elsewhere in the code as well: - PyList_GET_SIZE : PyListObject * - Py_ssize_t Yes, I know there are some unfortunate exceptions. ;-) I agree that it would be nice if something in the name hinted that the return type was Py_UCS4 though. Marc-Andre Lemburg wrote: The first argument of the macro can be any array, not just Py_UNICODE*, but also Py_UCS4* or even int*. It's true that macros in C do not have any type safety. While technically passing a Py_UCS4 * will work, on a UCS2 build it would needlessly check the Py_UCS4 data for surrogates. I think we should discourage that. You can also technically pass a PyListObject * to PyTuple_GET_SIZE, but that's also not a good idea. ;-) Alexander Belopolsky wrote: The issue is that once in in the process of reading the codepoint, it is determined whether the code point is BMP or non-BMP. Testing the result again in order to write it is somewhat wasteful. I don't think this would matter in practice, but would like to hear alternative opinions before moving further. If the common pattern is: ch = Py_UNICODE_NEXT(rp, end); uc = Py_UNICODE_SOME_TRANSFORMATION(ch); Py_UNICODE_PUT_NEXT(wp, uc); The second check for surrogates in Py_UNICODE_PUT_NEXT is necessary, unless you can prove that Py_UNICODE_SOME_TRANSFORMATION will never transform characters 0x1 into characters 0x1 or vice versa. Can we prove will always be the case, for current and future versions of Unicode, for all or almost-all of the transformations we care about? Answering that question and figuring out what to do about it are probably more trouble than it's worth. If a particularly point proves to be a bottleneck, we can always specialize the code there later. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10542 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10542] Py_UNICODE_NEXT and other macros for surrogates
Daniel Stutzbach stutzb...@google.com added the comment: +1 on the general idea of abstracting out repeated code. I will take a closer look at the details within the next few days. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10542 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach stutzb...@google.com added the comment: (I also noticed that the new methods from issue #9213 are not mentioned in the range() docs Wasn't that fixed in Issue9746? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2690 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach stutzb...@google.com added the comment: The descriptions of range's limitations there is no longer accurate (slicing is supported following this patch and containment testing is now efficient) Want to open a new issue for that? (or is there one already?) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2690 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach stutzb...@google.com added the comment: The descriptions of range's limitations in the docs still needs an update. -- assignee: georg.brandl - ncoghlan status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2690 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: Committed as r86937. Thanks again for reviewing! Although I do not anticipate any problems, I will keep an eye on the buildbots just in case. Antoine, regarding ms-alloced = (list_size + 1) / 2;, I ended up adding an extensive comment after all. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Changes by Daniel Stutzbach stutzb...@google.com: -- components: +Interpreter Core -Library (Lib) resolution: - accepted stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: The use of Py_LOCAL_INLINE is new to me since we usually use #define instead, but this has a cleaner look to it. I am unclear on whether all the our target compilers support an inline keyword. If you're sure it works everywhere, that's great. I fixed ./configure to properly set up Py_LOCAL_INLINE in Issue5553. :-) It will expand to static inline under both MSVC and gcc. On older compilers, it may expand to static __inline__, static __inline, or whatever else is needed to get the job done. As a last resort, it will expand to simply static, but I don't know of any 32-bit (or 64-bit) compilers where that would actually happen. Also note that this patch puts a lot of faith in branch prediction. If some target processor doesn't support it, or has limited ability to remember predictions, or mispredicts, then the code will be slower. I think even a limited amount of memory dedicated to branch prediction should be sufficient. There are two cases: 1) Sorting a simple type, like an int: the comparison is lightweight, and the CPU should have plenty of memory to remember which branch to take in the sorting code. 2) Sorting a complex type (i.e., calling a __lt__ method written in Python): the processor might not be able to remember which branch to take, but the performance impact will be small (as a percentage) since most of the CPU is being consumed by the comparisons. Thanks for taking the time to review this. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: Just wanted to post this so there weren't any illusions about the patch being a big win. When a key function is defined, this is all you can possibly shave off the time for a comparison. I don't want to argue whether the patch is a big win or not (I recognize that it is a tradeoff), but when using a key it does shave off more than the call to sortwrapper_richcompare. Stack with sortwrapper: long_richcompare do_richcompare PyObject_RichCompare sortwrapper_richcompare do_richcompare PyObject_RichCompare PyObject_RichCompareBool count_run list_sort Stack without: long_richcompare do_richcompare PyObject_RichCompare PyObject_RichCompareBool count_run list_sort -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10576] Add a progress callback to gcmodule
Daniel Stutzbach stutzb...@google.com added the comment: How about a string and a dict? the string can be start and stop and we can add interesting information to the dict as you suggest. I like where this is headed. How about putting the string in the dict, too? d['phase'] = 'start' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10576 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
New submission from Daniel Stutzbach stutzb...@google.com: The I/O ABC documentation has a blanket disclaimer at the top: The abstract base classes also provide default implementations of some methods in order to help implementation of concrete stream classes. For example, BufferedIOBase provides unoptimized implementations of readinto() and readline(). Which effectively means that to subclass one of them requires digging into Lib/_pyio.py to figure out which methods provide useful implementations. It would be handy to have a table, similar to the one for the collections ABCs [1], that spells out which methods are provided. [1]: http://docs.python.org/py3k/library/collections.html#abcs-abstract-base-classes I hope to have a patch ready sometime within the next week. -- assignee: stutzbach components: Documentation messages: 122930 nosy: stutzbach priority: normal severity: normal stage: needs patch status: open title: I/O ABC docs should specify which methods have implementations type: feature request versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10589 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
Daniel Stutzbach stutzb...@google.com added the comment: +io and doc people Attached is a simple patch to add a table to the documentation summarizing the I/O ABCs. -- keywords: +patch nosy: +benjamin.peterson, ezio.melotti, georg.brandl Added file: http://bugs.python.org/file19885/io-abc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10589 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
Daniel Stutzbach stutzb...@google.com added the comment: What does unsupported mean? Abstract would look more exact. It means they raise io.UnsupportedOperation when called (unless the subclass overrides them to do something else). They are not marked with @abstractmethod, so Abstract would be wrong. It should be a bit more clear in context. Here's a link to the point in the documentation where the table would fit: http://docs.python.org/dev/py3k/library/io.html#class-hierarchy The most relevant bit is: implementations are allowed to raise UnsupportedOperation if they do not support a given operation. E.g. readinto is listed as provided by RawIOBase in the doc text. The doc text describes the API. That is, it describes what the readinto method should do if a subclass of RawIOBase decides to implement it. The existing text doesn't contain any clues as to which methods contain useful implementations versus which will raise io.UnsupportedOperation if not overridden. Hence the need for the table. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10589 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
Daniel Stutzbach stutzb...@google.com added the comment: Other suggestions for a better name for that column are certainly welcome. :-) Stub Methods? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10589 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8743] set() operators don't work with collections.Set instances
Daniel Stutzbach stutzb...@google.com added the comment: Yes, I can take a stab at it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8743 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10576] Add a progress callback to gcmodule
Daniel Stutzbach stutzb...@google.com added the comment: These functions will be very useful for any long-running program. Thank you for the patch. Would you be willing to write tests and documentation? Would it make more sense for the callback to take a boolean instead of an integer as the first argument? -- nosy: +stutzbach ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10576 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: Antoine Pitrou wrote: Next time, please upload a single patch. Really. I haven't used Rietveld that much yet, and I'm still learning best-practices. I apologize for the painful experience. For anyone else planning to take a look at this, here it is as one big patch: http://codereview.appspot.com/3290041 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: Antoine, My original patch was much more focused, but had a slightly larger performance penalty for sorting random keys (see http://bugs.python.org/msg122178). Do you think the performance tradeoff there was still worthwhile? Ihave uploaded my original patch (as one patch :) ) here: http://codereview.appspot.com/3291041 If you think the original performance looks like a worthwhile tradeoff, could you spend about 1 minute looking over that version of patch and give your general impression? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10437] ThreadPoolExecutor should accept max_workers=None
Daniel Stutzbach stutzb...@google.com added the comment: Amaury Forgeot d'Arc said: pypy does have a GIL! D'oh. That shows you how much I know about PyPy. Make that keep in mind that IronPython doesn't have a GIL. ;) Brian Quinlan said: I think that using the number of CPUs for max_workers makes sense for you but won't for most users. So I wouldn't make it a default. Fair enough. Marking as invalid. -- resolution: - invalid stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10437 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: I'm starting to get settled in here at Google and finding time to follow up on everything that got put on hold while moving. Based on the feedback everyone gave me (thanks!), I greatly revised my script for comparing the speed of sort(). It's now compares speeds more quickly and more accurately. I discovered a few things along the way: 1) Don't try to run timings on a virtual server with crummy clock resolution. 2) Make sure that CPU frequency scaling is turned off. It increases the standard deviation of timings by an order of magnitude or two. On Linux systems, the new version of my script will check to see if CPU frequency scaling is turned on and provide suggestions on how to turn it off. On other operating systems, you are on your own. ;) Usage: filfre:~/py3k-patched$ ./python Tools/sort_speed/sort_speed.py -r 3 --maxn 10 ../py3k-pristine . sort_random Using my new-and-improved timing script and running on real hardware with CPU frequency scaling turned off, I was able to detect a slight slowdown on sorting random integers. Here's a summary for sorting random integers: Original patch (increasing execution time is bad): n in [700 .. 100,000]: 1% to 2% increase in execution time n in [70 to 700]: 2% to 3% increase n in [10 to 70]: 3% to 4% increase n in [5 to 7]: 7% to 9% decrease n in [0 to 3]: 18+% decrease I went back to the code and managed to squeeze out a bit more performance. Here's a summary of the new patch: New patch (increasing execution time is bad): n in [30,000 .. 100,000]: 1% increase in execution time n in [35 .. 30,000]: 1% to 2% increase n in [20 .. 35]: about the same n in [10 .. 20]: 1 to 2% decrease n in [4 .. 7]: 15+% decrease n in [0 .. 3]: 22+% decrease All of the above results are for sorting without a key. For sorting *with* a key, there's a big performance boost across the board (15% to 45% decrease in execution time, when the key function is simply int). Overall, the patch shrinks Objects/listobject.c by 10 lines. Executive Summary - Good: - Dramatically cuts execution time of sorting with a key (15% to 45%) - Significantly cuts execution time of sorting small lists without a key (15+%) Bad: - Very slightly increases execution of sorting large lists without a key (1% to 2%; less than 1% for very large lists) Worthwhile trade? -- Added file: http://bugs.python.org/file19779/sort-speed-test.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Changes by Daniel Stutzbach stutzb...@google.com: Added file: http://bugs.python.org/file19780/sort-faster.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Changes by Daniel Stutzbach stutzb...@google.com: Added file: http://bugs.python.org/file19781/detailed-speed-results.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: Antoine Pitrou rep...@bugs.python.org wrote: Why don't you contribute a list sorting benchmark to the suite in http://hg.python.org/benchmarks/? I considered that, but I want to separately benchmark sorting different kinds and quantities of data. AFAIK, there isn't an easy way to do that with perf.py. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: Antoine Pitrou rep...@bugs.python.org wrote: Right, that wouldn't suit your present purposes. But apparently you are proposing to add a list sorting benchmark to the Tools directory, with lots of duplicated code from that repo... Oh, I just stuck that under Tools because it was convenient for testing. The timing script is in a separate patch because I'm -0 on committing the timing script itself. The perf.py that my timing script uses is based on the one from benchmarks/, but with 90% of the code removed, so the amount of duplicated code is less than it may at first appear. My script could go in benchmarks/ if there was a convenient way for it to import perf.py. I wouldn't want to clutter the top-level directory with my script though. Any suggestions? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: Raymond Hettinger rhettin...@users.sourceforge.net added the comment: That result is surprising though -- I thought the concept was manipulate the key and value arrays at the same time instead of just the keys If the key parameter was not used, then the values pointer is a null pointer. Each time elements must be moved, the code needs to check if the values pointer is NULL. In other words, the overhead is one conditional branch per move or group of moves (we only have to check once for memmove()). Since the branch will always be the same throughout any given call to sort(), CPU branch prediction is effective making the branches fairly inexpensive. -- did you do more than this, perhaps changing the logic or pattern of comparisons? In the original version of the patch, I tried hard to avoid unrelated changes. In the new version, I made many small not-strictly-related optimizations. However, I have not changed the order in which comparisons occur. Why did the variable names change, pa/pb to ssa/ssb, etc.? I took pa to mean pointer to array A, but it's now a sortslice structure instead of a pointer. comment lines are rewrapped The comment rewrapping aren't completely spurious. The comments really have changed and in some cases the constraints on the inputs to a function have changed slightly. For what it's worth, my methodology when working on this patch went like this: 1) Make an improvement 2) Measure the performance relative to the previous reference point 3) If it was a statistically significant an improvement, commit the change to my local DVCS 4) Goto 1 I can upload the patch to Rietveld to facilitate review. If I can get Rietveld to show each of my local commits, would it be helpful to see the patch in incremental pieces? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach stutzb...@google.com added the comment: How did it get *faster* than the original (in the case with no key-function)? I was able to shave off some instructions in countrun(), binarysort(), and the setup and cleanup code in listsort() proper. For small n, these made a difference. Is there any chance you can persuade Uncle Timmy to review this? This is all his code and he's likely to have some good insights. Any suggestions on how he might be persuaded? ;-) Also, is there anyone else who is knowledgeable about Python on less common platforms? ISTM part of the optimization is dependent on the branch-prediction and other nuances that vary from environment to environment. That being said, doing fewer memory allocations is always a win. I do have a Windows machine I can test on. It's not exactly a less common platform, but at least it's a completely different compiler. I'll post the results once I have them. The Rietveld issue is here: http://codereview.appspot.com/3269041 I ended up loading my incremental patches in, but it's easy enough to diff the base with the last patch. If for some reasons it doesn't work as conveniently as I expect, let me know and I will upload it to Rietveld again as one big patch. If there's anything else I can do to make reviewing easier, please let me know. For that matter, if there's other code you'd like me to review in exchange or straightforward bugs you'd like to pawn off, I would be happy to help out. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9915 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10437] ThreadPoolExecutor should accept max_workers=None
Daniel Stutzbach stutzb...@google.com added the comment: In my case, each thread launches a CPU-intensive process via subprocess, then waits for it to report output and exit. Also, keep in mind that PyPy and IronPython don't have a GIL (although I'm not a PyPy or IronPython user myself). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10437 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
