Re: [Python-Dev] [Python-checkins] cpython: Add optional *func* argument to itertools.accumulate().
On Sun, Mar 27, 2011 at 10:53 PM, Nick Coghlan ncogh...@gmail.com wrote: On Mon, Mar 28, 2011 at 2:11 PM, Daniel Stutzbach stutzb...@google.com wrote: Is there a good use-case for the func argument? The examples that Raymond gives in the docs (cumulative multiplication, running min/max, cash flow accumulation) look fairly solid to me. (I had the nagging suspicion that I was making a blunder in my email, but I couldn't see it despite rereading my email several times before sending. My blunder was in not rereading the patch to see the examples. Anyway...) When would a running product, min, or max be useful? for running_min in accumulate(data, min): # Do what? -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Add optional *func* argument to itertools.accumulate().
On Mon, Mar 28, 2011 at 12:49 AM, Raymond Hettinger raymond.hettin...@gmail.com wrote: Do a google code search for R's builtin functions cumsum, cumprod, cummin, and cummax. Look at mumpy's accumulate ufunc which works with many operators. APL and K also have an accumulate tool which takes arbitrary functions. Thanks. I had not been thinking along numeric lines. I can see how these would be useful for working with matrices, vectors, and similar constructs. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Add optional *func* argument to itertools.accumulate().
On Sun, Mar 27, 2011 at 6:52 PM, raymond.hettinger python-check...@python.org wrote: -.. function:: accumulate(iterable) +.. function:: accumulate(iterable[, func]) Make an iterator that returns accumulated sums. Elements may be any addable -type including :class:`Decimal` or :class:`Fraction`. Equivalent to:: +type including :class:`Decimal` or :class:`Fraction`. If the optional +*func* argument is supplied, it should be a function of two arguments +and it will be used instead of addition. Is there a good use-case for the func argument? I can only think of bad use-cases (where func does something that does not remotely resemble addition). I fear that people will actually implement these bad use-cases, and I will have to try to read and understand their code. Adding the func argument seems analogous to adding a func argument to sum(), which would give it all of the power of reduce(). -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
On Mon, Mar 21, 2011 at 8:38 AM, Antoine Pitrou solip...@pitrou.net wrote: We cannot emulate svnmerge for porting between branches, though - and I doubt bzr can do it. That's because merges in common DVCSes are based on the DAG, while svnmerge is a prettily ad-hoc free-form thing. The equivalent way to how we had been using svnmerge would be to use hg transplant to move patches between branches (and never merging the branches). Conversely, the current hg workflow would be similar to committing changes to the earliest applicable svn branch, then doing a full svnmerge to later branches. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hg: inter-branch workflow
On Sun, Mar 20, 2011 at 9:39 AM, Georg Brandl g.bra...@gmx.net wrote: Now, hg pull --rebase prevents that by re-basing the A-B history line onto the latest remote head. After rebasing, the history looks like this: ... --- X --- C --- D --- E --- A' --- B' Rebasing also allows you to collapse the commits, so you can make the tree look like this: ... --- X --- C --- D --- E --- AB' Collapsing commits is useful for keeping the repository clean. For example, I make commit A and send it for review. Someone points out some flaws and I make commit B. I don't want the flawed version checked in to the main repository, so I collapse the commits into one commit AB'. Keeping the repository clean makes it easier to use a bisection search to hunt down the introduction of a bug. If every developer's intermediate commits make it into the main repository, it's hard to go back to an older revision to test something, because many of the older revisions will be broken in some way. In reality it works fine if you know the limits: rebasing really only should be applied if the changesets are not already known somewhere else, only in the local repo you're working with. The changesets must only be in the local repo *and* they must not have been merged into another branch yet. On Sun, Mar 20, 2011 at 9:21 AM, Guido van Rossum gu...@python.org wrote: Why does everyone want it and hate it at the same time? People love it because it's a very powerful tool. People hate it because it allows you to shoot yourself in the foot. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 regret about deleting list.sort(cmp=...)
On Sat, Mar 12, 2011 at 3:44 PM, Guido van Rossum gu...@python.org wrote:
I recently advised a Googler who was sorting a large dataset and
running out of memory. My analysis of the situation was that he was
sorting a huge list of short lines of the form shortstring,integer
with a key function that returned a tuple of the form (shortstring,
integer).
As Raymond pointed out, a change I made for 3.2 significantly shrinks the
memory footprint of sorting with a key (although it's still more
memory-intensive than sorting with cmp).
He could reduce the memory footprint further by sorting in two passes
instead of using a tuple, leveraging the fact that Python guarantees a
stable sort. In 3.2 or later, this technique will require roughly twice as
much memory as just storing the list:
biglist.sort(key=lambda s: int(s.split(',')[1])) # Sort by the integer
biglist.sort(key=lambda s: s.split(',')[0]) # Sort by the shortstring
I think the use cases are pretty narrow where there's plenty of memory for
storing the list but not enough to store two copies.
--
Daniel Stutzbach
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 regret about deleting list.sort(cmp=...)
On Sat, Mar 12, 2011 at 9:17 PM, Terry Reedy tjre...@udel.edu wrote: But in this case, they are much slower. To be faster, one would need something like key=lambda p,q:p*(lcm//q), where lcm is the least common multiple of of all the q's (denominators). For the example above, lcm = 700. But for longer lists, it could be outrageously large. You can get away with less precision. It's okay if the key function loses some information, as long as it still has enough to distinguish each pair of numbers. In other words, you just need a number that's at least as large as the largest lcm between any pair: max_denominator_sq = max(q for p, q in fraction_list) ** 2 # Strictly larger than the lcm between any pair fraction_list.sort(key=lambda p, q: p*max_denominator_sq//q) Using the above, key is 4 times faster than cmp in Python2.6: stutzbach-glaptop:~$ python2.6 -m timeit -s 'fractions = [(i, j) for i in range(100) for j in range(1, 100)]' 'sorted(fractions, cmp=lambda (p,q),(r,s): cmp(p*s, q*r))' 10 loops, best of 3: 23.3 msec per loop stutzbach-glaptop:~$ python2.6 -m timeit -s 'fractions = [(i, j) for i in range(100) for j in range(1, 100)]' 'max_denominator_sq = max(q for p, q in fractions)**2' (1,2), (3,4), (0,5), (9,100), (3,7), (2,8)'sorted(fractions, key=lambda t: t[0]*max_denominator_sq//t[1])' 100 loops, best of 3: 5.52 msec per loop with a further speed improvement in 3.2: stutzbach-glaptop:~$ ~/python/cpython-3.2/python -m timeit -s 'fractions = [(i, j) for i in range(100) for j in range(1, 100)]' 'max_denominator_sq = max(q for p, q in fractions)**2' 'sorted(fractions, key=lambda t: t[0]*max_denominator_sq//t[1])' 100 loops, best of 3: 4.89 msec per loop and more speed improvements with my experimental changes targeting 3.3 (not yet in the bug tracker): :-) stutzbach-glaptop:~$ ~/python/cpython/python -m timeit -s 'fractions = [(i, j) for i in range(100) for j in range(1, 100)]' 'max_denominator_sq = max(q for p, q in fractions)**2' 'sorted(fractions, key=lambda t: t[0]*max_denominator_sq//t[1])' 100 loops, best of 3: 3.73 msec per loop -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hg diff
On Tue, Mar 8, 2011 at 6:30 PM, Éric Araujo mer...@netwok.org wrote: What’s the command you use with git? Maybe someone will find the Mercurial one. Something like the following, assuming we're both working on branch master to begin with. git fetch their-repository master:experimental-branch git diff master...experimental-branch The idea is to pull their remote branch but not merge it, which will create multiple heads locally. Then find the common ancestor of my regular local head and the new head, and diff the ancestor with the new head. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hg diff
On Tue, Mar 8, 2011 at 12:34 AM, Martin v. Löwis mar...@v.loewis.dewrote: With a branch you can easily view the full patch, making a branch strictly more general. I just asked this before: how *exactly* do you do that? I confess: I'm not sure exactly how to do it in hg. I know it's easy in git; I assume it's easy in hg. I did some searching but was unable to come up with the right incantation. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hg diff
On Antoine Pitrou solip...@pitrou.net wrote: How do you review a branch? Below is an example from github (because that's where my experience with reviewing DCVS branches comes from), but I think it communicates the idea well. The user hsoft forked my blist project, made some changes, and sent me a request to pull from his branch. This is what I see: https://github.com/DanielStutzbach/blist/pull/30/files On Antoine Pitrou solip...@pitrou.net wrote: You are assuming that I, as a reviewer, want to know about the history of changesets that led to the patch, but I don't: I want to read the patch as a cohesive whole, and I couldn't care less about the various typo fixes, style changes, performance tweaks that were checked in all along. With a branch you can easily view the full patch, making a branch strictly more general. The advantage of having a branch comes when you want to review the second or third iteration of a proposed change. With a branch, you can view the diff between the latest iteration and the branch point to view the change as a cohesive whole. Or you can view the diff between the latest iteration and the last iteration you reviewed, to see if they have addressed your earlier comments or not. You can also just tweak a few things and push the changes back to them. They can easily merge your changes with any changes they've made in the meantime (which is hard to do if you're pushing patch files around). -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .hgignore (was: Mercurial conversion repositories)
On Sat, Mar 5, 2011 at 5:54 AM, Tim Delaney timothy.c.dela...@gmail.comwrote: If those were to be removed from .hgignore then there would be a high likelihood of someone doing hg addremove and inadvertently tracking them. The purpose of .hgignore is to prevent inadventently tracking files that shouldn't be tracked. If the goal is to prevent something from being committed, shouldn't the check go in a pre-commit hook instead? -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] of branches and heads
On Sun, Feb 27, 2011 at 2:42 AM, Adrian Buehlmann adr...@cadifra.comwrote: On 2011-02-26 23:26, Greg Ewing wrote: There are *some* topological restrictions, because hg won't let you assign a branch name that's been used before to a node unless one of its parents has that name. So you can't create two disconnected subgraphs whose nodes have the same branch name. That's not completely correct. You *can* do that. Can we create a hook on the server to reject changesets like that? -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] hooks: Fix checkbranch hook
On Sat, Feb 26, 2011 at 11:48 PM, Nick Coghlan ncogh...@gmail.com wrote: On Sun, Feb 27, 2011 at 5:26 PM, Martin v. Löwis mar...@v.loewis.de wrote: If you think that it's a likely problem that people create named branches by mistake, then we should have a hook. I suspect the concern is that people may create named branches locally as part of their own workflow, then mistakenly push those branches instead of collapsing back to a single commit against the relevant line of development. +1 -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial conversion repositories
On Fri, Feb 25, 2011 at 6:32 PM, Nick Coghlan ncogh...@gmail.com wrote: Would it be possible to name trunk as 2.x instead? Otherwise I could see people getting confused and asking why trunk was closed, and/or not the same as default. Can we just get rid of trunk altogether? It's history is a strict subset of the 2.7 branch's history, isn't it? -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial conversion repositories
On Sat, Feb 26, 2011 at 8:44 AM, Antoine Pitrou solip...@pitrou.net wrote: Le samedi 26 février 2011 à 08:38 -0800, Daniel Stutzbach a écrit : Can we just get rid of trunk altogether? It's history is a strict subset of the 2.7 branch's history, isn't it? Named branches are exclusive, they can't be a subset of each other ;) (in other words: 2.7 starts where trunk stops; trunk changesets are strict ancestors of 2.7) Maybe I don't fully understand how Mercurial branches work or how it defines terminology (in fact, that is likely :) ). What's the difference between the statement trunk changesets are strict ancestors of 2.7 and the statement trunk's history is a strict subset of 2.7's history? -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial conversion repositories
On Sat, Feb 26, 2011 at 9:55 AM, Antoine Pitrou solip...@pitrou.net wrote: There is no such thing as an unnamed branch. What would hg branches show? An empty space? I understand now why I was confused. I had previously read the sentence Both Git and Mercurial support unnamed local branches. at http://mercurial.selenic.com/wiki/BranchingExplained But as I dig deeper, I see that there is only one unnamed branch, and it actually does have an implicit name: default. It appears Mercurial supports at least three different kinds of branching: cloning (similar to Bazaar), bookmarks (similar to git), and named branches. So a named branch can contain more than one branch. Were there reasons for going with named branches over bookmarks? PEP 385 discusses only cloning and named branches. I'm just curious, not trying to start a long discussion. :-) -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial conversion repositories
On Sat, Feb 26, 2011 at 1:37 PM, Brett Cannon br...@python.org wrote: There is hg-git, but that is hg on top of git. Actually, hg-git is bidirectional. The hg-git documentation is written from the perspective of an hg client talking to a git server, but for a DVCS client and server are a matter of perspective. I spent some time on Friday setting up hg-git on my workstation and making a few test commits. It took me awhile to figure out how to get everything working, but it seems to work smoothly now. At some point I'll update http://wiki.python.org/moin/Git with instructions. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sat, Feb 12, 2011 at 8:20 PM, exar...@twistedmatrix.com wrote: What part do you think is a hard problem? Convincing people to switch to a new API? I think the hard parts is coming up with an API that's simple enough to learn quickly but powerful enough for to cover most use-cases and cleanly extendable to cover use-cases we haven't thought of. If we go with something based on or inspired by Twisted, that solves some problems, but creates others. Will users be able to later migrate to using Twisted proper? Will the standard library module and Twisted go out of sync? What happens if a user tries to use both the standard library module and Twisted? Please understand that I'm not saying these are insurmountable problems. I'm just suggesting things that might be good to address in a PEP. *Defining* the new API doesn't seem very hard to me. If you have the skills and experience so that designing a async API is not as hard for you, please run with it. :-) Personally, I would love to see asyncore deprecated in favor of something better. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sat, Feb 12, 2011 at 4:22 PM, exar...@twistedmatrix.com wrote: Do people want to seriously consider deprecating asyncore and adding a replacement for it to the stdlib? (Hey, PyCon is coming up. How convenient. :) The desire is there, but it's a hard problem. There was a similar discussion before PyCon 2009, but not much came of it: http://mail.python.org/pipermail/python-dev/2009-March/086678.html -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Fri, Feb 11, 2011 at 8:06 AM, Guido van Rossum gu...@python.org wrote: And finally remember that asyncore is the most monkey-patched module in the world. :-) I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Fri, Feb 11, 2011 at 10:36 AM, Antoine Pitrou solip...@pitrou.netwrote: Daniel Stutzbach stutzb...@google.com wrote: I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. Would that be a Mapping or a Sequence? Before or after monkey-patching? :-) -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87903 - python/branches/py3k/Doc/whatsnew/3.2.rst
Thanks for catching the misspelling of my name! If you have a moment, could you look over my patch for Issue 8743http://bugs.python.org/issue8743 ? On Mon, Jan 10, 2011 at 1:26 PM, raymond.hettinger python-check...@python.org wrote: Author: raymond.hettinger Date: Mon Jan 10 22:26:49 2011 New Revision: 87903 Log: Misspelling. Modified: python/branches/py3k/Doc/whatsnew/3.2.rst Modified: python/branches/py3k/Doc/whatsnew/3.2.rst == --- python/branches/py3k/Doc/whatsnew/3.2.rst (original) +++ python/branches/py3k/Doc/whatsnew/3.2.rst Mon Jan 10 22:26:49 2011 @@ -553,7 +553,7 @@ range(0, 100, 2)[0:5] range(0, 10, 2) - (Contributed by Daniel Stutzback in :issue:`9213` and by Alexander Belopolsky + (Contributed by Daniel Stutzbach in :issue:`9213` and by Alexander Belopolsky in :issue:`2690`.) * The :func:`callable` builtin function from Py2.x was resurrected. It provides @@ -1514,7 +1514,7 @@ and it saves time lost during comparisons which were delegated by the sort wrappers. - (Patch by Daniel Stutzback in :issue:`9915`.) + (Patch by Daniel Stutzbach in :issue:`9915`.) * JSON decoding performance is improved and memory consumption is reduced whenever the same string is repeated for multiple keys. Also, JSON encoding ___ Python-checkins mailing list python-check...@python.org http://mail.python.org/mailman/listinfo/python-checkins -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FYI: Python 2.7.1 + gcc 4.6 (experimental) probable optimizer problem
On Sat, Jan 8, 2011 at 12:03 PM, Ralf W. Grosse-Kunstleve r...@yahoo.comwrote: g++ (GCC) 4.6.0 20101206 (experimental) % make /bin/sh: line 1: 41686 Segmentation fault (core dumped) CC='gcc -pthread' LDSHARED='gcc -pthread -shared ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py build make: *** [sharedmods] Error 139 Does that version of gcc emit any warnings during compilation? -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On Tue, Jan 4, 2011 at 9:33 AM, Lukas Lueg lukas.l...@googlemail.comwrote: The keys are immutable anyway so the instances of PyDict could manage a opaque value (in fact, a counter) that changes every time a new value is written to any key. Once we get a reference out of the dict, we can can do very fast lookups by passing the key, the reference we know from the last lookup and our last state. The lookup returns a new reference and the new state. If the dict has not changed, the state doesnt change and the reference is simply taken from the passed value passed to the lookup. That way the code remains the same no matter if the dict has changed or not. I have had similar ideas in the past but have never found time to explore them. The same mechanism could also be used to speed up attribute access on objects. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't compile regex module with Python 3.2
On Wed, Dec 8, 2010 at 6:56 PM, MRAB pyt...@mrabarnett.plus.com wrote: Is this change intentional? If so, why does unicodeobject.h still do the mapping? In 3.2b1, unicodeobject.h doesn't map _PyUnicode_IsWhitespace: http://svn.python.org/view/python/tags/r32b1/Include/unicodeobject.h?view=markup Do you have an old unicodeobject.h somehow? -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] I/O ABCs
The documentation for the collections Abstract Base Classes (ABCs) [1] contains a table listing all of the collections ABCs, their parent classes, their abstract methods, and the methods they provide. This table makes it very easy to figure out which methods I must override when I derive from one of the ABCs, as well as which methods will be provided for me. I'm working on a similar table for the I/O ABCs ( http://bugs.python.org/issue10589). The existing documentation [2] describes the methods of each class but doesn't describe which methods provide a meaningful implementation and which methods a user should override. If I want to inherit from one of the I/O ABCs, I have to go poking into Lib/_pyio.py to figure out which methods I need to override. While starting to examine the I/O ABCs, I discovered that there are some inconsistencies. For example, RawIOBase provides .read() if the subclass provides .readinto(). BufferedIOBase does the opposite; it provides .readinto() if the subclass provides .read() [3]. I would like to fix some of these inconsistencies. However, this will be a backwards-incompatible change. A Google Code Search suggests that the ABCs are currently only used within the standard library [4]. Just to be clear, the changes would NOT impact code that merely uses I/O objects; they would only impact code that implements I/O by subclassing one of the I/O ABCs and depending on features that are currently undocumented. Does anyone have any categorical objections? [1]: http://docs.python.org/py3k/library/collections.html#abcs-abstract-base-classes [2]: http://docs.python.org/py3k/library/io.html#class-hierarchy [3]: Possibly hurting performance by forcing .readinto() to perform the extra allocations, copies, and deallocations required by .read(). [4]: http://www.google.com/codesearch?hl=ensa=Nq=BufferedIOBase++lang:pythonct=rrcs_r=lang:python -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
2010/10/27 Kristján Valur Jónsson krist...@ccpgames.com Svn.python.org already plays host to some other, less official, projects such as stackless, so why not this? What are the benefits of hosting such a project on svn.python.org instead of somewhere else? (such as GitHub or BitBucket) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
2010/10/27 Kristján Valur Jónsson krist...@ccpgames.com Firstly, the ease of integrating changes. It would be possible to port those bugfixes that release-27 gets, and also backport selected things from py3k using the tools already in place such as svnmerge. py3k will soon be moving to Mercurial, so svnmerge would not be helpful for much longer. On the plus side, since Mercurial is a Distributed Version Control System, if you setup an unofficial continuation of Python 2 on the host of your choice, it will be easy for you to pull patches from py3k. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure
On Thu, Oct 14, 2010 at 12:38 PM, barry.warsaw python-check...@python.orgwrote: -# Generated by GNU Autoconf 2.65 for python 3.2. +# Generated by GNU Autoconf 2.67 for python 3.2. Was the change in autoconf versions intentional and/or is it a problem? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Fri, Oct 8, 2010 at 4:44 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: Hehe. What's the story behind changing the name from Cheeseshop to PyPI btw ? I found the first one much nicer A through investigation revealed that the Cheeseshop did not in fact have any cheese at all. Not even Wensleydale. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rietveld integration into Roundup
On Sat, Oct 2, 2010 at 3:55 PM, Martin v. Löwis mar...@v.loewis.dewrote: I'll have to come up with a better way to determine the branch which a patch was created on. That would also be helpful for those of us using DVCS software to talk to the svn server. :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for binary operations on Sets
On Wed, Sep 29, 2010 at 11:29 PM, Terry Reedy tjre...@udel.edu wrote: Does this violate the Sequence ABC (assuming there is one)? There is a Sequence ABC, but it does not define __add__. It only defines the following methods: __contains__, __getitem__, __iter__, __len__, __reversed__, count, and index tuple, range, and str types all register as following the Sequence ABC. list and bytearray types register as following the MutableSequence ABC, which is a subclass of the Sequence ABC. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We should be using a tool for code reviews
On Thu, Sep 30, 2010 at 9:52 AM, exar...@twistedmatrix.com wrote: Of course, this is only true if the core developers *do* submit to the same rules. Is anyone proposing that current core committers have all their work reviewed before it is accepted? I think most would welcome (or at least tolerate ;) ) additional review of their code. The hard part is encouraging contributors to find the time and motivation to thoroughly review code that they aren't personally interested in (and perhaps not even familiar with). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We should be using a tool for code reviews
On Thu, Sep 30, 2010 at 10:48 AM, Martin v. Löwis mar...@v.loewis.dewrote: Not sure how well 'tit for tat' schemes work - we *could* require that people don't commit unreviewed changes, and also require that you can't commit unless you have reviewed somebody else's changes. I wonder if a reputation scheme would work better. Track and publicize patch submissions, reviews, and the review/patch ratio, but do not enforce any particular ratios. Perhaps provide a roundup query showing patches awaiting review sorted by the patch submitter's review/patch ratio? (in descending order) Obviously there would be many non-trivial details to work out. I'm just brainstorming. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We should be using a tool for code reviews
On Wed, Sep 29, 2010 at 1:41 PM, Antoine Pitrou solip...@pitrou.net wrote: He, several of us would like it too (although for short patches it doesn't really make a difference), but what's missing is some kind of Roundup integration. Something as trivial as a start review button in front of every uploaded patch file would do the trick; it has been suggested several times already, but what's needed is someone to write the code :) Most patches won't import cleanly into Rietveld, because the patch typical does not contain precise information about the base revision for the patch. How about the opposite approach: make a Python-specific version of upload.py that lets the user attach the patch to an issue with an optional message? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Atlassian and bitbucket merge
On Wed, Sep 29, 2010 at 4:25 PM, Martin v. Löwis mar...@v.loewis.dewrote: Of course, with a hosted service, you often can't run hooks at all, so the effort to write them is also reduced :-) It should be easy to write an automated script that pulls the latest changes from the hosted service and then runs hooks. Obviously, it would not be possible to write hooks that reject changesets, but it would be possible to write hooks that send email or notify buildbots. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyObject_GC_UnTrack() no longer reliable in 2.7?
On Fri, Sep 24, 2010 at 4:09 PM, Martin v. Löwis mar...@v.loewis.dewrote: I think it would be possible to have two versions of _PyGC_REFS_UNTRACKED, one being, say, -5. _PyGC_REFS_UNTRACKED_AND_KEEP_IT_THAT_WAY would be what you get when you call PyObject_GC_UnTrack; the code to do automatic tracking/untracking based on contents would use some other new API (which would be non-public in 2.7.x). Where would the extra state information be stored? (to distinguish untracked and untracked-and-keep-it-that-way) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new LRU cache API in Py3.2
On Sat, Sep 4, 2010 at 3:28 AM, Stefan Behnel stefan...@behnel.de wrote: What about adding an intermediate namespace called cache, so that the new operations are available like this: I had been thinking that the lru_cache should be a class (with a dict-like interface), so it can be used explicitly and not just as a decorator. It could provide a wrap() method to be used as a decorator (or implement __call__ to keep the current semantics, but explicit is better than implicit) widget_cache = lru_cache() widget_cache[name] = widget @lru_cache().wrap def get_thingy(name): return something(name) # get_thingy.cache is an lru_cache instance print(get_thingy.cache.hits) I have been using a similar LRU cache class to store items retrieved from a database. In my case, a decorator-paradigm wouldn't work well because I only want to cache a few of the columns from a much larger query, plus there are multiple functions that want to talk to the cache. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r84430 - in python/branches/py3k: Include/unicodeobject.h Objects/unicodeobject.c
On Thu, Sep 2, 2010 at 6:26 PM, Victor Stinner victor.stin...@haypocalc.com wrote: But I didn't found any doc for other Py_UNICODE_str*() functions in Doc/c-api/*.rst. http://bugs.python.org/issue8649 - Py_UNICODE_* functions are undocumented -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Tue, Aug 31, 2010 at 4:54 AM, M.-A. Lemburg m...@egenix.com wrote: Is it possible to have multiple versions of the lib C loaded on Windows ? Yes, and it's a pretty common situation. The fopen() that I call within a DLL may not be the same fopen() called by another DLL. When writing a DLL for Windows, the API must be designed with the assumption that anything returned by the C library cannot be passed a different C library. For example, suppose I a expose a function in my DLL that allocates some memory, populates it with useful information, and returns a pointer. I must also supply a function to free the memory. I cannot ask the caller to simply call free(), because their free() may not be using the same heap as my malloc(). Likewise, a FILE * isn't safe to pass around, unless I can guarantee that the application really is one big happy family compiled against the same version of the C library. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for commits and/or privileges
On Sat, Aug 21, 2010 at 12:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: +1 from me too. When you start, olease do leave me as the primary maintainer for issues relating to sets and set ABCs. I see myself as the primary maintainer of nothing. :-) I would not commit anything related to sets or set ABCs unless you have signed off on it in some way. Perhaps in time there will be some piece of Python that I've modified so heavily that I become ipso facto the primary maintainer, but I'm in no hurry. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for commits and/or privileges
On Sat, Aug 21, 2010 at 3:47 AM, Martin v. Löwis mar...@v.loewis.de wrote: Please send me your SSH key. Done. I have also subscribed to python-committers and python-checkins. I will add my interests to Misc/maintainers.rst. Are there any other initial start-up tasks I should perform? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shared Folders Under Vista
On Sun, Aug 22, 2010 at 9:01 PM, Steve Holden st...@holdenweb.com wrote: Folders that I create with Cygwin using mkdir appear to be shared - sort of. They are denoted with the shared folder icon, and when I use Windows Explorer to delete them I get the warning about the folder being shared. When you say mkdir, what exactly do you mean? /usr/bin/mkdir? If so, what does this question have to do with Python? Python's os.mkdir? If so, which version of Python are you using? What version of Windows are you running? For what it's worth, I just tried both of the above on my XP/Cygwin system and did not see the shared folder icon in Explorer. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Request for commits and/or privileges
Several issues that I'm involved with (listed below) are ready for commit, as far as I can tell. They have a patch, and either a core developer has positively reviewed the patch or the patch is a straightforward implementation of a core developer's suggested approach. They are all bug-fixes or optimizations. May I have commit privileges so that I can commit these patches and future patches in a similar state? If the consensus is negative for any reason, I completely respect that decision and will continue to contribute patches just as I am now (but in that case, consider this my humble request for someone to commit these changes :) ). If positive, I would start by committing just one (after writing an appropriate NEWS entry) and soliciting feedback to make sure that I had done it right. http://bugs.python.org/issue8781 - 32-bit wchar_t doesn't need to be unsigned to be usable http://bugs.python.org/issue9214 - Most Set methods of KeysView and ItemsView do not work right http://bugs.python.org/issue8750 - Many of MutableSet's methods assume that the other parameter is not self http://bugs.python.org/issue5553 - Py_LOCAL_INLINE(type) doesn't actually inline except using MSC http://bugs.python.org/issue2521 - ABC caches should use weak refs http://bugs.python.org/issue808164 - socket.close() doesn't play well with __del__ Many more in the pipeline :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cProfile and threads
2010/8/17 Kristján Valur Jónsson krist...@ccpgames.com These patches are the result of work that we have done in profiling Stackless Python server applications at runtime, but they apply just as well to C Python. The first patch makes _lsprof, the engine behind cProfile, multi-stack aware. This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered. +1 The second patch allows to set the trace/profile function in python * globally*, so that all threads are affected. This is essential if you want to take a profililng snapshot of a running application. +1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klose d...@ubuntu.com wrote: 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP should define a non-configurable way for the tag. The tag should include all information which currently makes an extension ABI incompatible: - the python implementation (cpython, PyPy, ...) - the python version (3,2, 3.3, ...) - unicode configure option (--with-wide-unicode, 16 or 32) - platform information (necessary?) I'm not sure it's that easy to enumerate all of the ways to end up with an incompatible ABI. There are quite a lot of configure options listed with configure --help, and it's not always obvious if an option changes the ABI. On top of that, there are compilation flags that can change the ABI, as Kristján discovered in the following thread: http://mail.python.org/pipermail/python-dev/2010-June/100583.html On the flip side, a fully enumerated ABI signature could be used to identify (in)compatible binary eggs, which is basically impossible now. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration readiness
On Sat, Jul 3, 2010 at 11:36 AM, Tres Seaver tsea...@palladion.com wrote: - - Create a pristine clone of the trunk (one where I never commit any changes): $ cd $python_repo $ hg clone http://code.python.org/hg/trunk/ pytrunk-upstream - - Create a local clone from that repository: $ hg clone pytrunk-upstream pytrunk-work $ ./configure make My question is basically the same as Terry Reedy's, but I'm going to phrase it a bit differently: This is perhaps a naive question, but why do you create a second local clone instead of just creating a branch? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration)
On Thu, Jul 1, 2010 at 7:52 AM, anatoly techtonik techto...@gmail.comwrote: 4. Even if I make patch in my Mercurial clone - you still can't pull it and I have to attach it to tracker. No gain. Was there ever any discussion about hosting the central repository on a site such as bitbucket or github? I tried searching the python-dev archives but was unable to find much. Anyway... assuming there's a at least a clone of the central repository on one of those sites, you can fork it and work on your patch there. A core developer can then pull your patch to their local repository, tweak it as needed, then commit it to the central repository. I would put accent on keeping mirror of Subversion as easy way to contribute for those who are not yet ready for DVCS. Subversion also provides greater interoperability. Assuming that any modern DVCS tool may act as Subversion client, we will gain more contributors if we won't try to force people use Python and Mercurial. The hg-git tool allows Mercurial and git to interoperate, so that's not as much of an issue as it once was. It's geared toward using a Mercurial client to talk to a git server, but I'm told it can work the other way around with a bit of work. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Thu, Jun 24, 2010 at 10:50 AM, Barry Warsaw ba...@python.org wrote: The idea is to put the Python version number in the shared library file name, and extend .so lookup to find these extended file names. So for example, we'd see foo.3.2.so instead, and Python would know how to dynload both that and the traditional foo.so file too (for backward compatibility). What use case does this address? PEP 3147 addresses the fact that the user may have different versions of Python installed and each wants to write a .pyc file when loading a module. .so files are not generated simply by running the Python interpreter, ergo .so files are not an issue for that use case. If you want to make it so a system can install a package in just one location to be used by multiple Python installations, then the version number isn't enough. You also need to distinguish debug builds, profiling builds, Unicode width (see issue8654), and probably several other ./configure options. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bug tracker permissions request
May I have enhanced permissions on the bug tracker, so that I can perform the following tasks? - Assign issues to myself that I plan to write a patch for - Update the Stage to patch review after writing a patch - Occasional bug triage My login name on the tracker is stutzbach. I only find the time to produce patches once in awhile, but when I have the time I usually produce more than one. Assigning bugs to myself will increase my motivation to write patches, as I will feel that I've made a commitment to fixing them. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bug tracker permissions request
On Tue, Apr 27, 2010 at 10:14 AM, R. David Murray rdmur...@bitdance.comwrote: Done. I agree with Brian, Daniel has been making valuable contributions for quite some time now. I/we will keep an eye on his triage, of course. Thanks. Is there a document that describes the meaning of all of the different fields in the bug tracker? I've read http://www.python.org/dev/workflow/, but it doesn't cover everything. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bug tracker permissions request
On Tue, Apr 27, 2010 at 11:16 AM, R. David Murray rdmur...@bitdance.comwrote: I think it is on Brett's list to update that doc, but maybe we should help him out :). Can you list what's missing? We should fill in the gaps. Sure, here's what I've noticed: The page doesn't document the Resolution or Status fields. For the Keywords field, the page only documents the easy keyword. Also, some of the headings in the page are enclosed in square brackets, while others are not. It's not clear to me what the brackets are intended to designate. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://bugs.python.org/ is down
On Sat, Apr 17, 2010 at 12:17 PM, Victor Stinner victor.stin...@haypocalc.com wrote: http://bugs.python.org/ displays Service Temporarily Unavailable. Is it normal? It's working fine for me. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
On Fri, Apr 16, 2010 at 4:11 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: ISTM that making it illegal costs cycles with giving any real benefit. It is reasonably common to accept **kwds and then pass it down to another function. Do we want to validate the keys of every kwds dict on every call? Why do we even care? IIRC, there's a performance hack in dictobject.c that keeps track of whether all of the keys are strings or not. The hack is designed so that lookup operations can call the string compare/hash functions directly if possible, rather than going through the slower PyObject_ functions. Consequently, validating **kwds should be cheap. I don't know if the the current validating of **kwds with Python functions already leverages that hack or not. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
On Fri, Apr 16, 2010 at 4:51 PM, Benjamin Peterson benja...@python.orgwrote: 2010/4/16 Daniel Stutzbach dan...@stutzbachenterprises.com: IIRC, there's a performance hack in dictobject.c that keeps track of whether all of the keys are strings or not. The hack is designed so that lookup operations can call the string compare/hash functions directly if possible, rather than going through the slower PyObject_ functions. That won't work. You could put non-string keys in a dictionary and remove them, but the dictionary would still be in the less optimized state. It would be enough to make the common case fast (1 pointer comparison). The error case would need to check all of the keys, true, but that's not really a performance concern. Unless you're saying you often create a dictionary, add non-string keys, remove the non-string keys, then pass it as a **kwds? ;-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tuning Python dicts
On Sat, Apr 10, 2010 at 1:06 PM, Reid Kleckner r...@mit.edu wrote: Looking at dictnotes.txt, I can see that people have experimented with taking advantage of cache locality. I was wondering what benchmarks were used to glean these lessons before I write my own. Python obviously has very particular workloads that need to be modeled appropriately, such as namespaces and **kwargs dicts. I don't know what benchmarks were used to write dictnotes.txt, but moving forward I would recommend implementing your changes on trunk (i.e., Python 2.x) and running the Unladen Swallow Benchmarks, which you can get from the link below: http://code.google.com/p/unladen-swallow/wiki/Benchmarks They are macro-benchmarks on real applications. You will probably also want to write some micro-benchmarks of your own so that you can pinpoint any bottlenecks in your code and determine where you are ahead of the current dict implementation and where you are behind. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] stdlib socket usage and keepalive
On Mon, Apr 12, 2010 at 5:34 PM, Jesus Cea j...@jcea.es wrote: The problem is: linux doesn't uses KEEPALIVE by default. If you believe the problem is with the Linux kernel, perhaps you should take up your case on a more appropriate mailing list? Python's socket module is a fairly low-level module, as it's just a thin wrapper around the corresponding operating system calls. Anyone using it has to be prepared to deal with a certain amount of exposed operating system details. If you want to use TCP KEEPALIVE on Linux, then just call: my_socket_object.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1) Most non-trivial applications use select() or poll() to avoid blocking calls and do their own timeout-checking at the application layer, so they don't need KEEPALIVE. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] WeakSet in Python 2.7
On Mon, Mar 29, 2010 at 12:16 PM, Michael Foord mich...@voidspace.org.ukwrote: Python 3 includes a WeakSet implementation. Any objections to me backporting this to 2.7? http://docs.python.org/py3k/library/weakref.html#weakref.WeakSet Backporting WeakSet would also make it possible to backport the fix for this reference leak: http://bugs.python.org/issue2521 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] WeakSet in Python 2.7
On Mon, Mar 29, 2010 at 2:21 PM, Michael Foord mich...@voidspace.org.ukwrote: It should be possible to fix it with a WeakKeyDictionary instead of WeakSet. True. I should have said Backporting WeakSet would make it *easier* to backport the fix ... :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing the new GIL
On Sat, Mar 13, 2010 at 3:46 PM, Antoine Pitrou solip...@pitrou.net wrote: - the second mechanism dynamically computes the interactiveness of a thread and allows interactive threads to steal the GIL quickly. In this approach, IO methods don't have to be modified at all. I like the second approach as well, assuming interactiveness can be computed cheaply. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Sat, Mar 6, 2010 at 6:43 AM, Nick Coghlan ncogh...@gmail.com wrote: You may want to consider providing global thread and process executors in the futures module itself. Code which just wants to say do this in the background without having to manage the lifecycle of its own executor instance is then free to do so. +1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Sat, Mar 6, 2010 at 5:38 PM, Michael Foord fuzzy...@voidspace.org.ukwrote: On 06/03/2010 23:37, Greg Ewing wrote: I've been thinking for a while that it would be a big help if there were one, standardised module in the stdlib for handling async events, and all the other gui toolkits etc. were made to use it. Wouldn't it have to be the Tcl event loop then? I image he means a standardized Abstract Base Class, which each GUI toolkit would subclass. That's more or less how Twisted's reactors work. That way non-GUI async code can just use the ABC and not worry about what event loop is running underneath (be it TCL, GTK, or just poll()). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Fri, Mar 5, 2010 at 12:03 AM, Brian Quinlan br...@sweetapp.com wrote: import futures +1 on the idea, -1 on the name. It's too similar to from __future__ import Also, the PEP should probably link to the discussions on stdlib-sig? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Fri, Mar 5, 2010 at 11:03 AM, Jesse Noller jnol...@gmail.com wrote: http://java.sun.com/javase/6/docs/api/java/util/concurrent/Future.html According to that link, Java has a module named Concurrent with an interface named Future. You're proposing a module named Futures with a class named Future. Why not name your module concurrent? That would eliminate the confusion with from __future__. I don't see a problem with keeping the class name. Plus, a concurrent module might be useful for things other than Futures, in the future. ;-) Just my 0.02 cents, -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching function pointers in type objects
On Tue, Mar 2, 2010 at 9:06 PM, Reid Kleckner r...@mit.edu wrote: I don't think this will help you solve your problem, but one thing we've done in unladen swallow is to hack PyType_Modified to invalidate our own descriptor caches. We may eventually want to extend that into a callback interface, but it probably will never be considered an API that outside code should depend on. Thanks Reid and Benjamin for the information. I think I see a way to dramatically speed up PyObject_RichCompareBool when comparing immutable, built-in, non-container objects (int, float, str, etc.). It would speed up list.sort when the key is one of those types, as well as most operations on the ubiquitous dictionary with str keys. Is that a worthwhile avenue to pursue, or is it likely to be redundant with Unladen Swallow's optimizations? If I can find time to pursue it, would it be best for me to implement it as a patch to Unladen Swallow, CPython trunk, or CPython py3k? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching function pointers in type objects
On Wed, Mar 3, 2010 at 3:29 PM, Benjamin Peterson benja...@python.orgwrote: 2010/3/3 Daniel Stutzbach dan...@stutzbachenterprises.com: I think I see a way to dramatically speed up PyObject_RichCompareBool when comparing immutable, built-in, non-container objects (int, float, str, etc.). It would speed up list.sort when the key is one of those types, as well as most operations on the ubiquitous dictionary with str keys. (correcting myself) I just noticed that CPython already optimizes dictionaries with str-only keys and skips PyObject_RichCompareBool, so no speed up there. I think it would be redundant with optimization I have in mind, so it could perhaps be taken out which would simplify the dict code a bit and save a pointer in the dict structure. Perhaps you could explain what exactly you want to do. :) That would help us make a judgment. It'd actually be a pretty small patch, so I think I should just explain it by actually writing it. ;-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching function pointers in type objects
On Wed, Mar 3, 2010 at 4:34 PM, Collin Winter collinwin...@google.comwrote: I would recommend patching py3k, with a backport to trunk. After thinking it over, I'm inclined to patch trunk, so I can run the Unladen Swallow macro-benchmarks, then forward-port to py3k. I'm correct in understanding that it will be a while before the Unladen Swallow benchmarks can support Python 3, right? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Caching function pointers in type objects
In CPython, is it safe to cache function pointers that are in type objects? For example, if I know that some_type-tp_richcompare is non-NULL, and I call it (which may execute arbitrary user code), can I assume that some_type-tp_richcompare is still non-NULL? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Platform extension for distutils on other interpreters than CPython
On Tue, Feb 23, 2010 at 12:50 PM, Maciej Fijalkowski fij...@gmail.comwrote: I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython. Also, it would be nice if the package could tell distutils that the compilation is option, in order to support modules where the C version simply replaces functions for speed. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial repository for Python benchmarks
On Sun, Feb 21, 2010 at 2:28 PM, Collin Winter collinwin...@google.comwrote: Would it be possible for us to get a Mercurial repository on python.org for the Unladen Swallow benchmarks? Maciej and I would like to move the benchmark suite out of Unladen Swallow and into python.org, where all implementations can share it and contribute to it. PyPy has been adding some benchmarks to their copy of the Unladen benchmarks, and we'd like to have as well, and Mercurial seems to be an ideal solution to this. If and when you have a benchmark repository set up, could you announce it via a reply to this thread? I'd like to check it out. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3147: PYC Repository Directories
On Sat, Jan 30, 2010 at 8:21 PM, Vitor Bosshard algor...@gmail.com wrote: Putting the files into a separate dir also makes it much harder to work with external tools; e.g. VCSes already ignore .pyc and .pyo files, but not unknown directories. Can't a VCS be configured to ignore a .pyr directory just as easily as it can be configured to ignore a .pyc file? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Wed, Jan 27, 2010 at 7:13 AM, Steve Howell showel...@yahoo.com wrote: My concept of Python lists is that they should have at least the same performance characteristics as an ordinary to-do list that you make with pencil, paper, and an eraser. When you complete the first task on your to-do list, you can just erase it; no need to recopy the whole list. I don't think your analogy works, unless you recopy your to-do lists whenever you complete a task in the middle of the list. ;-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Wed, Jan 27, 2010 at 9:55 AM, Steve Howell showel...@yahoo.com wrote: Fair enough, but that's still wasteful of memory, keeping around a bunch of None elements because you can't inexpensively delete them. Even if there are many references to it, there is only one None element. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] O(1) random access to deque? (Re: patch to make list.pop(0) work in O(1) time)
On Wed, Jan 27, 2010 at 3:50 PM, Nick Coghlan ncogh...@gmail.com wrote: I'm actually wondering if you could apply some of the implementation strategies discussed here to grant O(1) random access to arbitrary elements of a deque. I haven't looked at the deque code in a long time, but I believe the memory structure is already larger than that for a basic list. The memory structure is a linked list of blocks, where each block can hold several elements. As a linked list, the current structure would have O(n) random access. A few years back, I had proposed an alternative way of implementing deque, that would allow O(1) random access: http://mail.python.org/pipermail/python-dev/2007-November/075242.html It's essentially identical to Steve's proposal for list, except that I made the array circular, so element i is at position items[(start + i) % len]. That way it doesn't have to regularly allocate and deallocate memory for an approximately-fixed-length FIFO queue (which Steve's list will need to do). Raymond's objections are here: http://mail.python.org/pipermail/python-dev/2007-November/075244.html -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Wed, Jan 27, 2010 at 3:49 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: Also, am not sure if this affects psyco or the other implementations such as Jython which may implement lists in terms of existing Java containers. Or Unladen Swallow. I'm -1 on mucking with any of the fundamental data structures until the Unladen Swallow patch lands (assuming it lands). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Tue, Jan 26, 2010 at 1:17 AM, Steve Howell showel...@yahoo.com wrote: Ok, I fixed the obvious segfaults, and I am now able to pass all the tests on my debug build. A new diff is attached. Just to be clear, when you say all tests do you mean all of the list tests or the full Python test suite? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Tue, Jan 26, 2010 at 3:32 PM, Steve Howell showel...@yahoo.com wrote: Even most tiny-ish lists are wasting memory, though, according to this sequence: 4, 8, 16, 25, ... Several operations will allocate a list of exactly the right size, wasting no memory. In particular, a fixed-size list will always be exactly the right size. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Mon, Jan 25, 2010 at 1:22 PM, Steve Howell showel...@yahoo.com wrote: I haven't completely worked out the best strategy to eventually release the memory taken up by the pointers of the unreleased elements, but the worst case scenario is that the unused memory only gets wasted until the time that the list itself gets garbage collected. FWIW, for a long-running FIFO queue, it's critical to release some of the memory along the way, otherwise the amount of wasted memory is unbounded. Good luck :) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] recursive closures - reference leak
2009/12/8 Maciej Fijalkowski fij...@gmail.com Note that disabling gc does not mean that you will not have unpredictable pauses. Consider for example that if you loose a reference to a very long chain of objects, you can have arbitrarily many frees being called before anything else can happen. That strikes me as a *predictable* long pause. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Too many Python accounts
On Sun, Nov 15, 2009 at 2:44 PM, Martin v. Löwis mar...@v.loewis.dewrote: But then, users can easily create as many fake accounts as they want to. Why not do something more robust, then? For example, when a user enters an OpenID that hasn't been seen by PyPi before, make them enter a CAPTCHA. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
On Mon, Nov 9, 2009 at 2:42 AM, Martin v. Löwis mar...@v.loewis.dewrote: Interesting. Something goes wrong, it seems: if items get removed over and over again, I think the set should shrink (not sure whether it actually does). Then, I think you should end up with an amortized O(1) for selecting an element (assuming that the underlying hashes don't collide). I'm not sure if Python's implementation shrinks or not, but even if it did shrink, it would still be amortized O(n). Imagine a completely full hash table that currently contains n elements in n slots (I know this cannot happen in Python's implementation but it's useful for illustrative purposes). Assume it will shrink when it gets down to n/2 elements. Here is my pathological algorithm again, for reference purposes: while s: for i in s: break # Imagine a complex algorithm here that depends on i still being in s s.remove(i) We repeatedly search through the slots sequentially and remove the first element we find. The first removal requires checking 1 slot, the second removal requires checking 2 slots, the third removal requires checking 3 slots, and so forth. The hash table will shrink after the n/2-th removal, when we have checked 1 + 2 + 3 + ... + n/2 = O(n**2) slots for n/2 removals (or amortized O(n) per removal). It's too late for shrinking to save us; we've already performed too much work. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
On Mon, Nov 9, 2009 at 3:50 PM, Martin v. Löwis mar...@v.loewis.dewrote: I think for regular removal, the same logic should not apply: if a series of removals is performed, then further (non-pop) removals see increasing costs, as do regular lookups. So I think that a removal should trigger shrinking (with appropriate thresholds) unless it's a .pop. Minor technicality, but it's the next() operation of the iterator that has increasing cost as the set/dict becomes sparse. Removals are always O(1). The removal uses the hash to find the item quickly. The iterator has to scan through the table for non-empty entries. (the above assumes a good hash function with few collisions, of course) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
On Tue, Nov 3, 2009 at 3:55 AM, David Cournapeau courn...@gmail.com wrote: - are two branches are necessary or not ? If two branches are necessary, I think we simply do not have the resources at the moment. - how to maintain a compatible C API across 2.x and 3.x - is it practically possible to support and maintain numpy from 2.4 to 3.x ? For example, I don't think the python 2.6 py3k warnings are very useful when you need to maintain compatibility with 2.4 and 2.5. I've already ported some of my Python extension modules to Python 3. Here's how I would answer your questions based on my experience. Writing C code that compiles with Python 2.4 through 3.1 is pretty easy. Python's C API hasn't changed much and you can use #ifdef and #define for any bits that must be version-specific. It's pretty easy to make Python source that works under 2.6 and 3.x. It's basically impossible to make Python source that works under 2.4/2.5 and 3.x. You may be able to write code that works under 2.4/2.5 and works cleanly with 2to3 to produce 3.x code. I haven't tried that route, though in theory it should work. All you really need is syntax compatibility. For the rest, you can check sys.version_info. In a nutshell, I don't think you need two branches to support an extension module on Python 2 and Python 3. YMMV. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it
On Tue, Nov 3, 2009 at 4:46 PM, Steven D'Aprano st...@pearwood.info wrote: def pick_two_cards(hand): assert isinstance(hand, (set, frozenset)) assert len(hand) == 5 return (hand.pick(), hand.pick()) Even if pick() chose random, you still might end up picking the same card twice. Is that really what you intended? FWIW, I've been working on an extension module that supplies a sortedset type [1]. In most ways, it's similar to a set except it's indexable like a list. The items are kept in sorted order, so index 0 is always the lowest item, index 1 is the next-to-lowest, etc. Because they're indexable, it's easy and efficient to retrieve random elements using the standard library's random module. With the sortedset type, that function would become: def pick_two_cards(hand): assert isinstance(hand, (set, frozenset)) assert len(hand) == 5 return random.choice(hand), random.choice(hand) or if you want to avoid duplicates: return random.sample(hand, 2) Would something like that fit your needs? [1] It's already implemented, along with sortedlist, weaksortedlist, and weaksortedset types. I'm just putting them through the paces in my own products before releasing them. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reworking the GIL
On Mon, Nov 2, 2009 at 11:27 AM, Martin v. Löwis mar...@v.loewis.dewrote: Hmm. This creates a busy wait loop; if you add larger sleep values, then it loses accuracy. I thought that at first, too, but then I checked the documentation for Sleep(0): A value of zero causes the thread to relinquish the remainder of its time slice to any other thread of equal priority that is ready to run. (this is not to say that I think the solution with Sleep is worthwhile, though...) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] MSDN subscribers: Using Visual Studio?
On Thu, Oct 29, 2009 at 6:57 AM, Antoine Pitrou solip...@pitrou.net wrote: If it's just a matter of building and testing it you don't need any MSDN subscription, you just need Visual Studio Express which is free (as in free beer...). I don't know if it allows you to build an installer though. It does. If I recall correctly, in addition to Visual Studio Express, I also needed the Windows SDK (which is also free as in beer). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reworking the GIL
On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou solip...@pitrou.netwrote: Er, I prefer to keep things simple. If you have lots of I/O you should probably use an event loop rather than separate threads. On Windows, sometimes using a single-threaded event loop is sometimes impossible. WaitForMultipleObjects(), which is the Windows equivalent to select() or poll(), can handle a maximum of only 64 objects. Do we really need priority requests at all? They seem counter to your desire for simplicity and allowing the operating system's scheduler to do its work. That said, if a thread's time budget is merely paused during I/O rather than reset, then a thread making frequent (but short) I/O requests cannot starve the system. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add const to python API - issue 6952
On Tue, Oct 20, 2009 at 4:03 PM, Barry Scott ba...@barrys-emacs.org wrote:
Checking my patch I have two functions that need to
have output params changed to const to avoid casting.
PyOS_strtoul- ptr
PyLong_FromString - pend
This is a no-win situation. If the string is const in the caller, they
currently need to cast it. If you make the change, then if string is not
const in the caller then they will need to cast it. I've provided a short
example below and marked the lines that generate an incompatible pointer
type warning with gcc.
I suggest following POSIX's lead and omitted the const in these cases.
void test_const(const char **foo);
void test_noconst(char **foo);
int main(void) {
char *noconst_var;
const char *const_var;
test_const(noconst_var); // generates a warning
test_const(const_var);
test_noconst(noconst_var);
test_noconst(const_var); // generates a warning
return 0;
}
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SIGCHECK() in longobject.c
On Sun, Oct 18, 2009 at 3:01 PM, Antoine Pitrou solip...@pitrou.net wrote: Can we remove this check, or are there people doing million-digits calculations they want to interrupt using Control-C ? I sometimes do million-digits calculations that I want to interrupt using Control-C. (particularly when I didn't *intend* to do a million-digits calculation... ;) ) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SIGCHECK() in longobject.c
On Sun, Oct 18, 2009 at 5:46 PM, Antoine Pitrou solip...@pitrou.net wrote: Daniel Stutzbach daniel at stutzbachenterprises.com writes: I sometimes do million-digits calculations that I want to interrupt using Control-C.(particularly when I didn't *intend* to do a million-digits calculation... ;) )-- Sure, but it's no different than doing, e.g.: list(range(1)).sort() That's a good point, although I can't recall the last time I accidently created a painfully large list. I can recall the last time I started a painfully large integer computation. Being able to stop the interpretter with Control-C instead of kill -9 is a minor convenience, though. I could live without it. (Although I can't speak for everyone, of course) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Better module shutdown procedure
On Wed, Oct 14, 2009 at 3:16 PM, Neil Schemenauer n...@arctrix.com wrote: The current shutdown code in pythonrun.c zaps module globals by setting them to None (an attempt to break reference cycles). That causes problems since __del__ methods can try to use the globals after they have been set to None. The procedure implemented by http://bugs.python.org/issue812369 seems to be a better idea. References to modules are replaced by weak references and the GC is allowed to cleanup reference cycles. Based on the description, it still resorts to zapping module globals by setting them to None. It zaps them to weakrefs first, which means that globals are more likely to be valid during __del__, but it still cannot make any guarantees and referencing globals from __del__ is still a bad idea. Is that a correct synopsis? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Better module shutdown procedure
On Wed, Oct 14, 2009 at 4:42 PM, Neil Schemenauer n...@arctrix.com wrote: Yes, it does still resort to setting globals to None. However, the weakref step makes it much more likely that __del__ methods run before that happens. After this change, referencing global variables from __del__ methods is okay. The first and last sentences seem like a contradiction to me. If I cannot guarantee that globals will be valid when __del__ is executed, then I must not reference globals from __del__. I think I'm missing something here. Is there some way the programmer can determine that referencing a global variable in __del__ will be 100% safe? (not just likely) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Better module shutdown procedure
On Wed, Oct 14, 2009 at 6:05 PM, Neil Schemenauer n...@arctrix.com wrote: def __del__(): print sys.version the global variable reference to 'sys' is not a reference on the GC referencing counting sense. IOW, it does not result in a a Py_INCREF while the function is not being executed and therefore should be safe after the proposed change. Currently, it could result in 'None' being printed. Currently it throws an exception since sys is None. :-) Here is my understanding of the proposed procedure: 1. Replace modules in sys.modules with weakrefs 2. Run the garbage collector 3. Replace globals in any remaining modules with None 4. Run the garbage collector Is it possible for a __del__ method to be called in step 4 or not? I am still unclear on this point. :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Weak dict iterators are fragile
On Sun, Oct 11, 2009 at 10:50 AM, Antoine Pitrou solip...@pitrou.netwrote: In py3k, the weak dict methods keys(), values() and items() have been changed to return iterators (they returned lists in 2.x). However, it turns out that it makes these methods quite fragile, because a GC collection can occur whenever during iterating, destroy one of the weakref'ed objects, and trigger a resizing of the underlying dict, which in turn raises an exception (RuntimeError: dictionary changed size during iteration). Ouch! The iterator from __iter__ is also affected. 1. Add the safe methods listkeys(), listitems(), listvalues() which would behave as the keys(), etc. methods from 2.x 2. Make it so that keys(), items(), values() atomically build a list of items internally, which makes them more costly for large weak dicts, but robust. -1 on 1. +0 on 2. It'd be nice if we could postpone the resize if there are active iterators, but I don't think there's a clean way to track the iterators. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package install failures in 2.6.3
On Mon, Oct 5, 2009 at 9:32 AM, Barry Warsaw ba...@python.org wrote: If, as I hope, the answer to that is yes, then I strongly support releasing a fixed setuptools instead of reverting the change to Python. How do your propose to get the author of setuptools to release a new version? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
.On Wed, Sep 30, 2009 at 3:33 PM, R. David Murray rdmur...@bitdance.comwrote: 1) do not add another class, just pass around (IPvXAddress, IPVXNetwork) tuples when the address needs to be retained (or write your own tailored trivial class, like I did in my example). I've been puzzled by objections to forcing the user to decide what metadata to store. Every time I've ever needed to store an IP address or a network, I've always need to store a bunch of other affiliated data with it. E.g.: class GnutellaPeer: # IP address, port, timestamp, and dozen of other fields class NetworkInterface: # IP address, subnet mask, name, MAC address, etc. class Route: # interface, IP address, subnet mask, gateway address, route priority Is it such a burden for the programmer to store one extra field in the class they will inevitably write? For that matter, I do not see the advantage to an IPNetwork class that saves the programmer from having to separately store the IP address and subnet mask. Maybe I am biased by my background: I learned network programming in C, and I think of an IP address as little more than an integer type with a special string representation. People have voiced support for the IPNetwork class and for the occasional utility of an .ip field. I assume they have good use cases. It would be nice if the use cases were collected into the PEP, in a clear and articulate way. Preferably by someone other than ipaddr author, for whom the use cases are understandably a bit too obvious to explain at length with exasperation. It aught to be easy to justify the functionality of the library, if the use cases are clearly articulated. The current PEP begins by noting that many other IP address libraries are available, but doesn't describe what makes ipaddr unique or superior other than a claim to being lightweight. After downloading several of the other IP address libraries (http://bit.ly/483Yw4), ipaddr appears to be the second largest, after the huge netaddr package.http://packages.python.org/netaddr/ I worry that this discussion has focused too much on the details of ipaddr (and the false dichotomy of ipaddr versus nothing), without properly tackling the question of What use-cases for IP addresses are sufficiently universal* that they belong in the standard library? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan ncogh...@gmail.com wrote:
I should note that I've softened my position slightly from what I posted
yesterday. I could live with the following compromise:
x = IPv4Network('192.168.1.1/24')
y = IPv4Network('192.168.1.0/24')
x == y # Equality is the part I really want to see changed
True
x.ip
IPv4Address('192.168.1.1')
y.ip
IPv4Address('192.168.1.0')
With those semantics, IPv4Network objects with distinct IP addresses (but
the same network) could no longer be stored in a dictionary or set. IMO, it
is a little counter-intuitive for objects to compare equal yet have
different properties. I don't think this is a good compromise.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Mon, Sep 28, 2009 at 9:35 AM, R. David Murray rdmur...@bitdance.comwrote: A little googling produced this hit: http://tools.ietf.org/wg/mip4/draft-ietf-mip4-nemo-v4-base/draft-ietf-mip4-nemo-v4-base-11-from-10.diff.txt which while not a standard itself clearly believes that non-contiguous netmasks are valid, and is dated 2008. Earlier RFCs (950 and 1219) give them as valid but discouraged. That's a draft for RFC 5177. In the RFC, all references to non-contiguous subnets have been removed. http://tools.ietf.org/html/rfc5177 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Mon, Sep 28, 2009 at 4:29 PM, Martin v. Löwis mar...@v.loewis.dewrote: I guess a close case would be rational numbers: clearly, 3÷2 == 6÷4; would a Python library still remember (and repr) the original numerator and denominator? No need for a hypothetical, rational numbers were added in Python 2.6: Python 2.6.2 (r262:71600, Apr 15 2009, 07:20:39) [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin Type help, copyright, credits or license for more information. import fractions fractions.Fraction(6,4) Fraction(3, 2) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Sat, Sep 26, 2009 at 2:07 PM, DrKJam drk...@gmail.com wrote: The current version of the PEP and reference implementation do not mention or deal with IPv4 classful addressing (A, B, C, D and E). It would be good to know if any of this (admittedly older yet no less important) functionality is going to be supported. If the library is to concentrate solely on classless addressing (i.e. CIDR) please can this be stated in future revisions of the PEP. Classful addressing was deprecated more than 15 years ago! Quoting RFC 4632: With the full deployment of CIDR on the Internet, such scenarios are no longer operationally relevant. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com The plumage don't enter into it. It's stone dead. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
