Re: [Python-Dev] PySequence_Concat for dicts

2008-01-12 Thread Jared Flatow 
> On Jan 11, 2008 5:21 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
>> When does it come-up that you want a third summed dict
>> while keeping the two originals around unchanged?  Does
>> it matter that the addition is non-commutative?  Would
>> a + b + c produce an intermediate a/b combo and then
>> another new object for a/b/c so that the entries in
>> a get copied twice and memory usage has to hold a, b,
>> a/b, c, and a/b/c in memory all at the same time?

There is no way around it, this will be less efficient than the  
inplace operation. If there were a precedent for calling a method  
like update and returning itself instead of None, I would suggest  
that, but since this is the way that has already been established for  
lists, it seems a natural extension.

On Jan 11, 2008, at 9:22 PM, Guido van Rossum wrote:
> It does suggest that we have two choices for the proposed operation:
> d1+d2 or d1|d2. I think the latter may be more appropriate:
> len(seq1+seq2) == len(seq1) ++len(seq2), but no such invariant exists
> for set union.

This might be way out there but I suppose you could consider that a  
dict is actually two different things, depending on what you are  
doing, and that these operations might actually do different things.  
Interpreted as a sequence, it is a sequence of key mappings. As  
confirmed in another recent discussion, Python guarantees consistent  
(read: repeatable) ordering of iteration through a dict, so in this  
sense it really is a sequence. (On a side note, I frequently rely on  
the repeatability of ordering when interacting with the Python shell).

The notable sequence operation being + for concatenation, would  
perform an update of the keys (thus for the sequence op the mappings  
aren't guaranteed to be preserved, only the keys).

The other interpretation of dict is obviously as a set of (key,  
value) pairs. For sets, the four major operations could behave  
exactly as any other set of (key, value) tuples (i.e. if you  
transform it to a list and then apply the same ops you should get the  
same result).

jared

p.s. If I were to get approval to implement some version of this,  
which version of Python would be appropriate to work with?
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PySequence_Concat for dicts

2008-01-12 Thread Christian Heimes 
Jared Flatow wrote:
> p.s. If I were to get approval to implement some version of this,  
> which version of Python would be appropriate to work with?

The minimum version is 2.6. 2.5 is in maintenance mode. Changes made to
2.6 are merged into 3.0 so you don't have to create two patches. It'd
stick to 2.6 because it's easy to port a 2.6 patch to 3.0, if the new
feature won't make it into 2.6.

Christian

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] inst_persistent_id

2008-01-12 Thread Jim Fulton 

Recently, I reviewed the documentation source for pickle and came  
across the following comment:

BAW: Both pickle and cPickle support something called  
inst_persistent_id()
which appears to give unknown types a second shot at producing a  
persistent
id.  Since Jim Fulton can't remember why it was added or what it's  
for, I'm
leaving it undocumented.

I couldn't remember this and decided to dig into it and realized that  
this was a very useful experimental feature I added way back when  
cPickle was in its infancy.  This is a fairly useful optimization that  
I never got around to evaluating.  (Yeah, I know.) It is a hook, like  
the persistent_id hook that is called with objects to determine if  
they should be pickled by persistent reference.  Unlike persistent_id,  
it isn't called for built-in types (really types for which pickle has  
specific handlers), like strings, numbers, lists, tuples and so on. It  
is only called for "instances" (types for which pickle doesn't have  
specific handlers).  This vastly reduces the number of calls to the  
hook.  Some tests with ZODB indicated significant improvements in  
pickling speed when a hook is used.

If there are no objections, I'll update the Python 2 documentation to  
describe this and add a test.  The comment above suggests that this  
hook is in pickle and cPickle.  It is in cPickle, but was removed from  
pickle.  I propose to add it back to pickle -- mainly for consistency  
with cPickle.  I'll need to double check how this interacts with the  
type dispatching in pickle protocol 2.

Any objections?

Jim

--
Jim Fulton
Zope Corporation


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PySequence_Concat for dicts

2008-01-12 Thread Guido van Rossum 
On Jan 12, 2008 8:51 AM, Jared Flatow <[EMAIL PROTECTED]> wrote:
> > On Jan 11, 2008 5:21 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> >> When does it come-up that you want a third summed dict
> >> while keeping the two originals around unchanged?  Does
> >> it matter that the addition is non-commutative?  Would
> >> a + b + c produce an intermediate a/b combo and then
> >> another new object for a/b/c so that the entries in
> >> a get copied twice and memory usage has to hold a, b,
> >> a/b, c, and a/b/c in memory all at the same time?
>
> There is no way around it, this will be less efficient than the
> inplace operation. If there were a precedent for calling a method
> like update and returning itself instead of None, I would suggest
> that, but since this is the way that has already been established for
> lists, it seems a natural extension.
>
> On Jan 11, 2008, at 9:22 PM, Guido van Rossum wrote:
> > It does suggest that we have two choices for the proposed operation:
> > d1+d2 or d1|d2. I think the latter may be more appropriate:
> > len(seq1+seq2) == len(seq1) ++len(seq2), but no such invariant exists
> > for set union.
>
> This might be way out there but I suppose you could consider that a
> dict is actually two different things, depending on what you are
> doing, and that these operations might actually do different things.
> Interpreted as a sequence, it is a sequence of key mappings. As
> confirmed in another recent discussion, Python guarantees consistent
> (read: repeatable) ordering of iteration through a dict, so in this
> sense it really is a sequence. (On a side note, I frequently rely on
> the repeatability of ordering when interacting with the Python shell).
>
> The notable sequence operation being + for concatenation, would
> perform an update of the keys (thus for the sequence op the mappings
> aren't guaranteed to be preserved, only the keys).
>
> The other interpretation of dict is obviously as a set of (key,
> value) pairs. For sets, the four major operations could behave
> exactly as any other set of (key, value) tuples (i.e. if you
> transform it to a list and then apply the same ops you should get the
> same result).

No, a dict is not a set of (key, value) pairs. The keys form a set,
you cannot have duplicate keys.

> jared
>
> p.s. If I were to get approval to implement some version of this,
> which version of Python would be appropriate to work with?
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How to change path at compile time?

2008-01-12 Thread Martin v. Löwis 
> Is there any reasonable way to change the default sys.path at compile
> time?  (ie. add a directory).

[this is off-topic for python-dev]

Edit PYTHONPATH in Modules/Setup, e.g. by setting SITEPATH.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP: per user site-packages directory

2008-01-12 Thread Christian Heimes 
Christian Heimes wrote:
> MA Lemburg has suggested a per user site-packages directory in the
> "pkgutil, pkg_resource and Python 3.0 name space packages" thread. I've
> written a short PEP about it for Python 2.6 and 3.0.

Addition:
An user has requested a new option to suppress the user site packages
directory:

-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE

Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PySequence_Concat for dicts

2008-01-12 Thread Raymond Hettinger 
[Raymond]
>>> When does it come-up that you want a third summed dict
>>> while keeping the two originals around unchanged?  Does
>>> it matter that the addition is non-commutative?  Would
>>> a + b + c produce an intermediate a/b combo and then
>>> another new object for a/b/c so that the entries in
>>> a get copied twice and memory usage has to hold a, b,
>>> a/b, c, and a/b/c in memory all at the same time?

[Jared]
> There is no way around it, this will be less efficient than the  
> inplace operation. If there were a precedent for calling a method  
> like update and returning itself instead of None, I would suggest  
> that, but since this is the way that has already been established for  
> lists, it seems a natural extension.

Not natural, just inefficient and cute.  Also, there was no answer
to the question about use cases.  AFAICT, this feature has never 
been requested.  The closest was a feature request for a
variant of update() that avoided overwrites when a duplicate
key was encountered -- Guido rejected that one a long time ago.

Your previous note suggests that there are alternative interpretations
of what the syntax could mean and that's not good a good thing.
That sort of ambiguity damages the language. It is not even
clear where the appropriate operators would be +-* or the
set operators &|^-.  How about we keep sets for set operations 
and dict for mapping operations and not foolishly conflate the two
just because we can.  The mapping API is central to the language.
Altering it should be approached with a great deal of care.

Also, the argument that we used + for lists so now we have
to do it for dicts is a weak one -- they are completely different animals.
Operators are not the solution to all problems.  In this case, we
don't even have a problem to be solved; there is just an urge
to hypergeneralize what was done for other datatypes where
it was appropriate.  The .update() method we have now is explicit, 
clear about its intent, and efficient.

IMO, the only thing this proposal has going for it is that it is cute.


Raymond
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] inst_persistent_id

2008-01-12 Thread Alexandre Vassalotti 
On Jan 12, 2008 12:25 PM, Jim Fulton <[EMAIL PROTECTED]> wrote:
> If there are no objections, I'll update the Python 2 documentation to
> describe this and add a test.  The comment above suggests that this
> hook is in pickle and cPickle.  It is in cPickle, but was removed from
> pickle.  I propose to add it back to pickle -- mainly for consistency
> with cPickle.  I'll need to double check how this interacts with the
> type dispatching in pickle protocol 2.
>
> Any objections?
>

Well, in Python 3K, inst_persistent_id() won't be usable, since
PyInstance_Type was removed. Adding (actually supporting) this feature
in Python 2.x will make it slightly harder to port code. So, I think
it would probably best to leave it as it is right now -- i.e.,
undocumented and unsupported.

By the way, you might be interested to look at my work on pickle [1]
for Python 3K. As part of last year Summer of Code, I removed the
differences between the Python and C implementation of pickle, and
thus allowing the C version to be transparently used, instead of the
Python one, when it is available. Currently, I am polishing the code
for inclusion into the main branch. I also started to work on the next
version of the pickle protocol, that will make it more suitable for
Python 3K. If you are interested to help out, just send me an email
and I will explain you what needs to be done.


-- Alexandre

.. [1] http://peadrop.com/alex-py3k/?file/91639e5487dc/Modules/_picklemodule.c
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP: per user site-packages directory

2008-01-12 Thread Gregory P. Smith 
On 1/12/08, Christian Heimes <[EMAIL PROTECTED]> wrote:
>
> Christian Heimes wrote:
> > MA Lemburg has suggested a per user site-packages directory in the
> > "pkgutil, pkg_resource and Python 3.0 name space packages" thread. I've
> > written a short PEP about it for Python 2.6 and 3.0.
>
> Addition:
> An user has requested a new option to suppress the user site packages
> directory:
>
> -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE


+0.5  Thanks for writing this up as a PEP.

My main suggestion was going to be the ability to turn it off as you already
mentioned.  However, please consider leaving it off by default to avoid
problems for installed python scripts importing user supplied code.  For
shared hosting environments where this becomes really useful users can
easily add the -s (or whatever flag is chosen) to their programs
themselves.  I don't know what that'd mean on windows where #! lines don't
exist.  Yet another file extension to imply the flag (yuck)?  A .cmd wrapper
script to run python with the flag (ugh)?

For security reasons we also need it disabled when the getuid() != geteuid()
to avoid user supplied code being executed as another user.  Defaulting to
disabled would mean that security could be left up to the end user to mess
up.  (many systems do not allow setuid #! scripts but this issue would still
apply to things run under sudo)

-gps
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Rounding Decimals

2008-01-12 Thread Jeffrey Yasskin 
During the discussion about the new Rational implementation
(http://bugs.python.org/issue1682), Guido and Raymond decided that
Decimal should not implement the new Real ABC from PEP 3141. So I've
closed http://bugs.python.org/issue1623 and won't be pursuing any of
the extra rounding methods mentioned on this thread.

-- 
Namasté,
Jeffrey Yasskin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python] Re: PEP: per user site-packages directory

2008-01-12 Thread Michael Foord 
Gregory P. Smith wrote:
>
> On 1/12/08, *Christian Heimes* <[EMAIL PROTECTED] 
> > wrote:
>
> Christian Heimes wrote:
> > MA Lemburg has suggested a per user site-packages directory in the
> > "pkgutil, pkg_resource and Python 3.0 name space packages"
> thread. I've
> > written a short PEP about it for Python 2.6 and 3.0.
>
> Addition:
> An user has requested a new option to suppress the user site packages
> directory:
>
> -s : don't add user site directory to sys.path; also
> PYTHONNOUSERSITE
>
>
> +0.5  Thanks for writing this up as a PEP.
>
> My main suggestion was going to be the ability to turn it off as you 
> already mentioned.  However, please consider leaving it off by default 
> to avoid problems for installed python scripts importing user supplied 
> code.  For shared hosting environments where this becomes really 
> useful users can easily add the -s (or whatever flag is chosen) to 
> their programs themselves.  I don't know what that'd mean on windows 
> where #! lines don't exist.  Yet another file extension to imply the 
> flag (yuck)?  A .cmd wrapper script to run python with the flag (ugh)?

+1 from me on implementing it and having it on by default for Windows.

Why do you think the user namespace overriding the system namespace be 
more of a problem for Windows than other platforms?

This would be a really useful feature for me and it would be a shame for 
it not to be on by default on Windows (and another set of complexities 
for setuptools I suspect).

Michael Foord

>
> For security reasons we also need it disabled when the getuid() != 
> geteuid() to avoid user supplied code being executed as another user.  
> Defaulting to disabled would mean that security could be left up to 
> the end user to mess up.  (many systems do not allow setuid #! scripts 
> but this issue would still apply to things run under sudo)
>
> -gps
>
> 
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Rounding Decimals

2008-01-12 Thread Guido van Rossum 
On Jan 12, 2008 5:09 PM, Jeffrey Yasskin <[EMAIL PROTECTED]> wrote:
> During the discussion about the new Rational implementation
> (http://bugs.python.org/issue1682), Guido and Raymond decided that
> Decimal should not implement the new Real ABC from PEP 3141. So I've
> closed http://bugs.python.org/issue1623 and won't be pursuing any of
> the extra rounding methods mentioned on this thread.

Well, I didn't really decide anything. I suggested that if the
developers of Decimal preferred it, it might be better if Decimal did
not implement the Real ABC, and Raymond said he indeed preferred it.

Since this reduces the usefulness of numeric.Real, I'm somewhat
disappointed in this outcome (as I imagine you are). However, Decimal
has very little of my own sweat in it, and a lot of Raymond, Facundo
(and others, including Mark), and the ABC thing is still somewhat
experimental, so it wouldn't make sense for me to impose my ideas onto
Decimal unilaterally, and I'm sure Raymond etc. have their reasons.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] OSCON 2008 Call for Proposals

2008-01-12 Thread Aahz 
The O'Reilly Open Source Convention (OSCON) is accepting proposals for
tutorials and presentations.  The submission period ends Feb 4.

OSCON 2008 will be in Portland, Oregon July 21-25.  For more information
and to submit a proposal, see

http://conferences.oreilly.com/oscon/
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com