Re: [Python-Dev] accept string in a2b and base64?

[M.-A. Lemburg]

Nick Coghlan wrote:
> The reason Python 2's implicit str<->unicode conversions are so
> problematic isn't just because they're implicit: it's because they
> effectively assume *latin-1* as the encoding on the 8-bit str side.

The implicit conversion in Python2 only works with ASCII content,
pretty much like what you describe here.

Note that e.g. UTF-16 is not an ASCII super set, but the ASCII
assumption still works:

>>> u'abc'.encode('utf-16-le').decode('ascii')
u'a\x00b\x00c\x00'

Apart from that nit (which can be resolved in most cases by
disallowing 0 bytes), I still believe that the Python2 implicit
conversion between Unicode and 8-bit strings is a very useful
feature in practice.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 21 2012)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2012-02-13: Released eGenix pyOpenSSL 0.13http://egenix.com/go26
2012-02-09: Released mxODBC.Zope.DA 2.0.2 http://egenix.com/go25
2012-02-06: Released eGenix mx Base 3.2.3 http://egenix.com/go24

::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP czar for PEP 3144?

[Robert Kern]

On 2/21/12 4:52 AM, Guido van Rossum wrote:

On Mon, Feb 20, 2012 at 4:53 PM, Stephen J. Turnbull  wrote:

Steven D'Aprano writes:

  >  Also, "Czar" is commonly used in US politics as an informal term for the 
top
  >  official responsible for an area.

I think here the most important connotation is that in US parlance a
"czar" does not report to a committee, and with the exception of a
case where Sybil is appointed czar, cannot bikeshed.  Decisions get
made (what a concept!)


I'm curious how old that usage is. I first encountered it around '88
when I interned for a summer at DEC SRC (long since subsumed into HP
Labs); the person in charge of deciding a particular aspect of their
software or organization was called a czar, e.g. the documentation
czar.


From the Wikipedia article Steven cited:

"""
The earliest known use of the term for a U.S. government official was in the 
administration of Franklin Roosevelt (1933–1945), during which eleven unique 
positions (or twelve if one were to count "Economic Czar" and "Economic Czar of 
World War II" as separate) were so described. The term was revived, mostly by 
the press, to describe officials in the Nixon and Ford administrations and 
continues today.

"""

http://en.wikipedia.org/wiki/List_of_U.S._executive_branch_%27czars%27

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Eli Bendersky]

On Tue, Feb 21, 2012 at 03:59, Nick Coghlan  wrote:

> On Tue, Feb 21, 2012 at 11:39 AM, Eli Bendersky  wrote:
> > So the two choices here are either change the documentation or the C
> > implementation to actually make Element a class. The first is of course
> > simpler. However, someone somewhere may have written code that knowingly
> > forces the Python implementation to be used and subclasses Element. Such
> > code will break in 3.3, so it probably makes sense to invest in making
> > Element a class in the C implementation as well.
>
> Yeah, that's my take as well (especially since, in 3.2 and earlier,
> "forcing" use of the pure Python version was just a matter of
> importing ElementTree instead of cElementTree).
>
>
I can't fathom why someone would do it though, since bar tiny differences
(like this one) cET is just a faster ET and it's available practically
everywhere with CPython. I mean, is it really important to be able to
subclass ET.Element? What goal does it serve?

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] accept string in a2b and base64?

[Antoine Pitrou]

On Tue, 21 Feb 2012 12:51:08 +1000
Nick Coghlan  wrote:
> 
> My one concern with the base64 patch is that it doesn't test that
> mixing types triggers TypeError. While this shouldn't require any
> extra code (the error should arise naturally from the method
> implementation), it should still be tested explicitly to ensure type
> mismatches fail as expected.

I don't think mixing types is a concern. The extra parameters to the
base64 functions aren't mixed into the original string, they are used to
modify the decoding algorithm.

So it's like typing `open(b"LICENSE", "r")`: the fast that `b"LICENSE"`
is bytes while `"r"` is str isn't really a problem.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] accept string in a2b and base64?

[Nick Coghlan]

On Tue, Feb 21, 2012 at 10:28 PM, Antoine Pitrou  wrote:
> So it's like typing `open(b"LICENSE", "r")`: the fast that `b"LICENSE"`
> is bytes while `"r"` is str isn't really a problem.

Ah, right - I misunderstood how the different arguments were being used.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of PEP 397 - Python launcher for Windows

[Vinay Sajip]

Mark Hammond  gmail.com> writes:

> think there is something that could be added to those docs - the use of 
> PATHEXT and the fact that once the shebang line is in place, a 
> command-prompt could do just "hello.py" rather than needing "py hello.py".

Or even just "hello" should work.

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] hash randomization in 3.3

[Antoine Pitrou]

Hello,

Shouldn't it be enabled by default in 3.3?
It's currently disabled.

$ ./python -c "print(hash('aa'))"
12416074593111936
[44297 refs]
$ ./python -c "print(hash('aa'))"
12416074593111936
[44297 refs]

Thanks

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] What do you want me to discuss at PyCon's keynote?

[Guido van Rossum]

I'm starting to think about my annual PyCon keynote. I don't want it
to be just a feel-good motivational speech (I'm no good at those), nor
a dry "state of the Python union" talk (I'm bored with those), but I'd
like to hear what Python users care about. I've created a Google+ post
for feedback: 
https://plus.google.com/u/0/115212051037621986145/posts/P8XZ5Zxvpxk

Post your questions as comments there, or vote on others' questions
using your +1 button.

If you don't have a G+ account, you can mail questions to
gu...@python.org-- I will then copy them here anonymously for voting,
unless you ask me not to, or ask me to add your name.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A panel with Guido/python-dev on scientific uses and Python 3 at Google HQ, March 2nd

[Fernando Perez]

On Tue, 21 Feb 2012 07:44:41 +, Fernando Perez wrote:

> I wanted to point out to you folks, and invite any of you who could make
> it in person, to a panel discussion we'll be having on Friday March 2nd,
> at 3pm, during the PyData workshop that will take place at Google's
> headquarters in Mountain View:
> 
> http://pydataworkshop.eventbrite.com

as luck would have it, it seems that *today* eventbrite revamped their url 
handling and the url I gave yesterday no longer works; it's now:

http://pydataworkshop-esearch.eventbrite.com/?srnk=1

Sorry for the hassle, folks.

Ah, Murphy's law, web edition...

Cheers,

f

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Benjamin Peterson]

2012/2/21 Antoine Pitrou :
>
> Hello,
>
> Shouldn't it be enabled by default in 3.3?

Should you be able to disable it?


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Antoine Pitrou]

On Tue, 21 Feb 2012 14:58:41 -0500
Benjamin Peterson  wrote:
> 2012/2/21 Antoine Pitrou :
> >
> > Hello,
> >
> > Shouldn't it be enabled by default in 3.3?
> 
> Should you be able to disable it?

PYTHONHASHSEED=0 should disable it.  Do we also need a command-line
option?

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Benjamin Peterson]

2012/2/21 Antoine Pitrou :
> On Tue, 21 Feb 2012 14:58:41 -0500
> Benjamin Peterson  wrote:
>> 2012/2/21 Antoine Pitrou :
>> >
>> > Hello,
>> >
>> > Shouldn't it be enabled by default in 3.3?
>>
>> Should you be able to disable it?
>
> PYTHONHASHSEED=0 should disable it.  Do we also need a command-line
> option?

I don't think so. I was just wondering if we should force people to use it.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Barry Warsaw]

On Feb 21, 2012, at 02:58 PM, Benjamin Peterson wrote:

>2012/2/21 Antoine Pitrou :
>>
>> Hello,
>>
>> Shouldn't it be enabled by default in 3.3?

Yes.

>Should you be able to disable it?

No, but you should be able to provide a seed.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Glenn Linderman]

On 2/21/2012 11:58 AM, Benjamin Peterson wrote:

2012/2/21 Antoine Pitrou:

Hello,

Shouldn't it be enabled by default in 3.3?

Should you be able to disable it?


Yes, absolutely.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Brett Cannon]

On Tue, Feb 21, 2012 at 15:05, Barry Warsaw  wrote:

> On Feb 21, 2012, at 02:58 PM, Benjamin Peterson wrote:
>
> >2012/2/21 Antoine Pitrou :
> >>
> >> Hello,
> >>
> >> Shouldn't it be enabled by default in 3.3?
>
> Yes.
>
> >Should you be able to disable it?
>
> No, but you should be able to provide a seed.


I think that's inviting trouble if you can provide the seed. It leads to a
false sense of security in that providing some seed secures them instead of
just making it a tad harder for the attack. And it won't help with keeping
compatibility with Python 2.7 installations that don't have randomization
turned on by default. If we are going to allow people to turn this off then
it should be basically the inverse of the default under Python 2.7 and no
more.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] CPU vs Wall time Profiling

[Sümer Cip]

Hi all,

Is there a reason behind the fact that the Python profilers work with Wall
time by default? There are OS-dependent ways to get the CPU time of a
thread, and giving that choice to the user _somehow_ ( to use wall vs cpu
time) might be a good feature?

--
Sumer Cip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Benjamin Peterson]

2012/2/21 Antoine Pitrou :
>
> Hello,
>
> Shouldn't it be enabled by default in 3.3?

I've now enabled it by default in 3.3.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Xavier Morel]

On 2012-02-21, at 21:24 , Brett Cannon wrote:
> On Tue, Feb 21, 2012 at 15:05, Barry Warsaw  wrote:
> 
>> On Feb 21, 2012, at 02:58 PM, Benjamin Peterson wrote:
>> 
>>> 2012/2/21 Antoine Pitrou :
 
 Hello,
 
 Shouldn't it be enabled by default in 3.3?
>> 
>> Yes.
>> 
>>> Should you be able to disable it?
>> 
>> No, but you should be able to provide a seed.
> 
> I think that's inviting trouble if you can provide the seed. It leads to a
> false sense of security in that providing some seed secures them instead of
> just making it a tad harder for the attack.

I might have misunderstood something, but wouldn't providing a seed always 
make it *easier* for the attacker, compared to a randomized hash?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CPU vs Wall time Profiling

[Maciej Fijalkowski]

On Tue, Feb 21, 2012 at 2:00 PM, Sümer Cip  wrote:
> Hi all,
>
> Is there a reason behind the fact that the Python profilers work with Wall
> time by default? There are OS-dependent ways to get the CPU time of a
> thread, and giving that choice to the user _somehow_ ( to use wall vs cpu
> time) might be a good feature?

What would you use for linux for example?

Cheers,
fijal
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Brett Cannon]

On Tue, Feb 21, 2012 at 15:58, Xavier Morel  wrote:

> On 2012-02-21, at 21:24 , Brett Cannon wrote:
> > On Tue, Feb 21, 2012 at 15:05, Barry Warsaw  wrote:
> >
> >> On Feb 21, 2012, at 02:58 PM, Benjamin Peterson wrote:
> >>
> >>> 2012/2/21 Antoine Pitrou :
> 
>  Hello,
> 
>  Shouldn't it be enabled by default in 3.3?
> >>
> >> Yes.
> >>
> >>> Should you be able to disable it?
> >>
> >> No, but you should be able to provide a seed.
> >
> > I think that's inviting trouble if you can provide the seed. It leads to
> a
> > false sense of security in that providing some seed secures them instead
> of
> > just making it a tad harder for the attack.
>
> I might have misunderstood something, but wouldn't providing a seed always
> make it *easier* for the attacker, compared to a randomized hash?
>

Yes, that was what I was trying to convey.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Barry Warsaw]

On Feb 21, 2012, at 09:58 PM, Xavier Morel wrote:

>On 2012-02-21, at 21:24 , Brett Cannon wrote:
>> On Tue, Feb 21, 2012 at 15:05, Barry Warsaw  wrote:
>> 
>>> On Feb 21, 2012, at 02:58 PM, Benjamin Peterson wrote:
>>> 
 2012/2/21 Antoine Pitrou :
> 
> Hello,
> 
> Shouldn't it be enabled by default in 3.3?
>>> 
>>> Yes.
>>> 
 Should you be able to disable it?
>>> 
>>> No, but you should be able to provide a seed.
>> 
>> I think that's inviting trouble if you can provide the seed. It leads to a
>> false sense of security in that providing some seed secures them instead of
>> just making it a tad harder for the attack.
>
>I might have misunderstood something, but wouldn't providing a seed always 
>make it *easier* for the attacker, compared to a randomized hash?

I don't think so.  You'd have to somehow coerce the sys.hash_seed out of the
process.  Not impossible perhaps, but unlikely unless the application isn't
written well and leaks that information (which is not Python's fault).

Plus, with randomization enabled, that won't help you much past the current
invocation of Python.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Martin v. Löwis]

Am 21.02.2012 20:59, schrieb Antoine Pitrou:
> On Tue, 21 Feb 2012 14:58:41 -0500
> Benjamin Peterson  wrote:
>> 2012/2/21 Antoine Pitrou :
>>>
>>> Hello,
>>>
>>> Shouldn't it be enabled by default in 3.3?
>>
>> Should you be able to disable it?
> 
> PYTHONHASHSEED=0 should disable it.  Do we also need a command-line
> option?

On the contrary. PYTHONHASHSEED should go in 3.3, as should any
facility to disable or otherwise fix the seed.

Regards,
martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Martin v. Löwis]

>> Should you be able to disable it?
> 
> No, but you should be able to provide a seed.

Why exactly is that?

We should take an attitude that Python hash values
are completely arbitrary and can change at any point
without notice. The only strict requirement should be
that hashing must be consistent with equality; everything
else should be an implementation detail.

With that attitude, supporting explicit seeds is counter-productive.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Martin v. Löwis]

Am 21.02.2012 11:41, schrieb Eli Bendersky:
> 
> 
> On Tue, Feb 21, 2012 at 03:59, Nick Coghlan  > wrote:
> 
> On Tue, Feb 21, 2012 at 11:39 AM, Eli Bendersky  > wrote:
> > So the two choices here are either change the documentation or the C
> > implementation to actually make Element a class. The first is of
> course
> > simpler. However, someone somewhere may have written code that
> knowingly
> > forces the Python implementation to be used and subclasses
> Element. Such
> > code will break in 3.3, so it probably makes sense to invest in making
> > Element a class in the C implementation as well.
> 
> Yeah, that's my take as well (especially since, in 3.2 and earlier,
> "forcing" use of the pure Python version was just a matter of
> importing ElementTree instead of cElementTree).
> 
> 
> I can't fathom why someone would do it though, since bar tiny
> differences (like this one) cET is just a faster ET and it's available
> practically everywhere with CPython. I mean, is it really important to
> be able to subclass ET.Element? What goal does it serve?

Statements like this make me *extremely* worried. Please try to adopt
a position of much higher caution, accepting that a change is
"incompatible" if there is a remote possibility that someone might
actually rely on the original behavior. Otherwise, I predict that you
will get flooded with complaints that you broke ET for no good reason.

In the specific case, I tried to write a script that determines the
memory usage of ET. As Element is lacking gc.get_referents support,
I tried isinstance(o, Element), which failed badly.

Feel free to dismiss this application as irrelevant, but I do wish
that somebody was in charge of ET who was taking backwards
compatibility as serious as Fredrik Lundh.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CPU vs Wall time Profiling

[Victor Stinner]

Python 3.3 has two new functions in the time module: monotonic() and
wallclock().

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Antoine Pitrou]

On Tue, 21 Feb 2012 12:41:17 +0200
Eli Bendersky  wrote:
> On Tue, Feb 21, 2012 at 03:59, Nick Coghlan  wrote:
> 
> > On Tue, Feb 21, 2012 at 11:39 AM, Eli Bendersky  wrote:
> > > So the two choices here are either change the documentation or the C
> > > implementation to actually make Element a class. The first is of course
> > > simpler. However, someone somewhere may have written code that knowingly
> > > forces the Python implementation to be used and subclasses Element. Such
> > > code will break in 3.3, so it probably makes sense to invest in making
> > > Element a class in the C implementation as well.
> >
> > Yeah, that's my take as well (especially since, in 3.2 and earlier,
> > "forcing" use of the pure Python version was just a matter of
> > importing ElementTree instead of cElementTree).
> >
> >
> I can't fathom why someone would do it though, since bar tiny differences
> (like this one) cET is just a faster ET and it's available practically
> everywhere with CPython. I mean, is it really important to be able to
> subclass ET.Element? What goal does it serve?

It probably wouldn't be very difficult to make element_new() the tp_new
of Element_Type, and expose that type as "Element".
That would settle the issue nicely and avoid compatibility concerns :)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Antoine Pitrou]

On Tue, 21 Feb 2012 22:51:48 +0100
"Martin v. Löwis"  wrote:
> Am 21.02.2012 20:59, schrieb Antoine Pitrou:
> > On Tue, 21 Feb 2012 14:58:41 -0500
> > Benjamin Peterson  wrote:
> >> 2012/2/21 Antoine Pitrou :
> >>>
> >>> Hello,
> >>>
> >>> Shouldn't it be enabled by default in 3.3?
> >>
> >> Should you be able to disable it?
> > 
> > PYTHONHASHSEED=0 should disable it.  Do we also need a command-line
> > option?
> 
> On the contrary. PYTHONHASHSEED should go in 3.3, as should any
> facility to disable or otherwise fix the seed.

Being able to reproduce exact output is useful to chase sporadic test
failures (as with the --randseed option to regrtest).

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Nick Coghlan]

On Wed, Feb 22, 2012 at 7:47 AM, "Martin v. Löwis"  wrote:
> Am 21.02.2012 11:41, schrieb Eli Bendersky:
>> I can't fathom why someone would do it though, since bar tiny
>> differences (like this one) cET is just a faster ET and it's available
>> practically everywhere with CPython. I mean, is it really important to
>> be able to subclass ET.Element? What goal does it serve?
>
> Statements like this make me *extremely* worried. Please try to adopt
> a position of much higher caution, accepting that a change is
> "incompatible" if there is a remote possibility that someone might
> actually rely on the original behavior. Otherwise, I predict that you
> will get flooded with complaints that you broke ET for no good reason.

Indeed. It's a *major* PITA at times (and has definitely led to some
ugly workarounds), but we have to take documented API compatibility
very seriously.

We're often even reluctant to change long-standing *de facto*
behaviour, let alone things that are written up in the docs as being
explicitly supported. In Python 3, merely saying "this class" or "this
type" is as good as saying "this instance of the type metaclass" as
far as API guarantees go. That's the reason for the awkward phrasing
in the functools docs - specifically to *avoid* saying that
functools.partial is a class, as we want to allow closure-based
implementations as well.

The key thing to remember is that the web-style "eh, just change it,
people can fix their code to cope" mentality is a tiny *minority* in
the global software development community. There's a huge amount of
Python code out there, and a lot of it is hidden behind corporate
firewalls. Our attention to backward compatibility concerns is one of
the reasons why Python's reach extends into so many different areas.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Andrew McNabb]

On Tue, Feb 21, 2012 at 10:47:43PM +0100, "Martin v. Löwis" wrote:
> > I can't fathom why someone would do it though, since bar tiny
> > differences (like this one) cET is just a faster ET and it's available
> > practically everywhere with CPython. I mean, is it really important to
> > be able to subclass ET.Element? What goal does it serve?
> 
> Statements like this make me *extremely* worried. Please try to adopt
> a position of much higher caution, accepting that a change is
> "incompatible" if there is a remote possibility that someone might
> actually rely on the original behavior. Otherwise, I predict that you
> will get flooded with complaints that you broke ET for no good reason.

I'm happy to stand up as an example of someone who uses a custom Element
class.  My specific use case is loading the project Gutenberg database,
which is a 210MB XML file.  I created a custom Element class which I use
for the top-level element (a custom element_factory passed to
TreeBuilder distinguishes between the top-level element and all others).
The custom Element class doesn't add children, so it keeps ElementTree
from storing all of the elements its seen so far.  On a system with 1 GB
of RAM, there was no other way to get the file to load.

So, I would be one of those people who would flood in the complaints. :)

--
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Nick Coghlan]

On Wed, Feb 22, 2012 at 8:07 AM, Antoine Pitrou  wrote:
> On Tue, 21 Feb 2012 22:51:48 +0100
> "Martin v. Löwis"  wrote:
>> On the contrary. PYTHONHASHSEED should go in 3.3, as should any
>> facility to disable or otherwise fix the seed.
>
> Being able to reproduce exact output is useful to chase sporadic test
> failures (as with the --randseed option to regrtest).

I'm with Antoine here - being able to force a particular seed still
matters for testing purposes. However, the documentation of the option
may need to be updated for 3.3 to emphasise that it should only be
used to reproduce sporadic failures. Using it to work around
applications that can't cope with randomised hashes would be rather
ill-advised.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Nick Coghlan]

On Wed, Feb 22, 2012 at 8:16 AM, Andrew McNabb  wrote:
> So, I would be one of those people who would flood in the complaints. :)

As another "you don't know what you're going to break" war story:

In Python 2.5, using "-m" with a package appeared to work, but
actually slightly corrupted the import state (mostly in a benign way,
but if it ever bit you it would lead to some very confusing
behaviour).

Since I'd never intended to allow that to happen (as I knew about the
state corruption problem), for 2.6 I added back the "this doesn't work
properly" guard that had been present in the earlier versions of 2.5,
but had been lost when some duplicate code in pkgutil and runpy was
merged into a single version.

Doing that broke things: http://bugs.python.org/issue4195

The basic rule is, if it's documented to work a certain way and the
current implementation does work that way, then, someone, somewhere is
relying on it working as documented. If it *doesn't* actually work
that way (or the behaviour isn't explicitly documented at all), then
we have some leeway to decide whether to bring the docs in line with
the actual behaviour or vice-versa. For the Element case though,
there's no such discrepancy - the docs and implementation have been
consistent for years, so we need to maintain the current behaviour if
the C acceleration is going to be used implicitly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (2.6): ensure no one tries to hash things before the random seed is found

[Nick Coghlan]

On Wed, Feb 22, 2012 at 2:24 AM, benjamin.peterson
 wrote:
> http://hg.python.org/cpython/rev/357e268e7c5f
> changeset:   75133:357e268e7c5f
> branch:      2.6
> parent:      75124:04738f35e0ec
> user:        Benjamin Peterson 
> date:        Tue Feb 21 11:08:50 2012 -0500
> summary:
>  ensure no one tries to hash things before the random seed is found

Won't this trigger in the -Wd case that led to the PyStr_Fini workaround?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (2.6): ensure no one tries to hash things before the random seed is found

[Nick Coghlan]

On Wed, Feb 22, 2012 at 8:49 AM, Nick Coghlan  wrote:
> Won't this trigger in the -Wd case that led to the PyStr_Fini workaround?

Never mind, just saw the later series of checkins that fixed it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: enable hash randomization by default

[Nick Coghlan]

On Wed, Feb 22, 2012 at 7:08 AM, benjamin.peterson
 wrote:
> +      Changing hash values affects the order in which keys are retrieved 
> from a
> +      dict.  Although Python has never made guarantees about this ordering 
> (and
> +      it typically varies between 32-bit and 64-bit builds), enough 
> real-world
> +      code implicitly relies on this non-guaranteed behavior that the
> +      randomization is disabled by default.

That last sentence needs to change for 3.3

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of PEP 397 - Python launcher for Windows

[Mark Hammond]

On 22/02/2012 2:50 AM, Vinay Sajip wrote:

Mark Hammond  gmail.com>  writes:


think there is something that could be added to those docs - the use of
PATHEXT and the fact that once the shebang line is in place, a
command-prompt could do just "hello.py" rather than needing "py hello.py".


Or even just "hello" should work.


Ooops - right.  IIRC, "hello.py" will work without anything special in 
PATHEXT and just "hello" would work with a modified PATHEXT.


Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CPU vs Wall time Profiling

[Guido van Rossum]

On Tue, Feb 21, 2012 at 1:00 PM, Sümer Cip  wrote:
> Is there a reason behind the fact that the Python profilers work with Wall
> time by default? There are OS-dependent ways to get the CPU time of a
> thread, and giving that choice to the user _somehow_ ( to use wall vs cpu
> time) might be a good feature?

The original reason was that the Unix wall clock was more accurate
than its CPU clock. If that's changed we should probably (perhaps in a
platform-dependent way) change the default to the most accurate clock
available.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Eli Bendersky]

> I'm happy to stand up as an example of someone who uses a custom Element
> class.  My specific use case is loading the project Gutenberg database,
> which is a 210MB XML file.  I created a custom Element class which I use
> for the top-level element (a custom element_factory passed to
> TreeBuilder distinguishes between the top-level element and all others).
> The custom Element class doesn't add children, so it keeps ElementTree
> from storing all of the elements its seen so far.  On a system with 1 GB
> of RAM, there was no other way to get the file to load.
>
> So, I would be one of those people who would flood in the complaints. :)
>

Andrew, could you elaborate on your use case? Are you using cElementTree to
do the parsing, or ElementTree (the Python implementation). Can you show a
short code sample?

Thanks in advance,
Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[Eli Bendersky]

> > I can't fathom why someone would do it though, since bar tiny
> > differences (like this one) cET is just a faster ET and it's available
> > practically everywhere with CPython. I mean, is it really important to
> > be able to subclass ET.Element? What goal does it serve?
>
> Statements like this make me *extremely* worried. Please try to adopt
> a position of much higher caution, accepting that a change is
> "incompatible" if there is a remote possibility that someone might
> actually rely on the original behavior. Otherwise, I predict that you
> will get flooded with complaints that you broke ET for no good reason.
>

No need to be worried, Martin. If you read back in this thread you'll see
that I agree that backwards compatibility should be preserved, by making
the Element exposed from _elementtree also a type. I was simply trying to
have a discussion to better understand the use cases and implications. I
hope that's alright.


>
> In the specific case, I tried to write a script that determines the
> memory usage of ET. As Element is lacking gc.get_referents support,
> I tried isinstance(o, Element), which failed badly.
>

Thanks for describing the use case. By the way, when working with ET I also
wanted to track the memory usage of the package a couple of times, which
made me lament that there's no useful memory profiler in the stdlib.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Windows build - fixing compile warnings before VS2010

[Brian Curtin]

While some effort has gone on to get the 32-bit build to compile
without warnings (thanks for that!), 64-bit still has numerous
warnings. Before I push forward on more of the VS2010 port, I'd like
to have a clean 2008 build all around so we can more easily track what
may have changed. In completing that effort, I'm using a guideline
Martin set out in #9566 [0], and please let me know if there are any
others to follow. I kind of doubt anyone is against this, but if you
are, please speak up before I start pushing changes.

While I have your attention, I'd like to throw two other things out
there to follow up the above effort:
1. Is anyone opposed to moving up to Level 4 warnings?
...take a deep breath...
2. Is anyone opposed to enabling warnings as errors?


[0] http://bugs.python.org/issue9566#msg113574
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[martin]

I'm with Antoine here - being able to force a particular seed still
matters for testing purposes. However, the documentation of the option
may need to be updated for 3.3 to emphasise that it should only be
used to reproduce sporadic failures. Using it to work around
applications that can't cope with randomised hashes would be rather
ill-advised.


In the tracker, someone proposed that the option is necessary to synchronize
the seed across processes in a cluster. I'm sure people will use it for that
if they can.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

[martin]

Thanks for describing the use case. By the way, when working with ET I also
wanted to track the memory usage of the package a couple of times, which
made me lament that there's no useful memory profiler in the stdlib.


A memory profiler can be a ten-line Python function which, however, does need
to be tuned to the application. So I'm not sure it can be provided by the
stdlib in a reasonable fashion beyond what's already there, but it may not
be necessary to have it in the stdlib, either.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Windows build - fixing compile warnings before VS2010

[martin]

Zitat von Brian Curtin :


While some effort has gone on to get the 32-bit build to compile
without warnings (thanks for that!), 64-bit still has numerous
warnings. Before I push forward on more of the VS2010 port, I'd like
to have a clean 2008 build all around so we can more easily track what
may have changed.


Does that *really* have to be a prerequisite for porting to VS 2010?
If yes, then my hopes that we can move to VS 2010 before 3.3 are
falling...


While I have your attention, I'd like to throw two other things out
there to follow up the above effort:
1. Is anyone opposed to moving up to Level 4 warnings?


Not sure what this means. What kind of warnings would this get us?

MS says "This option should be used only to provide "lint" level
warnings and is not recommended as your usual warning level setting."

Usually, following MS recommendations is a good thing to do on Windows.
But then, the documentation goes on saying

"For a new project, it may be best to use /W4 in all compilations.
This will ensure the fewest possible hard-to-find code defects."


...take a deep breath...
2. Is anyone opposed to enabling warnings as errors?


The immediate consequence would be that the Windows buildbots
break when somebody makes a checkin on Unix, and they cannot
easily figure out how to rewrite the code to make the compiler
happy. So I guess I'm -1.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CPU vs Wall time Profiling

[Sümer Cip]

>
> The original reason was that the Unix wall clock was more accurate
> than its CPU clock. If that's changed we should probably (perhaps in a
> platform-dependent way) change the default to the most accurate clock
> available.
>
>
Currently it seems clock_gettime() APIs have nanosecond resolution and OTOH
gettimeofday() have microsecond. Other than that, clock_gettime() has a
significant advantage: it has per-process timer available which will
increase the accuracy of the timing information of the profiled
application.

--
Sumer Cip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] hash randomization in 3.3

[Nick Coghlan]

On Wed, Feb 22, 2012 at 3:20 PM,   wrote:
>> I'm with Antoine here - being able to force a particular seed still
>> matters for testing purposes. However, the documentation of the option
>> may need to be updated for 3.3 to emphasise that it should only be
>> used to reproduce sporadic failures. Using it to work around
>> applications that can't cope with randomised hashes would be rather
>> ill-advised.
>
>
> In the tracker, someone proposed that the option is necessary to synchronize
> the seed across processes in a cluster. I'm sure people will use it for that
> if they can.

Yeah, that use case sounds reasonable, too. Another example is that,
even within a machine, if two processes are using shared memory rather
than serialised IPC, synchronising the hashes may be necessary. The
key point is that there *are* valid use cases for forcing a particular
seed, so we shouldn't take that ability away.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] accept string in a2b and base64?

[Stephen J. Turnbull]

R. David Murray writes:

 > If most people agree with Antoine I won't fight it, but it seems to me
 > that accepting unicode in the binascii and base64 APIs is a bad
 > idea.

First, I agree with David that this change should have been brought up
on python-dev before committing it.  The distinctions Python 3 has
made between APIs for bytes and those for str are both obviously
controversial and genuinely delicate.

Second, if Unicode is to be accepted in these APIs, there is a doc
issue (which I haven't checked).  It must be made clear that the
"printable ASCII" is question is the set represented by the *integers*
33 to 126, *not* the ASCII characters ! to ~.  Those characters are
present in the Unicode repertoire in many other places (specifically
the "full-width ASCII" compatibility character set around U+FF20, but
also several Greek and Cyrillic characters, and possibly others.)

I'm going to side with Antoine and Nick on these particular changes
because in practice (except maybe in the email module :-( ) the
BASE-encoded "text" to be decoded is going to be consistently defined
by the client as either str or bytes, but not both.  The fact that the
repr of the encoded text is identical (except for the presence or
absence of a leading "b") is very suggestive here.  I do harbor a
slight niggle that I think there is more room for confusion here than
in Nick's urllib work.

However, once we clarify that confusion in *our* minds, I don't think
there's much potential for dangerous confusion for API clients.  (I
agree with Antoine on that point.)  The BASE## decoding APIs in
abstract are "text" to bytes.  Pedantically in Python that suggests a
str -> bytes signature, but RFC 4648 doesn't anywhere require a 1-byte
representation of ASCII, only that the representation be interpreted
as integers in the ASCII coding.  However, an RFC-4648-conforming
implementation MUST reject any string containing characters not
allowed in the representation, so it's actually stricter than
requiring ASCII.  I see no problem with allowing str-or-bytes -> bytes
polymorphism here.

The remaining issue to my mind is we'd also like bytes -> str-or-bytes
polymorphism for symmetry, but this is not Haskell, we can't have it.

The same is true for binascii, I suppose -- assuming that the module
is specified (as the name suggests) to produce and consume only ASCII
text as a representation of bytes.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com