Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread martin
It doesn't change anything, you will still get collisions. That depends right? If the collision is because they all have the same hash(), yes. It might be different if it is because the secondary hashing (or whatever it's called :-) causes collisions. But Python deals with the latter case

Re: [Python-Dev] Backwards incompatible sys.stdout.write() behavior in Python 3 (Was: [Python-ideas] Pythonic buffering in Py3 print())

2012-01-17 Thread anatoly techtonik
On Fri, Jan 13, 2012 at 7:19 PM, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 13 Jan 2012 17:00:57 +0100 Xavier Morel python-...@masklinn.net wrote: FWIW this is not restricted to Linux (the same behavior change can be observed in OSX), and the script is overly complex you can expose

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Stephen J. Turnbull
mar...@v.loewis.de writes: It doesn't change anything, you will still get collisions. That depends right? If the collision is because they all have the same hash(), yes. It might be different if it is because the secondary hashing (or whatever it's called :-) causes collisions.

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Victor Stinner
I thought that the original problem was that with N insertions in the dictionary, by repeatedly inserting different keys generating the same hash value an attacker could arrange that the cost of finding an open slot is O(N), and thus the cost of N insertions is O(N^2). If so, frequent

Re: [Python-Dev] Backwards incompatible sys.stdout.write() behavior in Python 3 (Was: [Python-ideas] Pythonic buffering in Py3 print())

2012-01-17 Thread Ronald Oussoren
On 17 Jan, 2012, at 11:59, anatoly techtonik wrote: If you track this more closely, you'll notice there are four issues (surprises) from the user point of view: 1. print() buffers output on Python3 2. print() also buffers output on Python2, but only on Linux 3. there is some useless

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Victor Stinner
I finished my patch transforming hash(str) to a randomized hash function, see random-8.patch attached to the issue: http://bugs.python.org/issue13703 The remaining question is which random number generator should be used on Windows to initialize the hash secret (CryptoGen adds an overhead of 10%,

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Jeremy Sanders
Victor Stinner wrote: If hash(str1)DICT_MASK == hash(str2)DICT_MASK but hash(str1)!=hash(str2), strings are not compared (this is a common optimization in Python), and the so the attack would not be successful (it would be slow, but not as slow as comparing two strings). It's a shame the

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Jeremy Sanders
Jeremy Sanders wrote: Victor Stinner wrote: If hash(str1)DICT_MASK == hash(str2)DICT_MASK but hash(str1)!=hash(str2), strings are not compared (this is a common optimization in Python), and the so the attack would not be successful (it would be slow, but not as slow as comparing two

Re: [Python-Dev] [Python-checkins] cpython: add str.casefold() (closes #13752)

2012-01-17 Thread Éric Araujo
Hi, changeset: d4669f43d05f user:Benjamin Peterson benja...@python.org date:Sat Jan 14 13:23:30 2012 -0500 summary: add str.casefold() (closes #13752) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++

Re: [Python-Dev] [Python-checkins] cpython: provide a common method to check for RETR_DATA validity, first checking the

2012-01-17 Thread Éric Araujo
Hi Giampaolo, changeset: 53a5a5b8859d user:Giampaolo Rodola' g.rod...@gmail.com date:Mon Jan 09 17:10:10 2012 +0100 summary: provide a common method to check for RETR_DATA validity, first checking the expected len and then the actual data content; this way we get a failure

Re: [Python-Dev] Sphinx version for Python 2.x docs

2012-01-17 Thread Sandro Tosi
On Mon, Jan 16, 2012 at 16:42, Éric Araujo mer...@netwok.org wrote: Hi, Le 14/01/2012 15:31, Sandro Tosi a écrit : On Sat, Jan 14, 2012 at 04:24, Éric Araujo mer...@netwok.org wrote: Doc/glossary.rst:520: WARNING: unknown keyword: nonlocal That’s a mistake I did in cefe4f38fa0e.  This

Re: [Python-Dev] Sphinx version for Python 2.x docs

2012-01-17 Thread Georg Brandl
Am 17.01.2012 19:02, schrieb Sandro Tosi: I should have given more info, as I wanted the opposite result :) file.next should not link to the next function but to the file.next method. Because Sphinx does not differentiate between meth/func/class/mod roles, :meth:`next` is not resolved to the

Re: [Python-Dev] Script(s) for building Python on Windows

2012-01-17 Thread Martin v. Löwis
Are you suggesting creating vs10to9, which would be congruent to vs9to8, or vs9to10? After reconsidering, I don't think I want anything like this in the tree at this point. The code will be outdated by the time Python 3.3 is released, as Python 3.3 will be built with a Visual Studio different

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Martin v. Löwis
I thought that the original problem was that with N insertions in the dictionary, by repeatedly inserting different keys generating the same hash value an attacker could arrange that the cost of finding an open slot is O(N), and thus the cost of N insertions is O(N^2). If so, frequent

[Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Antoine Pitrou
Hello, We would like to propose the following PEP to change (C)Python's release cycle. Discussion is welcome, especially from people involved in the release process, and maintainers from third-party distributions of Python. Regards Antoine. PEP: 407 Title: New release cycle and introducing

[Python-Dev] Switching to Visual Studio 2010

2012-01-17 Thread Martin v. Löwis
It seems a number of people are interested that the Python trunk switches to Visual Studio 2010 *now*. I've been hesitant to agree to such a change, as I still hope that Python can skip over VS 2010 (a.k.a. VS 10), and go straight to VS 11. However, I just learned that VS 11 supposed ready VS 10

Re: [Python-Dev] Switching to Visual Studio 2010

2012-01-17 Thread Brian Curtin
On Tue, Jan 17, 2012 at 14:43, Martin v. Löwis mar...@v.loewis.de wrote: It seems a number of people are interested that the Python trunk switches to Visual Studio 2010 *now*. I've been hesitant to agree to such a change, as I still hope that Python can skip over VS 2010 (a.k.a.  VS 10), and

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Martin v. Löwis
I plan to commit my fix to Python 3.3 if it is accepted. Then write a simplified version to Python 3.2 and backport it to 3.1. I'm opposed to any change to the hash values of strings in maintenance releases, so I guess I'm opposed to your patch in principle. See my next message for an

[Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Martin v. Löwis
I'd like to propose a different approach to seeding the string hashes: only do so for dictionaries involving only strings, and leave the tp_hash slot of strings unchanged. Each string would get two hashes: the public hash, which is constant across runs and bugfix releases, and the dict-hash,

Re: [Python-Dev] Switching to Visual Studio 2010

2012-01-17 Thread Martin v. Löwis
I previously completed the port at my old company (but could not release it), and I have a good bit of it completed for us at http://hg.python.org/sandbox/vs2010port/. That repo is a little bit behind 'default' but updating it shouldn't pose any problems. So: do you agree that we switch? Do

Re: [Python-Dev] Python as a Metro-style App

2012-01-17 Thread Martin v. Löwis
Just wondering, do Metro apps define UNDER_CE or _WIN32_WCE? The point is that the old ANSI functions (CreateFileA etc) have been removed from the embedded MS Windows CE long ago, too, and MS Windows Mobile used to be a custom CE variant or at least strongly related. In any case, it could

Re: [Python-Dev] Switching to Visual Studio 2010

2012-01-17 Thread Brian Curtin
On Tue, Jan 17, 2012 at 15:01, Martin v. Löwis mar...@v.loewis.de wrote: I previously completed the port at my old company (but could not release it), and I have a good bit of it completed for us at http://hg.python.org/sandbox/vs2010port/. That repo is a little bit behind 'default' but

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Antoine Pitrou
On Tue, 17 Jan 2012 21:59:28 +0100 Martin v. Löwis mar...@v.loewis.de wrote: I'd like to propose a different approach to seeding the string hashes: only do so for dictionaries involving only strings, and leave the tp_hash slot of strings unchanged. I think Python 3 would be better with a clean

[Python-Dev] Coroutines and PEP 380

2012-01-17 Thread Mark Shannon
Hi all. Lets start controversially: I don't like PEP 380, I think it's a kludge. I think that CPython should have proper coroutines, rather than add more bits and pieces to generators in an attempt to make them more like coroutines. I have mentioned this before, but this time I have done

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Victor Stinner
2012/1/17 Martin v. Löwis mar...@v.loewis.de: I'd like to propose a different approach to seeding the string hashes: only do so for dictionaries involving only strings, and leave the tp_hash slot of strings unchanged. The real problem is in dict (or any structure using an hash table), so if it

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Victor Stinner
There is a simpler solution: bucket_index = (hash(str) ^ secret) DICT_MASK. Oops, hash^secret doesn't add any security. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Eric Snow
On Tue, Jan 17, 2012 at 1:34 PM, Antoine Pitrou solip...@pitrou.net wrote: Under the proposed scheme, there would be two kinds of feature versions (sometimes dubbed minor versions, for example 3.2 or 3.3): normal feature versions and long-term support (LTS) versions. ... A new feature version

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Victor Stinner
Each string would get two hashes: the public hash, which is constant across runs and bugfix releases, and the dict-hash, which is only used by the dictionary implementation, and only if all keys to the dict are strings. The distinction between secret (private, secure) and public hash

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Matt Joiner
If minor/feature releases are introducing breaking changes perhaps it's time to adopt accelerated major versioning schedule. For instance there are breaking ABI changes between 3.0/3.1, and 3.2, and while acceptable for the early adoption state of Python 3, such changes should normally be reserved

Re: [Python-Dev] Coroutines and PEP 380

2012-01-17 Thread Matt Joiner
Just to clarify, this differs in functionality from enhanced generators by allowing you to yield from an arbitrary call depth rather than having to yield from through a chain of calling generators? Furthermore there's no syntactical change except to the bottommost frame doing a co_yield? Does this

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Antoine Pitrou
Hello, On Wed, 18 Jan 2012 10:04:19 +1100 Matt Joiner anacro...@gmail.com wrote: If minor/feature releases are introducing breaking changes perhaps it's time to adopt accelerated major versioning schedule. The PEP doesn't propose to accelerate compatibility breakage. So I don't think a change

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Victor Stinner
I plan to commit my fix to Python 3.3 if it is accepted. Then write a simplified version to Python 3.2 and backport it to 3.1. I'm opposed to any change to the hash values of strings in maintenance releases, so I guess I'm opposed to your patch in principle. If randomized hash cannot be

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Terry Reedy
On 1/17/2012 3:34 PM, Antoine Pitrou wrote: Hello, We would like to propose the following PEP to change (C)Python's release cycle. Discussion is welcome, especially from people involved in the release process, and maintainers from third-party distributions of Python. Regards Antoine. PEP:

Re: [Python-Dev] Coroutines and PEP 380

2012-01-17 Thread Ethan Furman
Mark Shannon wrote: I think that CPython should have proper coroutines, rather than add more bits and pieces to generators in an attempt to make them more like coroutines. I have mentioned this before, but this time I have done something about it :) I have a working, portable, (asymmetric)

Re: [Python-Dev] Coroutines and PEP 380

2012-01-17 Thread Glyph
On Jan 17, 2012, at 5:03 PM, Mark Shannon wrote: Lets start controversially: I don't like PEP 380, I think it's a kludge. Too late; it's already accepted. There's not much point in making controversial statements about it now. I think that CPython should have proper coroutines, rather than

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Antoine Pitrou
On Tue, 17 Jan 2012 18:29:11 -0500 Terry Reedy tjre...@udel.edu wrote: To me, as I understand the proposal, the title is wrong. Our current feather releases already are long-term support versions. They get bugfix releases at close to 6 month intervals for 1 1/2 -2 years and security fixes

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Ezio Melotti
Hi, On 17/01/2012 22.34, Antoine Pitrou wrote: [...] Proposal Under the proposed scheme, there would be two kinds of feature versions (sometimes dubbed minor versions, for example 3.2 or 3.3): normal feature versions and long-term support (LTS) versions. Normal feature versions

Re: [Python-Dev] Backwards incompatible sys.stdout.write() behavior in Python 3 (Was: [Python-ideas] Pythonic buffering in Py3 print())

2012-01-17 Thread Terry Reedy
On 1/17/2012 5:59 AM, anatoly techtonik wrote: 1. print() buffers output on Python3 2. print() also buffers output on Python2, but only on Linux No, print() does not buffer output. It merely sends it to a file. 4. print() is not guilty - it is sys.stdout.write() that buffers output Oh,

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Jeff Hardy
On Tue, Jan 17, 2012 at 3:50 PM, Ezio Melotti ezio.melo...@gmail.com wrote: * What is the effect on PyPy/Jython/IronPython?  Can they just skip the feature releases and focus on the LTS ones? At least for IronPython it's unlikely we'd be able track the feature releases. We're still trying to

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread martin
Zitat von Victor Stinner victor.stin...@haypocalc.com: Each string would get two hashes: the public hash, which is constant across runs and bugfix releases, and the dict-hash, which is only used by the dictionary implementation, and only if all keys to the dict are strings. The distinction

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread martin
If randomized hash cannot be turned on by default, an alternative is to switch them off by default, and add an option (command line option, environment variable, etc.) to enable it. That won't really fix the problem. If people install a new release because it fixes a vulnerability, it better

Re: [Python-Dev] [Python-checkins] cpython: Refactored logging rotating handlers for improved flexibility.

2012-01-17 Thread Éric Araujo
Hi, changeset: 57295c4d81ac user:Vinay Sajip vinay_sa...@yahoo.co.uk date:Wed Jan 04 12:02:26 2012 + summary: Refactored logging rotating handlers for improved flexibility. diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst ---

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Martin v. Löwis
Am 17.01.2012 22:26, schrieb Antoine Pitrou: On Tue, 17 Jan 2012 21:59:28 +0100 Martin v. Löwis mar...@v.loewis.de wrote: I'd like to propose a different approach to seeding the string hashes: only do so for dictionaries involving only strings, and leave the tp_hash slot of strings unchanged.

[Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Stephen J. Turnbull
Executive summary: My take is show us the additional resources, and don't be stingy! Sorry, Antoine, I agree with your goals, but I think you are too optimistic about the positive effects and way too optimistic about the costs. Antoine Pitrou writes: Finding a release cycle for an open-source

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Terry Reedy
On 1/17/2012 6:42 PM, Antoine Pitrou wrote: On Tue, 17 Jan 2012 18:29:11 -0500 Terry Reedytjre...@udel.edu wrote: To me, as I understand the proposal, the title is wrong. Our current feather releases already are long-term support versions. They get bugfix releases at close to 6 month

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Gregory P. Smith
On Tue, Jan 17, 2012 at 12:59 PM, Martin v. Löwis mar...@v.loewis.dewrote: I'd like to propose a different approach to seeding the string hashes: only do so for dictionaries involving only strings, and leave the tp_hash slot of strings unchanged. Each string would get two hashes: the public

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Gregory P. Smith
On Tue, Jan 17, 2012 at 12:52 PM, Martin v. Löwis mar...@v.loewis.dewrote: I plan to commit my fix to Python 3.3 if it is accepted. Then write a simplified version to Python 3.2 and backport it to 3.1. I'm opposed to any change to the hash values of strings in maintenance releases, so I

Re: [Python-Dev] Status of the fix for the hash collision vulnerability

2012-01-17 Thread Martin v. Löwis
Am 18.01.2012 07:06, schrieb Gregory P. Smith: On Tue, Jan 17, 2012 at 12:52 PM, Martin v. Löwis mar...@v.loewis.de mailto:mar...@v.loewis.de wrote: I plan to commit my fix to Python 3.3 if it is accepted. Then write a simplified version to Python 3.2 and backport it to 3.1.

Re: [Python-Dev] Hashing proposal: change only string-only dicts

2012-01-17 Thread Martin v. Löwis
+1 Absolutely. We can and should make 3.3 change hashes across runs (behavior that can be disabled via a flag or environment variable). I think the issue of doctests and such breaking even in 2.7 due to hash order changes is a being overblown. Code like that has already needs to fix its

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Georg Brandl
Am 18.01.2012 05:32, schrieb Terry Reedy: On 1/17/2012 6:42 PM, Antoine Pitrou wrote: On Tue, 17 Jan 2012 18:29:11 -0500 Terry Reedytjre...@udel.edu wrote: To me, as I understand the proposal, the title is wrong. Our current feather releases already are long-term support versions. They get

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Paul Moore
On 18 January 2012 04:32, Terry Reedy tjre...@udel.edu wrote: It's really about making feature releases more frequent, not making previews available during development. Given the difficulty of making a complete windows build, it would be nice to have one made available every 6 months,

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Georg Brandl
Am 18.01.2012 01:24, schrieb Jeff Hardy: On Tue, Jan 17, 2012 at 3:50 PM, Ezio Melotti ezio.melo...@gmail.com wrote: * What is the effect on PyPy/Jython/IronPython? Can they just skip the feature releases and focus on the LTS ones? At least for IronPython it's unlikely we'd be able track

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Paul Moore
On 18 January 2012 07:46, Georg Brandl g.bra...@gmx.net wrote: But I am dubious that releases that are obsolete in 6 months and lack 3rd party support will see much production use. Whether people would use the releases is probably something that only they can tell us -- that's why a community

Re: [Python-Dev] PEP 407: New release cycle and introducing long-term support versions

2012-01-17 Thread Georg Brandl
Am 18.01.2012 00:50, schrieb Ezio Melotti: Hi, On 17/01/2012 22.34, Antoine Pitrou wrote: [...] Proposal Under the proposed scheme, there would be two kinds of feature versions (sometimes dubbed minor versions, for example 3.2 or 3.3): normal feature versions and long-term