[Python-Dev] Compiling 2.7.2 on OS/2

2012-01-05 Thread Paul Smedley
Hi All, I'm working on updating my port of Python 2.6.5 to v2.7.2 for the OS/2 platform. I have python.exe and python27.dll compiling find, but when starting to build sharedmods I'm getting the following error: running build running build_ext Traceback (most recent call last): File "./setu

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Antoine Pitrou
On Thu, 5 Jan 2012 15:26:27 +1100 Andrew Bennetts wrote: > > I don't think that's news either. > http://mail.python.org/pipermail/python-dev/2003-May/035907.html and > http://twistedmatrix.com/pipermail/twisted-python/2003-June/004339.html for > instance show that in 2003 it was clearly known to

[Python-Dev] Testing the tests by modifying the ordering of dict items.

2012-01-05 Thread Mark Shannon
Hi, Python code should not depend upon the ordering of items in a dict. Unfortunately it seems that a number of tests in the standard library do just that. Changing PyDict_MINSIZE from 8 to either 4 or 16 causes the following tests to fail: test_dis test_email test_inspect test_nntplib tes

Re: [Python-Dev] Testing the tests by modifying the ordering of dict items.

2012-01-05 Thread Antoine Pitrou
On Thu, 05 Jan 2012 13:46:52 + Mark Shannon wrote: > > Should I: > > 1. Submit one big bug report? > > 2. Submit a bug report for each "failing" test separately? I would say a separate bug report for each failing test file, i.e. one report for test_dis, one for test_email etc. Hope this do

Re: [Python-Dev] Compiling 2.7.2 on OS/2

2012-01-05 Thread Amaury Forgeot d'Arc
2012/1/5 Paul Smedley > Hi All, > > I'm working on updating my port of Python 2.6.5 to v2.7.2 for the OS/2 > platform. > > I have python.exe and python27.dll compiling find, but when starting to > build sharedmods I'm getting the following error: > running build > running build_ext > Traceback (m

Re: [Python-Dev] Testing the tests by modifying the ordering of dict items.

2012-01-05 Thread Barry Warsaw
On Jan 05, 2012, at 01:46 PM, Mark Shannon wrote: >2. Submit a bug report for each "failing" test separately? I'm sure it will be a pain, but this is really the best thing to do. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.pyth

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Maciej Fijalkowski
On Thu, Jan 5, 2012 at 3:39 PM, Antoine Pitrou wrote: > On Thu, 5 Jan 2012 15:26:27 +1100 > Andrew Bennetts wrote: >> >> I don't think that's news either. >> http://mail.python.org/pipermail/python-dev/2003-May/035907.html and >> http://twistedmatrix.com/pipermail/twisted-python/2003-June/004339.

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Glenn Linderman
On 1/5/2012 9:34 AM, Maciej Fijalkowski wrote: Also consider that new 2.6.x would go as a security fix to old ubuntu, but all other packages won't, because they'll not contain security fixes. Just so you know Why should CPython by constrained by broken policies of Ubuntu? If the other package

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Antoine Pitrou
On Thu, 5 Jan 2012 19:34:13 +0200 Maciej Fijalkowski wrote: > > Just to make things clear - stdlib itself has 1/64 of tests relying on > dict order. Changing dict order in *older* pythons will break > everyone's tests and some peoples code. Breaking tests is not a problem: they are typically not

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread David Malcolm
On Thu, 2012-01-05 at 19:34 +0200, Maciej Fijalkowski wrote: > On Thu, Jan 5, 2012 at 3:39 PM, Antoine Pitrou wrote: > > On Thu, 5 Jan 2012 15:26:27 +1100 > > Andrew Bennetts wrote: > >> > >> I don't think that's news either. > >> http://mail.python.org/pipermail/python-dev/2003-May/035907.html a

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2012 02:14 PM, Glenn Linderman wrote: > 1) the security problem is not in CPython, but rather in web servers > that use dict inappropriately. Most webapp vulnerabilities are due to their use of Python's cgi module, which it uses a dict to ho

Re: [Python-Dev] Compiling 2.7.2 on OS/2

2012-01-05 Thread Paul Smedley
Hi Amaury, On 06/01/12 00:32, Amaury Forgeot d'Arc wrote: 2012/1/5 Paul Smedley mailto:p...@smedley.id.au>> Hi All, I'm working on updating my port of Python 2.6.5 to v2.7.2 for the OS/2 platform. I have python.exe and python27.dll compiling find, but when starting to buil

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Glenn Linderman
On 1/5/2012 11:49 AM, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2012 02:14 PM, Glenn Linderman wrote: 1) the security problem is not in CPython, but rather in web servers that use dict inappropriately. Most webapp vulnerabilities are due to their use of Python's

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Ethan Furman
Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2012 02:14 PM, Glenn Linderman wrote: 1) the security problem is not in CPython, but rather in web servers that use dict inappropriately. Most webapp vulnerabilities are due to their use of Python's cgi module, which it

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Paul Moore
On 5 January 2012 19:33, David Malcolm wrote: > We have similar issues in RHEL, with the Python versions going much > further back (e.g. 2.3) > > When backporting the fix to ancient python versions, I'm inclined to > turn the change *off* by default, requiring the change to be enabled via > an env

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Barry Warsaw
On Jan 05, 2012, at 02:33 PM, David Malcolm wrote: >We have similar issues in RHEL, with the Python versions going much >further back (e.g. 2.3) > >When backporting the fix to ancient python versions, I'm inclined to >turn the change *off* by default, requiring the change to be enabled via >an env

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Barry Warsaw
On Jan 05, 2012, at 08:35 PM, Paul Moore wrote: >Uh, surely no-one is suggesting backporting to "ancient" versions? I >couldn't find the statement quickly on the python.org website (so this >is via google), but isn't it true that 2.6 is in security-only mode >and 2.5 and earlier will never get the

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Toshio Kuratomi
On Thu, Jan 05, 2012 at 08:35:57PM +, Paul Moore wrote: > On 5 January 2012 19:33, David Malcolm wrote: > > We have similar issues in RHEL, with the Python versions going much > > further back (e.g. 2.3) > > > > When backporting the fix to ancient python versions, I'm inclined to > > turn the

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread David Malcolm
On Thu, 2012-01-05 at 20:35 +, Paul Moore wrote: > On 5 January 2012 19:33, David Malcolm wrote: > > We have similar issues in RHEL, with the Python versions going much > > further back (e.g. 2.3) > > > > When backporting the fix to ancient python versions, I'm inclined to > > turn the change

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Georg Brandl
On 01/05/2012 09:45 PM, Barry Warsaw wrote: > On Jan 05, 2012, at 02:33 PM, David Malcolm wrote: > >>We have similar issues in RHEL, with the Python versions going much >>further back (e.g. 2.3) >> >>When backporting the fix to ancient python versions, I'm inclined to >>turn the change *off* by de

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Christian Heimes
Am 05.01.2012 21:45, schrieb Barry Warsaw: > This sounds like a reasonable compromise for all stable Python releases. It > can be turned on by default for Python 3.3. If you also make the default > setting easy to change (i.e. parameterized in one place), then distros can > make their own decisio

[Python-Dev] usefulness of Python version of threading.RLock

2012-01-05 Thread Charles-François Natali
Hi, Issue #13697 (http://bugs.python.org/issue13697) deals with a problem with the Python version of threading.RLock (a signal handler which tries to acquire the same RLock is called right at the wrong time) which doesn't affect the C version. Whether such a use case can be considered good practis

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Christian Heimes
Am 05.01.2012 21:10, schrieb Ethan Furman: > Tres Seaver wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 01/05/2012 02:14 PM, Glenn Linderman wrote: >>> 1) the security problem is not in CPython, but rather in web servers >>> that use dict inappropriately. >> >> Most webapp vuln

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Antoine Pitrou
On Thu, 05 Jan 2012 22:40:58 +0100 Christian Heimes wrote: > Am 05.01.2012 21:45, schrieb Barry Warsaw: > > This sounds like a reasonable compromise for all stable Python releases. It > > can be turned on by default for Python 3.3. If you also make the default > > setting easy to change (i.e. pa

Re: [Python-Dev] usefulness of Python version of threading.RLock

2012-01-05 Thread Eric Snow
2012/1/5 Charles-François Natali : > Hi, > > Issue #13697 (http://bugs.python.org/issue13697) deals with a problem > with the Python version of threading.RLock (a signal handler which > tries to acquire the same RLock is called right at the wrong time) > which doesn't affect the C version. > Whethe

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Christian Heimes
Am 05.01.2012 22:59, schrieb Antoine Pitrou: > I don't think we (python-dev) are really concerned with 2.3, 2.4, > 2.5 and 3.0. They're all unsupported, and people do what they want > with their local source trees. Let me reply with a quote from Barry: > Correct, although there's no reason why a

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Serhiy Storchaka
05.01.12 21:14, Glenn Linderman написав(ла): So, fixing the vulnerable packages could be a sufficient response, rather than changing the hash function. How to fix? Each of those above allocates and returns a dict. Simply have each of those allocate and return and wrapped dict, which has the fo

Re: [Python-Dev] usefulness of Python version of threading.RLock

2012-01-05 Thread Antoine Pitrou
On Thu, 5 Jan 2012 15:02:42 -0700 Eric Snow wrote: > 2012/1/5 Charles-François Natali : > > Hi, > > > > Issue #13697 (http://bugs.python.org/issue13697) deals with a problem > > with the Python version of threading.RLock (a signal handler which > > tries to acquire the same RLock is called right

Re: [Python-Dev] Compiling 2.7.2 on OS/2

2012-01-05 Thread Terry Reedy
On 1/5/2012 3:01 PM, Paul Smedley wrote: File "./setup.py", line 1154, in detect_modules for arg in sysconfig.get_config_var("__CONFIG_ARGS").split()] AttributeError: 'NoneType' object has no attribute 'split' make: *** [sharedmods] Error 1 File "./setup.py", line 1368, in detect_modules if '

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Nick Coghlan
On Fri, Jan 6, 2012 at 8:15 AM, Serhiy Storchaka wrote: > 05.01.12 21:14, Glenn Linderman написав(ла): >> >> So, fixing the vulnerable packages could be a sufficient response, >> rather than changing the hash function.  How to fix?  Each of those >> above allocates and returns a dict.  Simply have

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Terry Reedy
On 1/5/2012 3:10 PM, Ethan Furman wrote: Tres Seaver wrote: 1) the security problem is not in CPython, but rather in web servers that use dict inappropriately. Most webapp vulnerabilities are due to their use of Python's cgi module, which it uses a dict to hold the form / query string data b

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Steven D'Aprano
David Malcolm wrote: When backporting the fix to ancient python versions, I'm inclined to turn the change *off* by default, requiring the change to be enabled via an environment variable: I want to avoid breaking existing code, even if such code is technically relying on non-guaranteed behavior.

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Barry Warsaw
On Jan 05, 2012, at 10:40 PM, Christian Heimes wrote: >Hey Barry, stop stealing my ideas! :) I've argued for these default >settings for days. :) >I've suggested the env var PYRANDOMHASH. It's easy to set env vars in >Apache. For example Debian/Ubuntu has /etc/apache2/envvars. For consistency,

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Nick Coghlan
On Fri, Jan 6, 2012 at 10:07 AM, Steven D'Aprano wrote: > Surely the way to verify the behaviour is to run this from the shell: > > python -c print(hash("abcde")) > > twice, and see that the calls return different values. (Or have I > misunderstood the way the fix is going to work?) > > In any cas

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Victor Stinner
2012/1/6 Barry Warsaw : >>Settings for PYRANDOMHASH: >> >> PYRANDOMHASH=1 >>   enable randomized hashing function >> >> PYRANDOMHASH=/path/to/seed >>   enable randomized hashing function and read seed from 'seed' >> >> PYRANDOMHASH=0 >>   disable randomed hashing function > > Why not PYTHONHASHSEED

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Christian Heimes
Am 06.01.2012 01:34, schrieb Nick Coghlan: > On Fri, Jan 6, 2012 at 10:07 AM, Steven D'Aprano wrote: >> Surely the way to verify the behaviour is to run this from the shell: >> >> python -c print(hash("abcde")) >> >> twice, and see that the calls return different values. (Or have I >> misunderstoo

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Benjamin Peterson
2012/1/5 Nick Coghlan : > On Fri, Jan 6, 2012 at 10:07 AM, Steven D'Aprano wrote: >> Surely the way to verify the behaviour is to run this from the shell: >> >> python -c print(hash("abcde")) >> >> twice, and see that the calls return different values. (Or have I >> misunderstood the way the fix i

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Antoine Pitrou
On Fri, 06 Jan 2012 01:50:00 +0100 Christian Heimes wrote: > Am 06.01.2012 01:34, schrieb Nick Coghlan: > > On Fri, Jan 6, 2012 at 10:07 AM, Steven D'Aprano > > wrote: > >> Surely the way to verify the behaviour is to run this from the shell: > >> > >> python -c print(hash("abcde")) > >> > >> tw

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Nick Coghlan
On Fri, Jan 6, 2012 at 10:59 AM, Benjamin Peterson wrote: > What exactly is the disadvantage of a sys attribute? That would seem > preferable to an obscure incarnation like that. Adding sys attributes in maintenance (or security) releases makes me nervous. However, Victor and Christian are right

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Steven D'Aprano
Benjamin Peterson wrote: 2012/1/5 Nick Coghlan : On Fri, Jan 6, 2012 at 10:07 AM, Steven D'Aprano wrote: Surely the way to verify the behaviour is to run this from the shell: python -c print(hash("abcde")) twice, and see that the calls return different values. (Or have I misunderstood the wa

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Benjamin Peterson
2012/1/5 Steven D'Aprano : > Benjamin Peterson wrote: >> >> 2012/1/5 Nick Coghlan : >>> >>> On Fri, Jan 6, 2012 at 10:07 AM, Steven D'Aprano >>> wrote: Surely the way to verify the behaviour is to run this from the shell: python -c print(hash("abcde")) twice, and see

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Christian Heimes
Am 06.01.2012 03:04, schrieb Benjamin Peterson: > It's obscure because hash('') != 0 doesn't necessarily mean the hashes > are randomized. A different hashing algorithm could be in effect. Also in 1 of 2**32 or 2**64 tries hash('') is 0 although randomizing is active. Christian __

Re: [Python-Dev] usefulness of Python version of threading.RLock

2012-01-05 Thread Robert Collins
On Fri, Jan 6, 2012 at 11:17 AM, Antoine Pitrou wrote: >> >From what I understand, the biggest motivation for pure Python >> versions is cooperation with the other Python implementations.  See >> http://www.python.org/dev/peps/pep-0399/ > > Apologies, I didn't remember it was written down in PEP.

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Steven D'Aprano
Benjamin Peterson wrote: 2012/1/5 Steven D'Aprano : [...] There's nothing obscure about directly testing the hash. That's about as far from obscure as it is possible to get: you are directly testing the presence of a feature by testing the feature. It's obscure because hash('') != 0 doesn't n

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Glenn Linderman
On 1/5/2012 5:52 PM, Steven D'Aprano wrote: At some point, presuming that there is no speed penalty, the behaviour will surely become not just enabled by default but mandatory. Python has never promised that hashes must be predictable or consistent, so apart from backwards compatibility conce

Re: [Python-Dev] usefulness of Python version of threading.RLock

2012-01-05 Thread Matt Joiner
I'm pretty sure the Python version of RLock is in use in several alternative implementations that provide an alternative _thread.lock. I think gevent would fall into this camp, as well as a personal project of mine in a similar vein that operates on python3. 2012/1/6 Charles-François Natali > Hi

Re: [Python-Dev] Hash collision security issue (now public)

2012-01-05 Thread Serhiy Storchaka
06.01.12 02:10, Nick Coghlan написав(ла): Not a good idea - a lot of the 3rd party tests that depend on dict ordering are going to be using those modules anyway, so scattering our solution across half the standard library is needlessly creating additional work without really reducing the incompat

Re: [Python-Dev] Compiling 2.7.2 on OS/2

2012-01-05 Thread Paul Smedley
Hi Terry, On 06/01/12 10:25, Terry Reedy wrote: On 1/5/2012 3:01 PM, Paul Smedley wrote: File "./setup.py", line 1154, in detect_modules for arg in sysconfig.get_config_var("__CONFIG_ARGS").split()] AttributeError: 'NoneType' object has no attribute 'split' make: *** [sharedmods] Error 1 Fi