[issue13804] Python library structure creates hard to read code when using higher order functions
Martin Häcker spamfaen...@gmx.de added the comment: Jup - oh the joys of writing code in a bugtracker :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13804 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13823] xml.etree.ElementTree.ElementTree.write - argument checking
New submission from patrick vrijlandt patrick.vrijla...@gmail.com: (1) The docs say: xml_declaration controls if an XML declaration should be added to the file. Use False for never, True for always, None for only if not US-ASCII or UTF-8 or Unicode (default is None). The method also accepts other values, like xml_declaration = yes. This behavior should be documented, or raise a ValueError (up to effbot, I think) (2) The docs say (in a note): The encoding string included in XML output should conform to the appropriate standards. For example, “UTF-8” is valid, but “UTF8” is not. See http://www.w3.org/ But the method accepts both values, (“UTF-8” and “UTF8”). Since this will result in invalid xml, (but not invalid python) it should probably raise ValueError too. (3) Open issue 9458 also refers to this method. It might be wise to raise ValueError if the encoding does not match the (mode of the) file target (binary or text). -- assignee: docs@python components: Documentation, XML messages: 151612 nosy: docs@python, patrick.vrijlandt priority: normal severity: normal status: open title: xml.etree.ElementTree.ElementTree.write - argument checking versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13823 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11948] Tutorial/Modules - small fix to better clarify the modules search path
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset d01208ba482f by Sandro Tosi in branch '2.7': Issue #11948: clarify modules search path http://hg.python.org/cpython/rev/d01208ba482f New changeset 6d663db63705 by Sandro Tosi in branch '3.2': Issue #11948: clarify modules search path http://hg.python.org/cpython/rev/6d663db63705 New changeset 93769b8ff40b by Sandro Tosi in branch 'default': Issue #11948: merge with 3.2 http://hg.python.org/cpython/rev/93769b8ff40b -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11948 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11948] Tutorial/Modules - small fix to better clarify the modules search path
Sandro Tosi sandro.t...@gmail.com added the comment: Thanks Terry for the ping, I've just committed it - thanks again to everyone for the help/suggestions. -- resolution: - fixed stage: commit review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11948 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13824] argparse.FileType opens a file without excepting resposibility for closing it
New submission from David Layton dmlay...@gmail.com: argparse.FileType.__call__ opens the specified file and returns it. This is well documented as an anit-idiom in http://docs.python.org/howto/doanddont.html#exceptions. ...a serious problem — due to implementation details in CPython, the file would not be closed when an exception is raised until the exception handler finishes; and, worse, in other implementations (e.g., Jython) it might not be closed at all regardless of whether or not an exception is raised. Disregarding the above, handling a file which may or may not have been opened depending the users input requires a bit of boilerplate code compared to the usual with-open idiom. Additionally, there is no way to prevent FileType from clobbering an existing file when used with write mode. Given these issues and others, it seems to me that the usefulness of FileType is outweighed by propensity to encourage bad coding. Perhaps, it would be best if FileType (or some replacement) simply checked that the file exists (when such a check is appropriate), it can be opened in the specified mode, and, curry the call to open (i.e. return lambda: open(string, self._mode, self._bufsize)) -- components: Library (Lib) messages: 151615 nosy: David.Layton, Paolo.Elvati, Stefan.Pfeiffer, bethard, manveru priority: normal severity: normal status: open title: argparse.FileType opens a file without excepting resposibility for closing it type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13824 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13817] deadlock in subprocess while running several threads using Popen
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +neologix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13817 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13813] sysconfig.py and distutils/util.py redundancy
Éric Araujo mer...@netwok.org added the comment: Background on the distutils freeze? Right now I don’t have the time and I’m going to be offline until the end of the month. You can look for Tarek Ziadé’s blog posts after PyCon 2010, or wait until I come back and put the links on a wiki page :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13813 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Éric Araujo mer...@netwok.org added the comment: Even Lib/packaging/create.py change is related to fixing tests. The test can be changed differently, but I like the idea of having always the same output in packaging (e.g. it is more readable for the user if files are sorted). See #13712 for why this is a fake fix. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9631] Python 2.7 installation issue for Linux gcc-4.1.0-3 (Fedora Core 5?)
mike mikaelpetters...@gmail.com added the comment: Hi, I downloaded source and did the following instructions. We use Red Hat Enterprise Linux Server release 5.5. ./configure --prefix=/home/mike/python_rh_32 make make install I also changed the line in site.py from: s = os.path.join(os.path.dirname(sys.path.pop()), s) to: s = os.path.join(os.path.dirname(sys.path[-1]), s) but I still get: /usr/bin/install -c -m 644 ./LICENSE /home/mike/python_rh_32/lib/python2.7/LICENSE.txt PYTHONPATH=/home/mike/python_rh_32/lib/python2.7 \ ./python -Wi -tt /home/mike/python_rh_32/lib/python2.7/compileall.py \ -d /home/mike/python_rh_32/lib/python2.7 -f \ -x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \ /home/mike/python_rh_32/lib/python2.7 Traceback (most recent call last): File /home/mike/python_rh_32/lib/python2.7/compileall.py, line 16, in module import struct File /home/mike/python_rh_32/lib/python2.7/struct.py, line 1, in module from _struct import * ImportError: No module named _struct make: *** [libinstall] Error 1 Did anyone slove this problem? -- nosy: +eraonel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9631 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13763] Potentially hard to understand wording in devguide
Éric Araujo mer...@netwok.org added the comment: Thanks for commenting. Please commit your wording, or this alternate version: “``hg`` is the name of the Mercurial command-line program, and is often used as an abbreviation for Mercurial.” (Just to avoid repetition and to add “command-line”.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13763 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
STINNER Victor victor.stin...@haypocalc.com added the comment:
I tried the collision counting with a low number of collisions:
less than 15 collisions
---
Fail at startup.
5 collisions (32 buckets, 21 used=65.6%): hash=ceb3152f = f
10 collisions (32 buckets, 21 used=65.6%): hash=ceb3152f = f
dict((str(k), 0) for k in range(200))
-
15 collisions (32,768 buckets, 18024 used=55.0%): hash=0e4631d2 = 31d2
20 collisions (131,072 buckets, 81568 used=62.2%): hash=12660719 = 719
25 collisions (1,048,576 buckets, 643992 used=61.4%): hash=6a1f6d21 = f6d21
30 collisions (1,048,576 buckets, 643992 used=61.4%): hash=6a1f6d21 = f6d21
35 collisions = ? (more than 10,000,000 integers)
random_dict('', 5, charset, 1, 3)
--
charset = 'abcdefghijklmnopqrstuvwxyz0123456789'
15 collisions (8192 buckets, 5083 used=62.0%): hash=1526677a = 77a
20 collisions (32768 buckets, 19098 used=58.3%): hash=5d7760e6 = 60e6
25 collisions = unable to generate a new key
random_dict('', 5, charset, 1, 3)
--
charset =
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.=+_(){}%'
15 collisions (32768 buckets, 20572 used=62.8%): hash=789fe1e6 = 61e6
20 collisions (2048 buckets, 1297 used=63.3%): hash=2052533d = 33d
25 collisions = nope
random_dict('', 5, charset, 1, 10)
--
charset =
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.=+_(){}%'
15 collisions (32768 buckets, 18964 used=57.9%): hash=94d7c4f5 = 44f5
20 collisions (32768 buckets, 21548 used=65.8%): hash=acb5b39e = 339e
25 collisions (8192 buckets, 5395 used=65.9%): hash=04d367ae = 7ae
30 collisions = nope
random_dict() comes from the following script:
***
import random
def random_string(charset, minlen, maxlen):
strlen = random.randint(minlen, maxlen)
return ''.join(random.choice(charset) for index in xrange(strlen))
def random_dict(prefix, count, charset, minlen, maxlen):
dico = {}
keys = set()
for index in xrange(count):
for tries in xrange(1):
key = prefix + random_string(charset, minlen, maxlen)
if key in keys:
continue
keys.add(key)
break
else:
raise ValueError(unable to generate a new key)
dico[key] = None
charset =
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.=+_(){}%'
charset = 'abcdefghijklmnopqrstuvwxyz0123456789'
random_dict('', 5, charset, 1, 3)
***
I ran the Django test suite. With a limit of 20 collisions, 60 tests
fail. With a limit of 50 collisions, there is no failure. But I don't
think that the test suite uses large data sets.
I also triend the Django test suite with a randomized hash function.
There are 46 failures. Many (all?) are related to the order of dict
keys: repr(dict) or indirectly in a HTML output. I didn't analyze all
failures. I suppose that Django can simply run the test suite using
PYTHONHASHSEED=0 (disable the randomized hash function), at least in a
first time.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13817] deadlock in subprocess while running several threads using Popen
Charles-François Natali neolo...@free.fr added the comment: Here's the backtrace: #0 0x003bfb20c9b1 in sem_wait () from /lib64/libpthread.so.0 #1 0x0051a7c3 in PyThread_acquire_lock (lock=0x17db0750, waitflag=1) at Python/thread_pthread.h:321 #2 0x0051a9b4 in find_key (key=1, value=0x0) at Python/thread.c:268 #3 0x0051abdc in PyThread_get_key_value (key=1) at Python/thread.c:360 #4 0x005025b1 in PyGILState_GetThisThreadState () at Python/pystate.c:598 #5 0x005024f5 in _PyGILState_Reinit () at Python/pystate.c:547 #6 0x00521fc7 in PyOS_AfterFork () at ./Modules/signalmodule.c:979 #7 0x0052461d in posix_fork (self=0x0, noargs=0x0) at ./Modules/posixmodule.c:3695 It's stuck in _PyGILState_Reinit(), when calling PyGILState_GetThisThreadState(). That's because in 2.7, TLS is emulated (see Python/thread.c), and it uses a global mutex. If this mutex is locked at the time of fork(), then the next call to TLS primitives (even PyGILState_GetThisThreadState()) will deadlock. Now, this particular bug is fixed in 2.7 since #13156, which backed-out _PyGILState_Reinit() because it was only relevant for native TLS implementations. The code is still present in 3.2 and and default, but this problem doesn't affect native TLS implementations. Just to be extra safe, we PyThread_ReInitTLS() - which resets this global mutex on emulated implementations, and is just a no-op on pthread and windows - should be moved earlier in PyOS_AfterFork(), to avoid this type of deadlock (I mean, PyGILState_GetThisThreadState() deadlock after fork() is bad). Patch attached. -- keywords: +needs review, patch stage: - patch review versions: +Python 3.2, Python 3.3 -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13817 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13825] Datetime failing while reading active directory time attribute
New submission from Chris Gill chrisfg...@gmail.com: I believe I am having a similar issue to this: http://bugs.python.org/issue7150 I am in the middle of programming a quick script and now I cannot seem to get beyond this issue; as it is printing up the expiration times from the AD user listings (many of which print 1601 year) it finally fails after the same user account, I have compared accounts and the expiration is the same as other accounts in AD: which is set to (never) any ideas of what's going on here and how I can bypass this error? error report: Traceback (most recent call last): ... line 14, in module print user.name + : + str(user.accountExpires) File C:\Python27\lib\site-packages\active_directory.py, line 425, in __getattr__ self._delegate_map[name] = converter (attr) File C:\Python27\lib\site-packages\active_directory.py, line 335, in convert_to_datetime return ad_time_to_datetime (item) File C:\Python27\lib\site-packages\active_directory.py, line 319, in ad_time_to_datetime return BASE_TIME + delta OverflowError: date value out of range code: import active_directory from datetime import datetime,timedelta ##check AD for account expirations users = active_directory.AD_object (LDAP://ou=administration,dc=domain,dc=com) for user in users.search (objectCategory='Person'): dn = user.distinguishedName dn = dn.encode(utf-8) #for the occasional apostrophe if Adjuncts in str(dn): print user.name + : + str(user.accountExpires) print done example output: CN=John Hancock: 1601-01-01 00:00:00 CN=Jane Smith: 1601-01-01 00:00:00 ... -- components: IDLE, Windows messages: 151622 nosy: Chris.Gill priority: normal severity: normal status: open title: Datetime failing while reading active directory time attribute versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13825 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13817] deadlock in subprocess while running several threads using Popen
Antoine Pitrou pit...@free.fr added the comment: I think you forgot to attach the patch :) -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13817 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13826] Having a shlex example in the subprocess.Popen docs is confusing
New submission from Julian Berman julian+python@grayvines.com: The example at http://docs.python.org/dev/library/subprocess.html#popen-constructor seems a bit misplaced, as it seems to suggest that one should use the shlex module. Most of the other examples in the module seem to use a list to provide the args, so if there was a need to just point out that shlex could be used for a corner case perhaps it'd be better suited as a footnote or another subsection somewhere. -- assignee: docs@python components: Documentation messages: 151624 nosy: Julian, docs@python priority: normal severity: normal status: open title: Having a shlex example in the subprocess.Popen docs is confusing type: enhancement versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13826 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment:
STINNER Victor wrote:
...
So I expect something similar in applications: no change in the
applications, but a lot of hacks/tricks in tests.
Tests usually check output of an application given a certain
input. If those fail with the randomization, then it's likely
real-world application uses will show the same kinds of failures
due to the application changing from deterministic to
non-deterministic via the randomization.
BTW: The patch still includes the unnecessary _Py_unicode_hash_secret.suffix
which needlessly complicates the code and doesn't any additional
protection against hash value collisions
How does it complicate the code? It adds an extra XOR to hash(str) and
4 or 8 bytes in memory, that's all. It is more difficult to compute
the secret from hash(str) output if there is a prefix *and* a suffix.
If there is only a prefix, knowning a single hash(str) value is just
enough to retrieve directly the secret.
The suffix only introduces a constant change in all hash values
output, so even if you don't know the suffix, you can still
generate data sets with collisions by just having the prefix.
I don't think it affects more than 0.01% of applications/users :)
It would help to try a patched Python on a real world application like
Django to realize how much code is broken (or not) by a randomized
hash function.
That would help for both approaches, indeed.
Please note, that you'd have to extend the randomization to
all other Python data types as well in order to reach the same level
of security as the collision counting approach.
As-is the randomization patch does not solve the integer key attack and
even though parsers such as JSON and XML-RPC aren't directly affected,
it is well possible that stringified integers such as IDs are converted
back to integers later during processing, thereby triggering the
attack.
Note that the integer attack also applies to other number types
in Python:
(3, 3, 3)
See Tim's post I referenced earlier on for the reasons. Here's
a quick summary ;-) ...
{3: 3}
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Antoine Pitrou pit...@free.fr added the comment: Please note, that you'd have to extend the randomization to all other Python data types as well in order to reach the same level of security as the collision counting approach. You also have to extend the collision counting to sets, by the way. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13826] Having a shlex example in the subprocess.Popen docs is confusing
Changes by R. David Murray rdmur...@bitdance.com: -- priority: normal - low ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13826 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13826] Having a shlex example in the subprocess.Popen docs is confusing
R. David Murray rdmur...@bitdance.com added the comment: It is not particularly intuitive what goes in to a Popen non-shell argument list, unless you are an experienced programmer. The real purpose of the note is to convey a lot of information about how tokenization works in a short example, and it also demonstrates how to investigate other complex cases the user may have to deal with. Because of the first part of that (showing tokenization quirks) I don't think it should be relegated to a footnote. That said, the example could perhaps be reworded slightly to make its expositional purpose clearer. Suggestions welcome. -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13826 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: Antoine Pitrou wrote: Antoine Pitrou pit...@free.fr added the comment: Please note, that you'd have to extend the randomization to all other Python data types as well in order to reach the same level of security as the collision counting approach. You also have to extend the collision counting to sets, by the way. Indeed, but that's easy, since the set implementation derives from the dict implementation. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Alex Gaynor alex.gay...@gmail.com added the comment: Django's tests will *not* be run with HASHEED=0, if they're broken with hash randomization then they are likely broken on random.choice([32-bit, 64-bit, pypy, jython, ironpython]) and we strive to run on all those platforms. If our tests are order dependent then they're broken, and we'll fix the tests. Further, most of the failures I can think of would be failures in the tests that wouldn't actually be failures in a real application, such as the rendered HTML being different because a tag's attributes are in a different order. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13826] Having a shlex example in the subprocess.Popen docs is confusing
Sandro Tosi sandro.t...@gmail.com added the comment: Maybe we can add a very small example before the whole note to show just how to use Popen in simple situation, and so the shlex part below will add more details for more advanced cases. -- nosy: +sandro.tosi versions: -Python 2.6, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13826 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13817] deadlock in subprocess while running several threads using Popen
Charles-François Natali neolo...@free.fr added the comment: Here's the patch. It's probably possible to add a test for this, however I don't have access to my development machine, so I can't write it now. -- Added file: http://bugs.python.org/file24275/reinit_tls.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13817 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13752] add a str.casefold() method
Changes by Giampaolo Rodola' g.rod...@gmail.com: -- nosy: +giampaolo.rodola ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13752 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: I tried the collision counting with a low number of collisions: ... no false positives with a limit of 50 collisions ... Thanks for running those tests. Looks like a limit lower than 1000 would already do just fine. Some timings showing how long it would take to hit a limit: # 100 python2.7 -m timeit -n 100 dict((x*(2**64 - 1), 1) for x in xrange(1, 100)) 100 loops, best of 3: 297 usec per loop # 250 python2.7 -m timeit -n 100 dict((x*(2**64 - 1), 1) for x in xrange(1, 250)) 100 loops, best of 3: 1.46 msec per loop # 500 python2.7 -m timeit -n 100 dict((x*(2**64 - 1), 1) for x in xrange(1, 500)) 100 loops, best of 3: 5.73 msec per loop # 750 python2.7 -m timeit -n 100 dict((x*(2**64 - 1), 1) for x in xrange(1, 750)) 100 loops, best of 3: 12.7 msec per loop # 1000 python2.7 -m timeit -n 100 dict((x*(2**64 - 1), 1) for x in xrange(1, 1000)) 100 loops, best of 3: 22.4 msec per loop These timings have to matched against the size of the payload needed to trigger those limits. In any case, the limit needs to be configurable like the hash seed in the randomization patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment:
[Reposting, since roundup removed part of the Python output]
M.-A. Lemburg wrote:
Note that the integer attack also applies to other number types
in Python:
-- (hash(3), hash(3.0), hash(3+0j)
(3, 3, 3)
See Tim's post I referenced earlier on for the reasons. Here's
a quick summary ;-) ...
-- {3:1, 3.0:2, 3+0j:3}
{3: 3}
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13478] No documentation for timeit.default_timer
Sandro Tosi sandro.t...@gmail.com added the comment: I propose these 2 patches: I'm not exactly proud of - 'in a platform specific manner' in the first hunk - the second hunk I know that there should be a better way to express it, but since I can't get to it I'd just ask for suggestions :) -- keywords: +patch stage: needs patch - patch review Added file: http://bugs.python.org/file24276/issue13478-py27.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13478 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13478] No documentation for timeit.default_timer
Changes by Sandro Tosi sandro.t...@gmail.com: Added file: http://bugs.python.org/file24277/issue13478-py32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13478 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13827] Unexecuted import changes namespace
New submission from Michael Hipp mich...@redmule.com: A local *unexecuted* import appears to be changing the namespace. Attached files are ready to run. # over.py SOMETHING = overridden # main.py OVERRIDE = False SOMETHING = original def main(): #global SOMETHING # uncomment and it works if OVERRIDE: from over import SOMETHING # comment out and it works pass print SOMETHING # UnboundLocalError: local variable 'SOMETHING' referenced before assignment The SOMETHING variable has a value from the module global namespace, but it gets lost due to an import that is never executed. I would think an unexecuted statement shouldn't have any effect on anything. The second file will have to be submitted in a follow-on, it appears -- components: None files: main.py messages: 151635 nosy: hippmr priority: normal severity: normal status: open title: Unexecuted import changes namespace type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file24278/main.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13827 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13827] Unexecuted import changes namespace
Michael Hipp mich...@redmule.com added the comment: Add'l over.py file -- Added file: http://bugs.python.org/file24279/over.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13827 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13827] Unexecuted import changes namespace
Benjamin Peterson benja...@python.org added the comment: Not a bug. Basically, import is an explicit assignment statement. -- nosy: +benjamin.peterson resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13827 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13827] Unexecuted import changes namespace
Michael Hipp mich...@redmule.com added the comment: Even an *unexecuted* import assignment statement? -- resolution: invalid - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13827 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13827] Unexecuted import changes namespace
Michael Foord mich...@voidspace.org.uk added the comment: hippmr: the problem is that by importing SOMETHING inside that function you're creating a *local variable* called SOMETHING. If the override isn't executed, and SOMETHING isn't global, then that local variable doesn't exist - which is why you get that error. So even if the import isn't executed, its existence in the function tells Python that name is local to the function. -- nosy: +michael.foord resolution: - invalid stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13827 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13827] Unexecuted import changes namespace
Ezio Melotti ezio.melo...@gmail.com added the comment: OVERRIDE = False SOMETHING = original def main(): ... if OVERRIDE: ... SOMETHING = None ... print SOMETHING ... main() Traceback (most recent call last): File stdin, line 1, in module File stdin, line 4, in main UnboundLocalError: local variable 'SOMETHING' referenced before assignment http://docs.python.org/faq/programming.html#why-am-i-getting-an-unboundlocalerror-when-the-variable-has-a-value -- nosy: +ezio.melotti resolution: invalid - stage: committed/rejected - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13827 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13827] Unexecuted import changes namespace
Changes by Ezio Melotti ezio.melo...@gmail.com: -- components: -None resolution: - invalid stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13827 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11551] test_dummy_thread.py test coverage improvement
Sandro Tosi sandro.t...@gmail.com added the comment: Hi Denver, do you have the time to address the review on rietveld and propose a new patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11551 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13825] Datetime failing while reading active directory time attribute
scape chrisfg...@gmail.com added the comment: I dug a little deeper using an error trap and found some of the problematic accounts in AD have their attribute set to a wildly long number and not 0 (as are others when 'never' is specified.) i'll dig further, it also does not seem to be an issue with python but more of an issue with the module I am using (active_directory) and its datetime handling (likely not fixed as was Python) I don't think the issue is necessarily solved, but I'll close it anyways as I think I have atleast my solution now -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13825 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11948] Tutorial/Modules - small fix to better clarify the modules search path
Eric Snow ericsnowcurren...@gmail.com added the comment: FYI: unless importlib took undue liberties (unlikely), frozen modules also precede path-based modules. See the implicit additions to sys.meta_path in Lib/importlib/_bootstrap.py. Whether or not to include a mention of frozen modules in the tutorial...I'll leave that to you. :) -- nosy: +eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11948 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13828] Further improve casefold documentation
New submission from Jim Jewett jimjjew...@gmail.com: http://hg.python.org/cpython/rev/0b5ce36a7a24 changeset: 74515:0b5ce36a7a24 + Casefolding is similar to lowercasing but more aggressive because it is + intended to remove all case distinctions in a string. For example, the German + lowercase letter ``'ß'`` is equivalent to ``ss``. Since it is already + lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold` + converts it to ``ss``. Perhaps add the recommendation to canonicalize as well. A complete, but possibly too long, try is below: Casefolding is similar to lowercasing but more aggressive because it is intended to remove all case distinctions in a string. For example, the German lowercase letter ``'ß'`` is equivalent to ``ss``. Since it is already lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold` converts it to ``ss``. Note that most case-insensitive matches should also match compatibility equivalent characters. The casefolding algorithm is described in section 3.13 of the Unicode Standard. Per D146, a compatibility caseless match can be achieved by from unicodedata import normalize def caseless_compat(string): nfd_string = normalize(NFD, string) nfkd1_string = normalize(NFKD, nfd_string.casefold()) return normalize(NFKD, nfkd1_string.casefold()) -- assignee: docs@python components: Documentation messages: 151644 nosy: Jim.Jewett, benjamin.peterson, docs@python priority: normal severity: normal status: open title: Further improve casefold documentation versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13828 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13828] Further improve casefold documentation
Jim Jewett jimjjew...@gmail.com added the comment: Frankly, I do think that sample code is too long, but correctness matters ... perhaps a better solution would be to add either a method or a unicodedata function that does the work, then the extra note could just say Note that most case-insensitive matches should also match compatibility equivalent characters; see unicodedata.compatibity_casefold -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13828 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13641] decoding functions in the base64 module could accept unicode strings
Antoine Pitrou pit...@free.fr added the comment: Thanks for the updated patch! Two comments: - I see no tests for map01 and altchars being passed as an str, is this supported by the patch or am I reading it wrong? - apparently b16decode is not tackled, is it deliberate? Thanks again. -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13641 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Frank Sievertsen pyt...@sievertsen.de added the comment: The suffix only introduces a constant change in all hash values output, so even if you don't know the suffix, you can still generate data sets with collisions by just having the prefix. That's true. But without the suffix, I can pretty easy and efficient guess the prefix by just seeing the result of a few well-chosen and short repr(dict(X)). I suppose that's harder with the suffix. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13829] exception error
New submission from Dan kamp roont...@gmail.com: Get this error when trying to run Moviegrabber on a mac running v2.7. Crash report below. Process: Python [2444] Path: /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/C ontents/MacOS/Python Identifier: Python Version: ??? (???) Code Type: X86-64 (Native) Parent Process: Python [2431] Date/Time: 2011-12-21 20:32:41.233 -0500 OS Version: Mac OS X 10.7.2 (11C74) Report Version: 9 Interval Since Last Report: 357638 sec Crashes Since Last Report: 17 Per-App Crashes Since Last Report: 405 Anonymous UUID: 3C387DB7-0AA3-4F34-A100-31D1736D2668 Crashed Thread: 0 Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x0108 VM Regions Near 0x108: -- __TEXT 00010c5c5000-00010c5c6000 [4K] r-x/rwx SM=COW /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/C ontents/MacOS/Python Application Specific Information: objc[2429]: garbage collection is OFF Thread 0 Crashed: 0 libdispatch.dylib 0x7fff8b14ace9 _dispatch_wakeup + 108 1 libdispatch.dylib 0x7fff8b14d876 _dispatch_resume_slow + 20 2 com.apple.CoreFoundation0x7fff8af8262c -[NSXPCConnection start] + 92 3 com.apple.CoreFoundation0x7fff8af84ca3 __CFXNotificationCenterSetupConnection + 387 4 com.apple.CoreFoundation0x7fff8af84b11 __CFXNotificationCenterCreate + 273 5 com.apple.CoreFoundation0x7fff8af849ea __CFNotificationCenterGetDistributedCenter_block_invoke_1 + 26 6 libdispatch.dylib 0x7fff8b14d224 dispatch_once_f + 53 7 com.apple.CoreFoundation0x7fff8af73e0a CFNotificationCenterGetDistributedCenter + 74 8 com.apple.CoreFoundation0x7fff8afb2eb8 CFXPreferencesGetSourceForTriplet_block_invoke_1 + 40 9 libdispatch.dylib 0x7fff8b14d224 dispatch_once_f + 53 10 com.apple.CoreFoundation0x7fff8af7fa4a __CFXPreferencesGetSourceForTriplet + 58 11 com.apple.CoreFoundation0x7fff8af8be67 __CFXPreferencesGetSearchListForBundleID + 215 12 com.apple.CoreFoundation0x7fff8af8bd48 ___CFXPreferencesCopyAppValue_block_invoke_1 + 24 13 com.apple.CoreFoundation0x7fff8af8bcea CFPreferencesCopyAppValue + 218 14 com.apple.SystemConfiguration 0x7fff928f388c SCDynamicStoreCopyProxies + 43 15 _scproxy.so 0x00010ceae9ba 0x10ceae000 + 2490 16 org.python.python 0x00010c651b58 PyEval_EvalFrameEx + 13318 17 org.python.python 0x00010c654df7 0x10c5ca000 + 568823 18 org.python.python 0x00010c651e0a PyEval_EvalFrameEx + 14008 19 org.python.python 0x00010c654df7 0x10c5ca000 + 568823 20 org.python.python 0x00010c651e0a PyEval_EvalFrameEx + 14008 21 org.python.python 0x00010c654cd8 PyEval_EvalCodeEx + 1996 22 org.python.python 0x00010c5f2abf 0x10c5ca000 + 166591 23 org.python.python 0x00010c5d1d32 PyObject_Call + 97 24 org.python.python 0x00010c5e06e9 0x10c5ca000 + 91881 25 org.python.python 0x00010c5d1d32 PyObject_Call + 97 26 org.python.python 0x00010c64dc40 PyEval_CallObjectWithKeywords + 180 27 org.python.python 0x00010c5dd489 PyInstance_New + 273 28 org.python.python 0x00010c5d1d32 PyObject_Call + 97 29 org.python.python 0x00010c651f63 PyEval_EvalFrameEx + 14353 30 org.python.python 0x00010c654cd8 PyEval_EvalCodeEx + 1996 31 org.python.python 0x00010c654e6c 0x10c5ca000 + 568940 32 org.python.python 0x00010c651e0a PyEval_EvalFrameEx + 14008 33 org.python.python 0x00010c654cd8 PyEval_EvalCodeEx + 1996 34 org.python.python 0x00010c654e6c 0x10c5ca000 + 568940 35 org.python.python 0x00010c651e0a PyEval_EvalFrameEx + 14008 36 org.python.python 0x00010c654cd8 PyEval_EvalCodeEx + 1996 37 org.python.python 0x00010c5f2abf 0x10c5ca000 + 166591 38 org.python.python 0x00010c5d1d32 PyObject_Call + 97 39 org.python.python 0x00010c6525ec PyEval_EvalFrameEx + 16026 40 org.python.python 0x00010c654df7 0x10c5ca000 + 568823 41 org.python.python
[issue13826] Having a shlex example in the subprocess.Popen docs is confusing
Julian Berman julian+python@grayvines.com added the comment: Sounds reasonable to me. I'll take a look at adding one unless someone manages to beat me to it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13826 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13830] codecs error handler is called with a UnicodeDecodeError with the same args
New submission from Amaury Forgeot d'Arc amaur...@gmail.com:
The script below shows that the error handler is always called with the same
error object. The 'start', 'end', and 'reason' properties are correctly
updated, but the 'args' is always the same and holds the values used for the
first call.
It's a bit weird that error.args[2] is not equal to error.start, for example.
All versions are affected: 2.7, 3.2, 3.3.
And by the way, I could not find where these are attributes documented.
def custom_handler(error):
print(error.args,
(error.start, error.end, error.reason))
return b'?'.decode(), error.end
import codecs
codecs.register_error('custom', custom_handler)
b'\x80\xd0'.decode('utf-8', 'custom')
--
components: Unicode
messages: 151650
nosy: amaury.forgeotdarc, ezio.melotti
priority: normal
severity: normal
status: open
title: codecs error handler is called with a UnicodeDecodeError with the same
args
type: behavior
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13830
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
New submission from Faheem Mitha fah...@faheem.info: The documentation in http://docs.python.org/library/multiprocessing.html#module-multiprocessing.pool says class multiprocessing.pool.AsyncResult¶ The class of the result returned by Pool.apply_async() and Pool.map_async(). get([timeout]) Return the result when it arrives. If timeout is not None and the result does not arrive within timeout seconds then multiprocessing.TimeoutError is raised. If the remote call raised an exception then that exception will be reraised by get(). Consider the example code from multiprocessing import Pool def go(): print 1 raise Exception(foobar) print 2 p = Pool() x = p.apply_async(go) x.get() p.close() p.join() ### The traceback from this is Traceback (most recent call last): File stdin, line 10, in module File /usr/lib/python2.6/multiprocessing/pool.py, line 422, in get raise self._value Exception: foobar 1 As is clear in this example, this is not a full traceback - it only shows the traceback to the line where get is located and gives no further information. This is the case in all the other places I have used get. It seems to me that it *should* return the full traceback, which may contain important information missing in such a partial one. I don't know whether one would call this a feature request or a bug report. Maybe there is some technical reason why this is not possible, but I can't think of one. -- components: Library (Lib) messages: 151651 nosy: fmitha priority: normal severity: normal status: open title: get method of multiprocessing.pool.Async should return full traceback type: enhancement versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13832] tokenization assuming ASCII whitespace; missing multiline case
New submission from Jim Jewett jimjjew...@gmail.com: Parser/parsetok.c was recently changed (e.g. http://hg.python.org/cpython/rev/2bd7f40108b4 ) to raise an error if multiple statements were found in a single-statement compile call. It sensibly ignores trailing whitespace and comments. Unfortunately, (1) It looks only at (c == ' ' || c == '\t' || c == '\n' || c == '\014') as opposed to using Py_UNICODE_ISSPACE(ch) (2) It assumes that a # means the rest of the line is OK, instead of looking for additional linebreaks. Not sure whether to mark this a bug or an enhancement, since it is already strictly better than the 3.2 behavior of never warning about extra text. -- components: Interpreter Core messages: 151652 nosy: Jim.Jewett priority: normal severity: normal status: open title: tokenization assuming ASCII whitespace; missing multiline case versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13832 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11948] Tutorial/Modules - small fix to better clarify the modules search path
Terry J. Reedy tjre...@udel.edu added the comment: Definitely out-of-scope for the tutorial. I consider this akin to monkey patching imported modules. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11948 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13605] document argparse's nargs=REMAINDER
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 35665f6f3674 by Sandro Tosi in branch '2.7': Issue #13605: add documentation for nargs=argparse.REMAINDER http://hg.python.org/cpython/rev/35665f6f3674 New changeset 6f3d55f5a31e by Sandro Tosi in branch '3.2': Issue #13605: add documentation for nargs=argparse.REMAINDER http://hg.python.org/cpython/rev/6f3d55f5a31e New changeset 6b4cec0719a3 by Sandro Tosi in branch 'default': Issue #13605: merge with 3.2 http://hg.python.org/cpython/rev/6b4cec0719a3 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13605 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13605] document argparse's nargs=REMAINDER
Changes by Sandro Tosi sandro.t...@gmail.com: -- nosy: +sandro.tosi resolution: - fixed stage: needs patch - committed/rejected status: open - closed versions: +Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13605 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13605] document argparse's nargs=REMAINDER
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 1b481e76cd16 by Sandro Tosi in branch '2.7': Issue #13605: more meaningful example + fixes http://hg.python.org/cpython/rev/1b481e76cd16 New changeset d6e53d1f46eb by Sandro Tosi in branch '3.2': Issue #13605: more meaningful example + fixes http://hg.python.org/cpython/rev/d6e53d1f46eb New changeset 4c3271527794 by Sandro Tosi in branch 'default': Issue #13605: merge with 3.2 http://hg.python.org/cpython/rev/4c3271527794 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13605 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13832] tokenization assuming ASCII whitespace; missing multiline case
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13832 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13020] structseq.c: refleak
Torsten Landschoff t.landsch...@gmx.net added the comment: +1 for the patch. All the error paths above the change do Py_DECREF(arg); return NULL; arg is initialized with PySequence_Fast, which returns a new reference. Hard to create a test case for this... -- nosy: +torsten ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13020 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13833] No documentation for PyStructSequence
New submission from Torsten Landschoff t.landsch...@gmx.net: While writing a C extension I wanted to create a namedtuple like object as os.statvfs and friends do. I was unable to find a simple way to do this from C and was wondering how the posixmodule does it. It turned out that there is a PyStructSequence type for this. It would be nice to have it documented. First draft for the documentation update attached. -- assignee: docs@python components: Documentation files: structseq_doc.diff keywords: patch messages: 151657 nosy: docs@python, torsten priority: normal severity: normal status: open title: No documentation for PyStructSequence type: enhancement versions: Python 3.3 Added file: http://bugs.python.org/file24280/structseq_doc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13833 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13832] tokenization assuming ASCII whitespace; missing multiline case
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 00c4efbf57c3 by Benjamin Peterson in branch 'default': check after comments, too (#13832) http://hg.python.org/cpython/rev/00c4efbf57c3 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13832 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13832] tokenization assuming ASCII whitespace; missing multiline case
Benjamin Peterson benja...@python.org added the comment: The tokenizer doesn't consider unicode spaces, either. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13832 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13763] Potentially hard to understand wording in devguide
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 081106c142ec by Terry Jan Reedy in branch 'default': #13763 Clarify 'hg' usage. http://hg.python.org/devguide/rev/081106c142ec -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13763 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13763] Potentially hard to understand wording in devguide
Terry J. Reedy tjre...@udel.edu added the comment: I do not much like sentences starting with lowercase, so I combined our sentences. The result is good enough, I think. Closing. -- assignee: - terry.reedy resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13763 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment:
Frank Sievertsen wrote:
Frank Sievertsen pyt...@sievertsen.de added the comment:
The suffix only introduces a constant change in all hash values
output, so even if you don't know the suffix, you can still
generate data sets with collisions by just having the prefix.
That's true. But without the suffix, I can pretty easy and efficient guess
the prefix by just seeing the result of a few well-chosen and short
repr(dict(X)). I suppose that's harder with the suffix.
Since the hash function is known, it doesn't make things much
harder. Without suffix you just need hash('') to find out what
the prefix is. With suffix, two values are enough.
Say P is your prefix and S your suffix. Let's say you can get the
hash values of A = hash('') and B = hash('\x00').
With Victor's hash function you have (IIRC):
A = hash('') = P ^ (07) ^ 0 ^ S = P ^ S
B = hash('\x00') = ((P ^ (07)) * 103) ^ 0 ^ 1 ^ S = (P * 103) ^ 1 ^ S
Let X = A ^ B, then
X = P ^ (P * 103) ^ 1
since S ^ S = 0 and 0 ^ Y = Y (for any Y), i.e. the suffix doesn't
make any difference.
For P 50, you can then easily calculate P from X
using:
P = X // 102
(things obviously get tricky once overflow kicks in)
Note that for number hashes the randomization doesn't work at all,
since there's no length or feedback loop involved.
With Victor's approach hash(0) would output the whole seed,
but even if the seed is not known, creating an attack data
set is trivial, since hash(x) = P ^ x ^ S.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13811] In str.format, if invalid fill and alignment are specified, the text of the ValueError message is misleading.
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 5c33ebb50702 by Eric V. Smith in branch 'default': Improve exception text. Closes issue 13811. http://hg.python.org/cpython/rev/5c33ebb50702 -- nosy: +python-dev resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13811 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
STINNER Victor victor.stin...@haypocalc.com added the comment:
Since the hash function is known, it doesn't make things much
harder. Without suffix you just need hash('') to find out what
the prefix is. With suffix, two values are enough.
With my patch, hash('') always return zero. I don't remember who asked
me to do that, but it avoids to leak too easily the secret :-) I wrote
some info how to compute the secret:
http://bugs.python.org/issue13703#msg150706
I don't see how to compute the secret, but it doesn't mean that it is
impossible :-) I suppose that you have to brute force some bits, at
least if you only have repr(dict) which gives only (indirectly) the
lower bits of the hash.
(things obviously get tricky once overflow kicks in)
hash() doesn't overflow: if you know the string, you can run the
algorithm backward. To divide, you can compute 1/103 mod 2^32 (or
mod 2^64): 2021759595 and 16109806864799210091. So x/103 mod 2^32
= x*2021759595 mod 2^32.
See my invert_mod() function of:
https://bitbucket.org/haypo/misc/src/tip/python/mathfunc.py
With Victor's approach hash(0) would output the whole seed,
but even if the seed is not known, creating an attack data
set is trivial, since hash(x) = P ^ x ^ S.
I suppose that it would be too simple to compute the secret of a
randomized integer hash, so it is maybe better to leave them
unchanged. Using a different secret from strings and integer would not
protect Python against an attack only using integers, but integer keys
are less common than string keys (especially on web applications).
Anyway, I changed my mind about randomized hash: I now prefer counting
collisions :-)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13828] Further improve casefold documentation
Benjamin Peterson benja...@python.org added the comment: It's a bit unfriendly to launch into discussion of compatiblity caseless matching when the new reader probably has no idea what compatibility-equivalence is. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13828 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10278] add time.wallclock() method
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 8502a9236c2e by Victor Stinner in branch 'default': Issue #10278: Be more explicit in tests than wallclock() is monotonic (cannot http://hg.python.org/cpython/rev/8502a9236c2e -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10278 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Changes by Jesús Cea Avión j...@jcea.es: Added file: http://bugs.python.org/file24281/5458412752d5.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Changes by Jesús Cea Avión j...@jcea.es: Added file: http://bugs.python.org/file24282/f86bb02fd8f4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Changes by Jesús Cea Avión j...@jcea.es: Added file: http://bugs.python.org/file24283/f86bb02fd8f4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Changes by Jesús Cea Avión j...@jcea.es: Removed file: http://bugs.python.org/file23920/f73be85b9a7e.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13807] logging.Handler.handlerError() may raise AttributeError in traceback.print_exception()
Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: -- assignee: - vinay.sajip nosy: +vinay.sajip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13807 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Changes by Jesús Cea Avión j...@jcea.es: Removed file: http://bugs.python.org/file23921/43d1a819a63d.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Jesús Cea Avión j...@jcea.es added the comment: Code ready for commit. Please, review. After the first commit, I will try to make it compatible with MacOS and FreeBSD. Current target is Solaris and derivatives (OpenIndiana, for instance) Stan Cox, if you want systemtap compatibility at this moment, please provide a patch. The first patch doesn't need ustack compatibility. In fact, MacOS dtrace, for instance, seems not to have ustack support at all (according to Google), so the feature is very convenient but optional. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Benjamin Peterson benja...@python.org added the comment: I'm -1 on this patch for essentially the same reasons as Charles-François. It introduces a lot of code (and hacks!) in critical pathways of the interpreter. Someone would have to be constantly maintaining and testing it. In return, what do we get? -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12949] Documentation of PyCode_New() lacks kwonlyargcount argument
Changes by Meador Inge mead...@gmail.com: -- keywords: +easy stage: - needs patch versions: -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12949 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Jesús Cea Avión j...@jcea.es added the comment: NOTE to myself: In OpenIndiana we could have this error: dtrace: failed to compile script ./Include/phelper.d: Preprocessor not found In that case, we must install the C preprocessor: pfexec pkg install cpp I confirm that current 3.3 patch works in the 32 bits OpenIndiana Python buildbots, after adding this patch (already available in my mercurial repository, but not in the patch published in this issue): diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py --- a/Lib/test/test_dtrace.py +++ b/Lib/test/test_dtrace.py @@ -173,7 +173,7 @@ # Verify that we are checking: opcodes = set([CALL_FUNCTION, CALL_FUNCTION_VAR, CALL_FUNCTION_KW, CALL_FUNCTION_VAR_KW]) -obj = compile(open(sample).read(), sample, exec) +obj = compile(open(sample, encoding=utf-8).read(), sample, exec) class dump() : def __init__(self) : self.buf = [] Also, of course, the user must have dtrace permissions, as documented in a previous message. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12949] Documentation of PyCode_New() lacks kwonlyargcount argument
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset b22a35c14a91 by Meador Inge in branch '3.2': Issue #12949: Document the kwonlyargcount argument for the PyCode_New C API function. http://hg.python.org/cpython/rev/b22a35c14a91 New changeset 218b167ff521 by Meador Inge in branch 'default': Issue #12949: Document the kwonlyargcount argument for the PyCode_New C API function. http://hg.python.org/cpython/rev/218b167ff521 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12949 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13806] Audioop decompression frames size check fix
Changes by Oleg Plakhotnyuk oleg...@gmail.com: Removed file: http://bugs.python.org/file24260/audioop_size_check.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13806 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12949] Documentation of PyCode_New() lacks kwonlyargcount argument
Meador Inge mead...@gmail.com added the comment: Fixed. Thanks for the report Stefan. -- nosy: +meador.inge resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12949 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Glyph Lefkowitz gl...@twistedmatrix.com added the comment: Charles-François: Also, I must admit I'm quite skeptical about the real benefit of explicit probes for user-land, especially for CPython which isn't used for performance-critical systems... I beg to differ. CPython is totally used on performance-critical systems, and I know I'm not the only user who thinks that. Performance-critical doesn't necessarily mean goes as fast as it ever possibly can, clearly PyPy is the place to go for that, but can process at least X work in Y time. Meeting performance goals with CPython is already challenging enough, please don't make it artificially hard by refusing to integrate tools which help users understand and improve performance. Benjamin: I'm -1 on this patch for essentially the same reasons as Charles-François. It introduces a lot of code (and hacks!) in critical pathways of the interpreter. Someone would have to be constantly maintaining and testing it. In return, what do we get? You get support for a highly sophisticated and low-impact profiling and tracing technology which provides support for illuminating performance problems *as well as* complicated behavioral problems that only happen under load, without slowing down the interpreter as a whole. Not to mention possible integration with a whole slew of tools that know how to deal with data from that system. I'm not saying that this is necessarily worth the maintenance burden; your analysis of the tradeoff may ultimately be correct. I can't presume to know that because I am not intimately familiar with all the code it touches. But it's definitely not nothing. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13806] Audioop decompression frames size check fix
Oleg Plakhotnyuk oleg...@gmail.com added the comment: Yep, you're right. Didn't noticed audioop_check_size() function at first. The fact that audioop accepts unicode strings seems weird to me too. I've replaced strings with bytes in tests. However, I'm afraid to add restrictions to library itself because of backward compatibility. -- Added file: http://bugs.python.org/file24284/audioop_size_check.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13806 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13806] Audioop decompression frames size check fix
Changes by Oleg Plakhotnyuk oleg...@gmail.com: Removed file: http://bugs.python.org/file24284/audioop_size_check.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13806 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13806] Audioop decompression frames size check fix
Changes by Oleg Plakhotnyuk oleg...@gmail.com: Added file: http://bugs.python.org/file24285/audioop_size_check.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13806 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
John Levon movem...@users.sourceforge.net added the comment: I would strongly suggest those of you who are not getting it to actually try Jesus's patch out in some real scenarios. You'll quickly see what the benefit is. I think some of you are missing that it's a *not* just about performance: it's a system comprehension tool. It's there to help answer questions, on a live system; sometimes those are performance questions, but more often they're *not*. I can't speak as to the maintenance burden. I'm sure we can all agree that the ceval.c changes are not pretty - it's why I never submitted my original changes. But speaking for the Solaris organization, the burden of maintaining our patches across the various Python releases has been minimal. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Benjamin Peterson benja...@python.org added the comment: As great as a tool it maybe, it's still only available on a minority platform. So I couldn't really try it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Dave Malcolm dmalc...@redhat.com added the comment: On Fri, 2012-01-20 at 04:46 +, Benjamin Peterson wrote: Benjamin Peterson benja...@python.org added the comment: As great as a tool it maybe, it's still only available on a minority platform. So I couldn't really try it. FWIW, the analogous systemtap patch works great on Linux, or, at least Fedora 13+/RHEL 6 (though beware: not all Linux distributions have systemtap working properly out of the box, alas). See: http://fedoraproject.org/wiki/Features/SystemtapStaticProbes#Python_2 and http://bugs.python.org/issue4111 though this stalled due to lack of documentation (I started writing some, but got bogged down in the differences between the tapsets and the static markers; sorry). Dave -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Frank Sievertsen pyt...@sievertsen.de added the comment:
That's true. But without the suffix, I can pretty easy and efficient
guess the prefix by just seeing the result of a few well-chosen and
short repr(dict(X)). I suppose that's harder with the suffix.
Since the hash function is known, it doesn't make things much
harder. Without suffix you just need hash('') to find out what
the prefix is. With suffix, two values are enough
This is obvious and absolutely correct!
But it's not what I talked about. I didn't talk about the result of
hash(X), but about the result of repr(dict([(str: val), (str:
val)])), which is more likely to happen and not so trivial
(if you want to know more than the last 8 bits)
IMHO this problem shows that we can't advice dict() or set() for
(potential dangerous) user-supplied keys at the moment.
I prefer randomization because it fixes this problem. The
collision-counting-exception prevents a software from becoming slow,
but it doesn't make it work as expected.
Sure, you can catch the exception. But when you get the exception,
probably you wanted to add the items for a reason: Because you want
them to be in the dict and that's how your software works.
Imagine an irc-server using a dict to store the connected users, using
the nicknames as keys. Even if the irc-server catches the unexpected
exception while connecting a new user (when adding his/her name to the
dict), an attacker could connect 999 special-named users to prevent a
specific user from connecting in future.
Collision-counting-exception can make it possible to inhibit a
specific future add to the dict. The outcome is highly application
dependent.
I think it fixes 95% of the attack-vectors, but not all and it adds a
few new risks. However, of course it's much better then doing nothing
to fix the problem.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13703
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
