[issue2607] why is (default)dict so slow on tuples???
Andreas Eisele [EMAIL PROTECTED] added the comment: Great, that really solves my problem. Thank you so much, Amaury! As you say, the problem is unrelated to dicts, and I observe it also when including the tuples to a set or keeping them in lists. Perhaps your GC thresholds would be better default values than what is currently in force. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2607 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2607] why is (default)dict so slow on tuples???
Andreas Eisele [EMAIL PROTECTED] added the comment: Even if they mean that creation of a huge number N of objects requires O(N*N) effort? __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2607 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2607] why is (default)dict so slow on tuples???
Andreas Eisele [EMAIL PROTECTED] added the comment:
Sorry for not giving a good example in the first place.
The problem seems to appear only in the presence of
sufficiently many distinct tuples. Then I see performance
that looks rather like O(n*n)
Here is an example that shows the problem:
from time import clock
d = {}
t0 = clock()
for i in range(5):
for j in range(i*100,(i+1)*100):
d[str(j),str(j)]=j
print clock()-t0
13.04
39.51
81.86
134.18
206.66
The same example with str(j)+str(j) works fine.
Sorry if this should be a non-issue. For me it is a
reason to implement functionality in C or Perl
that I would really love to do in Python.
I would call such a thing a performance bug, but
maybe I'm just too demanding...
Best regards,
Andreas
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2607
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1521] string.decode() fails on long strings
Andreas Eisele added the comment: How do you run the test? Do you specify a maximum available size? I naively assumed that running make test from the toplevel would be clever about finding plausible parameters. However, it runs the bigmem tests in a minimalistic way, skipping essentially all interesting bits. Thanks for the hints on giving the maximal available size explicitly, which work in principle, but make testing rather slow. Also, if the encode/decode test are decorated with @bigmemtest(minsize=_2G*2+2, memuse=3) one needs to specify at least -M 15g, otherwise the tests are still skipped. No wonder that people do not normally run them... __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1521 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1521] string.decode() fails on long strings
Andreas Eisele added the comment: Tried @bigmemtest(minsize=_2G*2+2, memuse=3) but no change; the test is done only once with a small size (5147). Apparently something does not work as expected here. I'm trying this with 2.6 (Revision 59231). __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1521 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1521] string.decode() fails on long strings
Andreas Eisele added the comment: Thanks a lot for the patch, which indeed seems to solve the issue. Alas, the extended test code still does not catch the problem, at least in my installation. Someone with a better understanding of how these tests work and with access to a 64bit machine should still have a look. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1521 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1521] string.decode() fails on long strings
Andreas Eisele added the comment: Then 7G is enough for the test to run. yes, indeed, thanks for pointing this out. It runs and detects an ERROR, and after applying your patch it succeeds. What else needs to be done to make sure your patch finds it's way to the Python core? __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1521 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1521] string.decode() fails on long strings
New submission from Andreas Eisele: s.decode(utf-8) sometimes silently truncates the result if s has more than 2E9 Bytes, sometimes raises a fairly incomprehensible exception: Traceback (most recent call last): File stdin, line 2, in module File /usr/lib64/python2.5/encodings/utf_8.py, line 16, in decode return codecs.utf_8_decode(input, errors, True) TypeError: utf_8_decode() argument 1 must be (unspecified), not str -- components: Unicode messages: 57932 nosy: eisele severity: normal status: open title: string.decode() fails on long strings type: behavior versions: Python 2.5 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1521 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1521] string.decode() fails on long strings
Andreas Eisele added the comment: For instance: Python 2.5.1 (r251:54863, Aug 30 2007, 16:15:51) [GCC 4.1.0 (SUSE Linux)] on linux2 Type help, copyright, credits or license for more information. __[1] s= *int(5E9) 6.05 sec __[1] u=s.decode(utf-8) 4.71 sec __[1] len(u) 705032704 __[2] len(s) 50 __[3] I would have expected both lengths to be 5E9 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1521 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1521] string.decode() fails on long strings
Andreas Eisele added the comment: An instance of the other problem: Python 2.5.1 (r251:54863, Aug 30 2007, 16:15:51) [GCC 4.1.0 (SUSE Linux)] on linux2 Type help, copyright, credits or license for more information. __[1] s= *int(25E8) 2.99 sec __[1] u=s.decode(utf-8) Traceback (most recent call last): File stdin, line 1, in module File /home/cl-home/eisele/lns-root-07/lib/python2.5/encodings/utf_8.py, line 16, in decode return codecs.utf_8_decode(input, errors, True) TypeError: utf_8_decode() argument 1 must be (unspecified), not str __[1] __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1521 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
