Andreas Eisele schrieb am Donnerstag, 30. März 2023 um 11:16:02 UTC+2:
> I sometimes make use of the fact that the built-in pow() function has an
> optional third argument for modulo calculation, which is handy when dealing
> with tasks from number theory, very large numbers, prob
I sometimes make use of the fact that the built-in pow() function has an
optional third argument for modulo calculation, which is handy when dealing
with tasks from number theory, very large numbers, problems from Project Euler,
etc. I was unpleasantly surprised that math.pow() does not have
.
Thanks a lot for your consideration, and best regards,
Andreas
--
Dr. Andreas Eisele,Senior Researcher
DFKI GmbH, Language Technology Lab, [EMAIL PROTECTED]
Saarland UniversityComputational Linguistics
Stuhlsatzenhausweg 3 tel: +49-681-302-5285
D-66123 Saarbrücken
I should have been more specific about possible fixes.
python2.5 -m timeit 'gc.disable();l=[(i,) for i in range(200)]'
10 loops, best of 3: 662 msec per loop
python2.5 -m timeit 'gc.enable();l=[(i,) for i in range(200)]'
10 loops, best of 3: 15.2 sec per loop
In the latter
Sorry, I have to correct my last posting again:
Disabling the gc may not be a good idea in a real application; I suggest
you to play with the gc.set_threshold function and set larger values, at
least while building the dictionary. (700, 1000, 10) seems to yield good
results.
python2.5
Martin said that the default settings for the cyclic gc works for most
people.
I agree.
Your test case has found a pathologic corner case which is *not*
typical for common application but typical for an artificial benchmark.
I agree that my corner is not typical, but I strongly disagree
Andreas Eisele [EMAIL PROTECTED] added the comment:
Great, that really solves my problem.
Thank you so much, Amaury!
As you say, the problem is unrelated to dicts,
and I observe it also when including the tuples to
a set or keeping them in lists.
Perhaps your GC thresholds would be better
Andreas Eisele [EMAIL PROTECTED] added the comment:
Even if they mean that creation
of a huge number N of objects
requires O(N*N) effort?
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2607
Andreas Eisele [EMAIL PROTECTED] added the comment:
Sorry for not giving a good example in the first place.
The problem seems to appear only in the presence of
sufficiently many distinct tuples. Then I see performance
that looks rather like O(n*n)
Here is an example that shows the problem
Andreas Eisele added the comment:
How do you run the test? Do you specify a maximum available size?
I naively assumed that running make test from the toplevel would be
clever about finding plausible parameters. However, it runs the bigmem
tests in a minimalistic way, skipping essentially all
Andreas Eisele added the comment:
Tried
@bigmemtest(minsize=_2G*2+2, memuse=3)
but no change; the test is done only once with a small
size (5147). Apparently something does not work as
expected here. I'm trying this with 2.6 (Revision 59231).
__
Tracker
Andreas Eisele added the comment:
Thanks a lot for the patch, which indeed seems to solve the issue.
Alas, the extended test code still does not catch the problem, at
least in my installation. Someone with a better understanding of
how these tests work and with access to a 64bit machine should
Andreas Eisele added the comment:
Then 7G is enough for the test to run.
yes, indeed, thanks for pointing this out.
It runs and detects an ERROR, and after applying your patch it succeeds.
What else needs to be done to make sure your patch finds it's way to the
Python core
New submission from Andreas Eisele:
s.decode(utf-8)
sometimes silently truncates the result if s has more than 2E9 Bytes,
sometimes raises a fairly incomprehensible exception:
Traceback (most recent call last):
File stdin, line 2, in module
File /usr/lib64/python2.5/encodings/utf_8.py
Andreas Eisele added the comment:
For instance:
Python 2.5.1 (r251:54863, Aug 30 2007, 16:15:51)
[GCC 4.1.0 (SUSE Linux)] on linux2
Type help, copyright, credits or license for more information.
__[1] s= *int(5E9)
6.05 sec
__[1] u=s.decode(utf-8)
4.71 sec
__[1] len(u)
705032704
Andreas Eisele added the comment:
An instance of the other problem:
Python 2.5.1 (r251:54863, Aug 30 2007, 16:15:51)
[GCC 4.1.0 (SUSE Linux)] on linux2
Type help, copyright, credits or license for more information.
__[1] s= *int(25E8)
2.99 sec
__[1] u=s.decode(utf-8)
Traceback (most
16 matches
Mail list logo
