I tried it with numpy and it was faster without it. I think most of the time
consumed is in getting the memory from the operating system.
I tried it in Delphi but kept on getting memory probs. Couldn't get it to
finish.
Perhaps the best way to approach it with a new generator or simply a method
where you pass the index and it returns the contents of the virtual table.
Attached is a small set of tests that go about the problem in different
ways, but I couldn't get anywhere near the GBA time. I think that GBA gets
around it by forward checking that the certain members of the array aren't
read so, it doesn't allocate memory for them? Is the GBA app running on
windows?
It must be to do with memory allocation. Any other ideas?
Matthew Sherborne
----- Original Message -----
From: "Bob Kline" <[EMAIL PROTECTED]>
To: "Ed Hopkins" <[EMAIL PROTECTED]>
Cc: <[EMAIL PROTECTED]>
Sent: Saturday, May 11, 2002 5:50 AM
Subject: Re: Conversion from GFA Basic to Python for SpeedTest
> On Tue, 9 Apr 2002, Bob Kline wrote:
>
> > On Tue, 9 Apr 2002, Ed Hopkins wrote:
> >
> > >
> > > The result of this code:
> > >
> > > starting time : 10:06:30
> > > finished time: 10:06:41
> > > i = 8145060
> > > 8145060 40 41 42 43 44 45
> > >
> > > Within 11 seconds all 8145060 combinations of 6 of 45 have been
> > > written into a table together with a line identification number at
> > > position R(i,7) , at a 800 MHz computer with 195 MB memory.
> >
> > I'll be curious to see what someone more familiar with NumPy (I assume
> > David A. reads this list) comes up with for Python, but this is the sort
> > of work I'd be more include to do using Fortran or C++. In C++, on my
> > 450MHz machine, with no compiler optimization options turned on:
> >
> > starting time: Tue Apr 09 05:10:08 2002
> > finished time: Tue Apr 09 05:10:10 2002
> > i = 8145060
> > 8145060 40 41 42 43 44 45
>
> Or even better, in Fortran:
>
> Fri May 10 13:34:57 2002
> Fri May 10 13:34:57 2002
> 8145060 40 41 42 43 44 45
>
> I'd still like to know how the NumPy experts would approach this
> problem. David Ascher: did you ever get a chance to look at this
> poster's original question? Seems like just the sort of thing a
> numerical extension package for a progamming language would be at home
> with.
>
> --
> Bob Kline
> mailto:[EMAIL PROTECTED]
> http://www.rksystems.com
>
> _______________________________________________
> ActivePython mailing list
> [EMAIL PROTECTED]
> To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
> Other options:
http://listserv.ActiveState.com/mailman/listinfo/ActivePython
>
>
>
import time
from Numeric import *
from struct import pack, unpack
from mmap import mmap
def doTest1():
print
print 'Numeric, getting all the mem at the start'
start = time.time()
R = zeros((8145060, 7))
mid = time.time()
i = 0
for a in range(1, 40 + 1):
for b in range( a + 1, 41 + 1):
for c in range( b + 1, 42 + 1):
for d in range( c + 1, 43 + 1):
for e in range( d + 1, 44 + 1):
for f in range( e + 1, 45 + 1):
R[i] = a,b,c,d,e,f,i
i += 1
end = time.time()
print "Seconds to get memory", mid - start
print "Seconds to fill", end - mid
print "Total", end - start
print "i =", i
print "Last row", R[-1]
def doTest2():
print
print 'Numeric, resize as we go'
start = time.time()
R = zeros((1,7))
mid = time.time()
i = 0
for a in range(1, 40 + 1):
for b in range( a + 1, 41 + 1):
for c in range( b + 1, 42 + 1):
for d in range( c + 1, 43 + 1):
for e in range( d + 1, 44 + 1):
for f in range( e + 1, 45 + 1):
R[i] = a,b,c,d,e,f,i
i += 1
R.resize((i+1, 7))
#if i % 10000 == 0: print i
end = time.time()
print "Seconds to get memory", mid - start
print "Seconds to fill", end - mid
print "Total", end - start
print "i =", i
print "Last row", R[-1]
def doTest3():
print
print 'Normal, getting all the mem at the start'
start = time.time()
R = [(0,0,0,0,0,0,0)] * 8145060
mid = time.time()
i = 0
for a in range(1, 40 + 1):
for b in range( a + 1, 41 + 1):
for c in range( b + 1, 42 + 1):
for d in range( c + 1, 43 + 1):
for e in range( d + 1, 44 + 1):
for f in range( e + 1, 45 + 1):
R[i] = a,b,c,d,e,f,i
i += 1
#if i % 10000 == 0: print i
end = time.time()
print "Seconds to get memory", mid - start
print "Seconds to fill", end - mid
print "Total", end - start
print "i =", i
print "Last row", R[-1]
def doTest4():
print
print 'Normal, getting mem as we go'
start = time.time()
R = []
mid = time.time()
i = 0
for a in range(1, 40 + 1):
for b in range( a + 1, 41 + 1):
for c in range( b + 1, 42 + 1):
for d in range( c + 1, 43 + 1):
for e in range( d + 1, 44 + 1):
for f in range( e + 1, 45 + 1):
R.append((a,b,c,d,e,f,i))
i += 1
#if i % 10000 == 0: print i
end = time.time()
print "Seconds to get memory", mid - start
print "Seconds to fill", end - mid
print "Total", end - start
print "i =", i
print "Last row", R[-1]
def doTest5():
print
print 'File based'
start = time.time()
R = open('data.dat', 'wb')
mid = time.time()
i = 0
for a in range(1, 40 + 1):
for b in range( a + 1, 41 + 1):
for c in range( b + 1, 42 + 1):
for d in range( c + 1, 43 + 1):
for e in range( d + 1, 44 + 1):
for f in range( e + 1, 45 + 1):
R.write(pack('BBBBBBI', a,b,c,d,e,f,i))
i += 1
#if i % 10000 == 0: print i
end = time.time()
print "Seconds to get memory", mid - start
print "Seconds to fill", end - mid
print "Total", end - start
print "i =", i
R.close()
R = open('data.dat', 'rb')
R.seek(8145059 * 7)
dat = R.read(7)
a,b,c,d,e,f,i = unpack('BBBBBBI')
print "Last row", R[-1]
def doTest6():
print
print 'mmap based'
start = time.time()
R = mmap(0, (8145060 * 6) + (8145060 * 7))
mid = time.time()
i = 0L
for a in range(1, 40 + 1):
for b in range( a + 1, 41 + 1):
for c in range( b + 1, 42 + 1):
for d in range( c + 1, 43 + 1):
for e in range( d + 1, 44 + 1):
for f in range( e + 1, 45 + 1):
R.write(pack('BBBBBBI', a,b,c,d,e,f,i))
i += 1
#if i % 10000 == 0: print i
end = time.time()
print "Seconds to get memory", mid - start
print "Seconds to fill", end - mid
print "Total", end - start
print "i =", i
R.seek(8145059 * 7)
dat = R.read(7)
a,b,c,d,e,f,i = unpack('BBBBBBI')
print "Last row", R[-1]
doTest6()
