>>>>> "Steve" == Steve Roggenkamp <[EMAIL PROTECTED]> writes:
Steve> I currently working on a database which consists mainly of
Steve> integers. I have to search large arrays of these integers
Steve> which may contain over a hundred million entries. With
Steve> this amount of data I'm looking at ways to compress it to a
Steve> minimum amount of physical space. I also do not want to
Steve> have to parse a string to read in an integer. Both
Steve> computational performance and space would suffer too much.
Steve> Thus the need for storing binary data.
Steve> I have considered using `mmap', but I think the database
Steve> will eventually outgrow the address space of a single
Steve> process.
If the representation is simple enough, you might consider accessing
the database via C (which will avoid problems with lazy closures and
with ghc trying to garbage collect the data) and using mmalloc (a
malloc based on mmap) to manage memory. You don't have to run it on a
32-bit box - an Alpha would give you plenty of room!
Of course, Binary or Native would be easier, but the start-up time
for a large database might be a problem.
Tim
--
----------------------------------------------------------------------
T.R.BARBOUR Email : [EMAIL PROTECTED]
----------------------------------------------------------------------
Department of Computer Science
The University of Melbourne
Parkville, Victoria 3052
Australia
----------------------------------------------------------------------
