On Aug 9, 2005, at 20:56, Brian Myers wrote:
Seems to me like it would be easier to just recompile Metakit with
large file support enabled. But maybe Metakit is using internal
structures that would be difficult to alter.
I'd like to know if that's the case. It would mean it's unlikely
Metakit will support large files anytime soon. On the other hand,
if large files were to materialize soon, it would be worth while
just to wait.
Short answer: forget >2Gb files in MK 2.4, it can't be done.
On 32-bit machines, MK does not support >2Gb files because of an
assumption in lots of places that the size must fit in an int. A
while back, I went through the source and concluded that there are a
couple of hairy issues preventing an easy fix.
Even on 64-bit machines, MK would need changes to support >2Gb files
- some in the API but also a change in the file format (there are
precisely three 32-bit ints in various header/footer fields which
need to be extended).
I've solved the file format design issues, and have experimental non-
MK code which confirms it works and remains backwards compatible.
But that is of no use to anyone today.
On Aug 9, 2005, at 11:30 AM, Brian Kelley wrote:
[...]
One really interesting thing about metakit is that you can do
joins on
databases that are stored in seperate files! This won't help you
much, since you still have a total 2GB limit I think.
The 2 Gb limit is due to exhaustion of 32-bit memory address space.
Opening multiple files won't help you get around the fact that only 2
Gb can be mapped (i.e. open) at any one time.
The *only* option I see with a pure MK solution today is to use 64-
bit env, open multiple MK files, each under 2 Gb, and join/concat
them as you describe.
For completeness: a workaround is to try and stay under 2 Gb, by
keeping the larger bits of the data in separate (non-MK) files and
use MK only to manage that data, not contain it. That is not always
a meaningful option - it depends on the application needs.
-jcw
_____________________________________________
Metakit mailing list - [email protected]
http://www.equi4.com/mailman/listinfo/metakit
