On May 10, 2007, at 3:04 PM, [EMAIL PROTECTED] wrote:
Kasper Daniel Hansen <[EMAIL PROTECTED]> wrote:
Hi
We are using SQLite for a fairly big (but simple) calculation, and
have some problems when creating an index on a database with
600.000.000 rows. Specifically it has not ended even after 5 days of
running. We have done it successfully on 25% of the full data base,
and are now wondering what we can do.
This is likely a locality of reference problem. It comes up
about every two weeks on this mailing list. You can read more
about it by looking at the mailing list archives.
I think I know how to fix the problem. But I have not yet had
an opportunity to implement the fix.
Thank you for that pointer. I have now read up a bit on the
references. From what I understand it has to do with the ordering of
the data and the disk cache and so on.
I could without too much trouble (although it will take some time)
sort the data in a way such that it is sorted in the right way for
the index (by right I mean if I create a (VAR1, VAR2) index I want
VAR2 to be sorted within VAR1 in increasing order). Would that help?
Or does the UNIQUE keyword make sqlite do some checking that will
slow down everything.
Having said that, I can understand that sorting and disk cache and so
on factors in - but my initial database is already very large (1.3GB
- 145.000.000 milion rows), and surely that disk cache would already
factor in at that stage?
Thanks, Kasper
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
