Good evening, We are trying to build a C++ deduper application using the latest
sqlite release. Our deduper runs in two phases. In the first phase it reads
the records to be deduped from a Microsoft DBF file and writes the records into
sqlite wal database. While writing the records to to the sqlite database we do
a commit every 10000 records. We think we understand that we also need to do a
sqlite pragma wal_checkpoint everytime we do a sqlite database commit.
However, we would like to know whether to use
SQLITE_CHECKPOINT_PASSIVE or SQLITE_CHECKPOINT_FULL or
SQLITE_CHECKPOINT_RESTART in our pragma wal_checkpoint. We are using the
asumption that now will be no sqlite database readers while we are writing to
the sqlite wal database. If we have to write a busy-handler function in C/C++,
how do we do it?
In the second phase, we read the sqlite WAL database and try to find out
the duplicates in our input records. Here, we are only reading the sqlite WAL
database. We would like to find out how to optimize the read performance of the
sqlite WAL database during the second phase of deduping? Please let us know if
you have any suggestions. Thank you.
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
