On 5/18/06, Brannon King <[EMAIL PROTECTED]> wrote:
The statement CREATE UNIQUE INDEX primkey ON 4dData (x,y,z) is too slow for me. It takes 13 sec on 700k records on my Athlon XP 2GHz. That's about four seconds longer than it took me to put the data in there. The two times together are still faster than inserting into a table with a primary key already declared. Is there some way I can create that unique index without the overhead of uniqueness checking? I know all my triples are unique when used together: I generated them that way. Or is there some other way I can create that index faster? Thanks. ______________________________ Brannon King ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Brannon, You are getting almost 54K index entries per second using sqlite. How fast do you need to go? In another post you said: I need to do billions (no joke) of consecutive inserts into this table.
Writing binary out with std::ofstream was using 26% of my code execution time according to the profiler. I changed to sqlite3, and now the sqlite3_step function uses 50% of my execution time. After reading the news groups, I removed the primary key. That dropped it down to about 41%. That was significant. However, I was still significantly slower than binary writes with ofstream.
A more general question is why are you trying to use sqlite? If you need the maximum possible speed you may be better off using in memory vectors and maps and highly tuned routines like the C++ STL algorithms instead of sqlite. Then you will can go back to using binary file I/O. SQLite adds overhead above and beyond binary file I/O, it will always be slower. If the benefits of sqlite don't outweigh the costs, you should stick with binary I/O. What benefit are you hoping to get out of using sqlite? Dennis Cote
