Hello, Thanks for your suggestion, yup the Speed with Transaction is increase multifold. In 5 minute it read half the data. But wondering what different the Transaction make as compare to normal insert query ? Why is it faster here. I didn't change anything, just wrap the Insert query or rather my read Binary Loop in Sqlite transaction only.
Any guidance is great. Looks like Sqlite finally solve my problem thus far :). With Regards, Sumit Gupta -----Original Message----- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Black, Michael (IS) Sent: 12 August 2011 17:48 To: General Discussion of SQLite Database Subject: Re: [sqlite] Bulk Insert It doesn't hold them in memory...they are still written to disk and just rolled back if you abort the transaction. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate ________________________________ From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Sumit Gupta [gamersu...@gmail.com] Sent: Friday, August 12, 2011 7:15 AM To: 'General Discussion of SQLite Database' Subject: EXT :Re: [sqlite] Bulk Insert But having Transaaction to hold 10M records in memory, well not sure if that help ? Sumit -----Original Message----- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Stephan Beal Sent: 12 August 2011 17:44 To: General Discussion of SQLite Database Subject: Re: [sqlite] Bulk Insert On Fri, Aug 12, 2011 at 2:09 PM, Sumit Gupta <gamersu...@gmail.com> wrote: > query is only reading about 400-500 record per minute. This way it is going > If you are not currently using a transaction to wrap the whole import, add a transaction and the speed will improve a great deal. -- ----- stephan beal http://wanderinghorse.net/home/stephan/ _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
