Re: [sqlite] SQLite version 3.7.6 testing

2011-02-13 Thread Sven L
Lol... we just had a long discussion about sqlite_stat2 and ANALYZE command. I removed these from my code, since the command should not be used except for rare cases :P Should I re-add my ANALYZE-code and ask my original question again? Confused :P From: d...@sqlite.org Date: Sat, 12 Feb

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Jean-Christophe Deschamps
So if the drive is 5400 rpm, 227 is much more than 5400/60=90 and even if it's 7200 (manufacturers sometimes upgrade drives inside portable hd without prior notice), it's still twice as much as 7200/60=120. 5400/60, 7200/60 ... those values rely on the assumption that successive LBAs are

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Jim Wilcoxson
On Sun, Feb 13, 2011 at 1:15 AM, Max Vlasov max.vla...@gmail.com wrote: On Sat, Feb 12, 2011 at 7:11 PM, Jim Wilcoxson pri...@gmail.com wrote: Unless I'm missing something, SQLite has to update the first page of the database on every commit, to update the change counter. Assuming you

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Jim Wilcoxson
On Sun, Feb 13, 2011 at 7:07 AM, Jean-Christophe Deschamps j...@antichoc.net wrote: So if the drive is 5400 rpm, 227 is much more than 5400/60=90 and even if it's 7200 (manufacturers sometimes upgrade drives inside portable hd without prior notice), it's still twice as much as 7200/60=120.

[sqlite] Query optimizer with attached databases

2011-02-13 Thread Felix Schlesinger
Hello, I have come across several situations in which the sqlite query optimizer chooses clearly suboptimal query plans when a query spans several attached databases. If I copy all involved tables into one sqlite database file, a better query plan is chosen. The difference can be quite dramatic

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Max Vlasov
On Sun, Feb 13, 2011 at 5:31 PM, Jim Wilcoxson pri...@gmail.com wrote: On Sun, Feb 13, 2011 at 1:15 AM, Max Vlasov max.vla...@gmail.com wrote: So the final results: - the db was always ok and contains the correct value (id=10001 for initial 1). - the speed was about 227 commits per

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Jean-Christophe Deschamps
But what I postulate is that you can't physically write *the same* record over and over more than 90 times per second on a 5400 rpm drive, unless the drive, OS, or filesystem implements something like wear-leveling, where the physical location of sectors is constantly changing. It's still

[sqlite] basic rtree question

2011-02-13 Thread Puneet Kishor
I have a simple table like so CREATE TABLE points ( id INTEGER PRIMARY KEY, lon REAL, lat REAL, tile INTEGER ); It has about 13.25 million rows. I want to be able to return rows given a bounding box (min_lon, min_lat, max_lon, max_lat). Is the following the right strategy? CREATE VIRTUAL

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Jim Wilcoxson
On Sun, Feb 13, 2011 at 11:55 AM, Max Vlasov max.vla...@gmail.com wrote: On Sun, Feb 13, 2011 at 5:31 PM, Jim Wilcoxson pri...@gmail.com wrote: On Sun, Feb 13, 2011 at 1:15 AM, Max Vlasov max.vla...@gmail.com wrote: So the final results: - the db was always ok and contains the correct

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Simon Slavin
On 13 Feb 2011, at 5:53pm, Jim Wilcoxson wrote: I also found this page, used to force a Linux system crash: http://evuraan.blogspot.com/2005/06/how-to-force-paniccrash-in-linux.html I seem to remember a post that SQLite commit/sync is tested with the kill command, but it seems like

Re: [sqlite] basic rtree question

2011-02-13 Thread Jay A. Kreibich
On Sun, Feb 13, 2011 at 11:47:48AM -0600, Puneet Kishor scratched on the wall: I have a simple table like so CREATE TABLE points ( id INTEGER PRIMARY KEY, lon REAL, lat REAL, tile INTEGER ); It has about 13.25 million rows. I want to be able to return rows given a bounding box

Re: [sqlite] basic rtree question

2011-02-13 Thread Puneet Kishor
On Sunday, February 13, 2011 at 4:15 PM, Jay A. Kreibich wrote: I have a simple table like so CREATE TABLE points ( id INTEGER PRIMARY KEY, lon REAL, lat REAL, tile INTEGER ); It has about 13.25 million rows. I want to be able to return rows given a bounding box (min_lon,

[sqlite] Question about having 2 db connections on same thread

2011-02-13 Thread jeff archer
I thought this would be OK but now that I am seeing it fail I am having my doubts.  I have created 2 db connections on the same thread, both on the same file.  i.e. 2 calls to sqlite3_open16() (use exact same filename both times) have created 2 sqlite3 instances.  I do this because I have 2

Re: [sqlite] Speed up count(distinct col)

2011-02-13 Thread BareFeetWare
On 13/02/2011, at 1:04 AM, Yuzem wrote: I am grabbing the data from the each movie imdb webpage. Does IMDB allow use of their data this way? After my brief reading of their site, I thought they charge a $15k minimum per year for data. The script is written in bash and I can give you the code

Re: [sqlite] Speed up count(distinct col)

2011-02-13 Thread Random Coder
On Feb 13, 2011, at 3:38 PM, BareFeetWare list@barefeetware.com wrote: On 13/02/2011, at 1:04 AM, Yuzem wrote: I am grabbing the data from the each movie imdb webpage. Does IMDB allow use of their data this way? After my brief reading of their site, I thought they charge a $15k

Re: [sqlite] Question about having 2 db connections on same thread

2011-02-13 Thread Vannus
Are you not able to use the same db connection in both classes? something like FirstTablesClass-dbConnection = MyDBConnectionClass SecondTablesClass-dbConnection = MyDBConnectionClass On 13 February 2011 23:04, jeff archer jarch...@yahoo.com wrote: I thought this would be OK but now that I am

Re: [sqlite] Question about having 2 db connections on same thread

2011-02-13 Thread Simon Slavin
On 14 Feb 2011, at 12:30am, Vannus wrote: Are you not able to use the same db connection in both classes? something like FirstTablesClass-dbConnection = MyDBConnectionClass SecondTablesClass-dbConnection = MyDBConnectionClass Nevertheless, having two connections should work too. His

Re: [sqlite] Question about having 2 db connections on same thread

2011-02-13 Thread Pavel Ivanov
I do a transaction on connection 1 using BEGIN IMMEDIATE, some rows updated, COMMIT. Then, when I attempt same sequence on connection 2, when I do first call to step to execute begin transaction and it never returns. Do you check result code from COMMIT and is it successful? My guess is you

Re: [sqlite] BUG: capi3e.test when compiled with SQLITE_OMIT_UTF16

2011-02-13 Thread Shane Harrelson
Thanks.Hopefully corrected here: http://www.sqlite.org/src/info/b04304b967 -Shane On Fri, Feb 11, 2011 at 4:59 PM, Noah Hart n...@lipmantpa.com wrote: capi3e.test needs      ifcapable utf16 logic before capi3e-2.1.$i to properly pass tests when compiled with SQLITE_OMIT_UTF16 ~ Noah

Re: [sqlite] basic rtree question

2011-02-13 Thread Jay A. Kreibich
On Sun, Feb 13, 2011 at 04:18:25PM -0600, Puneet Kishor scratched on the wall: On Sunday, February 13, 2011 at 4:15 PM, Jay A. Kreibich wrote: CREATE VIRTUAL TABLE points_rtree USING rtree ( id, min_lon, min_lat, max_lon, max_lat ); Almost. You need to match min/max pairs, so the

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-13 Thread Max Vlasov
On Sun, Feb 13, 2011 at 8:53 PM, Jim Wilcoxson pri...@gmail.com wrote: But I thought about how it would be possible to test this explanation . I'm going to do some tests that works like this. The same id updating, but in the middle of 10,000 operation I will unplug the cord, the sqlite