[sqlite] After ANALYZE all has become slow

2014-08-07 Thread Giuseppe Costanzi
Hi to everybody, I have the following problem, on a database, 20M on a server linux, when I make ANALYZE all the interrogations become slow. The database is constituted by 37 tables, 56 views and varied indexes. I use python database API 2.0 to talking with clients. Considering that the data base

Re: [sqlite] In-memory DB slower than disk-based?

2014-08-07 Thread Simon Slavin
On 7 Aug 2014, at 3:47am, Jim Callahan jim.callahan.orla...@gmail.com wrote: [various questions] In addition, is this real hardware, or is it a simulated or virtual machine ? Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] After ANALYZE all has become slow

2014-08-07 Thread Simon Slavin
On 7 Aug 2014, at 10:56am, Giuseppe Costanzi giuseppecosta...@gmail.com wrote: Hi to everybody, I have the following problem, on a database, 20M on a server linux, when I make ANALYZE all the interrogations become slow. I'm very sorry, but I'm having a little trouble with your English and

Re: [sqlite] SQLite version 3.8.6 coming soon

2014-08-07 Thread E.Pasma
Op 6 aug 2014, om 02:57 heeft Richard Hipp het volgende geschreven: Version 3.8.6 was originally scheduled for September. But the change log ( http://www.sqlite.org/draft/releaselog/current.html) is getting rather long and there are a number of important bug fixes. So we might try to get

Re: [sqlite] After ANALYZE all has become slow

2014-08-07 Thread Giuseppe Costanzi
On Thu, Aug 7, 2014 at 2:50 PM, Simon Slavin slav...@bigfraud.org wrote: On 7 Aug 2014, at 10:56am, Giuseppe Costanzi giuseppecosta...@gmail.com wrote: Hi to everybody, I have the following problem, on a database, 20M on a server linux, when I make ANALYZE all the interrogations become

Re: [sqlite] After ANALYZE all has become slow

2014-08-07 Thread Simon Slavin
On 7 Aug 2014, at 3:24pm, Giuseppe Costanzi giuseppecosta...@gmail.com wrote: I have finished running ANALYZE and SQLite commands are taking longer only when I use python db api such as [...] however I have done what you have pointed out me and in effects the speed of execution from the

Re: [sqlite] SQLite version 3.8.6 coming soon

2014-08-07 Thread jose isaias cabrera
Richard Hipp wrote... 32-bit and 64-bit DLLs are the latest 3.8.6 beta are now available on the download page. http://www.sqlite.org/download.html All works great. No visible speed increase for WHERE .. IN () as reported by someone, but nonetheless, all is well. Thanks. On Tue, Aug 5,

Re: [sqlite] SQLite version 3.8.6 coming soon

2014-08-07 Thread Will Parsons
Richard Hipp wrote: On Wed, Aug 6, 2014 at 8:33 PM, James K. Lowden jklow...@schemamania.org wrote: On Wed, 6 Aug 2014 07:40:43 -0400 Richard Hipp d...@sqlite.org wrote: PS: backward compatibility, no s, no? A google search shows that you see it both ways - with and without the

[sqlite] Improve query performance using a join

2014-08-07 Thread Paul Dillon
Hello, I would like to know if splitting a big table into two smaller ones, and then using a join in my queries would speed up performance. My table is 100-300GB in size and has about 150 columns. There are 8 fields that I frequently use in my queries, which require full table scans to

[sqlite] sqlite merge

2014-08-07 Thread Paolo Combi
Hello, i have one problem. i have 1 sqlite file on my ipad and when i download this file i find some other version of this file . *.sqlite *.sqlite-shm *-sqlite-wal i see if i use a firefox plugin this file will be merge together and i want to do the same things with php . is possibile? best

Re: [sqlite] Improve query performance using a join

2014-08-07 Thread Simon Slavin
On 7 Aug 2014, at 4:51am, Paul Dillon paul.dil...@gmail.com wrote: 1. Will moving these 8 query fields to a smaller table improve query performance when joined to the larger table? My logic is that this small table would only be about 5% the size of the full table, so the full table scan to

Re: [sqlite] sqlite merge

2014-08-07 Thread Simon Slavin
On 7 Aug 2014, at 2:34pm, Paolo Combi paolo.co...@ipratico.it wrote: i have 1 sqlite file on my ipad and when i download this file i find some other version of this file . *.sqlite *.sqlite-shm *-sqlite-wal i see if i use a firefox plugin this file will be merge together and i want to

Re: [sqlite] sqlite merge

2014-08-07 Thread Richard Hipp
On Thu, Aug 7, 2014 at 9:34 AM, Paolo Combi paolo.co...@ipratico.it wrote: Hello, i have one problem. i have 1 sqlite file on my ipad and when i download this file i find some other version of this file . *.sqlite *.sqlite-shm *-sqlite-wal The -shm and -wal files are journals left over

Re: [sqlite] SQLite version 3.8.6 coming soon

2014-08-07 Thread Richard Hipp
On Thu, Aug 7, 2014 at 10:01 AM, E.Pasma pasm...@concepts.nl wrote: I have a case where a primary key index is no longer used where it was used before. Thank you for the test case! This problem should now be addressed on trunk and in the pre-release snapshots. Please retry using the latest

Re: [sqlite] Improve query performance using a join

2014-08-07 Thread Jim Callahan
A few observations 1. My condolences with those dimensions you are heading for big data/hadoop land. 2. Worry about the number of rows; that's what feeds into the big-oh: O(n). Assuming your 150 columns translate into a 1.5k to 2k record length that means your 300 gigabyte file must have 150

Re: [sqlite] In-memory DB slower than disk-based?

2014-08-07 Thread Jensen, Vern
Jim, The primary table in the database has 5 columns, and has at most about 4,000 entries during stress-testing. (Although in tests involving even only about 600 entries, the results were the same.) There are a few other tables that would be much smaller. Nothing that should stress any

Re: [sqlite] In-memory DB slower than disk-based?

2014-08-07 Thread Jensen, Vern
Real hardware. 27” iMac (native BootCamp running Win 7) with 3.4 GHz Intel Core i7 quad-core processor. And my mistake: 24 GB of RAM, not 20. Anytime I’ve looked at the app while it’s running, it’ll take 5 to 15 MB at most, depending on how many threads I give it. -Vern

Re: [sqlite] SQLite version 3.8.6 coming soon

2014-08-07 Thread Eric Sink
FWIW, the Zumero test suite is fairly abusive and it passes all test cases with 3.8.6 beta. -- E On Thu, Aug 7, 2014 at 11:59 AM, Richard Hipp d...@sqlite.org wrote: On Thu, Aug 7, 2014 at 10:01 AM, E.Pasma pasm...@concepts.nl wrote: I have a case where a primary key index is no longer

Re: [sqlite] In-memory DB slower than disk-based?

2014-08-07 Thread Stephen Chrzanowski
It smells like that you're running into general timing issues, especially if you're consistent with how long it takes to put info into the database. If we were to take your scenario of 100 threads of writing to memory, bashing the hell out of the database for writes, the first thread is going to

Re: [sqlite] After ANALYZE all has become slow

2014-08-07 Thread Keith Medcalf
Giuseppe, What version of the sqlite3 library is python using? python Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import sqlite3 cn = sqlite3.connect(':memory:') sqlite3.version '2.6.0'