[sqlite] Is there available sample featured SQLite database file?

2013-06-28 Thread Sqlite Dog
Hi, we are developing (yet another :) SQLite manager. I'm wondering is there available sample feature-tight limit-striking SQLite database (or SQL script) for GUI manager testing purposes? Like: - identifier names contains spaces, UNICODE chars, are surrounded by [], by or not surrounded; -

Re: [sqlite] Is there available sample featured SQLite database file?

2013-06-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/06/13 01:12, Sqlite Dog wrote: I'm wondering is there available sample feature-tight limit-striking SQLite database (or SQL script) for GUI manager testing purposes? You can certainly measure coverage of sqlite3.c itself when running your

Re: [sqlite] Is there available sample featured SQLite database file?

2013-06-28 Thread Sqlite Dog
Here is your starter schema (and perfectly valid): create table ( ); Thanks for this snippet! Actually found a bug :) For performance testing this could be helpful with the --dump-sql arg: https://code.google.com/p/apsw/source/browse/tools/speedtest.py Well, performance is not an

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Eduardo
On Wed, 26 Jun 2013 14:23:50 -1000 Elan Feingold e...@plexapp.com wrote: The crux of the query which makes it pathologically slow in 3.7.15+ appears to be this bit: join metadata_item_settings on metadata_item_settings.guid=grandchild.guid and metadata_item_settings.account_id=1.

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Wolfgang Enzinger
Richard Hipp drh@... writes: Amalgamations for the latest SQLite containing the NGQP are available from the http://www.sqlite.org/draft/download.html page. This should be a drop-in replacement for the amalgamation you are currently using. There are no new APIs or compiler flags to fuss

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 6:50 AM, Wolfgang Enzinger sql...@enzinger.netwrote: Richard Hipp drh@... writes: Amalgamations for the latest SQLite containing the NGQP are available from the http://www.sqlite.org/draft/download.html page. This should be a drop-in replacement for the

Re: [sqlite] The next-generation query planner

2013-06-28 Thread E.Pasma
Op 26 jun 2013, om 16:08 heeft Richard Hipp het volgende geschreven: The next-generation query planner (NGQP) is a rewrite of the query planner for SQLite that is faster (reduced run-time for sqlite3_prepare()) and generates better plans for queries (reducing the run-time for

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 7:21 AM, E.Pasma pasm...@concepts.nl wrote: I tested the draft version ... Thanks for the test report! Unfortunately I also found a bug. For some reason the optimizer may ignore a where clause on an outerjoined table. The example below shows this, as far as it

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 6:32 AM, Eduardo emorr...@yahoo.es wrote: On Wed, 26 Jun 2013 14:23:50 -1000 Elan Feingold e...@plexapp.com wrote: The crux of the query which makes it pathologically slow in 3.7.15+ appears to be this bit: join metadata_item_settings on

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Wolfgang Enzinger
Richard Hipp drh@... writes: I try to compile with SQLITE_HAS_CODEC defined I get errors saying that sqlite3_key_v2 and sqlite3_rekey_v2 functions are undefined. Are these new APIs? Yes. You need to use your login and password to download the latest SEE source - the latest SEE

[sqlite] Large Database Windows vs Linux

2013-06-28 Thread Christopher W. Steenwyk
Hi, I have been struggling with a problem and was hoping I could get some insight. I have a rather large database (11 GB) that has two tables (one with approximately 500,000 rows and another with approximately 50,000,000 rows). In this database I am performing a query that joins these two

[sqlite] Differences from 3.7.11 to 3.7.16/17 ?

2013-06-28 Thread Marten Feldtmann
I have written a wrapper for VASmalltalk and I had tested it (under Windows 7/32bit - with the downloadable dll's) up to 3.7.11. Today I used the 3.7.16/17 and foudn out, that all the stuff, where callbacks to Smalltalk (from SQLite) are used (tracing, external functions) did not work any

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 8:45 AM, Wolfgang Enzinger sql...@enzinger.netwrote: Richard Hipp drh@... writes: Nevertheless, it is important that you test the NGQP in your application. Getting back at the issue described in http://article.gmane.org/gmane.comp.db.sqlite.general/81564 (which is

Re: [sqlite] Differences from 3.7.11 to 3.7.16/17 ?

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 9:06 AM, Marten Feldtmann itli...@schrievkrom.dewrote: I have written a wrapper for VASmalltalk and I had tested it (under Windows 7/32bit - with the downloadable dll's) up to 3.7.11. Today I used the 3.7.16/17 and foudn out, that all the stuff, where callbacks to

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread og
I had a similar problem and it was the antivirus (win 7 prof)... My table has about 63 million rows and a description very similar to yours... but now I use the same data for FTS4, etc... on Debian wheeze (simple workstation... and currently all ok...) example: sqlite select count(*) from parte;

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Richard Hipp
On Thu, Jun 27, 2013 at 6:50 PM, Elan Feingold e...@plexapp.com wrote: Q2 runs excruciatingly slowly in 3.7.15.2 (1m 28sec). We then observed that Q2 was fast in 3.7.14 (300ms), but slow in 3.7.15.2 and later (tested up until 3.7.17). Have you tried it with the NGQP snapshot? The

Re: [sqlite] Differences from 3.7.11 to 3.7.16/17 ?

2013-06-28 Thread Marten Feldtmann
Are the old prebuild binary versions (dll windows) available from somewhere ? Marten ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Wolfgang Enzinger
Richard Hipp drh@... writes: The combination of schema, sqlite_stat1, and query is normally sufficient, yes. Can you change (and shorten) the table and index and column names to obscure their meaning and origin, and give us written permission to include your case in the published test

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Stephan Beal
On Fri, Jun 28, 2013 at 6:55 PM, Christopher W. Steenwyk csteen...@gmail.com wrote: I also did try version 3.7.13 and that did run faster. So for whatever reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows than my 3.7.13 32-bit. Vaguely related to those observations:

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Christopher W. Steenwyk
Thanks for the input. I did recompile in 64-bit mode with no difference. I also did try version 3.7.13 and that did run faster. So for whatever reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows than my 3.7.13 32-bit. On Fri, Jun 28, 2013 at 10:30 AM, og

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Simon Slavin
On 28 Jun 2013, at 6:08pm, Stephan Beal sgb...@googlemail.com wrote: On Fri, Jun 28, 2013 at 6:55 PM, Christopher W. Steenwyk csteen...@gmail.com wrote: I also did try version 3.7.13 and that did run faster. So for whatever reason my shell 3.7.17 (32 or 64 bit) is significantly slower on

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Stephan Beal
On Fri, Jun 28, 2013 at 7:18 PM, Simon Slavin slav...@bigfraud.org wrote: Probably that your entire VM is in cache memory on your computer, but the program running on your hardware gets to write to a physical disk drive. That's certainly the most likely hypothesis i've heard so far. i hadn't

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Paul Corke
On 28 June 2013 18:08, Stephan Beal wrote: i've seen sqlite3 tests of mine in a 32-bit VM running on 64-bit hardware run twice as fast as that same code on the 64-bit hardware (outside the VM) One of our customers uses our product on a VM, and it appears that the hypervisor lies about having

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Scott Robison
He said using the NGQP snapshot the query indeed runs fast. I think you mentally inserted some negation in that statement, DRH. On Fri, Jun 28, 2013 at 8:47 AM, Richard Hipp d...@sqlite.org wrote: On Thu, Jun 27, 2013 at 6:50 PM, Elan Feingold e...@plexapp.com wrote: Q2 runs excruciatingly

Re: [sqlite] GUI for SQLite

2013-06-28 Thread nobre
I've been impressed by sqliteexpert Personal Edition (for Windows) I haven't found a great one for OSX yet. -- View this message in context: http://sqlite.1065341.n5.nabble.com/GUI-for-SQLite-tp11673p69729.html Sent from the SQLite mailing list archive at Nabble.com.

[sqlite] Shell: .IMPORT does not seem to obey .BAIL setting

2013-06-28 Thread tonyp
Trying to import data into a table the operation stops on first error (actually a rollback seems to be performed) because of column number mismatch. Why does it not follow the .BAIL ON/OFF setting? And, why rows before the error are removed? So, in .BAIL OFF mode, I expected errors to

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread RSmith
I think your assumption about the file system is correct - It is hard for the code to produce widely differing times under different systems as the basic algorithms do not change between systems, only dependencies on file-system or VFS specific api's etc. The NTFS file system in Windows (hoping

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/06/13 13:17, RSmith wrote: Best guess is some other system is trying to also look into that file, making the Windows file manager stutter quite possibly the Win7 Preview pane, a 3rd party file indexer service, an anti-virus system or some

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Keith Medcalf
That would explain why the best thing to be done with System Destroyer (System Restore) is the same as the best way to handle the Hardware Destroyer (Power Management) in Windows. Disable it completely. -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread David de Regt
FWIW, with our test and prod implementations, we find between a 3 and 10x (300-1000%) increase in almost all of our query times on Windows NTFS over OSX and iOS systems, depending on the query type. We've done a bunch of testing and can verify it every time. I started a thread on this ~7

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Walter Hurry
On Fri, 28 Jun 2013 15:22:57 -0600, Keith Medcalf wrote: That would explain why the best thing to be done with System Destroyer (System Restore) is the same as the best way to handle the Hardware Destroyer (Power Management) in Windows. Disable it completely. The best thing to do with

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread David de Regt
It's the kind of useful help like this that makes me love the FOSS movement. -David -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Walter Hurry Sent: Friday, June 28, 2013 5:09 PM To: sqlite-users@sqlite.org Subject: Re:

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Howard Chu
David de Regt wrote: It's the kind of useful help like this that makes me love the FOSS movement. All based in facts, of course. http://blog.zorinaq.com/?e=74 -David -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Simon Slavin
On 29 Jun 2013, at 1:22am, Howard Chu h...@symas.com wrote: David de Regt wrote: It's the kind of useful help like this that makes me love the FOSS movement. All based in facts, of course. http://blog.zorinaq.com/?e=74 Nevertheless, the remark was not helpful and is therefore best