[sqlite] Is there available sample "featured" SQLite database file?

2013-06-28 Thread Sqlite Dog
Hi, we are developing (yet another :) SQLite manager. I'm wondering is there available sample "feature-tight" "limit-striking" SQLite database (or SQL script) for GUI manager testing purposes? Like: - identifier names contains spaces, UNICODE chars, are surrounded by [], by "" or not surrounded

Re: [sqlite] Is there available sample "featured" SQLite database file?

2013-06-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/06/13 01:12, Sqlite Dog wrote: > I'm wondering is there available sample "feature-tight" > "limit-striking" SQLite database (or SQL script) for GUI manager > testing purposes? You can certainly measure coverage of sqlite3.c itself when running

Re: [sqlite] Is there available sample "featured" SQLite database file?

2013-06-28 Thread Sqlite Dog
> Here is your starter schema (and perfectly valid): > create table ""("" ""); Thanks for this snippet! Actually found a bug :) > For performance testing this could be helpful with the --dump-sql arg: > https://code.google.com/p/apsw/source/browse/tools/speedtest.py Well, performance is

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Eduardo
On Wed, 26 Jun 2013 14:23:50 -1000 Elan Feingold wrote: > The crux of the query which makes it pathologically slow in 3.7.15+ appears > to be this bit: "join metadata_item_settings on > metadata_item_settings.guid=grandchild.guid and > metadata_item_settings.account_id=1". If I take ou

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Wolfgang Enzinger
Richard Hipp writes: > Amalgamations for the latest SQLite containing the NGQP are available from > the http://www.sqlite.org/draft/download.html page. This should be a > drop-in replacement for the amalgamation you are currently using. There > are no new APIs or compiler flags to fuss with. E

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 6:50 AM, Wolfgang Enzinger wrote: > Richard Hipp writes: > > > Amalgamations for the latest SQLite containing the NGQP are available > from > > the http://www.sqlite.org/draft/download.html page. This should be a > > drop-in replacement for the amalgamation you are curren

Re: [sqlite] The next-generation query planner

2013-06-28 Thread E.Pasma
Op 26 jun 2013, om 16:08 heeft Richard Hipp het volgende geschreven: The next-generation query planner (NGQP) is a rewrite of the query planner for SQLite that is faster (reduced run-time for sqlite3_prepare()) and generates better plans for queries (reducing the run-time for sqlite3_step()).

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 7:21 AM, E.Pasma wrote: > > I tested the draft version ... > Thanks for the test report! > Unfortunately I also found a bug. For some reason the optimizer may ignore > a where clause on an outerjoined table. The example below shows this, as > far as it appears in the ex

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 6:32 AM, Eduardo wrote: > On Wed, 26 Jun 2013 14:23:50 -1000 > Elan Feingold wrote: > > > The crux of the query which makes it pathologically slow in 3.7.15+ > appears to be this bit: "join metadata_item_settings on > metadata_item_settings.guid=grandchild.guid and >

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Wolfgang Enzinger
Richard Hipp writes: > > I try to compile with SQLITE_HAS_CODEC defined I get errors saying that > > sqlite3_key_v2 and sqlite3_rekey_v2 functions are undefined. Are these new > > APIs? > > > > Yes. You need to use your login and password to download the latest SEE > source - the latest SEE sou

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Wolfgang Enzinger
Richard Hipp writes: > Nevertheless, it is important that you test the NGQP in your application. Getting back at the issue described in http://article.gmane.org/gmane.comp.db.sqlite.general/81564 (which is *not* critical because an ANALYZE solves it, but nevertheless might be interesting): Thi

[sqlite] Large Database Windows vs Linux

2013-06-28 Thread Christopher W. Steenwyk
Hi, I have been struggling with a problem and was hoping I could get some insight. I have a rather large database (11 GB) that has two tables (one with approximately 500,000 rows and another with approximately 50,000,000 rows). In this database I am performing a query that joins these two tables

[sqlite] Differences from 3.7.11 to 3.7.16/17 ?

2013-06-28 Thread Marten Feldtmann
I have written a wrapper for VASmalltalk and I had tested it (under Windows 7/32bit - with the downloadable dll's) up to 3.7.11. Today I used the 3.7.16/17 and foudn out, that all the stuff, where callbacks to Smalltalk (from SQLite) are used (tracing, external functions) did not work any more

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 8:45 AM, Wolfgang Enzinger wrote: > Richard Hipp writes: > > > Nevertheless, it is important that you test the NGQP in your application. > > Getting back at the issue described in > http://article.gmane.org/gmane.comp.db.sqlite.general/81564 > > (which is *not* critical be

Re: [sqlite] Differences from 3.7.11 to 3.7.16/17 ?

2013-06-28 Thread Richard Hipp
On Fri, Jun 28, 2013 at 9:06 AM, Marten Feldtmann wrote: > I have written a wrapper for VASmalltalk and I had tested it (under > Windows 7/32bit - with the downloadable dll's) up to 3.7.11. > > Today I used the 3.7.16/17 and foudn out, that all the stuff, where > callbacks to Smalltalk (from SQLit

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread og
I had a similar problem and it was the antivirus (win 7 prof)... My table has about 63 million rows and a description very similar to yours... but now I use the same data for FTS4, etc... on Debian wheeze (simple workstation... and currently all ok...) example: sqlite> select count(*) from parte;

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Richard Hipp
On Thu, Jun 27, 2013 at 6:50 PM, Elan Feingold wrote: > >> Q2 runs excruciatingly slowly in 3.7.15.2 (1m 28sec). We then observed > >> that Q2 was fast in 3.7.14 (300ms), but slow in 3.7.15.2 and later > (tested > >> up until 3.7.17). > >> > > > > Have you tried it with the NGQP snapshot? > > > >

Re: [sqlite] Differences from 3.7.11 to 3.7.16/17 ?

2013-06-28 Thread Marten Feldtmann
Are the old prebuild binary versions (dll windows) available from somewhere ? Marten ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] The next-generation query planner

2013-06-28 Thread Wolfgang Enzinger
Richard Hipp writes: > The combination of schema, sqlite_stat1, and query is normally sufficient, > yes. > > Can you change (and shorten) the table and index and column names to > obscure their meaning and origin, and give us written permission to include > your case in the published test suite

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Stephan Beal
On Fri, Jun 28, 2013 at 6:55 PM, Christopher W. Steenwyk < csteen...@gmail.com> wrote: > I also did try version 3.7.13 and that did run faster. So for whatever > reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows > than my 3.7.13 32-bit. > Vaguely related to those observatio

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Christopher W. Steenwyk
Thanks for the input. I did recompile in 64-bit mode with no difference. I also did try version 3.7.13 and that did run faster. So for whatever reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows than my 3.7.13 32-bit. On Fri, Jun 28, 2013 at 10:30 AM, og wrote: > I had a

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Simon Slavin
On 28 Jun 2013, at 6:08pm, Stephan Beal wrote: > On Fri, Jun 28, 2013 at 6:55 PM, Christopher W. Steenwyk < > csteen...@gmail.com> wrote: > >> I also did try version 3.7.13 and that did run faster. So for whatever >> reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows >> th

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Stephan Beal
On Fri, Jun 28, 2013 at 7:18 PM, Simon Slavin wrote: > Probably that your entire VM is in cache memory on your computer, but the > program running on your hardware gets to write to a physical disk drive. > That's certainly the most likely hypothesis i've heard so far. i hadn't considered the eff

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Paul Corke
On 28 June 2013 18:08, Stephan Beal wrote: > i've seen sqlite3 tests of mine in a 32-bit VM running on > 64-bit hardware run twice as fast as that same code on the > 64-bit hardware (outside the VM) One of our customers uses our product on a VM, and it appears that the hypervisor lies about havin

Re: [sqlite] Performance regression since 3.7.15

2013-06-28 Thread Scott Robison
He said using the NGQP snapshot the query indeed runs fast. I think you mentally inserted some negation in that statement, DRH. On Fri, Jun 28, 2013 at 8:47 AM, Richard Hipp wrote: > On Thu, Jun 27, 2013 at 6:50 PM, Elan Feingold wrote: > > > >> Q2 runs excruciatingly slowly in 3.7.15.2 (1m 28

Re: [sqlite] GUI for SQLite

2013-06-28 Thread nobre
I've been impressed by sqliteexpert Personal Edition (for Windows) I haven't found a great one for OSX yet. -- View this message in context: http://sqlite.1065341.n5.nabble.com/GUI-for-SQLite-tp11673p69729.html Sent from the SQLite mailing list archive at Nabble.com. ___

[sqlite] Shell: .IMPORT does not seem to obey .BAIL setting

2013-06-28 Thread tonyp
Trying to import data into a table the operation stops on first error (actually a rollback seems to be performed) because of column number mismatch. Why does it not follow the .BAIL ON/OFF setting? And, why rows before the error are removed? So, in .BAIL OFF mode, I expected errors to display,

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread RSmith
I think your assumption about the file system is correct - It is hard for the code to produce widely differing times under different systems as the basic algorithms do not change between systems, only dependencies on file-system or VFS specific api's etc. The NTFS file system in Windows (hoping

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/06/13 13:17, RSmith wrote: > Best guess is some other system is trying to also look into that file, > making the Windows file manager stutter quite possibly the Win7 > Preview pane, a 3rd party file indexer service, an anti-virus system or > som

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Keith Medcalf
That would explain why the best thing to be done with System Destroyer (System Restore) is the same as the best way to handle the Hardware Destroyer (Power Management) in Windows. Disable it completely. > -Original Message- > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread David de Regt
FWIW, with our test and prod implementations, we find between a 3 and 10x (300-1000%) increase in almost all of our query times on Windows NTFS over OSX and iOS systems, depending on the query type. We've done a bunch of testing and can verify it every time. I started a thread on this ~7 month

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Walter Hurry
On Fri, 28 Jun 2013 15:22:57 -0600, Keith Medcalf wrote: > That would explain why the best thing to be done with System Destroyer > (System Restore) is the same as the best way to handle the Hardware > Destroyer (Power Management) in Windows. Disable it completely. > The best thing to do with Wi

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread David de Regt
It's the kind of useful help like this that makes me love the FOSS movement. -David -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Walter Hurry Sent: Friday, June 28, 2013 5:09 PM To: sqlite-users@sqlite.org Subject: Re: [sq

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Howard Chu
David de Regt wrote: It's the kind of useful help like this that makes me love the FOSS movement. All based in facts, of course. http://blog.zorinaq.com/?e=74 -David -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Walter

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Simon Slavin
On 29 Jun 2013, at 1:22am, Howard Chu wrote: > David de Regt wrote: >> It's the kind of useful help like this that makes me love the FOSS movement. > > All based in facts, of course. http://blog.zorinaq.com/?e=74 Nevertheless, the remark was not helpful and is therefore best forgotten. If an