Re: [sqlite] Mutally dependent JOIN clauses

2014-01-19 Thread Hick Gunter
>-Ursprüngliche Nachricht- >Von: Rob Golsteijn [mailto:rob.golste...@mapscape.eu] >Gesendet: Freitag, 17. Jänner 2014 11:38 >An: sqlite-users@sqlite.org >Betreff: Re: [sqlite] Mutally dependent JOIN clauses > >... >My statement: > >SELECT * FROM C >LEFT JOIN A ON A.a*A.a + B.b*B.b = C

Re: [sqlite] Dramatic performance difference between different "PRAGMA synchronous" settings for bulk inserts

2014-01-19 Thread Mario M. Westphal
> Unrelated to your question, but, take a look at "external content" > FTS4 table they dramatically cut down the amount of duplicated data > [1]) Thanks for the tip. I'll definitely check that. Currently I build the contents for FTS dynamically from several other tables, combining, splitting, me

Re: [sqlite] Dramatic performance difference between different "PRAGMA synchronous" settings for bulk inserts

2014-01-19 Thread Mario M. Westphal
> If you want to try running with synchronous=NORMAL, you might try setting PRAGMA wal_autocheckpoint=10; (from the default of 1000) which will make for dramatically larger WAL files, but also dramatically fewer syncs. Then the syncs will use just 5 or 6 minutes instead of 4.5 hours. Hopefully

Re: [sqlite] Command line shell not flushing stderr when interactive

2014-01-19 Thread Simon Slavin
On 19 Jan 2014, at 7:32pm, Luuk wrote: > It is acceptable—and normal—for standard output and standard error to be > directed to the same destination, such as the text terminal. Messages appear > in the same order as the program writes them, unless buffering is involved. > (For example, a comm

Re: [sqlite] Command line shell not flushing stderr when interactive

2014-01-19 Thread Luuk
On 19-01-2014 19:59, Christopher Wellons wrote: When the shell is set to interactive (i.e. "-interactive"), the output (stdout) is flushed with every prompt (shell.c:422) but stderr is not. Stderr is suppose to be unbuffered so that flushing is not required. Or is that different for windows

Re: [sqlite] Solving the Sudoku with SQlite 3.8.3 trunk

2014-01-19 Thread Keith Medcalf
soduko1.sql and soduko2.sql are the two originals. soduko3.sql removes the digits view to an actual table (from soduko2.sql) and soduko3.sql puts digits back in as a CTE but is a select from the wholenumber module rather than generating the digits recursively. So, the fastest one uses digits p

Re: [sqlite] Command line shell not flushing stderr when interactive

2014-01-19 Thread Christopher Wellons
>> When the shell is set to interactive (i.e. "-interactive"), the output >> (stdout) is flushed with every prompt (shell.c:422) but stderr is not. > Stderr is suppose to be unbuffered so that flushing is not required. Or is > that different for windows? According to the stderr Linux man page s

Re: [sqlite] Dramatic performance difference between different "PRAGMA synchronous" settings for bulk inserts

2014-01-19 Thread Petite Abeille
On Jan 19, 2014, at 3:00 PM, Mario M. Westphal wrote: > Also FTS4 is used, which also creates large tables. (Unrelated to your question, but, take a look at "external content" FTS4 table… they dramatically cut down the amount of duplicated data [1]) > During an ingest phase, my application pu

Re: [sqlite] Dramatic performance difference between different "PRAGMA synchronous" settings for bulk inserts

2014-01-19 Thread Richard Hipp
In WAL mode with synchronous=NORMAL, SQLite only syncs (FlushFileBuffers() on windows) when it does a checkpoint operation. Checkpoints should be happening automatically whenever the WAL file exceeds about 1MB in size. For an 8GB database, probably there are about 8000 sync operations, therefore.

Re: [sqlite] Command line shell not flushing stderr when interactive

2014-01-19 Thread Richard Hipp
On Sun, Jan 19, 2014 at 11:10 AM, Christopher Wellons < well...@nullprogram.com> wrote: > > When the shell is set to interactive (i.e. "-interactive"), the output > (stdout) is flushed with every prompt (shell.c:422) but stderr is not. > Stderr is suppose to be unbuffered so that flushing is not

[sqlite] Sqlie from c#: encoding bug?

2014-01-19 Thread cinema cinema
I'm reading a text from c# saving it through a insert parameter in a varchar column of a table. Both the file and the sqlite db encoding is utf-8 but I see different characters (it seems due to a bad encoding) while reading data from the sqlite3 command line or from other clients... Any suggestion?

[sqlite] Command line shell not flushing stderr when interactive

2014-01-19 Thread Christopher Wellons
When the shell is set to interactive (i.e. "-interactive"), the output (stdout) is flushed with every prompt (shell.c:422) but stderr is not. In some situations this leads to no error messages being displayed until the stderr buffer fills. This happens when running the official sqlite3 binary as

Re: [sqlite] Dramatic performance difference between different "PRAGMA synchronous" settings for bulk inserts

2014-01-19 Thread Simon Slavin
On 19 Jan 2014, at 2:00pm, Mario M. Westphal wrote: > I logged the execution times of various operations in this phase to a text > file. Everything was fast, the processing, the INSERTs etc. > But COMMIT operations sometimes took 20s, then 0.2s, then again 10s. That's > the time SQLite spends in

[sqlite] Dramatic performance difference between different "PRAGMA synchronous" settings for bulk inserts

2014-01-19 Thread Mario M. Westphal
I have a performance effect which I don't quite understand. Maybe I'm using the wrong settings or something. Sorry for the long post, but I wanted to include all the info that may be important. My software is written in C++, runs on Windows 7/8, the SQLite database file is either on a local SATA

Re: [sqlite] Solving the Sudoku with SQlite 3.8.3 trunk

2014-01-19 Thread big stone
Hi Keith, Indeed just removing the CTE creation of the DIGITS makes Dan's version up to speed. Would the "wholenumber" external SQLite module help : - to make SQLite code cleaner ? (like "generate_series" of Postgresql, or "dual" of Oracle) - still provide the same speed-up ? Portfolio of typic