[sqlite] I/O error on creating index with 3.7.14

2012-10-25 Thread Jamie Norrish
Using 3.7.14, when creating an index on a 27G database (on the table that contains almost all of the data), I consistently (on Windows XP and Debian GNU/Linux, on three different machines) get a disk I/O error. This does not happen using 3.7.13 (only tested on Debian GNU/Linux), nor does it happen

[sqlite] FW: how to select " char in sqlite

2012-10-25 Thread YAN HONG YE
char bh1[320]; memset(bh1,0,320); strcpy(bh1,"sqlite3 -html -header t9_engine.db \"select id,partnumber,substr(\'\',1,180) as img,pcs from engine where id>7;\" >> n.html"); system(bh1); //here couldn't work error: sqlite3 -html -header t9_engine.db "select id,partnumber,

[sqlite] FW: how to select " char in sqlite

2012-10-25 Thread YAN HONG YE
char bh1[320]; memset(bh1,0,320); strcpy(bh1,"sqlite3 -html -header t9_engine.db \"select id,partnumber,substr(\'\',1,180) as img,pcs from engine where id>7;\" >> n.html"); system(bh1); //here couldn't work error: sqlite3 -html -header t9_engine.db "select id,partnumber,

Re: [sqlite] light weight write barriers

2012-10-25 Thread Theodore Ts'o
On Thu, Oct 25, 2012 at 11:03:13AM -0700, da...@lang.hm wrote: > I agree, this is why I'm trying to figure out the recommended way to > do this without needing to do full commits. > > Since in most cases it's acceptable to loose the last few chunks > written, if we had some way of specifying order

Re: [sqlite] light weight write barriers

2012-10-25 Thread david
On Thu, 25 Oct 2012, Theodore Ts'o wrote: Or does rsyslog *really* need to issue an fsync after each log message? Or could it batch updates so that every N seconds, it flushes writes to the disk? In part this depends on how paranoid the admin is. By default rsyslog doesn't do fsyncs, but adm

Re: [sqlite] Disk I/O Error

2012-10-25 Thread Joe Mistachkin
Caio Honma wrote: > > I was trying to query from a huge table (150,000,000 lines+) using a left > join statement, using mode csv and using an output to file.. after several > minutes I got the Disk I/O error. I searched this maillist for some > possible answers (tried to check_integrity, but the d

[sqlite] Disk I/O Error

2012-10-25 Thread Caio Honma
Hi! I was trying to query from a huge table (150,000,000 lines+) using a left join statement, using mode csv and using an output to file.. after several minutes I got the Disk I/O error. I searched this maillist for some possible answers (tried to check_integrity, but the db was ok, then I install

Re: [sqlite] segfault in 3.7.13 (and older) with UNION ALL

2012-10-25 Thread Richard Hipp
This bug was fixed on 2012-08-28 (by the checkin at http://www.sqlite.org/src/info/200a81358c). The fix is available in SQLite version 3.7.14 and 3.7.14.1 as well as on trunk. On Thu, Oct 25, 2012 at 11:50 AM, Steve Pomeroy wrote: > This bug was originally found on a version of sqlite3 (tested

[sqlite] segfault in 3.7.13 (and older) with UNION ALL

2012-10-25 Thread Steve Pomeroy
This bug was originally found on a version of sqlite3 (tested in 3.5.9 and 3.7.11) in Android (2.2.1 and 4.1.2 respectively), but I've been able to recreate it on sqlite3 3.7.13 from Debian/wheezy. Test case: CREATE TABLE t1(a,b,c); INSERT INTO t1 VALUES(1,2,3), (4,5,6); CREATE TABLE t2(d,e,f); I

Re: [sqlite] light weight write barriers

2012-10-25 Thread Simon Slavin
On 25 Oct 2012, at 2:04am, da...@lang.hm wrote: > But unless you are a filesystem, how can you make sure that the message data > is written to file1 before you write the metadata about the message to file2? Wait for long enough for the disk subsystem to clear its backlog of write commands. A

Re: [sqlite] light weight write barriers

2012-10-25 Thread Alan Cox
> > Hopefully, eventually the storage developers will realize the value > > behind ordered commands and learn corresponding SCSI facilities to > > deal with them. > > Eventually, drive manufacturers will realize that trying to price > guage people who want advanced features such as TCQ, DIF/DIX, i

Re: [sqlite] transaction commit is successful despite I/O error

2012-10-25 Thread Pavel Ivanov
I guess this can stand as one more argument for not using SQLite over NFS. Another argument could be found here: http://www.sqlite.org/lockingv3.html#how_to_corrupt. Pavel On Wed, Oct 24, 2012 at 2:01 PM, Efim Dyadkin wrote: > I was probably not quite specific. So I would like to rephrase the p

Re: [sqlite] light weight write barriers

2012-10-25 Thread Theodore Ts'o
On Wed, Oct 24, 2012 at 11:58:49PM -0700, da...@lang.hm wrote: > The frustrating thing is that when people point out how things like > sqlite are so horribly slow, the reply seems to be "well, that's > what you get for doing so many fsyncs, don't do that", when there is > a 'problem' like the KDE "

Re: [sqlite] light weight write barriers

2012-10-25 Thread Theodore Ts'o
On Thu, Oct 25, 2012 at 02:03:25PM +0100, Alan Cox wrote: > > I doubt they care. The profit on high end features from the people who > really need them I would bet far exceeds any other benefit of giving it to > others. Welcome to capitalism 8) Yes, but it's a question of pricing. If they had pr

Re: [sqlite] Which non-default options are used for the shell?

2012-10-25 Thread Clemens Ladisch
Baruch Burstein wrote: > I know FTS is enabled (SQLITE_ENABLE_FTS*), but are any other flags used in > compiling the default shell? sqlite> pragma compile_options; CURDIR ENABLE_FTS3 ENABLE_RTREE TEMP_STORE=1 THREADSAFE=0 Regards, Clemens ___ sqlite-us

[sqlite] Which non-default options are used for the shell?

2012-10-25 Thread Baruch Burstein
I know FTS is enabled (SQLITE_ENABLE_FTS*), but are any other flags used in compiling the default shell? -- ˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/l

Re: [sqlite] Bug report concerning the juliandate function

2012-10-25 Thread Richard Hipp
On Wed, Oct 24, 2012 at 1:06 PM, wrote: > Dear sir or madam, > > I have tested sqlite version 3.6. and version 3.7.14.1 with the following > inputs: > > select julianday('2005-02-29'); > select julianday('2005-02-30'); > > In both cases, it should have returned NULL, but it returned a number. >

Re: [sqlite] light weight write barriers

2012-10-25 Thread david
On Thu, 25 Oct 2012, Theodore Ts'o wrote: On Wed, Oct 24, 2012 at 03:03:00PM -0700, da...@lang.hm wrote: Like what is being described for sqlite, loosing the tail end of the messages is not a big problem under normal conditions. But there is a need to be sure that what is there is complete up t

Re: [sqlite] light weight write barriers

2012-10-25 Thread david
On Thu, 25 Oct 2012, Theodore Ts'o wrote: On Thu, Oct 25, 2012 at 12:18:47AM -0500, Nico Williams wrote: By trusting fsync(). And if you don't care about immediate Durability you can run the fsync() in a background thread and mark the associated transaction as completed in the next transactio

Re: [sqlite] light weight write barriers

2012-10-25 Thread Theodore Ts'o
On Thu, Oct 25, 2012 at 12:18:47AM -0500, Nico Williams wrote: > > By trusting fsync(). And if you don't care about immediate Durability > you can run the fsync() in a background thread and mark the associated > transaction as completed in the next transaction to be written after > the fsync() co

Re: [sqlite] light weight write barriers

2012-10-25 Thread Theodore Ts'o
On Wed, Oct 24, 2012 at 03:03:00PM -0700, da...@lang.hm wrote: > Like what is being described for sqlite, loosing the tail end of the > messages is not a big problem under normal conditions. But there is > a need to be sure that what is there is complete up to the point > where it's lost. > > this

Re: [sqlite] light weight write barriers

2012-10-25 Thread Theodore Ts'o
On Tue, Oct 23, 2012 at 03:53:11PM -0400, Vladislav Bolkhovitin wrote: > Yes, SCSI has full support for ordered/simple commands designed > exactly for that task: to have steady flow of commands even in case > when some of them are ordered. SCSI does, yes --- *if* the device actually implements

Re: [sqlite] light weight write barriers

2012-10-25 Thread david
On Wed, 24 Oct 2012, Nico Williams wrote: On Wed, Oct 24, 2012 at 5:03 PM, wrote: I'm doing some work with rsyslog and it's disk-baded queues and there is a similar issue there. The good news is that we can have a version that is linux specific (rsyslog is used on other OSs, but there is an e

Re: [sqlite] light weight write barriers

2012-10-25 Thread david
On Wed, 24 Oct 2012, Nico Williams wrote: Before that happens, people will keep returning again and again with those simple questions: why the queue must be flushed for any ordered operation? Isn't is an obvious overkill? That [cache flushing] is not what's being asked for here. Just a light-

Re: [sqlite] transaction commit is successful despite I/O error

2012-10-25 Thread Efim Dyadkin
I was probably not quite specific. So I would like to rephrase the problem and give more details. I run a default configuration of Sqlite on Linux and I don't use WAL or anything that changes transaction behavior. A database is located on a remote file server and accessed via NFS. There is a si

[sqlite] Bug report concerning the juliandate function

2012-10-25 Thread kagan
Dear sir or madam, I have tested sqlite version 3.6. and version 3.7.14.1 with the following inputs: select julianday('2005-02-29'); select julianday('2005-02-30'); In both cases, it should have returned NULL, but it returned a number. In the first case, the year 2005 is not a leap year. There

Re: [sqlite] how to select " char in sqlite

2012-10-25 Thread Kees Nuyt
On Thu, 25 Oct 2012 01:20:24 +, YAN HONG YE wrote: >char bh1[320]; >memset(bh1,0,320); >strcpy(bh1,"sqlite3 -html -header t9_engine.db \"select >id,partnumber,substr(\'\',1,180) as >img,pcs from engine where id>7;\" >> n.html"); >system(bh1); //here couldn't work A