[sqlite] v3.6.23.1 disk I/O error on Win7 64bit

2010-09-01 Thread yfrankiv
Hi, Some of our users are getting this error: Message: disk I/O error (Code: 10). It happens during executing the following statement PRAGMA cache_size=131072. This error shows up sporadically and is not caused by running out of disk space or permission problems. So far, we're seeing reports

Re: [sqlite] SQLITE_NO_SYNC, PRAGMA synchronous or asynchronous io module?

2010-09-01 Thread Pavel Ivanov
I have set up a test program without QtSql and could not reproduce any of the performance degradations. So the problem has nothing to do with sqlite. Sorry for the noise. I guess Qt either ignores some of your pragmas, or resets their values to what it thinks is better. Also I'm afraid it

[sqlite] sqlite 3.7.2 doesn't compile if SQLITE_OMIT_WAL is defined

2010-09-01 Thread Pierre AUBERT
Hello all, I can't compile sqlite 3.7.2 if SQLITE_OMIT_WAL is defined. In pager.c, the function pagerPagecount is defined only if SQLITE_OMIT_WAL is not, but this function is called from hasHotJournal regardless SQLITE_OMIT_WAL. Best regards ___

Re: [sqlite] sqlite 3.7.2 doesn't compile if SQLITE_OMIT_WAL is defined

2010-09-01 Thread Richard Hipp
On Wed, Sep 1, 2010 at 10:28 AM, Pierre AUBERT p.aub...@staubli.com wrote: Hello all, I can't compile sqlite 3.7.2 if SQLITE_OMIT_WAL is defined. In pager.c, the function pagerPagecount is defined only if SQLITE_OMIT_WAL is not, but this function is called from hasHotJournal regardless

Re: [sqlite] sqlite 3.7.2 doesn't compile if SQLITE_OMIT_WAL is defined

2010-09-01 Thread Pierre AUBERT
Richard Hipp a écrit : On Wed, Sep 1, 2010 at 10:28 AM, Pierre AUBERT p.aub...@staubli.com wrote: Hello all, I can't compile sqlite 3.7.2 if SQLITE_OMIT_WAL is defined. In pager.c, the function pagerPagecount is defined only if SQLITE_OMIT_WAL is not, but this function is called from

[sqlite] Feature request: copying vacuum

2010-09-01 Thread Taras Glek
Hi, Currently VACUUM takes care of sqlite-level fragmentation. Unfortunately it does little for fs-level fragmentation since the same file is being reused. It would be really beneficial for Mozilla performance if we could get a vacuum/hotcopy mode. As I understand it, currently vacuum works

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Max Vlasov
This yields two benefits: A less fragmented db ~50% vacuum speedup since the data is only copied once Currently we can copy the data to a new file, but it is a pretty invasive change to swap all of the current sqlite connections to the new file. Things like prepared statements, etc

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Richard Hipp
On Wed, Sep 1, 2010 at 2:41 PM, Taras Glek tg...@mozilla.com wrote: Hi, Currently VACUUM takes care of sqlite-level fragmentation. Unfortunately it does little for fs-level fragmentation since the same file is being reused. It would be really beneficial for Mozilla performance if we could

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Jay A. Kreibich
On Wed, Sep 01, 2010 at 11:41:00AM -0700, Taras Glek scratched on the wall: Hi, Currently VACUUM takes care of sqlite-level fragmentation. Unfortunately it does little for fs-level fragmentation since the same file is being reused. It would be really beneficial for Mozilla performance if

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Scott Hess
On Wed, Sep 1, 2010 at 12:46 PM, Jay A. Kreibich j...@kreibi.ch wrote: On Wed, Sep 01, 2010 at 11:41:00AM -0700, Taras Glek scratched on the wall: Currently VACUUM takes care of sqlite-level fragmentation. Unfortunately it does little for fs-level fragmentation since the same file is being

Re: [sqlite] upstreaming Chromium patches for file handle passing support

2010-09-01 Thread Paweł Hajdan , Jr .
2010/8/31 Richard Hipp d...@sqlite.org os_unix.c is very unlikely to change in ways that you care about. I think the biggest concern is just copying that code, please see http://groups.google.com/a/chromium.org/group/chromium-dev/browse_thread/thread/7f8e9bf0f034322f/b26e422d4aa711c7. I think

Re: [sqlite] Asynchronous I/O Module For SQLite vs Single T hread mode

2010-09-01 Thread Gilles
Hello, Here follows, my Async IO implementation (at the moment, I just have tested the Win32 part. I will test soon the pthreads part). The main thread creates a worker thread (in charge of running the sqlite3async_run function). In Build options/compiler/defines, it seems that the following is

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Ben Danper
On Wed, Sep 1, 2010 at 12:46 PM, Jay A. Kreibich j...@... wrote: There is no reason to assume the filesystem will over-write the existing allocations, rather than just create new ones, especially if the pages are shuffled in groups... Actually there's no reason to do the opposite, as it would

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Simon Slavin
On 2 Sep 2010, at 4:42am, Ben Danper wrote: On Wed, Sep 1, 2010 at 12:46 PM, Jay A. Kreibich j...@... wrote: Maybe there would be some way to pre-populate the rollback journal with the full contents of the original database. Then the file could be truncated before the copy-back procedure.

Re: [sqlite] upstreaming Chromium patches for file handle passing support

2010-09-01 Thread Dan Kennedy
What do you think about extending the SQLite VFS to make it possible to open a database having a file handle (fd on Unix, HANDLE on Windows)? Opening based on file path would still be there (to preserve compatibility). How do you know which journal file to open or where to create a

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Max Vlasov
I agree with Jay - while it is tempting to have SQLite bite off optimizing this kind of thing, it's pretty far out of scope. Next we'll be talking about running SQLite on raw partitions! Scott, thought about it, ironically sqlite vfs is flexible enough to implement raw partition sqlite in

Re: [sqlite] upstreaming Chromium patches for file handle passing support

2010-09-01 Thread Max Vlasov
I wonder whether it would be possible to extend the VFS in a way that would make our use case possible (transferring a file handle over process boundary). Please note that we do it on all platforms. On POSIX we pass an integer file descriptor, and on Windows we pass a HANDLE. I used vfs for

Re: [sqlite] Feature request: copying vacuum

2010-09-01 Thread Scott Hess
On Wed, Sep 1, 2010 at 10:09 PM, Max Vlasov max.vla...@gmail.com wrote: I agree with Jay - while it is tempting to have SQLite bite off optimizing this kind of thing, it's pretty far out of scope.  Next we'll be talking about running SQLite on raw partitions! Scott, thought about it,

Re: [sqlite] upstreaming Chromium patches for file handle passing support

2010-09-01 Thread Scott Hess
On Wed, Sep 1, 2010 at 10:20 PM, Max Vlasov max.vla...@gmail.com wrote: I wonder whether it would be possible to extend the VFS in a way that would make our use case possible (transferring a file handle over process boundary). Please note that we do it on all platforms. On POSIX we pass an