[sqlite] Question about turn off the synchronous flag

2009-08-14 Thread Michael Han
Hello, My application is using SQLite and I find a huge discrepancy of committing speed when synchronous flag is set on / off (via PRAGMA). My question is can I safely turn off this flag if the underlying file system supports journaling? The FAQ (http://www.sqlite.org/faq.html#q5) said Another

[sqlite] About the substr(x,y,z) core function

2009-08-14 Thread A.J.Millan
Hi List: May be of interest to add in the doc, that, when the second argument is negative, the the first character of the substring is found by counting from the right rather than the left, but remember that the last actual character is -2, because -1 is just the ending NULL. Of course it is

Re: [sqlite] SQLite version 3.6.17

2009-08-14 Thread Hugh Sasse
The upshot of my tests on Solaris9 was: WARNING: Multi-threaded tests skipped: Linked against a non-threadsafe Tcl build All memory allocations freed - no leaks Memory used: now 0 max5727984 max-size 171798754 Page-cache used: now 0 max 13 max-size

Re: [sqlite] SQLite version 3.6.17

2009-08-14 Thread Dan Kennedy
On Aug 14, 2009, at 5:14 PM, Hugh Sasse wrote: The upshot of my tests on Solaris9 was: WARNING: Multi-threaded tests skipped: Linked against a non- threadsafe Tcl build All memory allocations freed - no leaks Memory used: now 0 max5727984 max-size 171798754

Re: [sqlite] SQLite version 3.6.17

2009-08-14 Thread Hugh Sasse
On Fri, 14 Aug 2009, Dan Kennedy wrote: On Aug 14, 2009, at 5:14 PM, Hugh Sasse wrote: The upshot of my tests on Solaris9 was: WARNING: Multi-threaded tests skipped: Linked against a non- threadsafe Tcl build All memory allocations freed - no leaks Memory used: now

Re: [sqlite] About the substr(x,y,z) core function

2009-08-14 Thread Igor Tandetnik
A.J.Millan wrote: May be of interest to add in the doc, that, when the second argument is negative, the the first character of the substring is found by counting from the right rather than the left, but remember that the last actual character is -2, because -1 is just the ending NULL. select

Re: [sqlite] Question about turn off the synchronous flag

2009-08-14 Thread Jay A. Kreibich
On Fri, Aug 14, 2009 at 05:10:40PM +0800, Michael Han scratched on the wall: My question is can I safely turn off this flag if the underlying file system supports journaling? No. As the docs state, with synchronous=ON the system verifies the data actually hits disk before proceeding.

[sqlite] Scalability.

2009-08-14 Thread Dmitri Priimak
Hi. I have a database with few simple tables. Database is updated regularly and than distributed to the clients, which only use it for reading. So, concurrency is not an issue. But the database is already quite large. The file is about 2.6GB and one table has about 1,800,000 rows and another

Re: [sqlite] Scalability.

2009-08-14 Thread P Kishor
On Fri, Aug 14, 2009 at 9:24 AM, Dmitri Priimakprii...@stanford.edu wrote: Hi. I have a database with few simple tables. Database is updated regularly and than distributed to the clients, which only use it for reading. So, concurrency is not an issue. But the database is already quite large.

Re: [sqlite] Scalability.

2009-08-14 Thread Jay A. Kreibich
On Fri, Aug 14, 2009 at 07:24:31AM -0700, Dmitri Priimak scratched on the wall: I have a database with few simple tables. Database is updated regularly and than distributed to the clients, which only use it for reading. So, concurrency is not an issue. But the database is already quite

Re: [sqlite] Question about turn off the synchronous flag

2009-08-14 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Michael Han wrote: My question is can I safely turn off this flag if the underlying file system supports journaling? By default journaling file systems only use their journal for meta-data (eg directory entries and allocation information). They

Re: [sqlite] Scalability.

2009-08-14 Thread Simon Slavin
On 14 Aug 2009, at 3:24pm, Dmitri Priimak wrote: I have a database with few simple tables. Database is updated regularly and than distributed to the clients, which only use it for reading. So, concurrency is not an issue. But the database is already quite large. The file is about 2.6GB

Re: [sqlite] Scalability.

2009-08-14 Thread Clark Christensen
Your only problem is that you're at Stanford and Dr Hipp was at Duke so he hates you. Not much humor on this list, but you made my day :-) Very funny. -Clark ___ sqlite-users mailing list sqlite-users@sqlite.org

[sqlite] Does PRAGMA synchronous=OFF ensure that no synching is done for the entire session?

2009-08-14 Thread Angus March
I need to know that if I turn of the synchronous that no synching will be done, up to, and including, when the session is closed. I'm asking, because my program just INSERTs once per session, so if a synch gets done when the session closes, that's pretty useless.

Re: [sqlite] Does PRAGMA synchronous=OFF ensure that no synching is done for the entire session?

2009-08-14 Thread Simon Slavin
On 14 Aug 2009, at 5:25pm, Angus March wrote: I need to know that if I turn of the synchronous that no synching will be done, up to, and including, when the session is closed. I'm asking, because my program just INSERTs once per session, so if a synch gets done when the session closes,

Re: [sqlite] Does PRAGMA synchronous=OFF ensure that no synching is done for the entire session?

2009-08-14 Thread Angus March
Simon Slavin wrote: On 14 Aug 2009, at 5:25pm, Angus March wrote: I need to know that if I turn of the synchronous that no synching will be done, up to, and including, when the session is closed. I'm asking, because my program just INSERTs once per session, so if a synch gets done when

Re: [sqlite] Does PRAGMA synchronous=OFF ensure that no synching is done for the entire session?

2009-08-14 Thread Simon Slavin
On 14 Aug 2009, at 5:33pm, Angus March wrote: I want my INSERT done right away, Then do not turn off synchronous ! I just don't want it to be flushed from the filesystem's write-behind cache until the kernel decides, not when SQLite decides. SQLite cannot control how your operating system

Re: [sqlite] Scalability.

2009-08-14 Thread Dmitri Priimak
Thanks, a lot guys. I will run a few tests with dummy database populated with my target amount rows and will let you know results. I realize now that I seem to have some unfounded fear of large files (FOLF) :) Hopefully it will pass. -- Dmitri Priimak

Re: [sqlite] Scalability.

2009-08-14 Thread Kees Nuyt
On Fri, 14 Aug 2009 07:24:31 -0700, Dmitri Priimak prii...@stanford.edu wrote: Hi. I have a database with few simple tables. Database is updated regularly and than distributed to the clients, which only use it for reading. So, concurrency is not an issue. But the database is already quite

Re: [sqlite] Scalability.

2009-08-14 Thread Dmitri Priimak
Kees Nuyt wrote: On Fri, 14 Aug 2009 07:24:31 -0700, Dmitri Priimak prii...@stanford.edu wrote: Hi. I have a database with few simple tables. Database is updated regularly and than distributed to the clients, which only use it for reading. So, concurrency is not an issue. But the

[sqlite] leaving database connection open

2009-08-14 Thread Ricardo Ayres Severo
Hi All, I'm using SQLite3 with a Embedded Linux application. The problem I'm facing is that there's a bug in the Linux kernel I'm using that the open files counter isn't being decremented when a file descriptor is closed. With that my application simply crashes after some time of usage. Because

[sqlite] 3.6.17 test failure

2009-08-14 Thread Ken
I'm not sure if this an issue or not. make test failed with the following: 2 errors out of 40872 tests Failures on these tests: rollback-2.3 tkt3457-1.4 All memory allocations freed - no leaks Memory used: now 0 max 102680 max-size2800336 Page-cache used: now

Re: [sqlite] Does PRAGMA synchronous=OFF ensure that no synching is done for the entire session?

2009-08-14 Thread Matt Sergeant
On Fri, 14 Aug 2009 12:33:30 -0400, Angus March wrote: I want my INSERT done right away, I just don't want it to be flushed from the filesystem's write-behind cache until the kernel decides, not when SQLite decides. Did you mean you do want it to be flushed from the filesystem's

Re: [sqlite] Scalability.

2009-08-14 Thread Kees Nuyt
On Fri, 14 Aug 2009 11:00:53 -0700, Dmitri Priimak prii...@stanford.edu wrote: Kees Nuyt wrote: On Fri, 14 Aug 2009 07:24:31 -0700, Dmitri Priimak prii...@stanford.edu wrote: Hi. I have a database with few simple tables. Database is updated regularly and than distributed to the clients,

[sqlite] SQLite locking problem with mozStorage

2009-08-14 Thread Sebastian Arcus
Hi all, I'm using the mozStorage implementation from Mozilla foundation, with Javascript and XUL (for the UI) to build a business-type app. I'm having this problem where I seem to get a lock of about 10 seconds after a read operation on a table, before I can perform a write operation on the

Re: [sqlite] SQLite locking problem with mozStorage

2009-08-14 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sebastian Arcus wrote: The SQLite documentation talks about entire database locks by operations of the order of milliseconds - 10 seconds seems a long way off. There is a possible but unlikely cause for what you are seeing. In order to ensure

Re: [sqlite] SQLite locking problem with mozStorage

2009-08-14 Thread Sebastian Arcus
Roger Binns wrote: Sebastian Arcus wrote: The SQLite documentation talks about entire database locks by operations of the order of milliseconds - 10 seconds seems a long way off. There is a possible but unlikely cause for what you are seeing. In order to ensure the database

Re: [sqlite] leaving database connection open

2009-08-14 Thread Igor Tandetnik
Ricardo Ayres Severo severo.rica...@gmail.com wrote: My question is if I would face any problem by opening the database when my application starts and actually never closing it. I don't do any access across threads. You shouldn't have any problems with this. It's a pretty common access

[sqlite] FTS3 and negation operator

2009-08-14 Thread cscs-sqlite
Did an FTS3 update change how many negation operators (dash/-) can be used in a match statement? For example, in sqlite3.dll version 3.5.7: colname match 'tetons -bend -jackson -oxbow* -parks' works as expected; bend, jackson, oxbow* and parks are all removed from the results. but With,

Re: [sqlite] SQLite locking problem with mozStorage

2009-08-14 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sebastian Arcus wrote: Hi and thanks for the suggestion. I did as you advised and ran 'vmstat 1' in a terminal. Very little activity - maybe 20-40kb every 6-7 seconds on the bo - pretty much nothing on bi. Also, zeros all around on so/si. That

Re: [sqlite] FTS3 and negation operator

2009-08-14 Thread Dan Kennedy
On Aug 15, 2009, at 9:32 AM, cscs-sql...@usa.net wrote: Did an FTS3 update change how many negation operators (dash/-) can be used in a match statement? For example, in sqlite3.dll version 3.5.7: colname match 'tetons -bend -jackson -oxbow* -parks' works as expected; bend,

Re: [sqlite] 3.6.17 test failure

2009-08-14 Thread Dan Kennedy
On Aug 15, 2009, at 2:14 AM, Ken wrote: I'm not sure if this an issue or not. make test failed with the following: 2 errors out of 40872 tests Failures on these tests: rollback-2.3 tkt3457-1.4 All memory allocations freed - no leaks Memory used: now 0 max 102680