Re: [sqlite] In memory only WAL file

ah ok, now I understand. So you would like some setting like "limit writes to x/s", keeping the intermediate writes in memory. Here is a great writeup on how to tune sqlite for writes: https://stackoverflow.com/q/1711631/124416 But depending on your app you might just group a ton of writes in a

Re: [sqlite] In memory only WAL file

I understand that the SQLite tells the OS all the correct things, but as I described earlier, that may just produce way too much IOPS, even if everything in the chain does the correct thing down to the last silicon transistor. That is why I was asking about a way how to go around it by holding

Re: [sqlite] In memory only WAL file

On 9 Apr 2018, at 8:41am, Pavel Cernohorsky wrote: > if you use bigger pro-level SSD, you get on a bit less that 1 year.. still > not good enough. Of course, big unknown in this is the constant representing > the ability of the SSD controller to merge requests

Re: [sqlite] In memory only WAL file

Hi, what prompts me? Well, I mentioned that in the original post - it is the number of IOPS going to the disk (write IOPS I am concerned about). Not that SQLite would be slow, the performance is OK (for now), but having IO in the range of thousands per second constant load just doesn't feel

Re: [sqlite] In memory only WAL file

Serious question: what prompts you to consider these things? Is sqlite being too slow for you? On Thu, Apr 5, 2018 at 10:00 AM Pavel Cernohorsky < pavel.cernohor...@appeartv.com> wrote: > Hello Dan, thank you very much for clearing this up, because that was my > important misunderstanding. > >

Re: [sqlite] In memory only WAL file

Hello Dan, thank you very much for clearing this up, because that was my important misunderstanding. Risking corruption when the OS crashes is not really an option for me. I will probably go for what David Raymond suggested in one of other posts, or I may also solve my whole problem on the

Re: [sqlite] In memory only WAL file

with what's actually possible and actually useful will now rip apart my suggestion. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Pavel Cernohorsky Sent: Wednesday, April 04, 2018 10:01 AM To: sqlite-users@mailinglists.sqlite.org Subjec

Re: [sqlite] In memory only WAL file

On 04/05/2018 02:08 PM, Pavel Cernohorsky wrote: Hello Dan, so you are saying that if WAL is somehow in memory only (not properly persisted) and app or OS crashes in the middle of the checkpoint operation, my main database file will get corrupted? And by corrupted you mean as in "you will

Re: [sqlite] In memory only WAL file

Well, so you are saying that I should use single transaction for the "checkpoints"? Unfortunately, that wouldn't play really well with those multiple database readers I mentioned - those readers would not see changes done by the writer until the transaction finishes (which can be several

Re: [sqlite] In memory only WAL file

And how do you change location of the WAL file? Plus, will the main database file really get corrupted (as in Consistency corrupted, not only Durability of the data which were in the WAL file) if the OS crashes? Pavel On 04/04/2018 07:07 PM, Peter Da Silva wrote: You could put the WAL in a

Re: [sqlite] In memory only WAL file

Hello Dan, so you are saying that if WAL is somehow in memory only (not properly persisted) and app or OS crashes in the middle of the checkpoint operation, my main database file will get corrupted? And by corrupted you mean as in "you will loose changes which were in the WAL file", or "you

Re: [sqlite] In memory only WAL file

Hello Jens and Warren, Performance is really one of the reasons, second is concurrent writer and readers (which can be in fact viewed as part of the performance). Pavel On 04/04/2018 06:57 PM, Jens Alfke wrote: On Apr 4, 2018, at 9:01 AM, Warren Young wrote: Why

Re: [sqlite] In memory only WAL file

On Behalf Of Pavel Cernohorsky Sent: Wednesday, April 04, 2018 10:01 AM To: sqlite-users@mailinglists.sqlite.org Subject: [sqlite] In memory only WAL file Hello, does anybody know if there is some possibility to not have WAL file as a normal file on the disk, but only in memory? I understand that a

Re: [sqlite] In memory only WAL file

On 4 Apr 2018, at 3:01pm, Pavel Cernohorsky wrote: > Hello, does anybody know if there is some possibility to not have WAL file as > a normal file on the disk, but only in memory? I understand that all the > modifications to the database would get lost in case

Re: [sqlite] In memory only WAL file

You could put the WAL in a tmpfs/ramfs so the DB would only get corrupted if the OS crashed, it'd still be there for recovering from application crashes. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] In memory only WAL file

> On Apr 4, 2018, at 9:01 AM, Warren Young wrote: > > Why turn on WAL mode at all, then? Performance, probably. An in-memory WAL would scream. —Jens ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] In memory only WAL file

On 04/04/2018 09:01 PM, Pavel Cernohorsky wrote: Hello, does anybody know if there is some possibility to not have WAL file as a normal file on the disk, but only in memory? I understand that all the modifications to the database would get lost in case of the application / OS crash, but for my

Re: [sqlite] In memory only WAL file

On Apr 4, 2018, at 8:01 AM, Pavel Cernohorsky wrote: > > Hello, does anybody know if there is some possibility to not have WAL file as > a normal file on the disk, but only in memory? Why turn on WAL mode at all, then? Are you maybe using a SQLite setup where

[sqlite] In memory only WAL file

Hello, does anybody know if there is some possibility to not have WAL file as a normal file on the disk, but only in memory? I understand that all the modifications to the database would get lost in case of the application / OS crash, but for my application, I only need the level of durability