Re: [sqlite] mmap, madvise, mlock and performance

2018-08-13 Thread Shevek
On 08/03/2018 12:55 PM, Simon Slavin wrote: On 3 Aug 2018, at 8:36pm, Shevek wrote: We are running a 100Gb sqlite database, which we mmap entirely into RAM. We are having trouble with parts of the disk file being evicted from RAM during periods of low activity causing slow responses, particu

Re: [sqlite] mmap, madvise, mlock and performance

2018-08-03 Thread Warren Young
On Aug 3, 2018, at 1:36 PM, Shevek wrote: > > the database is read-only In that case, I’d just create a :memory: DB on application startup, attach to the disk copy, use the INSERT FROM … SELECT pattern [1] to clone the data content within a single transaction, create the indexes, and detach fr

Re: [sqlite] mmap, madvise, mlock and performance

2018-08-03 Thread Simon Slavin
On 3 Aug 2018, at 8:36pm, Shevek wrote: > We are running a 100Gb sqlite database, which we mmap entirely into RAM. We > are having trouble with parts of the disk file being evicted from RAM during > periods of low activity causing slow responses, particularly before 9am. Has > anybody played w

[sqlite] mmap, madvise, mlock and performance

2018-08-03 Thread Shevek
Hi, We are running a 100Gb sqlite database, which we mmap entirely into RAM. We are having trouble with parts of the disk file being evicted from RAM during periods of low activity causing slow responses, particularly before 9am. Has anybody played with mlock and/or madvise within the sqlite