Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-27 Thread Jamie
> So I am guessing that this is probably expected behavior for .NET. You may be > able to tune some runtime settings, to tell it to limit heap growth. Or it > may be possible via the .NET SQLite API to read the blobs into existing byte > array objects, which would mostly eliminate the garbage pr

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-27 Thread Jens Alfke
> On May 26, 2017, at 5:46 PM, Jamie wrote: > > Without going into too much detail of explaining C#, everything is discarded > appropriately. All disposable objects are wrapped by using statements. And > byte arrays vanish once they're out of scope. They vanish from _your_ perspective, but th

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Jamie
; you just upset that the RAM for which money was paid is actually being used > for something? > > -- > ˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı > > > -Original Message- > > From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] >

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Stephen Chrzanowski
RAM for which money was paid is actually being used > for something? > > -- > ˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı > > > -Original Message- > > From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] > > On Behalf Of Jami

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Jamie
g] > On Behalf Of Jamie > Sent: Friday, 26 May, 2017 12:41 > To: SQLite mailing list > Subject: Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File > > > https://support.microsoft.com/en-us/help/2549369/performance-degrades- > when-accessing-large-files-with-f

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Keith Medcalf
ite.org] > On Behalf Of Jamie > Sent: Friday, 26 May, 2017 12:41 > To: SQLite mailing list > Subject: Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File > > > https://support.microsoft.com/en-us/help/2549369/performance-degrades- > when-accessing-large-files-with-

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Jamie
> https://support.microsoft.com/en-us/help/2549369/performance-degrades-when-accessing-large-files-with-file-flag-random-access > > Although the optimization is good, it sounds like it is the typical Microsoft > designed-by-flock-of-idiots software.  There is absolutely no way that this > should

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Keith Medcalf
On Friday, 26 May, 2017 08:27, Jamie said: >> >> says that there is a different kind of file cache for a random-access >> file, and that it shows up as active mapped pages. >> https://support.microsoft.com/en-us/help/976618

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Jamie
> > says that there is a different kind of file cache for a random-access > file, and that it shows up as active mapped pages. > https://support.microsoft.com/en-us/help/976618/you-experience-performance-issues-in-application

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Jamie
Thank you for your suggestion Clemens. I had responded about the pragma mmap_size to Mr. Mistachkin, but it ended up in the wrong place, probably because of a mistake I made. Here is the contents of that message below and sorry about the inconvenience: I tried changing it to several different val

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Richard Hipp
On 5/26/17, Clemens Ladisch wrote: > > SQLite does not use FILE_FLAG_RANDOM_ACCESS (except on Windows CE), but > it's possible that Windows is estimating that the accesses are random > (because the actually are). > IIRC, we experimented with using FILE_FLAG_RANDOM_ACCESS but found that it made th

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-26 Thread Clemens Ladisch
Jamie wrote: > this is not the normal Windows File Caching that you would typically > see. File Caching would be under STANDBY Mapped File To rule out SQLite's mmap, execute "PRAGMA mmap_size = 0", and then confirm with the output of "PRAGMA mmap_size".

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-25 Thread Jamie
> What you are observing is the Windows System File Cache. I already explained this is not the normal Windows File Caching that you would typically see. File Caching would be under STANDBY Mapped File and can be easily removed from RAM when memory is needed for something else. You can even conf

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-25 Thread Keith Medcalf
On Wednesday, 24 May, 2017 18:48, Jamie wrote: > When I'm performing a large amount of selects of GLOBs/thumbnails from an > ongoing SQLiteConnection, I'm having a problem where the Windows Active > Mapped File will constantly grow out of control in size (memory leak?). > This is under Windows 7,

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-25 Thread Jamie
I tried changing it to several different values for that pragma, but it did not seem to have any affect on how large the Active Mapped File could grow. The pragma was set before anything else was done on the connection. Although futile; I also tried attempting to use the pragma on the connection

Re: [sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-24 Thread Joe Mistachkin
Have you tried using the command "PRAGMA mmap_size=0;" on the connection? https://www.sqlite.org/pragma.html#pragma_mmap_size -- Joe Mistachkin @ https://urn.to/r/mistachkin ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org h

[sqlite] Memory Issue - GLOBs and Windows Active Mapped File

2017-05-24 Thread Jamie
When I'm performing a large amount of selects of GLOBs/thumbnails from an ongoing SQLiteConnection, I'm having a problem where the Windows Active Mapped File will constantly grow out of control in size (memory leak?). This is under Windows 7, using a .Net Framework 4.0 C# application with System