Thanks for your responses! On Thu, Feb 16, 2017 at 8:56 PM, Dominique Pellé <dominique.pe...@gmail.com> wrote: > > I think it's probably best to give a large cache_size to > each connection, and limit the total amount of memory > used by SQLite with sqlite3_soft_heap_limit64(). > This will effectively limit the global amount of cache > used if cache pages comes from the heap. > > For example, if you give 20MB to each connection > (PRAGMA cache_size=....) and limit the global SQLite > heap usage to 30 MB with sqlite3_soft_limit() then even > if you have say 100 connections, SQLite will not use > more than 30 MB of heap. If a connection needs to > cache pages, unpinned cache pages from other > connections will be discarded in LRU order. Strictly > speaking, SQLite could use more than the 30 MB > soft limit if it has no other choice to allocate memory > (hence a soft limit), but in general that does not happen.
We want to limit/avoid incremental growth of caches, i.e. we prefer to allocate X amount of memory up-front for caches, and have SQLite work primarily with that. If we let it use malloc directly and control it with the soft heap limit, we'll have a steadily rising memory curve until all caches are fully loaded (could take weeks.) Also, my mental model is that there's one memory arena, whether system malloc or configured with SQLITE_CONFIG_PAGECACHE, but any number of independent connection caches competing for memory from that arena. It seems to me the behavior you're describing would require coordination between caches, so maybe my assumptions are wrong here? Thank you, - Kim _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
