> Hi, > > The latest info I can find about SQLITE_ENABLE_MEMORY_MANAGEMENT is > from > mid 2006 - "FAQ said that it might be problematic to use more than one > thread with SQLITE_ENABLE_MEMORY_MANAGEMENT". > > Is this still the case in 3.6.14 ? - The latest FAQ mentions nothing > about SQLITE_ENABLE_MEMORY_MANAGEMENT being problematic anymore. > > Background: > > I have a single process that uses sqlite 3.5.1 and opens 350 sqlite > databases, each db has a dedicated write thread and there is a global > "reader" thread pool shared amongst the 350 databases. > The files are never closed because we aim to provide 99.999% > availability. > After a few days on rhel5 64bit, sqlite consumes all of the 20gigs of > physical memory and all of the swap, machine dies.
Why is SQLite eating up all this memory? How large are the configured page caches? First thing to do is upgrade to 3.6.14 if it is at all possible. 3.5.1 is officially ancient. The list of bugs fixed since then, thread related and otherwise, must be enormous. > Ive played around with SQLITE_ENABLE_MEMORY_MANAGEMENT in 3.5.1 on > windows (visual c++ 2005)- seems to work okay for a while but > eventually > the heap is corrupted - most probably because of the thread issue with > memory management. > > Looks like my options are: > > 1. Multi-threaded memory management now supported in SQLite 3.6.14 > (fingers crossed - don't like my chances). In 3.6.14 the APIs associated with SQLITE_ENABLE_MEMORY_MANAGEMENT work across threads. 3.5.1 was the first release to support this, incidentally. > 2. Close each database often - this is a bad approach for 5 x 9's. > 3. Re-architect the writer threads to do both writes and reads - bad > for > performance. Using the same thread/connection-handle for reading and writing a database might improve performance. Dan. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
