I wonder if HeapSetInformation (which can enable a low-fragmentation heap) would be helpful too. You can set it on the process and the CRT heaps. Note that it's not available in Win2K and earlier.
Doug -----Original Message----- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Marcus Grimm Sent: Friday, December 17, 2010 9:21 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Windows performance problems associated with malloc() > An SQLite user has brought to our attention a performance issue in > SQLite that seems to be associated with malloc(). If you have > insights or corroborating experience with this issue please let me know. We recently had a malloc/free slowdown issue after changing to VS2008 in combination with XP. Not sure if it applies in your case but for us this helps: -- if( _get_sbh_threshold() < 512 ) { _set_sbh_threshold(512); } --- I'm unable to run your sqlite3.exe: MSVCR100.dll no found. Anyway, maybe the above helps. KInd regards Marcus > > SQLite supports a "zero-malloc option" (see > http://www.sqlite.org/malloc.html#memsys5 for details) which uses its > own internal memory allocator rather than system malloc(). Earlier > today, we patched the command-line shell to allow the zero-malloc > option to be turned on. If you do: > > sqlite3 DATABASE > > then the regular system memory allocator is used, but if you say: > > sqlite3 -heap 100M DATABASE > > then the MEMSYS5 memory allocator will be used with a pool of 100MB of > memory to work with. (You can adjust the size of your memory pool for > whatever you need.) > > There are win32 and win64 builds of this updated command-line shell > compiled using vs2010 here: > > http://www.sqlite.org/draft/download.html > > For certain full-text search queries against a large database, we are > seeing speeds which are 3x faster when using "-heap 300M" (the memsys5 > memory > allocator) versus omitting the -heap option and thus using system > malloc(). > This is on windows7. Similar results are seen with both gcc and > vs2010 builds. > > If you have any large queries that you can run on windows using the > command-line shell, I would appreciate you timing those queries using > the new shells from the download page, both with "-heap 300M" and > without it, and letting me know about any performance differences you see. > > I also observe that compiling for 64-bit using vs2010 (not an option > with my ancient version 2.95.3 gcc cross-compiler) that the queries > are an additional 2x faster. I was surprised at the dramatic > performance increase in going from 32-bit to 64-bit. Is such a > speed-up typical? > > The use of "-heap 300M" seems to not make any performance difference > on Linux. > > Any insights into why this is, what we are doing wrong, or what we can > do to improve the performance of malloc() on windows will be > appreciated. > > -- > D. Richard Hipp > d...@sqlite.org > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
