An SQLite user has brought to our attention a performance issue in SQLite that seems to be associated with malloc(). If you have insights or corroborating experience with this issue please let me know.
SQLite supports a "zero-malloc option" (see http://www.sqlite.org/malloc.html#memsys5 for details) which uses its own internal memory allocator rather than system malloc(). Earlier today, we patched the command-line shell to allow the zero-malloc option to be turned on. If you do: sqlite3 DATABASE then the regular system memory allocator is used, but if you say: sqlite3 -heap 100M DATABASE then the MEMSYS5 memory allocator will be used with a pool of 100MB of memory to work with. (You can adjust the size of your memory pool for whatever you need.) There are win32 and win64 builds of this updated command-line shell compiled using vs2010 here: http://www.sqlite.org/draft/download.html For certain full-text search queries against a large database, we are seeing speeds which are 3x faster when using "-heap 300M" (the memsys5 memory allocator) versus omitting the -heap option and thus using system malloc(). This is on windows7. Similar results are seen with both gcc and vs2010 builds. If you have any large queries that you can run on windows using the command-line shell, I would appreciate you timing those queries using the new shells from the download page, both with "-heap 300M" and without it, and letting me know about any performance differences you see. I also observe that compiling for 64-bit using vs2010 (not an option with my ancient version 2.95.3 gcc cross-compiler) that the queries are an additional 2x faster. I was surprised at the dramatic performance increase in going from 32-bit to 64-bit. Is such a speed-up typical? The use of "-heap 300M" seems to not make any performance difference on Linux. Any insights into why this is, what we are doing wrong, or what we can do to improve the performance of malloc() on windows will be appreciated. -- D. Richard Hipp d...@sqlite.org _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
