Hi, Windows malloc is slow. There is nothing you can do about it. I have seen huge difference in the same code running on Linux vs Windows (Code which mostly do malloc/realloc/free). I am using google perftools (http://code.google.com/p/google-perftools/) to accelerate malloc. Actually, I am using a patched version which have debug malloc working on win32 too (In addition to being faster) - see details here http://code.google.com/p/google-perftools/issues/detail?id=284
Yoni. On 17/12/2010 6:29 PM, Richard Hipp wrote: > An SQLite user has brought to our attention a performance issue in SQLite > that seems to be associated with malloc(). If you have insights or > corroborating experience with this issue please let me know. > > SQLite supports a "zero-malloc option" (see > http://www.sqlite.org/malloc.html#memsys5 for details) which uses its own > internal memory allocator rather than system malloc(). Earlier today, we > patched the command-line shell to allow the zero-malloc option to be turned > on. If you do: > > sqlite3 DATABASE > > then the regular system memory allocator is used, but if you say: > > sqlite3 -heap 100M DATABASE > > then the MEMSYS5 memory allocator will be used with a pool of 100MB of > memory to work with. (You can adjust the size of your memory pool for > whatever you need.) > > There are win32 and win64 builds of this updated command-line shell compiled > using vs2010 here: > > http://www.sqlite.org/draft/download.html > > For certain full-text search queries against a large database, we are seeing > speeds which are 3x faster when using "-heap 300M" (the memsys5 memory > allocator) versus omitting the -heap option and thus using system malloc(). > This is on windows7. Similar results are seen with both gcc and vs2010 > builds. > > If you have any large queries that you can run on windows using the > command-line shell, I would appreciate you timing those queries using the > new shells from the download page, both with "-heap 300M" and without it, > and letting me know about any performance differences you see. > > I also observe that compiling for 64-bit using vs2010 (not an option with my > ancient version 2.95.3 gcc cross-compiler) that the queries are an > additional 2x faster. I was surprised at the dramatic performance increase > in going from 32-bit to 64-bit. Is such a speed-up typical? > > The use of "-heap 300M" seems to not make any performance difference on > Linux. > > Any insights into why this is, what we are doing wrong, or what we can do to > improve the performance of malloc() on windows will be appreciated. > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
