On Fri, Dec 17, 2010 at 7:29 PM, Richard Hipp <d...@sqlite.org> wrote:
> > For certain full-text search queries against a large database, we are > seeing > speeds which are 3x faster when using "-heap 300M" (the memsys5 memory > allocator) versus omitting the -heap option and thus using system malloc(). > This is on windows7. Similar results are seen with both gcc and vs2010 > builds. > > If you have any large queries that you can run on windows using the > command-line shell, I would appreciate you timing those queries using the > new shells from the download page, both with "-heap 300M" and without it, > and letting me know about any performance differences you see. > > Just tested a feature to see the number of memory requests for a given query. It is made by keeping a global variable incremented each time malloc is called (actually it is mapped to a different proc) and showing the difference between the values at the start and at the end of the query. Sqlite version 3.7.4 (fts-enabled) For general queries the number of requests is not great, from 10 to 10,000 and if recalculated to requests per second also not so big (100 per sec for example). But for fts3 a small difference in match operand can lead to huge difference in the number of memory requests and you can see that for the phrase query the request per second value is very big. The queries bellow was made against a db with English wikipedia abstracts. The times measured after several repeats so no disk waiting to be affecting. SELECT Count(*) FROM (SELECT * FROM WikiData WHERE Abstract LIKE "%official%" LIMIT 500) Result: 500, mem req: 2163 664 ms. SELECT count(docid) FROM WikiFTS3 where Abstract Match 'after before'; Result: 1539, mem requests: 10157 44 ms. SELECT count(docid) FROM WikiFTS3 where Abstract Match '"of the previous"' Result: 403, mem requests: 1,627,732 3400 ms I don't know what it means, maybe that some optimization might be possible in the internal implementation of fts3 to minimize the number of memory requests. Thanks Max Vlasov _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
