On Thu, Sep 11, 2014 at 9:58 AM, Dan Kennedy <danielk1...@gmail.com> wrote:
> Fts5 will use less memory and be faster than fts4 (I think - initial > testing has been positive). It will also be smaller, as we can do without a > bunch of code that is used to workaround problems inherent in the > file-format. > The only problem I knew about was memory related in extremely large databases (artificially large, in fact). When we were implementing full text search for our product, we would reach a point where inserting a new record in FTS tables would result in a failure to allocate memory. I worked around it at the time (not having time to fully understand FTS to try to fix the problem I had found) by creating FTS "buckets" to spread the load. The schema and use case allowed for a single search on an individual bucket to return what was needed and prevented the previously observed "worst case" scenario with artificial data. If FTS5 will use less memory in extreme cases, that is good. Note: Our use case was somewhat atypical I suspect, so I never really considered it a flaw that we were putting it to a use that resulted in memory exhaustion. There will always be limits, and we were indexing millions of documents. -- Scott Robison _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
