On 09/11/2014 02:51 AM, Kyle Shannon wrote:
I was browsing the source on the fossil repository, and I noticed that
there were files for fts5 in the trunk. I've been playing with fts3/4
and I'm curious about what new features/changes are incorporated into
fts5. Can anyone clarify that for me? Thanks.
Fts5 will use less memory and be faster than fts4 (I think - initial
testing has been positive). It will also be smaller, as we can do
without a bunch of code that is used to workaround problems inherent in
the file-format.
It uses incremental-merging by default, which limits the amount of
writing any single INSERT operation is required to perform.
And other stuff that makes the codebase smaller and easier to manage.
The most user-visible change is the addition of an API that allows users
to write their own auxiliary (i.e. snippet(), rank(), offsets()) functions:
http://www.sqlite.org/src/artifact/064f9bf705e59d
The included snippet() and rank() functions use this API.
There are also changes to allow things like this:
SELECT snippet(fts) FROM fts WHERE fts MATCH 'xyz'
ORDER BY rank(fts) LIMIT 10 OFFSET 20;
to be executed efficiently without jumping through hoops. Specifically,
so that even if 'xyz' matches a large number of documents, only 10
snippets need to be assembled (assembling snippets is expensive in both
IO and CPU).
Fts5 is still in the experimental stage at the moment.
If anybody has any ideas for useful features, or knows of problems with
FTS4 that could be fixed in FTS5, don't keep them to yourself!
Dan.
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
