On 08/17/2012 03:58 PM, Mohit Sindhwani wrote:
Hi Ralf,
On 17/8/2012 3:50 PM, Ralf Junker wrote:
On 17.08.2012 09:30, Mohit Sindhwani wrote:
We're using FTS4 and it works well for many things. One of the things
that we'd like to do is to see what terms are being created by the
tokenizer in use. What would be the easiest way to do that?
I tried looking through the fts_aux table and the segments and content
tables, but nothing struck me directly as usable. Any suggestions?
http://www.sqlite.org/fts3.html#fts4aux
I did look at this - but I couldn't figure out a way that allowed me to
see what terms were created by the tokenizer for a particular
expression. Example "SOME TEXT" becomes "SOME", "TEXT" - is there a way
to find that?
You could, I suppose, obtain a handle for the tables tokenizer using
the fts3_tokenizer() function:
http://www.sqlite.org/fts3.html#section_8_1
Then use it to tokenize your expression using the API in
fts3_tokenizer.h. See static function "testFunc()" in
fts3_tokenizer.c for an example.
Dan.
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
