On 09/14/2015 09:13 PM, Abilio Marques wrote: > ?Hi, > > I know I'm a newcomer into the SQLite project, but I'm excited about what > FTS5 has to offer. To me it seems simple and powerful, and has some really > nice ideas. > > Is it possible for me to contribute on the module, or is it too late for > that? > > I would like to mention two new ideas I would offer to introduce. First, a > customizable list of stopwords: > > https://en.wikipedia.org/wiki/Stop_words > ? > (I didn't find anything similar to that in the documentation, am I missing > something?) > > I know I can add it via a custom tokenizer, but wouldn't it be useful to > have it straight out of the box?
Hi, I think such a thing would be implemented using the custom tokenizer API even if it were shipped as part of FTS5. As a "wrapper tokenizer" similar to the built-in porter tokenizer perhaps. If we had code for a stop-words implementation that seemed like it would work for everybody and any licensing issues could be worked out then there's no reason something like that couldn't be made part of FTS5. Dan.
