Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

2018-09-20 Thread Scott Robison
On Fri, Sep 21, 2018 at 12:39 AM 邱朗 wrote: > > >I think it could be made to work, or at least, I have experience > >making it work with CJK based on functionality exposed via ICU. I > >don't know if the unicode tokenizer uses ICU or if the functionality > >in ICU that I used is available in the un

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

2018-09-20 Thread 邱朗
> >I think it could be made to work, or at least, I have experience >making it work with CJK based on functionality exposed via ICU. I >don't know if the unicode tokenizer uses ICU or if the functionality >in ICU that I used is available in the unicode tables. Not >understanding any of the language

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

2018-09-20 Thread Scott Robison
On Fri, Sep 21, 2018 at 12:02 AM 邱朗 wrote: > > https://www.sqlite.org/fts5.html said " The unicode tokenizer classifies all > unicode characters as either "separator" or "token" characters. By default > all space and punctuation characters, as defined by Unicode 6.1, are > considered separators

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

2018-09-20 Thread 邱朗
https://www.sqlite.org/fts5.html said " The unicode tokenizer classifies all unicode characters as either "separator" or "token" characters. By default all space and punctuation characters, as defined by Unicode 6.1, are considered separators, and all other characters as token characters... " I

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

2018-09-20 Thread Scott Robison
On Thu, Sep 20, 2018, 8:21 PM 邱朗 wrote: > Hi, > I had thought Unicode61 Tokenizer can support CJK -- Chinese Japanese > Korean I verify my sqlite supports fts5 > > {snipped} > > But to my surprise it can't find any CJK word at all. Why is that ? Based on my experience with such things, I suspec

[sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

2018-09-20 Thread 邱朗
Hi, I had thought Unicode61 Tokenizer can support CJK -- Chinese Japanese Korean I verify my sqlite supports fts5 sqlite> pragma compile_options; BUG_COMPATIBLE_20160819 COMPILER=clang-9.0.0 DEFAULT_CACHE_SIZE=2000 DEFAULT_CKPTFULLFSYNC DEFAULT_JOURNAL_SIZE_LIMIT=32768 DEFAULT_PAGE_SIZE=4096 DEF

Re: [sqlite] Another Partial Index optimization opportunity (INSERT)

2018-09-20 Thread Deon Brewis
You are of course very right. (The UPDATE loaded the existing column value from the row before it did the 'Le', which is why it was performing an index delete+insert later on. INSERT doesn't do that.). I was expecting it to be kicked out of the query plan completely with an unused column, but a

Re: [sqlite] SQlite 3 - bottleneck with rbuFindMaindb

2018-09-20 Thread Dan Kennedy
On 09/20/2018 03:05 AM, Roger Cuypers wrote: I think it does at some point. I’m at home right now so I have to check this again tomorrow when I have access to the source. Should there be rbu calls if the application is only _reading_ fro the database and not updating? Maybe. If the app calle

Re: [sqlite] Another Partial Index optimization opportunity (INSERT)

2018-09-20 Thread David Raymond
Could you show us the plans for the version you're looking at? When I'm looking at it with 3.25.0 the index inserts are guarded by conditionals such that that index is opened, but never actually accessed when run with the values in your examples. They query plan is a *generic one* that gets use

[sqlite] Another Partial Index optimization opportunity (INSERT)

2018-09-20 Thread Deon Brewis
Hi, I previously reported that an UPDATE of a table containing an Expression and/or Partial Index will unnecessarily touch the expression and/or partial index. I see both are now fixed in the 3.26.0 2018-09-19 codebase. Thank you so much - it works great! However, while testing it, I also notic

Re: [sqlite] [EXTERNAL] Last_row_id

2018-09-20 Thread Hick Gunter
True, but the information available is very limited, basically just the NEW database row and function calls. CREATE TRIGGER return_pono AFTER INSERT ON po BEGIN INSERT INTO inserts VALUES (datetime('now'),'po','return_pono',last_insert_rowid()); END Unless of course you start writing user d

Re: [sqlite] [EXTERNAL] Last_row_id

2018-09-20 Thread Jean-Luc Hainaut
On 17/09/2018 14:05, Hick Gunter wrote: A trigger program does not return any result rows. True. But a "select" query in the body of a trigger can be used to evaluate a user-defined function (in the "where" clause for instance) in which any action allowed by your host language can be executed