Re: [sqlite] FTS5: how to find special character sequence containing '.' and '"'

Hi sorry, just forget it. I have overseen a typo. > Am 2019-11-01 um 15:00 schrieb mailing lists : > > Assume the following statements > > CREATE VIRTUAL TABLE NamesFTS USING FTS5 (LastName); > INSERT INTO NamesFTS (LastName) VALUES('L.'); > INSERT INTO NamesFTS (LastName) VALUES('"L."'); > >

[sqlite] FTS5: how to find special character sequence containing '.' and '"'

Assume the following statements CREATE VIRTUAL TABLE NamesFTS USING FTS5 (LastName); INSERT INTO NamesFTS (LastName) VALUES('L.'); INSERT INTO NamesFTS (LastName) VALUES('"L."'); SELECT rowid FROM NamesFTS; > 1 > 2 SELECT rowid FROM NamesFTS WHERE NamesFTS MATCH '"L."'; > 1 > 2 How do I get

Re: [sqlite] [FTS5] Potential table name escape issue with ORDER BY rank

On 21/8/62 01:06, Matt Haynie wrote: Hello sqlite-users, Apologies if this isn’t formatted correctly, I’m not used to submitting bug reports via mailing lists. Although I’m sure some people will be shaking their head, I chose to use periods between words in table names. I’ve been careful to

[sqlite] [FTS5] Potential table name escape issue with ORDER BY rank

Hello sqlite-users, Apologies if this isn’t formatted correctly, I’m not used to submitting bug reports via mailing lists. Although I’m sure some people will be shaking their head, I chose to use periods between words in table names. I’ve been careful to escape table names everywhere, so this

Re: [sqlite] FTS5 Transaction Leads to NULL Pointer

On Tue, 19 Mar 2019 at 07:35, Dan Kennedy wrote: > > Now fixed here: > >https://sqlite.org/src/info/45c73deb440496e8 From that diff, it seems that you changed the documentation of the function's parameters where the function was defined (see fts5_hash.c line 489 at the right) but not where

Re: [sqlite] FTS5 Transaction Leads to OOB Read

YES. AFL with ASAN. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] FTS5 Transaction Leads to OOB Read

That's the error I got too. Now fixed here:   https://sqlite.org/src/info/b3fa58dd7403dbd4 Dan. On 19/3/62 04:22, Dominique Pellé wrote: Chu wrote: The code: ``` CREATE VIRTUAL TABLE t1 USING fts5(content); BEGIN; INSERT INTO t1 (content) VALUES('');

Re: [sqlite] FTS5 Transaction Leads to NULL Pointer

On 18/3/62 15:48, Chu wrote: The code: ``` CREATE VIRTUAL TABLE t1 USING fts5(content); INSERT INTO t1 VALUES(''); BEGIN ; DELETE FROM t1 WHERE rowid = 1; SELECT * FROM t1 WHERE content MATCH ''; INSERT INTO t1 VALUES('');

Re: [sqlite] FTS5 Transaction Leads to OOB Read

Chu wrote: > The code: > > ``` > CREATE VIRTUAL TABLE t1 USING fts5(content); > > BEGIN; > INSERT INTO t1 (content) VALUES(''); > SELECT * FROM > t1('*'); > END; > ``` > > As you

[sqlite] FTS5 Transaction Leads to NULL Pointer

The code: ``` CREATE VIRTUAL TABLE t1 USING fts5(content); INSERT INTO t1 VALUES(''); BEGIN ; DELETE FROM t1 WHERE rowid = 1; SELECT * FROM t1 WHERE content MATCH ''; INSERT INTO t1 VALUES(''); SELECT * FROM t1 WHERE

[sqlite] FTS5 Transaction Leads to OOB Read

The code: ``` CREATE VIRTUAL TABLE t1 USING fts5(content); BEGIN; INSERT INTO t1 (content) VALUES(''); SELECT * FROM t1('*'); END; ``` As you can see, it creates a virtual table

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

Hi Hideaki, Thanks for your reply which made me figure out why I said icu version does "not" support Chinese: b/c in Chinese '中文' can be tokenize as either '中文' or '中' or '文' so when query '中文' or '中*' I can get the result but no result when query '文'. The same goes to '为什么', which can be be

Re: [sqlite] FTS5 MATCH using "string*" matches just the exact string

Hi Then I can do nothing about it. Sent from my iPhone > On Shahrivar 31, 1397 AP, at 7:50 PM, Dan Kennedy > wrote: > >> On 09/22/2018 02:11 PM, Maziar Parsijani wrote: >> Hi,Dan >> I have trimmed it here you can download it: >>

Re: [sqlite] FTS5 MATCH using "string*" matches just the exact string

On 09/22/2018 02:11 PM, Maziar Parsijani wrote: Hi,Dan I have trimmed it here you can download it: https://drive.google.com/file/d/1D0xKF3DaOEbvfe9yFf0aFQvpo2nxWYqF/view?usp=sharing And the problem is searching with MATCH 'أعلم*' it must find 5 rows. it could be the same as I use LIKE "%أعلم%"

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

Hello, full text search index can be used to see how the text is tokenized for both FTS4 and FTS5. for FTS4, fts3tokenize can be used too. sqlite> CREATE VIRTUAL TABLE icu_zh_cn USING fts3tokenize(icu, zh_CN); sqlite> SELECT token, start, end, position FROM icu_zh_cn WHERE INPUT='为什么不支持中文 fts5

Re: [sqlite] FTS5 MATCH using "string*" matches just the exact string

Hi,Dan I have trimmed it here you can download it: https://drive.google.com/file/d/1D0xKF3DaOEbvfe9yFf0aFQvpo2nxWYqF/view?usp=sharing And the problem is searching with MATCH 'أعلم*' it must find 5 rows. it could be the same as I use LIKE "%أعلم%" On Tue, Sep 11, 2018 at 8:35 PM Dan Kennedy

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

Hi, It was exactly like you said, my bad, so now I have built an icu version. BUT unfortunately it still does not support CJK, why is that ? qiulangs-MacBook-Pro:sqlite-autoconf-3250100 qiulang$ ./sqlite3 SQLite version 3.25.1 2018-09-18 20:20:44 Enter ".help" for usage hints. Connected to a

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

> On Sep 20, 2018, at 11:01 PM, 邱朗 wrote: > > https://www.sqlite.org/fts5.html said " > The unicode tokenizer classifies all unicode characters as either "separator" > or "token" characters. By default all space and punctuation characters, as > defined by

Re: [sqlite] FTS5 minimum number of characters to index ?

> On Sep 21, 2018, at 3:26 AM, Domingo Alvarez Duarte > wrote: > > looking at some fts5 tables it seems that an option to limit the minimum > number of characters to at least 2 or 3 would be a good shot as stopwords, A real stop-word list is valuable, but I don’t think a simple

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

On 09/21/2018 09:44 PM, 邱朗 wrote: I actually first used ./configure CFLAGS="-DSQLITE_ENABLE_ICU `icu-config --cppflags`" LDFLAGS="`icu-config --ldflags`" But I got the error When you ran this configure command, is the first line out output something like the following? bash:

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

I actually first used ./configure CFLAGS="-DSQLITE_ENABLE_ICU `icu-config --cppflags`" LDFLAGS="`icu-config --ldflags`" But I got the error sqlite3.c:184184:10: fatal error: 'unicode/utypes.h' file not found #include Then I added -I -L switches and if I remembered correct I used brew to

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

On 09/21/2018 05:21 PM, 邱朗 wrote: Hi, Thanks for replying my question. Following are the error I got when compiling sqlite-autoconf-3250100.tar.gz . The error looks similar to this old discussion http://sqlite.1065341.n5.nabble.com/compiling-Sqlite-with-ICU-td40641.html I am using macOS

[sqlite] FTS5 min_word_size patch small error

Hello ! On my last post about a patch to fts5 to add an option "min_word_size" there is a small mistake on the comparison: Original with mistake: if(p->nMinWordSize && p->nMinWordSize >= wsz) continue; New with mistake fixed (it should be ">" instead of ">="): if(p->nMinWordSize &&

[sqlite] FTS5 min_word_size patch

Hello ! After reporting here previously about this issue I've got a working implementation of "min_word_size" option to Unicode61Tokenizer see patch bellow. With it here is the result of a simple test: ./sqlite3 SQLite version 3.26.0 2018-09-20 20:43:28 Enter ".help" for usage hints.

[sqlite] FTS5 minimum number of characters to index ?

Hello ! I'm looking in the documentation and it doesn't seem to mention any option to specify a minimum number of characters to index, looking at some fts5 tables it seems that an option to limit the minimum number of characters to at least 2 or 3 would be a good shot as stopwords, another

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

Hi, Thanks for replying my question. Following are the error I got when compiling sqlite-autoconf-3250100.tar.gz . The error looks similar to this old discussion http://sqlite.1065341.n5.nabble.com/compiling-Sqlite-with-ICU-td40641.html I am using macOS 10.13 & xcode 10 Undefined symbols

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

On 09/21/2018 01:38 PM, 邱朗 wrote: I think it could be made to work, or at least, I have experience making it work with CJK based on functionality exposed via ICU. I don't know if the unicode tokenizer uses ICU or if the functionality in ICU that I used is available in the unicode tables. Not

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

On Fri, Sep 21, 2018 at 12:39 AM 邱朗 wrote: > > >I think it could be made to work, or at least, I have experience > >making it work with CJK based on functionality exposed via ICU. I > >don't know if the unicode tokenizer uses ICU or if the functionality > >in ICU that I used is available in the

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

> >I think it could be made to work, or at least, I have experience >making it work with CJK based on functionality exposed via ICU. I >don't know if the unicode tokenizer uses ICU or if the functionality >in ICU that I used is available in the unicode tables. Not >understanding any of the

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

On Fri, Sep 21, 2018 at 12:02 AM 邱朗 wrote: > > https://www.sqlite.org/fts5.html said " The unicode tokenizer classifies all > unicode characters as either "separator" or "token" characters. By default > all space and punctuation characters, as defined by Unicode 6.1, are > considered

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

https://www.sqlite.org/fts5.html said " The unicode tokenizer classifies all unicode characters as either "separator" or "token" characters. By default all space and punctuation characters, as defined by Unicode 6.1, are considered separators, and all other characters as token characters... "

Re: [sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

On Thu, Sep 20, 2018, 8:21 PM 邱朗 wrote: > Hi, > I had thought Unicode61 Tokenizer can support CJK -- Chinese Japanese > Korean I verify my sqlite supports fts5 > > {snipped} > > But to my surprise it can't find any CJK word at all. Why is that ? Based on my experience with such things, I

[sqlite] Why sqlite fts5 Unicode61 Tokenizer does not support CJK(Chinese Japanese Krean)?

Hi, I had thought Unicode61 Tokenizer can support CJK -- Chinese Japanese Korean I verify my sqlite supports fts5 sqlite> pragma compile_options; BUG_COMPATIBLE_20160819 COMPILER=clang-9.0.0 DEFAULT_CACHE_SIZE=2000 DEFAULT_CKPTFULLFSYNC DEFAULT_JOURNAL_SIZE_LIMIT=32768 DEFAULT_PAGE_SIZE=4096

Re: [sqlite] FTS5 MATCH using "string*" matches just the exact string

On 09/11/2018 01:26 PM, Maziar Parsijani wrote: Thanks for your answer Eric,Sandu About the first idea of Eric I have to say its correct for me when I do a test with another fts5 table with no huge rows but my database has that problem.And about the other possibility I tested that before to set

Re: [sqlite] FTS5 MATCH using "string*" matches just the exact string

Thanks for your answer Eric,Sandu About the first idea of Eric I have to say its correct for me when I do a test with another fts5 table with no huge rows but my database has that problem.And about the other possibility I tested that before to set "*"at the first and end of the match but I have

Re: [sqlite] FTS5 MATCH using "string*" matches just the exact string

In Arabic you have a different direction of the text. Might be that the reason? Sandu Eric Minbiole schrieb am Di., 11. Sep. 2018, 00:23: > > SELECT rowid,text > > FROM table > > WHERE table MATCH 'أعلم*'; > > > > And I have to add this that my data is Arabic text. > > > > This method must

Re: [sqlite] FTS5 MATCH using "string*" matches just the exact string

> SELECT rowid,text > FROM table > WHERE table MATCH 'أعلم*'; > > And I have to add this that my data is Arabic text. > > This method must find words that contains 'أعلم' but it doesn't. What > should I do now. > > I just tried this in SQLite version 3.24.0 on a Mac, and it seems to work. Please

[sqlite] FTS5 MATCH using "string*" matches just the exact string

Hi I am new here and I am not an expert on sqlite my question is about match option that I have read from https://sqlite.org/fts5.html but the problem is when I want to not find the exact word so I use : SELECT rowid,text FROM table WHERE table MATCH 'أعلم*'; And I have to add this that my

Re: [sqlite] fts5 giving results for substring searches for Hindi content.

On 02/04/2018 11:39 AM, raj Singla wrote: Hi, -- create fts4 and fts5 tables create virtual table idx4 using "fts4" (content); create virtual table idx5 using "fts5" (content); -- insert 1 sample rows into eachinsert into idx4 (content) values ('नीरजा भनोट के कातिल पाकिस्तान की जेल में थे, फिर

Re: [sqlite] fts5 giving results for substring searches for Hindi content.

raj Singla wrote: > create virtual table idx4 using "fts4" (content); > create virtual table idx5 using "fts5" (content); > ... > select * from idx4 where idx4 match 'पाकि';-- no results returned > select * from idx5 where idx5 match 'पाकि';-- returns नीरजा भनोट के FTS4 and FTS5 have different

[sqlite] fts5 giving results for substring searches for Hindi content.

Hi, -- create fts4 and fts5 tables create virtual table idx4 using "fts4" (content); create virtual table idx5 using "fts5" (content); -- insert 1 sample rows into eachinsert into idx4 (content) values ('नीरजा भनोट के कातिल पाकिस्तान की जेल में थे, फिर वे एफबीआई आए?');insert into idx5 (content)

Re: [sqlite] FTS5 crash with NULL in snippet() function's 5th parameter

On 12/26/2017 03:16 AM, Tony Papadimitriou wrote: (SQLite v3.21.0 with FTS5 enabled) If instead of ‘’ (empty string) one uses NULL for the 5th parameter in the snippet() function it consistently crashes. Thanks for reporting this. Now fixed (so that NULL is equivalent to an empty string)

[sqlite] FTS5 crash with NULL in snippet() function's 5th parameter

(SQLite v3.21.0 with FTS5 enabled) If instead of ‘’ (empty string) one uses NULL for the 5th parameter in the snippet() function it consistently crashes. Thanks. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] FTS5 performance problem.

John Found wrote: > Pseudocode: ?1 = keyword + ' ThreadID:' + num2str(threadID); > > select count() from PostFTS where PostFTS match ?1; That works. > ?1 = keyword > ?2 = threadID > select count() from PostFTS where PostFTS match ?1 ThreadID:?2; That is not valid SQL. You have to create a

Re: [sqlite] FTS5 performance problem.

On Sun, 28 May 2017 14:23:47 +0200 Clemens Ladisch wrote: > It might be better to use a single FTS lookup for both words: > > select count() from PostFTS where PostFTS match 'innermost ThreadID:6'; > Thanks! This is indeed much faster. But I am passing the search query

Re: [sqlite] FTS5 performance problem.

John Found wrote: > the execution time increases by a factor of 100 > > select count() from PostFTS where PostFTS match 'innermost' and ThreadID = 6; This is the same as: select count() from PostFTS where PostFTS match 'innermost' and PostFTS match 'ThreadID:6'; It might be better to use a

[sqlite] FTS5 performance problem.

I have a FTS5 virtual external content table, defined following way: CREATE VIRTUAL TABLE PostFTS using fts5( ThreadID, Content, content=Posts, content_rowid=id, tokenize='porter unicode61 remove_diacritics 1'); The following query executes for 4..5ms: select count() from PostFTS where PostFTS

[sqlite] FTS5 bug in SQLite 3.18.0

When SQLIte is compiled with sanitizers they report invalid readouts and undefined-behavior: ``` sqlite3.c:188900:5: runtime error: index 11 out of bounds for type 'char [8]' SUMMARY: AddressSanitizer: undefined-behavior sqlite3.c:188900:5 in sqlite3.c:189142:30: runtime error: index 12 out of

Re: [sqlite] FTS5 query results different before/after MERGE command.

On 04/21/2017 02:36 AM, Jonathan Gaillard wrote: Here is the db https://ufile.io/q0314 Thanks for reporting this. It was a bug in the fts5 prefix query code. Now fixed here: http://www.sqlite.org/src/info/840042cb2bed2924 Dan. If you do: select rowid, name from filesfts where

Re: [sqlite] FTS5 query results different before/after MERGE command.

Here is the db https://ufile.io/q0314 If you do: select rowid, name from filesfts where filesfts match '"upload file漢_5"*'; you get back: 141| upload file漢_5 142| upload file漢_6 152| upload file漢_5 153| upload file漢_6 163| upload file漢_5 164| upload file漢_6 174| upload file漢_5 175| upload

Re: [sqlite] FTS5 query results different before/after MERGE command.

On 04/20/2017 07:11 AM, Jonathan Gaillard wrote: If you do: select rowid, name from filesfts where filesfts match '"upload file漢_5"*'; This certainly seems like a bug. Are you able to share the database that exhibits the problem? Thanks, Dan. you get back: 141| upload file漢_5

Re: [sqlite] FTS5 query results different before/after MERGE command.

If you do: select rowid, name from filesfts where filesfts match '"upload file漢_5"*'; you get back: 141| upload file漢_5 142| upload file漢_6 152| upload file漢_5 153| upload file漢_6 163| upload file漢_5 164| upload file漢_6 174| upload file漢_5 175| upload file漢_6 185| upload file漢_5 196| upload

Re: [sqlite] FTS5 query results different before/after MERGE command.

On 04/18/2017 02:00 AM, Jonathan Gaillard wrote: I have a db where I have a query returning results from something I believe is deleted already from a FTS5 table. This is a normal FTS5 table, not a contentless one so I used the normal DELETE. Only after I run: INSERT INTO

[sqlite] FTS5 query results different before/after MERGE command.

I have a db where I have a query returning results from something I believe is deleted already from a FTS5 table. This is a normal FTS5 table, not a contentless one so I used the normal DELETE. Only after I run: INSERT INTO MyFTSTable(MyFTSTable, rank) VALUES('automerge', 0); INSERT INTO

Re: [sqlite] FTS5 not working

On 03/15/2017 06:00 AM, Domingo Alvarez Duarte wrote: Hello ! I just downloaded the http://www.sqlite.org/snapshot/sqlite-snapshot-201703062044.tar.gz compiled it with fts5 enabled and then tested it with this: === CREATE VIRTUAL TABLE email USING fts5(body); insert into email(body)

[sqlite] FTS5 not working MY MISTAKE FORGET

Hello ! Sorry by my previous message, it was my mistake when quering the fts5. I was quering like normal sql referring to a specific column in the where clause instead of using the table name. Cheers ! ___ sqlite-users mailing list

[sqlite] FTS5 not working

Hello ! I just downloaded the http://www.sqlite.org/snapshot/sqlite-snapshot-201703062044.tar.gz compiled it with fts5 enabled and then tested it with this: === CREATE VIRTUAL TABLE email USING fts5(body); insert into email(body) values('hello over there'); select rowid, body from email

Re: [sqlite] FTS5 pagination

Well, the obvious answer is to use OFFSET and LIMIT … but of course that might slow down too much as the page number increases. Or you could avoid the need for pagination by leaving the statement active between page-loads, and just reading another N rows out of it to jump to the next page.

[sqlite] FTS5 pagination

Hi, How can I paginate fts5 queries when ordered by rank? Normailly I use an index on an expression that gives me something to put in a WHERE exp > X and then also ORDER BY exp. Exp always has to be deterministic so it might be a text field (name or whatever) concatenated to a fixed length text

[sqlite] FTS5 ranking across multiple databases

Hi All, I'm trying to work out if there is a way to use/abuse/extend SQLite and FTS5 to allow me to do a query across two (or more) separate databases and then sort the results in a meaningful manner. In order to do this I really need the 'global' elements of the ranking algorithm (e.g.

[sqlite] FTS5 query that to match all rows.

On 04/13/2016 11:24 PM, John Found wrote: > What FTS5 query should I use in order to match all rows in the table? > Can you use "SELECT * FROM fts_table;"? Dan.

[sqlite] FTS5 query that to match all rows.

On Wed, 13 Apr 2016 19:41:34 +0200 R Smith wrote: > Or did you mean you publish an interface of sorts in your application > that allows users to run their own queries, but it /has/ to be an "fts5" > query in some way, and now sometimes they want to list all rows? It is > quite baffling - but

[sqlite] FTS5 query that to match all rows.

On Wed, 13 Apr 2016 23:38:04 +0700 Dan Kennedy wrote: > On 04/13/2016 11:24 PM, John Found wrote: > > What FTS5 query should I use in order to match all rows in the table? > > > > Can you use "SELECT * FROM fts_table;"? > *I* can. But the users of the program where I use SQLite can't simply

[sqlite] FTS5 query that to match all rows.

On 2016/04/13 6:44 PM, John Found wrote: > On Wed, 13 Apr 2016 23:38:04 +0700 > Dan Kennedy wrote: > >> On 04/13/2016 11:24 PM, John Found wrote: >>> What FTS5 query should I use in order to match all rows in the table? >>> >> Can you use "SELECT * FROM fts_table;"? >> > *I* can. But the users

[sqlite] FTS5 problem ? more than 3,000 queries !

On 04/04/2016 05:43 AM, Domingo Alvarez Duarte wrote: > Hello ! > > I'm testing fts5 on a database with data from hacker news (around 13M > records) and it took more than 2 hours to index, I have this application to > log all queries executed for debugging and although the search isn't slow > it's

[sqlite] FTS5 problem ? more than 3,000 queries !

And the indexes are not supposed to be used ? Shouldn't a index cover the ranking too ? It seems a mistake for me. Cheers ! > Mon Apr 04 2016 11:07:51 AM CEST from "Dan Kennedy" > Subject: Re: [sqlite] FTS5 problem ? more than >3,000 >queries ! > > On

[sqlite] FTS5 problem ? more than 3,000 queries !

Hello ! I'm testing fts5 on a database with data from hacker news (around 13M records) and it took more than 2 hours to index, I have this application to log all queries executed for debugging and although the search isn't slow it's making a lot of queries. Is this normal ? More than 3,000

[sqlite] FTS5 problem ? more than 3,000 queries !

On 3 Apr 2016, at 11:43pm, Domingo Alvarez Duarte wrote: > > Is this normal ? More than 3,000 queries ? It looks to me as if your SELECT is returning 3000 results. If you do a search for a rarer term does it execute fewer queries ? For instance you might search for '' instead of ''.

[sqlite] FTS5 Prefix Indexes

Hello ! Is it possible to expose commands to create/remove prefix indexes ? Something like (or any other friendly way to do it without reindex the whole database): INSERT INTO ft(ft) VALUES('remove prefix 2 3'); INSERT INTO ft(ft) VALUES('remove prefix 2'); INSERT INTO ft(ft)

[sqlite] FTS5 "constraint failed"

On 03/23/2016 12:06 AM, Domingo Alvarez Duarte wrote: > Hello ! > > After seeing several times work/commits on fts5 I decided to try it on a > table shown bellow, and when trying to populate it I get this error message: > > > sqlite> INSERT INTO fts_idx_items(fts_idx_items) VALUES('rebuild'); >

[sqlite] FTS5 "constraint failed"

16 06:23:53 PM CET from "Dan Kennedy" > Subject: Re: [sqlite] FTS5 "constraint failed" > > On 03/23/2016 12:06 AM, Domingo Alvarez Duarte wrote: > >>Hello ! >> >> After seeing several times work/commits on fts5 I decided to try it on a >>

[sqlite] FTS5 "constraint failed"

? -ledit -ldl -lpthread -lm -Wl,-rpath -Wl,/usr/local/lib ? > Tue Mar 22 2016 06:06:55 PM CET from "Domingo Alvarez Duarte" > Subject: [sqlite] FTS5 "constraint failed" > > Hello ! > > After seeing several times work/commits on fts5 I decided to try it on a

[sqlite] FTS5 "constraint failed"

Hello ! After seeing several times work/commits on fts5 I decided to try it on a table shown bellow, and when trying to populate it I get this error message: sqlite> INSERT INTO fts_idx_items(fts_idx_items) VALUES('rebuild'); Error: constraint failed The table has 12,000,000 records and

[sqlite] FTS5 prefix index documentation may be incorrect

On 11/25/2015 05:11 AM, Charles Leifer wrote: > The FTS5 prefix index documentation[1] seems to not be working. I've tried > with SQLite 3.9.0 and 3.10.0 (2015-11-06) and both show the same error > messages. Thanks for reporting this. Now fixed here:

[sqlite] FTS5 prefix index documentation may be incorrect

Thank you for the quick fix, and thank you so much for SQLite! On Nov 25, 2015 5:57 AM, "Dan Kennedy" wrote: > On 11/25/2015 05:11 AM, Charles Leifer wrote: > >> The FTS5 prefix index documentation[1] seems to not be working. I've tried >> with SQLite 3.9.0 and 3.10.0 (2015-11-06) and both show

[sqlite] FTS5 prefix index documentation may be incorrect

The FTS5 prefix index documentation[1] seems to not be working. I've tried with SQLite 3.9.0 and 3.10.0 (2015-11-06) and both show the same error messages. Examples: sqlite> CREATE VIRTUAL TABLE ft USING fts5(a, b, prefix='2 3'); Error: malformed prefix=... directive sqlite> CREATE VIRTUAL TABLE

[sqlite] FTS5 rank configuration option

On 11/06/2015 02:26 PM, chromedout64 at yahoo.com wrote: > The documentation says that a persistent rank can be set with > INSERT INTO ft(ft, rank) VALUES('rank', 'bm25(10.0, 5.0)'); > However, I can only get it to work on the same database connection as the > query. Subsequent database

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

On 11/06/2015 12:36 PM, chromedout64 at yahoo.com wrote: > I noticed that this update is not included in the official 3.9.2 release. How > is it determined when an update such as this one gets included in a release? > Thanks for any info. Most likely as part of 3.10.0. You can always download

[sqlite] FTS5 rank configuration option

The documentation says that a persistent rank can be set with INSERT INTO ft(ft, rank) VALUES('rank', 'bm25(10.0, 5.0)'); However, I can only get it to work on the same database connection as the query. Subsequent database connections seem to use the standard default ranking. Is this by design?

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

Great, that's exactly what I need, I'll give it a try.

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

I noticed that this update is not included in the official 3.9.2 release. How is it determined when an update such as this one gets included in a release? Thanks for any info.

[sqlite] FTS5 explicitly set delimiter

On 11/02/2015 01:55 AM, chromedout64 at yahoo.com wrote: > Thanks, I figured that this might be the case. What is the best way to > specify all characters except whitespace as part of a CREATE VIRTUAL TABLE > statement? Should you simply list each of the literal ascii characters such > as

[sqlite] FTS5 explicitly set delimiter

On 11/01/2015 06:39 PM, chromedout64 at yahoo.com wrote: > Is there an easy way to set an FTS5 table so that the only > delimiter/separator is a space and everything else, including all > punctuation, is a token? Some searching revealed that there is an > undocumented feature as a part of

[sqlite] FTS5 explicitly set delimiter

Thanks, I figured that this might be the case. What is the best way to specify all characters except whitespace as part of a CREATE VIRTUAL TABLE statement? Should you simply list each of the literal ascii characters such as tokenchars '!@#$%' and so on. Or is it possible or would it be better

[sqlite] FTS5 explicitly set delimiter

Is there an easy way to set an FTS5 table so that the only delimiter/separator is a space and everything else, including all punctuation,?is a token? Some searching revealed that there is an undocumented?feature as a part of FTS3/4 that allows the actual delimiter to be specified as part of

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

On 10/27/2015 05:06 PM, chromedout64 at yahoo.com wrote: > Using the example from the documentation at https://www.sqlite.org/fts5.html > I thought this worked in a previous version. > > CREATE TABLE tbl(a INTEGER PRIMARY KEY, b, c); > CREATE VIRTUAL TABLE fts_idx USING fts5(b, c, content='tbl',

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

Thanks for fixing it, Dan. Do you know when this will be rolled into an official or preliminary autoconf file? Thanks again.

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

This is on v3.9.1. I'm not sure exactly when it stopped working. I was using the external content functionality with triggers to keep the virtual table updated as part of a larger program, but found that the referenced example from the docs exhibits the same issue. INSERT's work fine, but

[sqlite] FTS5 External Content Table - REPLACE - SQL logic error or missing database

Using the example from the documentation at https://www.sqlite.org/fts5.html I thought this worked in a previous version. CREATE TABLE tbl(a INTEGER PRIMARY KEY, b, c); CREATE VIRTUAL TABLE fts_idx USING fts5(b, c, content='tbl', content_rowid='a'); CREATE TRIGGER tbl_ai AFTER INSERT ON tbl BEGIN

[sqlite] FTS5 issue on OS X

Oh yeah, bingo :) Thanks? Gergely On 22/10/2015 17:05, "Scott Hess" wrote: >Dollars to donuts you're compiling SQLite but then linking against the >system version. > >-scott > > >On Thu, Oct 22, 2015 at 7:51 AM, Gergely Lukacsy (glukacsy) < >glukacsy at cisco.com> wrote: > >> Hi Simon, >> >>

[sqlite] FTS5 issue on OS X

Hi Simon, Thanks for coming back to me. I ran sqlite3_compileoption_get in a loop to enumerate all options (I assume this is the same as the pragma query suggested below) and I got the following: 2015-10-22T14:46:24.000Z [0x7fff7679d300] DBWrapper.cpp:33 DBInit:OptiomENABLE_FTS3

[sqlite] FTS5 issue on OS X

On 22 Oct 2015, at 2:38pm, Gergely Lukacsy (glukacsy) wrote: > ... I get an "error: no module: fts5" error message. This indicates that I > didn't compile SQLite with FTS5 support, but I am pretty sure that I am using > the right preprocessor flags Using that version of SQLite, can you

[sqlite] FTS5 issue on OS X

Hi, I am using Sqlite in a codebase shared between Windows and OS X and specifically I am using FTS4 for searching our DB for free text strings. I have just upgraded to the latest 3.9.1 and thought to give a try to FTS5, but encountered an issue. It works fine on Windows, but on OS X the

[sqlite] FTS5 issue on OS X

Dollars to donuts you're compiling SQLite but then linking against the system version. -scott On Thu, Oct 22, 2015 at 7:51 AM, Gergely Lukacsy (glukacsy) < glukacsy at cisco.com> wrote: > Hi Simon, > > Thanks for coming back to me. > > I ran sqlite3_compileoption_get in a loop to enumerate all

[sqlite] FTS5 returns "corrupt" plus trailing zero

On 09/21/2015 05:14 PM, Ralf Junker wrote: > My implementation of > > http://www.sqlite.org/src/artifact/400384798349d658?ln=94-96 > > returns "corrupt" plus a trailing zero, that is 8 characters in total. > > Maybe this line > > http://www.sqlite.org/src/artifact/4fdbc0a321e3a1d7?ln=5364 > >

[sqlite] FTS5 returns "corrupt" plus trailing zero

My implementation of http://www.sqlite.org/src/artifact/400384798349d658?ln=94-96 returns "corrupt" plus a trailing zero, that is 8 characters in total. Maybe this line http://www.sqlite.org/src/artifact/4fdbc0a321e3a1d7?ln=5364 should be corrected to sqlite3Fts5BufferSet(, , 7,

[sqlite] FTS5 stopwords

On 09/14/2015 09:13 PM, Abilio Marques wrote: > ?Hi, > > I know I'm a newcomer into the SQLite project, but I'm excited about what > FTS5 has to offer. To me it seems simple and powerful, and has some really > nice ideas. > > Is it possible for me to contribute on the module, or is it too late for

[sqlite] FTS5 stopwords

On 09/14/2015 09:13 PM, Abilio Marques wrote: > ?Hi, > > I know I'm a newcomer into the SQLite project, but I'm excited about what > FTS5 has to offer. To me it seems simple and powerful, and has some really > nice ideas. > > Is it possible for me to contribute on the module, or is it too late for

[sqlite] FTS5 documentation typo

On 09/14/2015 06:31 PM, Abilio Marques wrote: > While reading the documentation draft for FTS5, I spotted this on > section 4.3.3 . > I believe the first example is missing quotation marks around the word > 'porter': > > *-- Two ways to create an FTS5 table that uses the porter tokenizer to > --

[sqlite] FTS5 stopwords

I?ve implemented a custom ranker in SQLite that is similar to SPH_RANK_SPH04 using FTS4 (BM25 + word distance and distance to beginning of text). The only thing that wasn?t possible out of the box using FTS4 was to get the distance between found matches as distance between them (how many words are

  1   2   >