Re: [sqlite] FTS3 tokenize unicode61 does not remove diacritics correctly?

So, how to handle, figure this Uni bug out? -- Sent from: http://sqlite.1065341.n5.nabble.com/ ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] FTS3 tokenize unicode61 does not remove diacritics correctly?

not provide a decomposition mapping for ``ł'', or ``Ł''. Even if it is a bug, then it will concern the Unicode standard rather then SQLite FTS3 itself, as the latter is using the character database provided by the Unicode standard. -- best regards Cezary H. Noweta

[sqlite] FTS3 tokenize unicode61 does not remove diacritics correctly?

Hi all, I have an issue with FTS3 (http://www.sqlite.org/fts3.html). I am creating virtual table using fts3 to query tokens: CREATE VIRTUAL TABLE tok1 USING fts3tokenize(unicode61); Documentation says: „By default, "unicode61" also removes all diacritics from Latin script characters.”; When I

[sqlite] FTS3 - Unexpected SELECT Results

I created a table?with the Porter tokenizer: "CREATE VIRTUAL TABLE fts_translations USING fts3(name, tokenize=porter)" Then I execute a select statement, which before?text substitution looks like this: SELECT [symbols].`id`, [symbols].`rid` FROM [symbols] ?INNER JOIN [symbol_translations] ON

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

On 11 May 2014 00:01, Scott Robison-2 [via SQLite] < ml-node+s1065341n75608...@n5.nabble.com> wrote: > I suspect the primary use case it was designed and tested for (and in fact > the way we use it at my place of employment) was more for "only growing > datasets" and less for an environment where

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

On Fri, May 9, 2014 at 4:20 AM, Andrew Moss wrote: > I am really surprised that FTS behaves this way. To my mind this is a bug > in the FTS extension that makes it unusable for many applications. Was > anyone else aware of this problem or made attempts at resolving

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

I am now totally convinced that FTS3/4 does not work for this usage model. If you are deleting and inserting documents, the size of the FTS index will grow in a linear manner with no limit no matter what you do with the merge command (when you run it, what parameters you provide). I have

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

On 2 May 2014 10:22, Dan Kennedy wrote: > > > A query like: > > SELECT level, count(*) AS ntree FROM yourftstablename_segdir; > > will tell you how many b-trees there currently are at each level. Which > might help you figure out what is going on and when you might expect

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

On 05/02/2014 07:30 PM, Stadin, Benjamin wrote: Is the delete marker also set on old keys on UPDATE? Or just DELETE -> INSERT? Internally, FTS implements an UPDATE as a DELETE followed by an INSERT. Dan. I ran into the ever-growing FTS index issue last year. I’m creating DB diffs which

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

Is the delete marker also set on old keys on UPDATE? Or just DELETE -> INSERT? I ran into the ever-growing FTS index issue last year. I’m creating DB diffs which also contain some FTS3/4 tables. The tables get constantly updated for the checksum. The DBs were always vacuum’ed, but the growing FTS

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

On 05/02/2014 04:13 PM, Andrew Moss wrote: On 2 May 2014 07:57, Dan Kennedy wrote: On 05/01/2014 03:30 PM, andrewmo wrote: We are using the FTS3 extension to sqlite to store large numbers of short (~300 byte) documents. This is working very well and providing us with

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

On 2 May 2014 07:57, Dan Kennedy wrote: > On 05/01/2014 03:30 PM, andrewmo wrote: > >> We are using the FTS3 extension to sqlite to store large numbers of short >> (~300 byte) documents. This is working very well and providing us with >> very >> fast text search, but the

Re: [sqlite] FTS3/4 merge function behaviour when deleting rows

On 05/01/2014 03:30 PM, andrewmo wrote: We are using the FTS3 extension to sqlite to store large numbers of short (~300 byte) documents. This is working very well and providing us with very fast text search, but the behaviour around deletion of documents has me confused. Our system must control

[sqlite] FTS3/4 merge function behaviour when deleting rows

We are using the FTS3 extension to sqlite to store large numbers of short (~300 byte) documents. This is working very well and providing us with very fast text search, but the behaviour around deletion of documents has me confused. Our system must control the the size of the database and will

[sqlite] FTS3/4 bug with tokenize=icu and parentheses

A few days ago I suspected a bug with SQLite FTS4 parenthesis when using the ICU tokenizer. To rule out it was my slightly altered SQLite build environment to compile SQLite + ICU for the iPhone, I reproduced this issue today with the current version of SQLite on OS X. Steps to reproduce: SQLite

[sqlite] FTS3 prefix wildcard

Hi All, A user wants to be able to query my FTS3 database using a preceeding wildcard. From reading back through the mailing list it looks like SQLite may not support this (for very understandable reasons) but I cannot find any mention of this in the docs. Is this limitation? To get round this I

Re: [sqlite] FTS3/FTS4 - Finding the term(s) that completes the input

Hi Simon, Abhinav and Filip, On 21/11/2011 11:32 AM, Simon Slavin wrote: You have to look at the rows it returns and see how many of them there are. If there's only one, that's your hit. If there are more than one, see how many characters you can move along the row before they start to be

Re: [sqlite] FTS3/FTS4 - Finding the term(s) that completes the input

Hi Filip, On 21/11/2011 4:33 PM, Filip Navara wrote: On Mon, Nov 21, 2011 at 4:13 AM, Mohit Sindhwani wrote: I think my examples muddied the waters. I have looked at Section 3 of the FTS documents and that lets me bring back the "full result" that matches - so, if I search

Re: [sqlite] FTS3/FTS4 - Finding the term(s) that completes the input

On Mon, Nov 21, 2011 at 4:13 AM, Mohit Sindhwani wrote: > Hi Abhinav, > > > On 21/11/2011 2:52 AM, Abhinav Upadhyay wrote: > >> On Mon, Nov 21, 2011 at 12:17 AM, Mohit Sindhwani wrote: >> >>> What I'd like to be able to do is something like this: >>> >>> - let's

Re: [sqlite] FTS3/FTS4 - Finding the term(s) that completes the input

On 21 Nov 2011, at 3:13am, Mohit Sindhwani wrote: > I think my examples muddied the waters. I have looked at Section 3 of the > FTS documents and that lets me bring back the "full result" that matches - > so, if I search for 'ling j', it can tell me that the result that matches is > 'james

Re: [sqlite] FTS3/FTS4 - Finding the term(s) that completes the input

Hi Abhinav, On 21/11/2011 2:52 AM, Abhinav Upadhyay wrote: On Mon, Nov 21, 2011 at 12:17 AM, Mohit Sindhwani wrote: What I'd like to be able to do is something like this: - let's say that the FTS4 table has values such as: * mohit sindhwani, onghu * john doe, gmail * james

Re: [sqlite] FTS3/FTS4 - Finding the term(s) that completes the input

On Mon, Nov 21, 2011 at 12:17 AM, Mohit Sindhwani wrote: > Hi, I'm finding my way through FTS3/FTS4 to replace some of the old code > that we have for searching terms within titles.  I now know that FTS3/4 > should be the way to proceed. > > So far, I have this: > - an FTS4 table

[sqlite] FTS3/FTS4 - Finding the term(s) that completes the input

Hi, I'm finding my way through FTS3/FTS4 to replace some of the old code that we have for searching terms within titles. I now know that FTS3/4 should be the way to proceed. So far, I have this: - an FTS4 table that has two columns: title (main column), ext (certain conditions to match) -

Re: [sqlite] fts3 table name cannot be the same as column name

On 08/17/2011 10:10 PM, George Brink wrote: > I think this is a bug... If you trying to create FTS3/FTS4 table, make > sure that column name in FTS table is not the same as the table name. > > C:\Projects>sqlite3 aaa > SQLite version 3.7.7.1 2011-06-28 17:39:05 > Enter ".help" for instructions >

[sqlite] fts3 table name cannot be the same as column name

I think this is a bug... If you trying to create FTS3/FTS4 table, make sure that column name in FTS table is not the same as the table name. C:\Projects>sqlite3 aaa SQLite version 3.7.7.1 2011-06-28 17:39:05 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> create

Re: [sqlite] [FTS3] Understanding the Flow of data through the tokenizer

On Mon, Jul 25, 2011 at 9:54 AM, Dan Kennedy wrote: > On 07/24/2011 08:16 PM, Abhinav Upadhyay wrote: >> Hi, >> >> I am trying to write my own custom tokenizer to filter stopwords apart >> from doing normalization and stemming. I have gone through the >> comments in

Re: [sqlite] [FTS3] Understanding the Flow of data through the tokenizer

On 07/24/2011 08:16 PM, Abhinav Upadhyay wrote: > Hi, > > I am trying to write my own custom tokenizer to filter stopwords apart > from doing normalization and stemming. I have gone through the > comments in fts3_tokenizer.h and also read the implementation of the > simple tokenizer. While overall

[sqlite] [FTS3] Understanding the Flow of data through the tokenizer

Hi, I am trying to write my own custom tokenizer to filter stopwords apart from doing normalization and stemming. I have gone through the comments in fts3_tokenizer.h and also read the implementation of the simple tokenizer. While overall I am able to understand what I need to do to implement

Re: [sqlite] [FTS3] Header to include for a custom tokenizer

On Sun, Jul 24, 2011 at 1:40 AM, Abhinav Upadhyay wrote: > On Sat, Jul 23, 2011 at 11:00 PM, Richard Hipp wrote: >> On Sat, Jul 23, 2011 at 1:01 PM, Abhinav Upadhyay < >> er.abhinav.upadh...@gmail.com> wrote: >> >>> Hi, >>> >>>  I am using the

Re: [sqlite] [FTS3] Header to include for a custom tokenizer

On Sat, Jul 23, 2011 at 11:00 PM, Richard Hipp wrote: > On Sat, Jul 23, 2011 at 1:01 PM, Abhinav Upadhyay < > er.abhinav.upadh...@gmail.com> wrote: > >> Hi, >> >>  I am using the Sqlite3 amalgamation. I am trying to register a custom >> tokenizer with sqlite for my FTS

Re: [sqlite] [FTS3] Header to include for a custom tokenizer

On Sat, Jul 23, 2011 at 1:01 PM, Abhinav Upadhyay < er.abhinav.upadh...@gmail.com> wrote: > Hi, > > I am using the Sqlite3 amalgamation. I am trying to register a custom > tokenizer with sqlite for my FTS application. The custom tokenizer is > in it's separate source file. I have included

[sqlite] [FTS3] Header to include for a custom tokenizer

Hi, I am using the Sqlite3 amalgamation. I am trying to register a custom tokenizer with sqlite for my FTS application. The custom tokenizer is in it's separate source file. I have included sqlite3.h header with the tokenizer source but sqlite3.h does not contain the declaration of the various

Re: [sqlite] [FTS3] The Compress and Uncompress functions and extension

2011/7/22 Abhinav Upadhyay : > Thanks for pointing out that mail archive discussion. I wasn't using > compress/uncompress because uncompress requires you to store the size > of the compressed buffer which is returned by the compress function > while compressing. But

Re: [sqlite] [FTS3] The Compress and Uncompress functions and extension

On Fri, Jul 22, 2011 at 1:32 PM, Abhinav Upadhyay wrote: > On Fri, Jul 22, 2011 at 12:38 PM, Alexey Pechnikov > wrote: >> But why you don't use compress/uncompress functions from DRH? See >>

Re: [sqlite] [FTS3] The Compress and Uncompress functions and extension

On Fri, Jul 22, 2011 at 12:38 PM, Alexey Pechnikov wrote: > But why you don't use compress/uncompress functions from DRH? See > http://www.mail-archive.com/sqlite-users%40sqlite.org/msg17018.html > > I did wrap these into extension and add SQLITE_COMPRESS_MIN_LENGTH >

Re: [sqlite] [FTS3] The Compress and Uncompress functions and extension

But why you don't use compress/uncompress functions from DRH? See http://www.mail-archive.com/sqlite-users%40sqlite.org/msg17018.html I did wrap these into extension and add SQLITE_COMPRESS_MIN_LENGTH http://sqlite.mobigroup.ru/artifact/a5da96353bb851b34114052ba85041fdffb725cd

Re: [sqlite] [FTS3] The Compress and Uncompress functions

On Wed, Jul 20, 2011 at 7:51 PM, Abhinav Upadhyay wrote: > Hi, > > I have an FTS table with compress and uncompress options enabled. I am > using zlib(3) for doing the compression. The compression function > seems to be doing ok as I can see the size of the database

[sqlite] [FTS3] The Compress and Uncompress functions

Hi, I have an FTS table with compress and uncompress options enabled. I am using zlib(3) for doing the compression. The compression function seems to be doing ok as I can see the size of the database coming down drastically. But I the uncompress function is not working properly. For example if I

Re: [sqlite] FTS3: synonyms dictionary and tokens length

2011/7/15 Dan Kennedy : >> But queries to original text will not work: >> select text from fts where fts match 'sqlite educate'; > > I think it will. Query strings - like 'sqlite educate' - are > also parsed using the tokenizer. So the query will be transformed > to 'dbms

Re: [sqlite] FTS3: synonyms dictionary and tokens length

On 07/15/2011 01:10 PM, Alexey Pechnikov wrote: > 2011/7/15 Dan Kennedy: >> I think you could just have the tokenizer return "dbms" whenever >> it sees "sqlite" in the input. > > But queries to original text will not work: > select text from fts where fts match 'sqlite

Re: [sqlite] FTS3: synonyms dictionary and tokens length

2011/7/15 Dan Kennedy : > I think you could just have the tokenizer return "dbms" whenever > it sees "sqlite" in the input. But queries to original text will not work: select text from fts where fts match 'sqlite educate'; You can see synonyms dictionary in PostgreSQL

Re: [sqlite] FTS3: synonyms dictionary and tokens length

On 07/15/2011 03:28 AM, Alexey Pechnikov wrote: > I want to add the table CREATE TABLE ext_fts_synonyms(word text not > null unique, synonym text not null); insert into > ext_fts_synonyms('sqlite','sqlite dbms'); > > And replace in tokenizer the term 'sqlite' to 2 terms 'sqlite' and > 'dbms' for

Re: [sqlite] FTS3: synonyms dictionary and tokens length

I want to add the table CREATE TABLE ext_fts_synonyms(word text not null unique, synonym text not null); insert into ext_fts_synonyms('sqlite','sqlite dbms'); And replace in tokenizer the term 'sqlite' to 2 terms 'sqlite' and 'dbms' for search by queries like to > select text from fts where fts

Re: [sqlite] FTS3: synonyms dictionary and tokens length

On 07/14/2011 07:42 PM, Alexey Pechnikov wrote: >> No. Don't return anything for a stop word. Just advance to the next >> non stop-word token and return it. > > Thanks, I did and it's work. > > And another question... Is there any way to use multi-word synonyms? Like to: > sqlite -> Open Source

Re: [sqlite] FTS3: synonyms dictionary and tokens length

> No. Don't return anything for a stop word. Just advance to the next > non stop-word token and return it. Thanks, I did and it's work. And another question... Is there any way to use multi-word synonyms? Like to: sqlite -> Open Source SQLite DBMS I think the single token "Open Source SQLite

Re: [sqlite] FTS3: synonyms dictionary and tokens length

On 07/14/2011 05:29 PM, Alexey Pechnikov wrote: > With 0-length token in icuNext there is the error: > Error: SQL logic error or missing database > > May xNext returns 0 length when the token is stopword? No. Don't return anything for a stop word. Just advance to the next non stop-word token and

Re: [sqlite] FTS3: synonyms dictionary and tokens length

With 0-length token in icuNext there is the error: Error: SQL logic error or missing database May xNext returns 0 length when the token is stopword? -- Best regards, Alexey Pechnikov. http://pechnikov.tel/ ___ sqlite-users mailing list

Re: [sqlite] FTS3: synonyms dictionary and tokens length

On 07/13/2011 05:05 PM, Alexey Pechnikov wrote: > With synonyms dictionary the result token length can be more then > original token length. > Is it problem for current realization of FTS? I don't think so. If it is, it's a bug. Dan. ___ sqlite-users

[sqlite] FTS3: synonyms dictionary and tokens length

With synonyms dictionary the result token length can be more then original token length. Is it problem for current realization of FTS? -- Best regards, Alexey Pechnikov. http://pechnikov.tel/ ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] FTS3/FTS4+ICU segfault on NULLs

On Mon, Mar 07, 2011 at 02:21:39AM +0300, Alexey Pechnikov wrote: > This code produce segfault on current trunk SQLite and some previous > releases (3.7.0.1 and 3.7.3): > > CREATE VIRTUAL TABLE fts USING fts3(name,TOKENIZE icu ru_RU); > insert into fts (name) values (NULL); > insert into fts

[sqlite] FTS3/FTS4+ICU segfault on NULLs

This code produce segfault on current trunk SQLite and some previous releases (3.7.0.1 and 3.7.3): CREATE VIRTUAL TABLE fts USING fts3(name,TOKENIZE icu ru_RU); insert into fts (name) values (NULL); insert into fts (name) values (NULL); delete from fts; Or: CREATE VIRTUAL TABLE fts USING

[sqlite] FTS3 can not support double-quote character search.

An unmatched double-quote the right-hand argument to MATCH causes an error in FTS3. Example: CREATE VIRTUAL TABLE t1 USING fts3(x); INSERT INTO t1 VALUES('abc"def"123'); SELECT * FROM t1 WHERE t1 MATCH '"*'; The SELECT statement above returns: "malformed MATCH expression: ["*]. Can this feature

Re: [sqlite] FTS3 bug

When I got that error I, for some reason, interpreted it as an SQL-level error message, and not an FTS-level one. Now I realise that it indicates an FTS-level error, so it makes sense. Since I thought it was an SQL-level error I thought it was an SQL injection bug. Sorry for crying wolf. It would

Re: [sqlite] FTS3 bug

On Fri, Jan 14, 2011 at 8:18 PM, Jonas Bengtsson wrote: > Hi all, > > I've found a defect in FTS3 where it will accept matching with the > string "" (two quotes) but not """ (three quotes). > > Three double-quotes is not a valid query string, so FTS3/4 returns an error code.

Re: [sqlite] FTS3 bug

Hi Phil, I am binding the values, not just concatenating them into the query or manually escaping them. So that's why I believe there's a bug in FTS3. I'm binding the value, but still it fails depending on the value of the variable. In Python it's: cursor.execute('SELECT * FROM MyTable WHERE

Re: [sqlite] FTS3 bug

On 15 Jan 2011, at 01:18, Jonas Bengtsson wrote: > 2) Any suggestions as to how to sanitize the user's input to avoid > this problem? This is just the one I've found, but I'm assuming there > are more. Don't try and sanitize the user's input - you will never be able to prove you have accounted

[sqlite] FTS3 bug

Hi all, I've found a defect in FTS3 where it will accept matching with the string "" (two quotes) but not """ (three quotes). I first came across the issue on Android, but have since reproduced it in Python. 1) I assume this is the place to post FTS3 bug reports, please let me know otherwise.

[sqlite] FTS3 + SQLITE_OMIT_PAGER_PRAGMAS causes division by zero exception

Recent changes to FTS3 apparently require that SQLite must be compiled with pager pragmas, otherwise FTS3 will cause a division by zero exception as I have experienced right now. This means that the FTS3 extension can crash an application if the core SQLite library is compiled with

Re: [sqlite] FTS3 indexing in PHP < 5.3

> lk > - Original Message - > From: "Richard Hipp" <d...@sqlite.org> > To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org> > Sent: Monday, December 20, 2010 1:36:51 PM GMT -05:00 US/Canada Eastern > Subject: Re: [sqlite] FT

Re: [sqlite] FTS3 indexing in PHP < 5.3

M GMT -05:00 US/Canada Eastern Subject: Re: [sqlite] FTS3 indexing in PHP < 5.3 On Mon, Dec 20, 2010 at 1:31 PM, Lee Kulberda <lkulbe...@nxtbookmedia.com>wrote: > Hello all > > I am trying to run a FTS3 on text files on a server running PHP Version > 5.2.14. with sqlite 3.5.9 >

Re: [sqlite] FTS3 indexing in PHP < 5.3

On Mon, Dec 20, 2010 at 1:31 PM, Lee Kulberda wrote: > Hello all > > I am trying to run a FTS3 on text files on a server running PHP Version > 5.2.14. with sqlite 3.5.9 > > Going through the command line directly to sqlite3 the virtual tables and > FTS3 works fine but

[sqlite] FTS3 indexing in PHP < 5.3

Hello all I am trying to run a FTS3 on text files on a server running PHP Version 5.2.14. with sqlite 3.5.9 Going through the command line directly to sqlite3 the virtual tables and FTS3 works fine but I am trying to incorporate into a php app. I get the following error: SQLSTATE[HY000]:

Re: [sqlite] FTS3 bug?

Dan Kennedy wrote: > On 12/08/2010 04:18 AM, Iker Arizmendi wrote: >> The function that opens a cursor for the simple tokenizer, >> simpleOpen, does not set the "pTokenizer" member of the >> returned cursor. Ie, it appears the following line is >> missing: >> >> c->base.pTokenizer =

Re: [sqlite] FTS3 bug?

On 12/08/2010 04:18 AM, Iker Arizmendi wrote: > The function that opens a cursor for the simple tokenizer, > simpleOpen, does not set the "pTokenizer" member of the > returned cursor. Ie, it appears the following line is > missing: > > c->base.pTokenizer = pTokenizer; > > which causes

[sqlite] FTS3 bug?

The function that opens a cursor for the simple tokenizer, simpleOpen, does not set the "pTokenizer" member of the returned cursor. Ie, it appears the following line is missing: c->base.pTokenizer = pTokenizer; which causes problems in simpleNext . Possible bug? Regards, Iker -- Iker

[sqlite] FTS3: custom tokenizer filter over built-in tokenizer

Hello all, I'd like to create an FTS3 tokenizer that filters stop words by internally creating the simple or porter tokenizers and filtering its output as needed (ala Lucene's StandardAnalyzer). Eg, CREATE VIRTUAL TABLE test USING fts3(txt, tokenize=stopword simple w1 ... wN) However,

[sqlite] FTS3: custom tokenizer filter over built-in tokenizer (resend)

Hello all, I'd like to create an FTS3 tokenizer that filters stop words by internally creating the simple or porter tokenizers and filtering its output as needed (ala Lucene's StandardAnalyzer). Eg, CREATE VIRTUAL TABLE test USING fts3(txt, tokenize=stopword simple w1 ... wN) However,

Re: [sqlite] FTS3

no FTS3 is no "real" virtual table and can be saved into the database file if you open it the index is already generated. but if you add ne entrys these have to be processed and added to the index. so if you crate a new FTS3 virtual table and insert all the entrys of a normla table it will

[sqlite] FTS3

Hello, I've got a question: Is it correct, that using FTS3 with virtual tables that these tables need to be build everytime from scratch? Or are they persisted in one way or the other? Thanks Ralf ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] FTS3 snippets() grouping

On Nov 19, 2010, at 10:14 PM, Matthew Leffler wrote: > I thought of that but I get an error with that query: > >>> unable to use function snippet in the requested context > Pesky function :P Then, if everything else fails, you could write it down in a temp table, and group that sigh...

Re: [sqlite] FTS3 snippets() grouping

I thought of that but I get an error with that query: >> unable to use function snippet in the requested context Thanks! Matthew On Nov 19, 2010, at 2:08 PM, Petite Abeille wrote: > > On Nov 19, 2010, at 9:46 PM, Matthew Leffler wrote: > >> A question: Is is possible to group the data from

Re: [sqlite] FTS3 snippets() grouping

On Nov 19, 2010, at 9:46 PM, Matthew Leffler wrote: > A question: Is is possible to group the data from the snippets() column in a > result? The query would look something like the following (which doesn't > work): > >>> SELECT snippet(search, '', '', '...') as extract, count() as count

[sqlite] FTS3 snippets() grouping

A question: Is is possible to group the data from the snippets() column in a result? The query would look something like the following (which doesn't work): >> SELECT snippet(search, '', '', '...') as extract, count() as count FROM >> search WHERE content MATCH 'search term' GROUP BY extract

Re: [sqlite] sqlite/fts3 feature request: select snippet() from fts3 where docid=1

On Sat, Nov 6, 2010 at 7:45 AM, Simon Hefti wrote: > Hello > > Do I understand correctly that fts3 snippet function does only work > with match queries like > select snippet(text) from text where text match 'foo' > > I would be interested to have: > select snippet(text)

[sqlite] sqlite/fts3 feature request: select snippet() from fts3 where docid=1

Hello Do I understand correctly that fts3 snippet function does only work with match queries like select snippet(text) from text where text match 'foo' I would be interested to have: select snippet(text) from text where docid=1 In sqlite 3.7.2 this statement is valid but returns an empty

[sqlite] FTS3 simple tokenizer splits at underscore

The documentation at http://www.sqlite.org/fts3.html#tokenizer states about the tokenizer eligible characters are all alphanumeric characters, the "_" character, and all characters with UTF codepoints greater than or equal to 128 This suggests to me that an underscore is part of

Re: [sqlite] FTS3 MATCH syntax

Your suggestion seems to work fairly well. It might have some unexpected results, but I'll definitely give it a try. Thanks! From: Sam Roberts FTS3 tokenizes strings on whitespace (and other chars), so I think the best you can do would be something like given:john given:q.

Re: [sqlite] FTS3 MATCH syntax

On Fri, Sep 24, 2010 at 7:16 PM, GHCS Software wrote: > I'm just started with generating a table using FTS3 and understand > most of the query formats, but am stuck on one minor point of syntax. My > FTS3 query will make reference to a specific table, e.g.: > > MATCH

Re: [sqlite] FTS3 MATCH syntax

I'm not an authority, but I've been using FTS3. FTS3 tokenizes strings on whitespace (and other chars), so I think the best you can do would be something like given:john given:q. It doesn't work really well out of the box for substring matching. On Fri, Sep 24, 2010 at 4:16 PM, GHCS Software

[sqlite] FTS3 MATCH syntax

I'm just started with generating a table using FTS3 and understand most of the query formats, but am stuck on one minor point of syntax. My FTS3 query will make reference to a specific table, e.g.: MATCH 'surname:smith' The problem is that I can't figure out how to specify it if the

[sqlite] FTS3: Combining match-column, prefix query and phrase query

Hi, Is it somehow possible to combine these three things? (1) match column: ... MATCH 'column:foo' (2) prefix query: ... MATCH 'foo*' (3) phrase query: ... MATCH '"foo bar"' I think (1) and (2) is no problem. For example if I want to search all documents containing words beginning OR ending

Re: [sqlite] FTS3 finds too much: Slash special meaning? Something else?

Am 03.09.2010 13:27, schrieb Dan Kennedy: > > On Sep 2, 2010, at 6:37 PM, Lukas Haase wrote: > >> Hi, >> >> I use FTS3 (SQLITE_ENABLE_FTS3) with enhanced query syntax >> (SQLITE_ENABLE_FTS3_PARENTHESIS). >> >> Now if I search for a string like '2002/91/AH' there are lots of items >> which do NOT

Re: [sqlite] FTS3 finds too much: Slash special meaning? Something else?

On Sep 2, 2010, at 6:37 PM, Lukas Haase wrote: > Hi, > > I use FTS3 (SQLITE_ENABLE_FTS3) with enhanced query syntax > (SQLITE_ENABLE_FTS3_PARENTHESIS). > > Now if I search for a string like '2002/91/AH' there are lots of items > which do NOT contain this string. This is a query: > > SELECT

[sqlite] FTS3 finds too much: Slash special meaning? Something else?

Hi, I use FTS3 (SQLITE_ENABLE_FTS3) with enhanced query syntax (SQLITE_ENABLE_FTS3_PARENTHESIS). Now if I search for a string like '2002/91/AH' there are lots of items which do NOT contain this string. This is a query: SELECT rowid, content FROM fulltext WHERE content MATCH '2002/91/AH'; In

[sqlite] FTS3 snippet problem: function returns full document content

For some documents snippet function returns full document content: sqlite> select length(snippet(file_text)) from file_text where file_text match 'переоформление договора' and rowid=1015; 42312 sqlite> select substr(snippet(file_text),1,500) from file_text where file_text match 'переоформление

Re: [sqlite] FTS3 - NEAR operator

On Jul 3, 2010, at 2:37 AM, Patel, Vinit wrote: > Hi : > > I was wondering how the NEAR operator works for the SQL database. > In the SQLite test suite, in test file fts3near.test, One of the > test case is as follows > > //fts3near-3.1 test case > > db eval { > DELETE FROM t1; > INSERT INTO

[sqlite] FTS3 - NEAR operator

Hi : I was wondering how the NEAR operator works for the SQL database. In the SQLite test suite, in test file fts3near.test, One of the test case is as follows //fts3near-3.1 test case db eval { DELETE FROM t1; INSERT INTO t1(content) VALUES( 'one two three two four six three six nine

[sqlite] fts3 column-specific phrase search?

I find myself wanting to do an FTS3 phrase search restricted to content in a specific column. The docs are pretty clear that this doesn't work (column prefixes only apply to tokens), but I wonder how hard it would be to make it work. I'm happy to work on it if I can be pointed in the right

Re: [sqlite] FTS3 query question

> But I'm not getting any results when I combine these using OR: > > SELECT rowid FROM EntryHeaders eh WHERE ( eh.Paragraph MATCH '91' ) OR ( > eh.Title MATCH 'civil' ) See http://www.sqlite.org/fts3.html#section_3 . Your query should be more like: SELECT rowid FROM EntryHeaders WHERE

[sqlite] FTS3 query question

I'm using sqlite 3.6.23.1. I have a FTS3 table created using: CREATE VIRTUAL TABLE EntryHeaders USING FTS3( Paragraph, Title, tokenize=porter ); The following individual queries work and return rows: SELECT rowid FROM EntryHeaders eh WHERE ( eh.Paragraph MATCH '91' ) SELECT rowid FROM

[sqlite] FTS3 performance problem

Dears FTS3 performance is acceptable for SQLite 3.6.20 when the FTS virtual table contains 10K~100K records. (Query speed: 20K records/ 0.4831sec) but the performance is bad for SQLite 3.6.21~3.6.23.1 (Query speed: 20K records/ > 7sec) The query is simple: *SELECT docid FROM XTable WHERE XTable

Re: [sqlite] FTS3 stop words/chars

On Fri, Apr 30, 2010 at 2:18 PM, Richard Hipp wrote: > > > On Fri, Apr 30, 2010 at 4:43 PM, Paul Rigor (uci) wrote: > >> Hi, >> >> What are the default stop words and characters for the FTS3 simple >> tokenizer? >> >> > The tokenizers built into FTS3 do not

Re: [sqlite] FTS3 stop words/chars

On Fri, Apr 30, 2010 at 4:43 PM, Paul Rigor (uci) wrote: > Hi, > > What are the default stop words and characters for the FTS3 simple > tokenizer? > > The tokenizers built into FTS3 do not use any stop words. > Thanks, > Paul > > -- > Paul Rigor > Pre-doctoral BIT Fellow and

[sqlite] FTS3 stop words/chars

Hi, What are the default stop words and characters for the FTS3 simple tokenizer? Thanks, Paul -- Paul Rigor Pre-doctoral BIT Fellow and Graduate Student Institute for Genomics and Bioinformatics Donald Bren School of Information and Computer Sciences University of California, Irvine

[sqlite] FTS3 Japanese Tokenization

Hi All, Does anyone know if ICU supports tokenization of Japanese text? Thanks Ray This e-mail message, including any attachments, is for the sole use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended

Re: [sqlite] FTS3 Appropriate usage

> Sqlite is nice for all projects where are a lot of selects and a few > insert/update operations per second. For typical web-project a selects > operations > are dominate. > > See some tests here: > geomapx.blogspot.com/2010/04/sqlite-index-degradation-tests.html > geom

Re: [sqlite] FTS3 Appropriate usage

On Fri, Apr 16, 2010 at 3:24 AM, Alexey Pechnikov wrote: > And you can use my patches for zlib-compression for FTS3. I'm planning to make > the "fts3z" extension because I want to use as original FTS3 > as FTS3 with compression together. Back when I was working up fts1, I

Re: [sqlite] FTS3 Appropriate usage

f selects and a few insert/update operations per second. For typical web-project a selects operations are dominate. See some tests here: geomapx.blogspot.com/2010/04/sqlite-index-degradation-tests.html geomapx.blogspot.com/2010/01/sqlite-fts3.html geomapx.blogspot.com/2009/11/postgresql-81-vs-sqlit

Re: [sqlite] FTS3 Appropriate usage

Thanks for your feedback. I gonna keep my data in my main RDBMS instead (which supports full-text searching as well). If it's too slow, i still have to option of setting up a database cluster or something. On Apr 15, 1:41 pm, Simon Slavin wrote: > On 15 Apr 2010, at 8:21am,

Re: [sqlite] FTS3 Appropriate usage

On 15 Apr 2010, at 8:21am, Jens wrote: > I'd appreciate any feedback you might have one this. Also, does anyone > have experience with sqlite+ft3 and high-availability solutions? Has > anyone done any benchmarking of fts3? For a start, FTS3 is a very specific solution to a very specific

  1   2   3   >