> What I can easily do without breaking 4.0.x "gamma" status, is to add > command line switch --disable-fulltext-stopwords. It can help as a > temporary solution, untill a proper fix - per-index options, that is - > will be implemented.
That would be helpful for me, but what about Thomas Spahni's suggestion? > > Sergei, > > but then, could you also add a command line switch > > --read-stopwords-from-file="filename" ??? > > Please. That could solve half of my problem. > > Best regards, > Thomas Spahni I was mere wondering why the stopword list was 'hardcoded' since it seems to me that it's one of those things a user should be able to change/modify without to much hassle and on a more frequent basis than whenever one recompile MySQL. Also a stopword list is very dependent on what kind of text/data one wants to search in so a large system with multiple users and databases might want different stopword lists... > > > I remember working on a project when I was school where we > wrote this > > program using autogenerated stopword lists and N-gram > matching for the text > > and search string. By this the stopword list was not hard coded.. > > What is "N-gram matching" ? > I post this to the MySQL board, since maybe someone else has something to add/say about it too :) Don't know where I got these texts from, but it should give you a general idea about n-grams. ************************ n-grams are used to describe objects as vectors. This makes it possible to apply geometric, statistical and other mathematical techniques, which are well defined for vectors, but not for objects in general. For example, one of the most common uses is to define a similarity measure between textual documents based on the application of a mathematical function to the vector representations of the documents ************************ N-Grams String-similarity approaches to conflation involve the system calculating a measure of similarity between an input query term and each of the distinct terms in the database. Those database terms that have a high similarity to a query term are then displayed to the user for possible inclusion in the query. N-gram matching techniques are one of the most common of these approaches (Freund & Willett, 1982). An n-gram is a set of n consecutive characters extracted from a word. The main idea behind this approach is that, similar words will have a high proportion of n-grams in common. Typical values for n are 2 or 3, these corresponding to the use of digrams or trigrams, respectively. So if you have the word 'computer' you'll get the following digrams: *c, co, om, mp, pu, ut, te, er, r* and the trigrams: **c,*co,com,omp,mpu,put,ute,ter,er*,r** where '*' denotes a padding space. There are n+1 such digrams and n+2 such trigrams in a word containing n characters. Found this link after some 'googling about' http://web.umr.edu/~tauritzd/ngram/ This is probably the original text for the first text I had: http://web.umr.edu/~tauritzd/ngram/tutorial.html > Regards, > Sergei > > -- > MySQL Development Team > __ ___ ___ ____ __ > / |/ /_ __/ __/ __ \/ / Sergei Golubchik <[EMAIL PROTECTED]> > / /|_/ / // /\ \/ /_/ / /__ MySQL AB, http://www.mysql.com/ > /_/ /_/\_, /___/\___\_\___/ Osnabrueck, Germany > <___/ > --------------------------------------------------------------------- Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail <[EMAIL PROTECTED]> To unsubscribe, e-mail <[EMAIL PROTECTED]> Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
