Re: Full Text Indexs
On 10/14/02 10:00 AM, "Benoit Hediard" <[EMAIL PROTECTED]> wrote: .. > > I recommand you not to empty the noise words files, you'll loose some great > capabilities of full-text search. > For example, if someone search for a letter/number or a common word ('the', > 'or'...), those words should be ignored, otherwise you'll get plenty of > "noise" in your results. > Even if it is minimal, the noise words file should never be empty. > Make sure you know your subject domain. I was indexing popular song titles and using a stop word list. I tried to search for the Huey Lewis song "If This Is It" -- and of course I couldn't find it -- all four are stop words. Of course, in most domains, stop words are good. ~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk FAQ: http://www.thenetprofits.co.uk/coldfusion/faq This list and all House of Fusion resources hosted by CFHosting.com. The place for dependable ColdFusion Hosting.
RE: Full Text Indexs
great - thanks Benoit - I'll take all this on board and re-assess -Original Message- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 14 October 2002 18:00 To: CF-Talk Subject: RE: Full Text Indexs The noise word file used is the one corresponding to the language you specified during the full-text index creation. In the full-text indexing Wizard "Select Table Columns" step, you define a "Language for Word Breaker". For example, we are using "French (France)" in our application, and it works fine. I recommand you not to empty the noise words files, you'll loose some great capabilities of full-text search. For example, if someone search for a letter/number or a common word ('the', 'or'...), those words should be ignored, otherwise you'll get plenty of "noise" in your results. Even if it is minimal, the noise words file should never be empty. Perhaps, should you try to delete all the full-text indexes and re-create them from scratch? Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : lundi 14 octobre 2002 17:18 À : CF-Talk Objet : RE: Full Text Indexs this doesn't appear to be the case here. "After eight" still throws an error as "after" is reserved. eight however is ok. I have checked the noise.eng file and it is empty. Is there any weay of telling which noise file is being used? - I assume it's the English one -Original Message----- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 18:08 To: CF-Talk Subject: RE: Full Text Indexs It should only throw an error if all the search words are defined in the noise words file. For example : "The" will throw an error, "The foo" will return results based on "foo" search. In our application, we have customized the noise word file and put each full text SQL search in blocks. If an error is thrown we create an empty recordset. For example : select ID_NEWS as ID, TITLE_NEWS as Title, CREATEDBYID_NEWS as CreatedByID, CREATEDBY_NEWS as CreatedByLogin, CREATIONDATE_NEWS as CreationDate fromITEM_NEWS and (contains(TITLE_NEWS,'"#arguments.Text#"') or contains(DESCRIPTION_NEWS,'"#arguments.Text#"')) Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 18:41 À : CF-Talk Objet : RE: Full Text Indexs Hi We tried this but it didn't seem to work. We only changed the english one. When we search on these key words it still throws an ODBC error. Any ideas? Andy -Original Message- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 13:23 To: CF-Talk Subject: RE: Full Text Indexs If you are using SQL Server 2000, all the "noise words" are defined in a text files, one per language, you can easily modify them (add/remove words) : "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.eng" for english indexing "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.fra" for french indexing . Once modified, you'll have to rebuild your full-text indexes. Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 14:00 À : CF-Talk Objet : Full Text Indexs Bit OT but thought I'd test the SQL gurus on here - I have built some full text indexes and am trying to get it to ignore the "ignored" words if that makes sense. For exaple if I type "after eight" into a search box it gives an error as "after" is a reserved word. How to I get SQL to ignore these words when performing my fulltext query? Alternatively does anyone know where I can find a full list of these ignored words? Andy ~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk FAQ: http://www.thenetprofits.co.uk/coldfusion/faq This list and all House of Fusion resources hosted by CFHosting.com. The place for dependable ColdFusion Hosting.
RE: Full Text Indexs
The noise word file used is the one corresponding to the language you specified during the full-text index creation. In the full-text indexing Wizard "Select Table Columns" step, you define a "Language for Word Breaker". For example, we are using "French (France)" in our application, and it works fine. I recommand you not to empty the noise words files, you'll loose some great capabilities of full-text search. For example, if someone search for a letter/number or a common word ('the', 'or'...), those words should be ignored, otherwise you'll get plenty of "noise" in your results. Even if it is minimal, the noise words file should never be empty. Perhaps, should you try to delete all the full-text indexes and re-create them from scratch? Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : lundi 14 octobre 2002 17:18 À : CF-Talk Objet : RE: Full Text Indexs this doesn't appear to be the case here. "After eight" still throws an error as "after" is reserved. eight however is ok. I have checked the noise.eng file and it is empty. Is there any weay of telling which noise file is being used? - I assume it's the English one -Original Message----- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 18:08 To: CF-Talk Subject: RE: Full Text Indexs It should only throw an error if all the search words are defined in the noise words file. For example : "The" will throw an error, "The foo" will return results based on "foo" search. In our application, we have customized the noise word file and put each full text SQL search in blocks. If an error is thrown we create an empty recordset. For example : select ID_NEWS as ID, TITLE_NEWS as Title, CREATEDBYID_NEWS as CreatedByID, CREATEDBY_NEWS as CreatedByLogin, CREATIONDATE_NEWS as CreationDate fromITEM_NEWS and (contains(TITLE_NEWS,'"#arguments.Text#"') or contains(DESCRIPTION_NEWS,'"#arguments.Text#"')) Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 18:41 À : CF-Talk Objet : RE: Full Text Indexs Hi We tried this but it didn't seem to work. We only changed the english one. When we search on these key words it still throws an ODBC error. Any ideas? Andy -Original Message- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 13:23 To: CF-Talk Subject: RE: Full Text Indexs If you are using SQL Server 2000, all the "noise words" are defined in a text files, one per language, you can easily modify them (add/remove words) : "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.eng" for english indexing "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.fra" for french indexing . Once modified, you'll have to rebuild your full-text indexes. Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 14:00 À : CF-Talk Objet : Full Text Indexs Bit OT but thought I'd test the SQL gurus on here - I have built some full text indexes and am trying to get it to ignore the "ignored" words if that makes sense. For exaple if I type "after eight" into a search box it gives an error as "after" is a reserved word. How to I get SQL to ignore these words when performing my fulltext query? Alternatively does anyone know where I can find a full list of these ignored words? Andy ~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm
RE: Full Text Indexs
this doesn't appear to be the case here. "After eight" still throws an error as "after" is reserved. eight however is ok. I have checked the noise.eng file and it is empty. Is there any weay of telling which noise file is being used? - I assume it's the English one -Original Message- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 18:08 To: CF-Talk Subject: RE: Full Text Indexs It should only throw an error if all the search words are defined in the noise words file. For example : "The" will throw an error, "The foo" will return results based on "foo" search. In our application, we have customized the noise word file and put each full text SQL search in blocks. If an error is thrown we create an empty recordset. For example : select ID_NEWS as ID, TITLE_NEWS as Title, CREATEDBYID_NEWS as CreatedByID, CREATEDBY_NEWS as CreatedByLogin, CREATIONDATE_NEWS as CreationDate fromITEM_NEWS and (contains(TITLE_NEWS,'"#arguments.Text#"') or contains(DESCRIPTION_NEWS,'"#arguments.Text#"')) Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 18:41 À : CF-Talk Objet : RE: Full Text Indexs Hi We tried this but it didn't seem to work. We only changed the english one. When we search on these key words it still throws an ODBC error. Any ideas? Andy -Original Message----- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 13:23 To: CF-Talk Subject: RE: Full Text Indexs If you are using SQL Server 2000, all the "noise words" are defined in a text files, one per language, you can easily modify them (add/remove words) : "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.eng" for english indexing "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.fra" for french indexing . Once modified, you'll have to rebuild your full-text indexes. Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 14:00 À : CF-Talk Objet : Full Text Indexs Bit OT but thought I'd test the SQL gurus on here - I have built some full text indexes and am trying to get it to ignore the "ignored" words if that makes sense. For exaple if I type "after eight" into a search box it gives an error as "after" is a reserved word. How to I get SQL to ignore these words when performing my fulltext query? Alternatively does anyone know where I can find a full list of these ignored words? Andy ~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Signup for the Fusion Authority news alert and keep up with the latest news in ColdFusion and related topics. http://www.fusionauthority.com/signup.cfm
RE: Full Text Indexs
It should only throw an error if all the search words are defined in the noise words file. For example : "The" will throw an error, "The foo" will return results based on "foo" search. In our application, we have customized the noise word file and put each full text SQL search in blocks. If an error is thrown we create an empty recordset. For example : select ID_NEWS as ID, TITLE_NEWS as Title, CREATEDBYID_NEWS as CreatedByID, CREATEDBY_NEWS as CreatedByLogin, CREATIONDATE_NEWS as CreationDate fromITEM_NEWS and (contains(TITLE_NEWS,'"#arguments.Text#"') or contains(DESCRIPTION_NEWS,'"#arguments.Text#"')) Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 18:41 À : CF-Talk Objet : RE: Full Text Indexs Hi We tried this but it didn't seem to work. We only changed the english one. When we search on these key words it still throws an ODBC error. Any ideas? Andy -Original Message- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 13:23 To: CF-Talk Subject: RE: Full Text Indexs If you are using SQL Server 2000, all the "noise words" are defined in a text files, one per language, you can easily modify them (add/remove words) : "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.eng" for english indexing "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.fra" for french indexing . Once modified, you'll have to rebuild your full-text indexes. Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 14:00 À : CF-Talk Objet : Full Text Indexs Bit OT but thought I'd test the SQL gurus on here - I have built some full text indexes and am trying to get it to ignore the "ignored" words if that makes sense. For exaple if I type "after eight" into a search box it gives an error as "after" is a reserved word. How to I get SQL to ignore these words when performing my fulltext query? Alternatively does anyone know where I can find a full list of these ignored words? Andy ~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm
RE: Full Text Indexs
Hi We tried this but it didn't seem to work. We only changed the english one. When we search on these key words it still throws an ODBC error. Any ideas? Andy -Original Message- From: Benoit Hediard [mailto:[EMAIL PROTECTED]] Sent: 11 October 2002 13:23 To: CF-Talk Subject: RE: Full Text Indexs If you are using SQL Server 2000, all the "noise words" are defined in a text files, one per language, you can easily modify them (add/remove words) : "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.eng" for english indexing "../Microsoft SQL Server/MSSQL/FTDATA/SQLServer/Config/noise.fra" for french indexing . Once modified, you'll have to rebuild your full-text indexes. Benoit Hediard www.benorama.com -Message d'origine- De : Andy Ewings [mailto:[EMAIL PROTECTED]] Envoyé : vendredi 11 octobre 2002 14:00 À : CF-Talk Objet : Full Text Indexs Bit OT but thought I'd test the SQL gurus on here - I have built some full text indexes and am trying to get it to ignore the "ignored" words if that makes sense. For exaple if I type "after eight" into a search box it gives an error as "after" is a reserved word. How to I get SQL to ignore these words when performing my fulltext query? Alternatively does anyone know where I can find a full list of these ignored words? Andy ~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk FAQ: http://www.thenetprofits.co.uk/coldfusion/faq This list and all House of Fusion resources hosted by CFHosting.com. The place for dependable ColdFusion Hosting.