Re: struggling with solr.WordDelimiterFilterFactory and periods "." or dots
hello,
thank you for the reply.
yes - i did re-index after the changes to the schema.
also - thank you for the direction on using the analyzer - but i am not sure
if i am interpreting the feedback from the analyzer correctly.
here is what i did:
in the Field value (Index) box - i placed this: BP2.1UAA
in the Field value (Query) box - i placed this: BP21UAA
then after hitting the Analyze button - i see the following:
Under Index Analyzer for:
org.apache.solr.analysis.WordDelimiterFilterFactory {splitOnCaseChange=1,
generateNumberParts=1, catenateWords=1, luceneMatchVersion=LUCENE_33,
generateWordParts=1, catenateAll=1, catenateNumbers=1}
i see
position1 2 3 4
term text BP 2 1 UAA
21 BP21UAA
Under Query Analyzer for:
org.apache.solr.analysis.WordDelimiterFilterFactory {splitOnCaseChange=1,
generateNumberParts=1, catenateWords=1, luceneMatchVersion=LUCENE_33,
generateWordParts=1, catenateAll=1, catenateNumbers=1}
i see
position1 2 3
term text BP 21 UAA
BP21UAA
the above information leads me to believe that i "should" have BP21UAA as an
indexed term generated from the BP2.1UAA value coming from the database.
also - the query analysis lead me to believe that i "should" find a document
when i search on BP21UAA in the itemNo field
do i have this correct
am i missing something here?
i am still unable to get a hit when i search on BP21UAA in the itemNo field.
thank you,
mark
--
View this message in context:
http://lucene.472066.n3.nabble.com/struggling-with-solr-WordDelimiterFilterFactory-and-periods-or-dots-tp3724822p3726021.html
Sent from the Solr - User mailing list archive at Nabble.com.
struggling with solr.WordDelimiterFilterFactory and periods "." or dots
hello all, i am struggling with getting solr.WordDelimiterFilterFactory to behave as is indicated in the solr book (Smiley) on page 54. the example in the books reads like this: >> Here is an example exercising all options: WiFi-802.11b to Wi, Fi, WiFi, 802, 11, 80211, b, WiFi80211b << essentially - i have the same requirement with embedded periods and need to return a successful search on a field, even if the user does NOT enter the period. i have a field, itemNo that can contain periods ".". example content in the itemNo field: B12.0123 when the user searches on this field, they need to be able to enter an itemNo without the period, and still find the item. example: user enters: B120123 and a document is returned with B12.0123. unfortunately, the search will NOT return the appropriate document, if the user enters B120123. however - the search does work if the user enters B12 0123 (a space in place of the period). can someone help me understand what is missing from my configuration? this is snipped from my schema.xml file ... ... ** -- View this message in context: http://lucene.472066.n3.nabble.com/struggling-with-solr-WordDelimiterFilterFactory-and-periods-or-dots-tp3724822p3724822.html Sent from the Solr - User mailing list archive at Nabble.com.
linking query in DIH fails with sql syntax error when specific fields contain bad data
hello all, some of my records contain bad data i the orb_itm_id column. example: select * from prtxtps_prt_summ where orb_itm_id like '''%'; prd_gro_id spp_id orb_itm_id ds_tx rnk_no 0022 335 ' LONG. (TERMINAL ATTACH )' LONG. (TERMINAL ATTACH) 0 0042 596 ', FAN MOTOR CAPACITOR S TRAP 0 this is causing the indexing process to fail on the bad records - with a sql syntax error is there a way i can trap for this and cleans the "'" before the sql is constructed? mark 2012-01-13 12:27:38,912 SEVERE [org.apache.solr.handler.dataimport.DataImporter] (Thread-27) Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to execute query: SELECT pa.uom_hi, pa.att_val_hi, pa.uom_low, pa.att_val_low, a.att_nm FROM prtxtpa_att_val pa, prtxtat_att a WHERE pa.att_id = a.att_id and pa.orb_itm_id = '' LONG. (TERMINAL ATTACH' and pa.spp_id = '335' and pa.prd_gro_id = '0022' and pa.att_val_hi is not NULL Processing Document # 119 at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:253) at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210) at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39) at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59) at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73) at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:591) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:617) at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:267) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:186) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408) Caused by: java.sql.SQLException: A syntax error has occurred. -- View this message in context: http://lucene.472066.n3.nabble.com/linking-query-in-DIH-fails-with-sql-syntax-error-when-specific-fields-contain-bad-data-tp3657482p3657482.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: best way to force substitutions in data
thank you both for the information. Gora, when you mentioned: >> - For keeping both values, use synonyms. << what did you mean exactly. mark -- View this message in context: http://lucene.472066.n3.nabble.com/best-way-to-force-substitutions-in-data-tp3646195p3647920.html Sent from the Solr - User mailing list archive at Nabble.com.
