Re: Deleting on exact match
On 6/21/2020 1:04 PM, Scott Q. wrote: The task at hand is to remove all documents indexed the old way, but how can I do that ? user is of the form u...@domain.com and if I search for u...@domain.com it matches all of 'user' or 'domain.com' which has obvious unwanted consequences. Therefore, how can I remove older documents which were indexed with partial match ? If it were me, I would probably set up a new core/collection with any config changes you want to make and reindex into the new location from scratch. Then once the new index is available, you can switch your application to it and completely delete the old one. Thanks, Shawn
Re: Deleting on exact match
I would add a new field with the new behavior. Then any document with content in the new field would not need to be deleted. Find the deletable content with: *:* -new_field:* I generally add a field that records when the document was indexed or updated. That can be really handy. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jun 21, 2020, at 12:32 PM, Scott Q. wrote: > > Also note that I didn't apply the new schema yet because I don't > think it will let me change it mid-way like this without deleting all > data and starting anew... > > On Sunday, 21/06/2020 at 15:12 Scott Q. wrote: > > > My apologies, it appears the configuration tags were escaped and > completely removed from my original e-mail. > > I am including them via pastebin.com > > > https://pastebin.com/BSUqgEke > > > > > On Sunday, 21/06/2020 at 15:04 Scott Q. wrote: > > > Hello, > > I use Solr with Dovecot and I made a mistake when I initially created > my schema for my instance. I created the username field with partial > matches enabled. > Aka, like this: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I already indexed millions of documents using this schema before I > fixed it and changed it to > > > > > > > > > The task at hand is to remove all documents indexed the old way, but > how can I do that ? user is of the form u...@domain.com and if I > search for u...@domain.com it matches all of 'user' or 'domain.com' > which has obvious unwanted consequences. > > Therefore, how can I remove older documents which were indexed with > partial match ?
Re: Deleting on exact match
Also note that I didn't apply the new schema yet because I don't think it will let me change it mid-way like this without deleting all data and starting anew... On Sunday, 21/06/2020 at 15:12 Scott Q. wrote: My apologies, it appears the configuration tags were escaped and completely removed from my original e-mail. I am including them via pastebin.com https://pastebin.com/BSUqgEke On Sunday, 21/06/2020 at 15:04 Scott Q. wrote: Hello, I use Solr with Dovecot and I made a mistake when I initially created my schema for my instance. I created the username field with partial matches enabled. Aka, like this: I already indexed millions of documents using this schema before I fixed it and changed it to The task at hand is to remove all documents indexed the old way, but how can I do that ? user is of the form u...@domain.com and if I search for u...@domain.com it matches all of 'user' or 'domain.com' which has obvious unwanted consequences. Therefore, how can I remove older documents which were indexed with partial match ?
Re: Deleting on exact match
My apologies, it appears the configuration tags were escaped and completely removed from my original e-mail. I am including them via pastebin.com https://pastebin.com/BSUqgEke On Sunday, 21/06/2020 at 15:04 Scott Q. wrote: Hello, I use Solr with Dovecot and I made a mistake when I initially created my schema for my instance. I created the username field with partial matches enabled. Aka, like this: I already indexed millions of documents using this schema before I fixed it and changed it to The task at hand is to remove all documents indexed the old way, but how can I do that ? user is of the form u...@domain.com and if I search for u...@domain.com it matches all of 'user' or 'domain.com' which has obvious unwanted consequences. Therefore, how can I remove older documents which were indexed with partial match ?
Deleting on exact match
Hello, I use Solr with Dovecot and I made a mistake when I initially created my schema for my instance. I created the username field with partial matches enabled. Aka, like this: I already indexed millions of documents using this schema before I fixed it and changed it to The task at hand is to remove all documents indexed the old way, but how can I do that ? user is of the form u...@domain.com and if I search for u...@domain.com it matches all of 'user' or 'domain.com' which has obvious unwanted consequences. Therefore, how can I remove older documents which were indexed with partial match ?
Re: Scoring partial match in title field higher than exact match in description field
1. Do not use stopwords. Ever. Especially with names. 2. Do not use stemming with names. “Bill Gates” is not the same as “Bill Gate”. You aren’t, but I thought I’d include that. 3. Mixing partail word matches and full word matches in the same search is likely to give odd results. 4. Your requirements don’t say anything about phrase matches, which are very important. 5. What synonmys are you using, are they tuned for names, like “william, bill”? If not, remove that. 6. Why are you using WordDelimiterGraphFilterFactory? Do you want to split “DeForest” into “De Forest”? 7. What are you using payloads for? You can use weights in the edismax parser to give different fields different importance in matching. I would also put weights on phrase matches. I usually double the weights for phrases because the native weighting for phrases in Solr isn’t enough. Something like this: name^4 name_ngram^2 infotext name^8 name_ngram^4 infotext^2 Get rid of: * StopFilterFactory * SynonymFilterFactory * WordDelimiterFilterFactory With the remaining filters, you’ll never have duplicates, so you can also get rid of RemoveDupliicatsTokenFilterFactory if you want. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Mar 31, 2020, at 6:00 AM, nileshwagh24 wrote: > > In need to score partial match in NAME field higher than exact match in > INFOTEXT field.Actually I need to sort my SOLR result based on following > five conditions > > 1.First, results with a whole word match on the first or second word in the > NAME go on top. > > 2.Then, results with a whole word match elsewhere in the NAME. > > 3.Then, results with a partial word match anywhere in the NAME. > > 4.Then, results with a whole word match in the INFOTEXT. > > 5.Finally, results with a partial word match in the INFOTEXT. > > I have added descriptionExact field for INFOTEXT and titleExact field for > NAME. > > For this I have added four fields like > titleExact,titlePartial,descriptionExact,DescriptionPartial and boosted > titleExact score higher than all others. > > But I am not getting result as expected. > > Following are the field definitions for exact and partial field > > > positionIncrementGap="100" multiValued="true"> > > mapping="mapping-ISOLatin1Accent.txt"/> > > ignoreCase="true"/> > > > > > mapping="mapping-ISOLatin1Accent.txt"/> > > ignoreCase="true" synonyms="synonyms.txt"/> > ignoreCase="true"/> > > > > > positionIncrementGap="100" multiValued="true"> > > mapping="mapping-ISOLatin1Accent.txt"/> > > encoder="float"/> > synonyms="synonyms.txt"/> > ignoreCase="true"/> > catenateNumbers="1" generateNumberParts="1" protected="protwords.txt" > splitOnCaseChange="1" generateWordParts="0" preserveOriginal="1" > catenateAll="0" catenateWords="1"/> > minGramSize="3"/> > > > > > mapping="mapping-ISOLatin1Accent.txt"/> > > encoder="float"/> > ignoreCase="true" synonyms="synonyms.txt"/> > ignoreCase="true"/> > catenateNumbers="1" generateNumberParts="1" protected="protwords.txt" > splitOnCaseChange="1" generateWordParts="0" preserveOriginal="1" > catenateAll="0" catenateWords="1"/> > > > > > > Any help will be appreciated > > > > -- > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Scoring partial match in title field higher than exact match in description field
In need to score partial match in NAME field higher than exact match in INFOTEXT field.Actually I need to sort my SOLR result based on following five conditions 1.First, results with a whole word match on the first or second word in the NAME go on top. 2.Then, results with a whole word match elsewhere in the NAME. 3.Then, results with a partial word match anywhere in the NAME. 4.Then, results with a whole word match in the INFOTEXT. 5.Finally, results with a partial word match in the INFOTEXT. I have added descriptionExact field for INFOTEXT and titleExact field for NAME. For this I have added four fields like titleExact,titlePartial,descriptionExact,DescriptionPartial and boosted titleExact score higher than all others. But I am not getting result as expected. Following are the field definitions for exact and partial field Any help will be appreciated -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Exact match
Hi, Here's our example of exact match fields: https://github.com/NatLibFi/finna-solr/blob/master/vufind/biblio/conf/schema.xml#L48 textProper_l requires a partial match from the beginning. textProper_lr requires a full match. I'm not sure if this works for you, but at least we have this creative use of PathHierarchyTokenizerFactory allowing the left-anchored search. HTH, Ere Paras Lehana kirjoitti 3.12.2019 klo 13.49: > Hi Omer, > > If you mean exact match with same number of words (Emir's), you can also > add an identifier in the beginning and end of the some other field like > title_exact. This can be done in your indexing script or using Pattern > Replace. During query side, you can use this identifier. For example, > indexing "united states" with "exactStart united states exactEnd" and > querying with the same. Obviously, you can have scoring issues here so only > use if you want it to debug or retrieve docs. > > Just adding to the all possible ways. *Anyways, I like the Keyword method.* > > On Tue, 3 Dec 2019 at 03:59, Erick Erickson wrote: > >> There are two different interpretations of “exact match” going on here, >> don’t be confused! >> >> Emir’s version is “the text has to match the _entire_ input. So a field >> with “a b c d” will NOT match “a b” or “a b c” or “b c", but only “a b c d”. >> >> David’s version is “The text has to contain some sequence of words that >> exactly matches my query”, so a field with “a b c d” _would_ match “a b”, >> “a b c”, “a b c d”, “b c”, “c d”, etc. >> >> Both are entirely valid use-cases, depending on what you mean by “exact >> match" >> >> Best, >> Erick >> >>> On Dec 2, 2019, at 4:38 PM, Emir Arnautović < >> emir.arnauto...@sematext.com> wrote: >>> >>> Hi Omer, >>> From performance perspective, it is the best if you index title as a >> single token: KeywordTokenizer + LowerCaseFilter >>> >>> If you need to query that field in some other way, you can index it >> differently as some other field using copyField. >>> >>> HTH, >>> Emir >>> -- >>> Monitoring - Log Management - Alerting - Anomaly Detection >>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ >>> >>> >>> >>>> On 2 Dec 2019, at 21:43, OTH wrote: >>>> >>>> Hello, >>>> >>>> What would be the best way to get exact matches (if any) to a query? >>>> >>>> E.g.: Let's the document text is: "united states of america". >>>> Currently, any query containing one or more of the three words "united", >>>> "states", or "america" will match with the above document. I would >> like a >>>> way so that the document matches only and only if the query were also >>>> "united states of america" (case-insensitive). >>>> >>>> Document field type: TextField >>>> Index Analyzer: TokenizerChain >>>> Index Tokenizer: StandardTokenizerFactory >>>> Index Token Filters: StopFilterFactory, LowerCaseFilterFactory, >>>> SnowballPorterFilterFactory >>>> The Query Analyzer / Tokenizer / Token Filters are the same as the Index >>>> ones above. >>>> >>>> FYI I'm relatively novice at Solr / Lucene / Search. >>>> >>>> Much appreciated >>>> Omer >>> >> >> > -- Ere Maijala Kansalliskirjasto / The National Library of Finland
Re: Exact match
Hi Omer, If you mean exact match with same number of words (Emir's), you can also add an identifier in the beginning and end of the some other field like title_exact. This can be done in your indexing script or using Pattern Replace. During query side, you can use this identifier. For example, indexing "united states" with "exactStart united states exactEnd" and querying with the same. Obviously, you can have scoring issues here so only use if you want it to debug or retrieve docs. Just adding to the all possible ways. *Anyways, I like the Keyword method.* On Tue, 3 Dec 2019 at 03:59, Erick Erickson wrote: > There are two different interpretations of “exact match” going on here, > don’t be confused! > > Emir’s version is “the text has to match the _entire_ input. So a field > with “a b c d” will NOT match “a b” or “a b c” or “b c", but only “a b c d”. > > David’s version is “The text has to contain some sequence of words that > exactly matches my query”, so a field with “a b c d” _would_ match “a b”, > “a b c”, “a b c d”, “b c”, “c d”, etc. > > Both are entirely valid use-cases, depending on what you mean by “exact > match" > > Best, > Erick > > > On Dec 2, 2019, at 4:38 PM, Emir Arnautović < > emir.arnauto...@sematext.com> wrote: > > > > Hi Omer, > > From performance perspective, it is the best if you index title as a > single token: KeywordTokenizer + LowerCaseFilter > > > > If you need to query that field in some other way, you can index it > differently as some other field using copyField. > > > > HTH, > > Emir > > -- > > Monitoring - Log Management - Alerting - Anomaly Detection > > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > > > >> On 2 Dec 2019, at 21:43, OTH wrote: > >> > >> Hello, > >> > >> What would be the best way to get exact matches (if any) to a query? > >> > >> E.g.: Let's the document text is: "united states of america". > >> Currently, any query containing one or more of the three words "united", > >> "states", or "america" will match with the above document. I would > like a > >> way so that the document matches only and only if the query were also > >> "united states of america" (case-insensitive). > >> > >> Document field type: TextField > >> Index Analyzer: TokenizerChain > >> Index Tokenizer: StandardTokenizerFactory > >> Index Token Filters: StopFilterFactory, LowerCaseFilterFactory, > >> SnowballPorterFilterFactory > >> The Query Analyzer / Tokenizer / Token Filters are the same as the Index > >> ones above. > >> > >> FYI I'm relatively novice at Solr / Lucene / Search. > >> > >> Much appreciated > >> Omer > > > > -- -- Regards, *Paras Lehana* [65871] Development Engineer, Auto-Suggest, IndiaMART Intermesh Ltd. 8th Floor, Tower A, Advant-Navis Business Park, Sector 142, Noida, UP, IN - 201303 Mob.: +91-9560911996 Work: 01203916600 | Extn: *8173* -- * * <https://www.facebook.com/IndiaMART/videos/578196442936091/>
Re: Exact match
There are two different interpretations of “exact match” going on here, don’t be confused! Emir’s version is “the text has to match the _entire_ input. So a field with “a b c d” will NOT match “a b” or “a b c” or “b c", but only “a b c d”. David’s version is “The text has to contain some sequence of words that exactly matches my query”, so a field with “a b c d” _would_ match “a b”, “a b c”, “a b c d”, “b c”, “c d”, etc. Both are entirely valid use-cases, depending on what you mean by “exact match" Best, Erick > On Dec 2, 2019, at 4:38 PM, Emir Arnautović > wrote: > > Hi Omer, > From performance perspective, it is the best if you index title as a single > token: KeywordTokenizer + LowerCaseFilter > > If you need to query that field in some other way, you can index it > differently as some other field using copyField. > > HTH, > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > >> On 2 Dec 2019, at 21:43, OTH wrote: >> >> Hello, >> >> What would be the best way to get exact matches (if any) to a query? >> >> E.g.: Let's the document text is: "united states of america". >> Currently, any query containing one or more of the three words "united", >> "states", or "america" will match with the above document. I would like a >> way so that the document matches only and only if the query were also >> "united states of america" (case-insensitive). >> >> Document field type: TextField >> Index Analyzer: TokenizerChain >> Index Tokenizer: StandardTokenizerFactory >> Index Token Filters: StopFilterFactory, LowerCaseFilterFactory, >> SnowballPorterFilterFactory >> The Query Analyzer / Tokenizer / Token Filters are the same as the Index >> ones above. >> >> FYI I'm relatively novice at Solr / Lucene / Search. >> >> Much appreciated >> Omer >
Re: Exact match
Hi Omer, From performance perspective, it is the best if you index title as a single token: KeywordTokenizer + LowerCaseFilter If you need to query that field in some other way, you can index it differently as some other field using copyField. HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 2 Dec 2019, at 21:43, OTH wrote: > > Hello, > > What would be the best way to get exact matches (if any) to a query? > > E.g.: Let's the document text is: "united states of america". > Currently, any query containing one or more of the three words "united", > "states", or "america" will match with the above document. I would like a > way so that the document matches only and only if the query were also > "united states of america" (case-insensitive). > > Document field type: TextField > Index Analyzer: TokenizerChain > Index Tokenizer: StandardTokenizerFactory > Index Token Filters: StopFilterFactory, LowerCaseFilterFactory, > SnowballPorterFilterFactory > The Query Analyzer / Tokenizer / Token Filters are the same as the Index > ones above. > > FYI I'm relatively novice at Solr / Lucene / Search. > > Much appreciated > Omer
Re: Exact match
if the query is in quotes it will work. also, not sure if youve been following, but get rid of: StopFilterFactory and all stopwords, or just make your stop word file empty if you need it to work in non quotes, add them to the query post submission ? On Mon, Dec 2, 2019 at 3:44 PM OTH wrote: > Hello, > > What would be the best way to get exact matches (if any) to a query? > > E.g.: Let's the document text is: "united states of america". > Currently, any query containing one or more of the three words "united", > "states", or "america" will match with the above document. I would like a > way so that the document matches only and only if the query were also > "united states of america" (case-insensitive). > > Document field type: TextField > Index Analyzer: TokenizerChain > Index Tokenizer: StandardTokenizerFactory > Index Token Filters: StopFilterFactory, LowerCaseFilterFactory, > SnowballPorterFilterFactory > The Query Analyzer / Tokenizer / Token Filters are the same as the Index > ones above. > > FYI I'm relatively novice at Solr / Lucene / Search. > > Much appreciated > Omer >
Exact match
Hello, What would be the best way to get exact matches (if any) to a query? E.g.: Let's the document text is: "united states of america". Currently, any query containing one or more of the three words "united", "states", or "america" will match with the above document. I would like a way so that the document matches only and only if the query were also "united states of america" (case-insensitive). Document field type: TextField Index Analyzer: TokenizerChain Index Tokenizer: StandardTokenizerFactory Index Token Filters: StopFilterFactory, LowerCaseFilterFactory, SnowballPorterFilterFactory The Query Analyzer / Tokenizer / Token Filters are the same as the Index ones above. FYI I'm relatively novice at Solr / Lucene / Search. Much appreciated Omer
Re: attempting to get an exact match on a textField
I figured it out. It was a combination of problems.
1. not fully indexing the data. that made the result set return smaller
than expected.
2. using the join statement without adding a field at the end of it to
search the other core on.
On Fri, Nov 15, 2019 at 1:39 PM rhys J wrote:
>
> I am trying to use the API to get an exact match on clt_ref_no.
>
> At one point, I was using ""s to enclose the text such as:
>
> clt_ref_no: "OWL-2924-8", and I was getting 5 results. Which is accurate.
>
> Now when I use it, I only get one match.
>
> If I try to build the url in perl, and then post the url, my response is
> this:
>
>
> http://localhost:8983/solr/debt/select?indent=on&rows=1000&sort=id%20asc&q=(%20clt_ref_no:%22OWL-2924-8%E2%80%9D%20OR%20contract_number:%22OWL-2924-8%22%20)&fq={!join%20from=debtor_id%20to=debt_id%20fromIndex=dbtr}&cursorMark=*&debug=true
> <http://10.40.10.14:8983/solr/debt/select?indent=on&rows=1000&sort=id%20asc&q=(%20clt_ref_no:%22OWL-2924-8%E2%80%9D%20OR%20contract_number:%22OWL-2924-8%22%20)&fq=%7B!join%20from=debtor_id%20to=debt_id%20fromIndex=dbtr%7D&cursorMark=*&debug=true>
>
> Breaking that down, I've got:
>
> q=( clt_ref_no: "OWL-2924-8" OR contract_number: "OWL-2924-8" )
> fq= {!join from=debtor_id to=debt_id fromIndex=dbtr}
>
> "error":{
> "trace":"java.lang.NullPointerException\n\tat
> org.apache.solr.search.JoinQuery.hashCode(JoinQParserPlugin.java:584)\n\tat
> java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)\n\tat
>
> org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:130)\n\tat
> org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:165)\n\tat
> org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:815)\n\tat
>
> org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1026)\n\tat
>
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1541)\n\tat
>
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1421)\n\tat
>
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:568)\n\tat
>
> org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1484)\n\tat
>
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:398)\n\tat
>
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2578)\n\tat
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)\n\tat
>
> org.eclipse.jetty.server.
attempting to get an exact match on a textField
I am trying to use the API to get an exact match on clt_ref_no.
At one point, I was using ""s to enclose the text such as:
clt_ref_no: "OWL-2924-8", and I was getting 5 results. Which is accurate.
Now when I use it, I only get one match.
If I try to build the url in perl, and then post the url, my response is
this:
http://localhost:8983/solr/debt/select?indent=on&rows=1000&sort=id%20asc&q=(%20clt_ref_no:%22OWL-2924-8%E2%80%9D%20OR%20contract_number:%22OWL-2924-8%22%20)&fq={!join%20from=debtor_id%20to=debt_id%20fromIndex=dbtr}&cursorMark=*&debug=true
<http://10.40.10.14:8983/solr/debt/select?indent=on&rows=1000&sort=id%20asc&q=(%20clt_ref_no:%22OWL-2924-8%E2%80%9D%20OR%20contract_number:%22OWL-2924-8%22%20)&fq={!join%20from=debtor_id%20to=debt_id%20fromIndex=dbtr}&cursorMark=*&debug=true>
Breaking that down, I've got:
q=( clt_ref_no: "OWL-2924-8" OR contract_number: "OWL-2924-8" )
fq= {!join from=debtor_id to=debt_id fromIndex=dbtr}
"error":{
"trace":"java.lang.NullPointerException\n\tat
org.apache.solr.search.JoinQuery.hashCode(JoinQParserPlugin.java:584)\n\tat
java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)\n\tat
org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:130)\n\tat
org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:165)\n\tat
org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:815)\n\tat
org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1026)\n\tat
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1541)\n\tat
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1421)\n\tat
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:568)\n\tat
org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1484)\n\tat
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:398)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2578)\n\tat
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:505)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(E
Re: exact Match and Contains
First, attachments are aggressively stripped by the mail server, none of your images came through. Second, try adding &debug=query to the URL and look at the parsed query returned, that should provide some good hints. Best, Erick On Thu, Jul 5, 2018 at 7:18 AM, Rushikesh Garadade < rushikeshgarad...@gmail.com> wrote: > Small Correction in the mail above: attachmentType is managed-schema is: > indexed="true" stored="true"/> > > > > -- Forwarded message - > From: Rushikesh Garadade > Date: Thu, Jul 5, 2018 at 7:43 PM > Subject: exact Match and Contains > To: > > > Hi, > I have field attachmentType in my collection whose schema is as follows: > indexed="true" stored="true"/> > > when I search for attachmentType:application/pdf i.e. /select > *?q=attachmentType:application/pdf * > I get results > [image: image.png] > > > When I search for attachmentType:*application/pdf* i.e. /select? > *q=attachmentType:*application/pdf** > > I get 0 results > [image: image.png] > > I am not getting what I am missing. > What I think is result of *query* is subset of result of **query** > > Please let me know what I am missing, Do let me know if you need any other > details > > Thanks, > Rushikesh Garadade >
Fwd: exact Match and Contains
Small Correction in the mail above: attachmentType is managed-schema is: -- Forwarded message - From: Rushikesh Garadade Date: Thu, Jul 5, 2018 at 7:43 PM Subject: exact Match and Contains To: Hi, I have field attachmentType in my collection whose schema is as follows: when I search for attachmentType:application/pdf i.e. /select *?q=attachmentType:application/pdf * I get results [image: image.png] When I search for attachmentType:*application/pdf* i.e. /select? *q=attachmentType:*application/pdf** I get 0 results [image: image.png] I am not getting what I am missing. What I think is result of *query* is subset of result of **query** Please let me know what I am missing, Do let me know if you need any other details Thanks, Rushikesh Garadade
exact Match and Contains
Hi, I have field attachmentType in my collection whose schema is as follows: when I search for attachmentType:application/pdf i.e. /select *?q=attachmentType:application/pdf * I get results [image: image.png] When I search for attachmentType:*application/pdf* i.e. /select? *q=attachmentType:*application/pdf** I get 0 results [image: image.png] I am not getting what I am missing. What I think is result of *query* is subset of result of **query** Please let me know what I am missing, Do let me know if you need any other details Thanks, Rushikesh Garadade
Re: Grouping on Exact Match
Is it possible that this field used to be text_en field and later you changed it to string_lower but did not reindex. If you open schema explorer in your admin GUI, what tokens do you see for this field? Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 25 Jan 2018, at 12:25, Gopesh Sharma wrote: > > Hello Emir, > > Service.Solr/partners/select/?wt=json&fq=-partnerProjectCount_num:0&bf=searchScore_num&sort=score+desc&q=Construction%20Company&group=true&group.field=partnerName_text_en&group.ngroups=true&group.limit=100&rows=50&start=0&indent=true > > > stored="true" multiValued="false" /> > > I am using the above query. > > Thanks, > Gopesh Sharma > > -Original Message- > From: Emir Arnautović [mailto:emir.arnauto...@sematext.com] > Sent: Thursday, January 25, 2018 4:31 PM > To: solr-user@lucene.apache.org > Subject: Re: Grouping on Exact Match > > Hi Gopesh, > No it is not - at least not in a way I was thinking. I should have been more > precise - I meant tokenized. So you are grouping on the field that uses this > field type? Can you share query? > > Thanks, > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection Solr & > Elasticsearch Consulting Support Training - > https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsematext.com%2F&data=02%7C01%7CGopesh_Sharma%40gensler.com%7C04a138e3daf7423ba05f08d563e2e1ef%7C94a74758f2ff413c9f705725701b8d02%7C0%7C0%7C636524748464316159&sdata=wuZfMQmGJbnHkPVMjtnUysxHud4U9wP57tDh9kPKNig%3D&reserved=0 > > > >> On 25 Jan 2018, at 11:57, Gopesh Sharma wrote: >> >> Hello Emir, >> >> Thanks for the reply. >> >> So if I am using below field type for the field, that means its analyzed? >> >> > positionIncrementGap="100" autoGeneratePhraseQueries="true"> >> >> >> >> >> >> >> >> >> >> >> Thanks, >> Gopesh Sharma >> >> -Original Message- >> From: Emir Arnautović [mailto:emir.arnauto...@sematext.com] >> Sent: Thursday, January 25, 2018 4:16 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Grouping on Exact Match >> >> Hi Gopesh, >> You are probably grouping on field that is analysed so “Consulting” is group >> term. What you need to do is to have name field that is not alalysed and >> group on that field. If you want to “group” on query input, that is not >> grouping - you simply use phrase query and all results are of that group. >> >> HTH, >> Emir >> -- >> Monitoring - Log Management - Alerting - Anomaly Detection Solr & >> Elasticsearch Consulting Support Training - >> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsemate >> xt.com%2F&data=02%7C01%7CGopesh_Sharma%40gensler.com%7Cda7c34701f32465 >> 883cc08d563e0ddd3%7C94a74758f2ff413c9f705725701b8d02%7C0%7C0%7C6365247 >> 39799609813&sdata=yXhlw36459RoXTKifDyGPVy1bgTpcVWOdHfViuGJgg8%3D&reser >> ved=0 >> >> >> >>> On 25 Jan 2018, at 11:41, Gopesh Sharma wrote: >>> >>> Hello All, >>> >>> I am grouping the results but the groups are not happening on the >>> exact match. For Example : I have 5 documents with name Construction >>> Company, Construction Tower, Tower Company, Tower House and again >>> Construction Company. If I search for Construction Company with >>> grouping I am getting result as >>> >>> >>> * Construction having 3 documents - Construction Company, Construction >>> Tower and Construction Company with group value construction. >>> >>> Is there any way I can get only two results with group value as exact match >>> of the search item. >>> >>> Thanks, >>> Gopesh Sharma >> >
RE: Grouping on Exact Match
Hello Emir, Service.Solr/partners/select/?wt=json&fq=-partnerProjectCount_num:0&bf=searchScore_num&sort=score+desc&q=Construction%20Company&group=true&group.field=partnerName_text_en&group.ngroups=true&group.limit=100&rows=50&start=0&indent=true I am using the above query. Thanks, Gopesh Sharma -Original Message- From: Emir Arnautović [mailto:emir.arnauto...@sematext.com] Sent: Thursday, January 25, 2018 4:31 PM To: solr-user@lucene.apache.org Subject: Re: Grouping on Exact Match Hi Gopesh, No it is not - at least not in a way I was thinking. I should have been more precise - I meant tokenized. So you are grouping on the field that uses this field type? Can you share query? Thanks, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsematext.com%2F&data=02%7C01%7CGopesh_Sharma%40gensler.com%7C04a138e3daf7423ba05f08d563e2e1ef%7C94a74758f2ff413c9f705725701b8d02%7C0%7C0%7C636524748464316159&sdata=wuZfMQmGJbnHkPVMjtnUysxHud4U9wP57tDh9kPKNig%3D&reserved=0 > On 25 Jan 2018, at 11:57, Gopesh Sharma wrote: > > Hello Emir, > > Thanks for the reply. > > So if I am using below field type for the field, that means its analyzed? > > positionIncrementGap="100" autoGeneratePhraseQueries="true"> > > > > > > > > > > > Thanks, > Gopesh Sharma > > -Original Message- > From: Emir Arnautović [mailto:emir.arnauto...@sematext.com] > Sent: Thursday, January 25, 2018 4:16 PM > To: solr-user@lucene.apache.org > Subject: Re: Grouping on Exact Match > > Hi Gopesh, > You are probably grouping on field that is analysed so “Consulting” is group > term. What you need to do is to have name field that is not alalysed and > group on that field. If you want to “group” on query input, that is not > grouping - you simply use phrase query and all results are of that group. > > HTH, > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection Solr & > Elasticsearch Consulting Support Training - > https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsemate > xt.com%2F&data=02%7C01%7CGopesh_Sharma%40gensler.com%7Cda7c34701f32465 > 883cc08d563e0ddd3%7C94a74758f2ff413c9f705725701b8d02%7C0%7C0%7C6365247 > 39799609813&sdata=yXhlw36459RoXTKifDyGPVy1bgTpcVWOdHfViuGJgg8%3D&reser > ved=0 > > > >> On 25 Jan 2018, at 11:41, Gopesh Sharma wrote: >> >> Hello All, >> >> I am grouping the results but the groups are not happening on the >> exact match. For Example : I have 5 documents with name Construction >> Company, Construction Tower, Tower Company, Tower House and again >> Construction Company. If I search for Construction Company with >> grouping I am getting result as >> >> >> * Construction having 3 documents - Construction Company, Construction >> Tower and Construction Company with group value construction. >> >> Is there any way I can get only two results with group value as exact match >> of the search item. >> >> Thanks, >> Gopesh Sharma >
Re: Grouping on Exact Match
Hi Gopesh, No it is not - at least not in a way I was thinking. I should have been more precise - I meant tokenized. So you are grouping on the field that uses this field type? Can you share query? Thanks, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 25 Jan 2018, at 11:57, Gopesh Sharma wrote: > > Hello Emir, > > Thanks for the reply. > > So if I am using below field type for the field, that means its analyzed? > > positionIncrementGap="100" autoGeneratePhraseQueries="true"> > > > > > > > > > > > Thanks, > Gopesh Sharma > > -Original Message- > From: Emir Arnautović [mailto:emir.arnauto...@sematext.com] > Sent: Thursday, January 25, 2018 4:16 PM > To: solr-user@lucene.apache.org > Subject: Re: Grouping on Exact Match > > Hi Gopesh, > You are probably grouping on field that is analysed so “Consulting” is group > term. What you need to do is to have name field that is not alalysed and > group on that field. If you want to “group” on query input, that is not > grouping - you simply use phrase query and all results are of that group. > > HTH, > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection Solr & > Elasticsearch Consulting Support Training - > https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsematext.com%2F&data=02%7C01%7CGopesh_Sharma%40gensler.com%7Cda7c34701f32465883cc08d563e0ddd3%7C94a74758f2ff413c9f705725701b8d02%7C0%7C0%7C636524739799609813&sdata=yXhlw36459RoXTKifDyGPVy1bgTpcVWOdHfViuGJgg8%3D&reserved=0 > > > >> On 25 Jan 2018, at 11:41, Gopesh Sharma wrote: >> >> Hello All, >> >> I am grouping the results but the groups are not happening on the >> exact match. For Example : I have 5 documents with name Construction >> Company, Construction Tower, Tower Company, Tower House and again >> Construction Company. If I search for Construction Company with >> grouping I am getting result as >> >> >> * Construction having 3 documents - Construction Company, Construction >> Tower and Construction Company with group value construction. >> >> Is there any way I can get only two results with group value as exact match >> of the search item. >> >> Thanks, >> Gopesh Sharma >
RE: Grouping on Exact Match
Hello Emir, Thanks for the reply. So if I am using below field type for the field, that means its analyzed? Thanks, Gopesh Sharma -Original Message- From: Emir Arnautović [mailto:emir.arnauto...@sematext.com] Sent: Thursday, January 25, 2018 4:16 PM To: solr-user@lucene.apache.org Subject: Re: Grouping on Exact Match Hi Gopesh, You are probably grouping on field that is analysed so “Consulting” is group term. What you need to do is to have name field that is not alalysed and group on that field. If you want to “group” on query input, that is not grouping - you simply use phrase query and all results are of that group. HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsematext.com%2F&data=02%7C01%7CGopesh_Sharma%40gensler.com%7Cda7c34701f32465883cc08d563e0ddd3%7C94a74758f2ff413c9f705725701b8d02%7C0%7C0%7C636524739799609813&sdata=yXhlw36459RoXTKifDyGPVy1bgTpcVWOdHfViuGJgg8%3D&reserved=0 > On 25 Jan 2018, at 11:41, Gopesh Sharma wrote: > > Hello All, > > I am grouping the results but the groups are not happening on the > exact match. For Example : I have 5 documents with name Construction > Company, Construction Tower, Tower Company, Tower House and again > Construction Company. If I search for Construction Company with > grouping I am getting result as > > > * Construction having 3 documents - Construction Company, Construction > Tower and Construction Company with group value construction. > > Is there any way I can get only two results with group value as exact match > of the search item. > > Thanks, > Gopesh Sharma
Re: Grouping on Exact Match
Hi Gopesh, You are probably grouping on field that is analysed so “Consulting” is group term. What you need to do is to have name field that is not alalysed and group on that field. If you want to “group” on query input, that is not grouping - you simply use phrase query and all results are of that group. HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 25 Jan 2018, at 11:41, Gopesh Sharma wrote: > > Hello All, > > I am grouping the results but the groups are not happening on the exact > match. For Example : I have 5 documents with name Construction Company, > Construction Tower, Tower Company, Tower House and again Construction > Company. If I search for Construction Company with grouping I am getting > result as > > > * Construction having 3 documents - Construction Company, Construction > Tower and Construction Company with group value construction. > > Is there any way I can get only two results with group value as exact match > of the search item. > > Thanks, > Gopesh Sharma
Grouping on Exact Match
Hello All, I am grouping the results but the groups are not happening on the exact match. For Example : I have 5 documents with name Construction Company, Construction Tower, Tower Company, Tower House and again Construction Company. If I search for Construction Company with grouping I am getting result as * Construction having 3 documents - Construction Company, Construction Tower and Construction Company with group value construction. Is there any way I can get only two results with group value as exact match of the search item. Thanks, Gopesh Sharma
Re: Phrase Exact Match with Margin of Error
Agree, that's the challenge. Since ComplexPhraseQuery parser needs terms analyzed/tokenized and if don't, it can't really operate at individual tokens with fuzzy or wildcard matches. The solution I can think of is to execute query against both the fields (KeywordTokenized..) and Non-KeywordTokenized fields and then boost the KeywordTokenized field higher... On Thu, Jun 15, 2017 at 1:20 PM, Max Bridgewater wrote: > Thanks Susheel. The challenge is that if I search for the word "between" > alone, I still get plenty of results. In a way I want the query to match > the document title exactly (up to a few characters) and the document title > match the query exactly (up to a few characters). KeywordTokenizer allows > that. But complexphrase does not seem to work with KeywordTokenizer. > > On Thu, Jun 15, 2017 at 10:23 AM, Susheel Kumar > wrote: > > > CompledPhraseQuery parser is what you need to look > > https://cwiki.apache.org/confluence/display/solr/Other+ > > Parsers#OtherParsers-ComplexPhraseQueryParser. > > See below for e.g. > > > > > > > > http://localhost:8983/solr/techproducts/select? > debugQuery=on&indent=on&q= > > manu:%22Bridge%20the%20gat~1%20between%20your%20skills% > > 20and%20your%20goals%22&defType=complexphrase > > > > On Thu, Jun 15, 2017 at 5:59 AM, Max Bridgewater < > > max.bridgewa...@gmail.com> > > wrote: > > > > > Hi, > > > > > > I am trying to do phrase exact match. For this, I use > > > KeywordTokenizerFactory. This basically does what I want to do. My > field > > > type is defined as follows: > > > > > > > > positionIncrementGap="100"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In addition to this, I want to tolerate typos of two or three letters. > I > > > thought fuzzy search could allow me to accept this margin of error. But > > > this doesn't seem to work. > > > > > > A typical query I would have is: > > > > > > q=subjet:"Bridge the gap between your skills and your goals" > > > > > > Now, in this query, if I replace gap with gat, I was hoping I could do > > > something such as: > > > > > > q=subjet:"Bridge the gat between your skills and your goals"~0.8 > > > > > > But this doesn't quite do what I am trying to achieve. > > > > > > Any suggestion? > > > > > >
Re: Phrase Exact Match with Margin of Error
I think that's because the KeywordTokenizer by definition produces a single token (not a phrase). Perhaps you could create two fields by a copyField - the one you already have(field1), and one tokenized using StandardTokenizer or WhiteSpaceTokenizer(field2) which will produce a phrase with multiple tokens. Then construct a query which searches both field1 for an exact match, and field2 using ComplexQueryParser (use the localparams syntax) to combine them. Boost the field1 (exact match). HTH -Simon On Thu, Jun 15, 2017 at 1:20 PM, Max Bridgewater wrote: > Thanks Susheel. The challenge is that if I search for the word "between" > alone, I still get plenty of results. In a way I want the query to match > the document title exactly (up to a few characters) and the document title > match the query exactly (up to a few characters). KeywordTokenizer allows > that. But complexphrase does not seem to work with KeywordTokenizer. > > On Thu, Jun 15, 2017 at 10:23 AM, Susheel Kumar > wrote: > > > CompledPhraseQuery parser is what you need to look > > https://cwiki.apache.org/confluence/display/solr/Other+ > > Parsers#OtherParsers-ComplexPhraseQueryParser. > > See below for e.g. > > > > > > > > http://localhost:8983/solr/techproducts/select? > debugQuery=on&indent=on&q= > > manu:%22Bridge%20the%20gat~1%20between%20your%20skills% > > 20and%20your%20goals%22&defType=complexphrase > > > > On Thu, Jun 15, 2017 at 5:59 AM, Max Bridgewater < > > max.bridgewa...@gmail.com> > > wrote: > > > > > Hi, > > > > > > I am trying to do phrase exact match. For this, I use > > > KeywordTokenizerFactory. This basically does what I want to do. My > field > > > type is defined as follows: > > > > > > > > positionIncrementGap="100"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In addition to this, I want to tolerate typos of two or three letters. > I > > > thought fuzzy search could allow me to accept this margin of error. But > > > this doesn't seem to work. > > > > > > A typical query I would have is: > > > > > > q=subjet:"Bridge the gap between your skills and your goals" > > > > > > Now, in this query, if I replace gap with gat, I was hoping I could do > > > something such as: > > > > > > q=subjet:"Bridge the gat between your skills and your goals"~0.8 > > > > > > But this doesn't quite do what I am trying to achieve. > > > > > > Any suggestion? > > > > > >
Re: Phrase Exact Match with Margin of Error
Thanks Susheel. The challenge is that if I search for the word "between" alone, I still get plenty of results. In a way I want the query to match the document title exactly (up to a few characters) and the document title match the query exactly (up to a few characters). KeywordTokenizer allows that. But complexphrase does not seem to work with KeywordTokenizer. On Thu, Jun 15, 2017 at 10:23 AM, Susheel Kumar wrote: > CompledPhraseQuery parser is what you need to look > https://cwiki.apache.org/confluence/display/solr/Other+ > Parsers#OtherParsers-ComplexPhraseQueryParser. > See below for e.g. > > > > http://localhost:8983/solr/techproducts/select?debugQuery=on&indent=on&q= > manu:%22Bridge%20the%20gat~1%20between%20your%20skills% > 20and%20your%20goals%22&defType=complexphrase > > On Thu, Jun 15, 2017 at 5:59 AM, Max Bridgewater < > max.bridgewa...@gmail.com> > wrote: > > > Hi, > > > > I am trying to do phrase exact match. For this, I use > > KeywordTokenizerFactory. This basically does what I want to do. My field > > type is defined as follows: > > > > > positionIncrementGap="100"> > > > > > > > > > > > > > > > > > > > > > > > > In addition to this, I want to tolerate typos of two or three letters. I > > thought fuzzy search could allow me to accept this margin of error. But > > this doesn't seem to work. > > > > A typical query I would have is: > > > > q=subjet:"Bridge the gap between your skills and your goals" > > > > Now, in this query, if I replace gap with gat, I was hoping I could do > > something such as: > > > > q=subjet:"Bridge the gat between your skills and your goals"~0.8 > > > > But this doesn't quite do what I am trying to achieve. > > > > Any suggestion? > > >
Re: Phrase Exact Match with Margin of Error
CompledPhraseQuery parser is what you need to look https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-ComplexPhraseQueryParser. See below for e.g. http://localhost:8983/solr/techproducts/select?debugQuery=on&indent=on&q=manu:%22Bridge%20the%20gat~1%20between%20your%20skills%20and%20your%20goals%22&defType=complexphrase On Thu, Jun 15, 2017 at 5:59 AM, Max Bridgewater wrote: > Hi, > > I am trying to do phrase exact match. For this, I use > KeywordTokenizerFactory. This basically does what I want to do. My field > type is defined as follows: > > positionIncrementGap="100"> > > > > > > > > > > > > In addition to this, I want to tolerate typos of two or three letters. I > thought fuzzy search could allow me to accept this margin of error. But > this doesn't seem to work. > > A typical query I would have is: > > q=subjet:"Bridge the gap between your skills and your goals" > > Now, in this query, if I replace gap with gat, I was hoping I could do > something such as: > > q=subjet:"Bridge the gat between your skills and your goals"~0.8 > > But this doesn't quite do what I am trying to achieve. > > Any suggestion? >
Phrase Exact Match with Margin of Error
Hi, I am trying to do phrase exact match. For this, I use KeywordTokenizerFactory. This basically does what I want to do. My field type is defined as follows: In addition to this, I want to tolerate typos of two or three letters. I thought fuzzy search could allow me to accept this margin of error. But this doesn't seem to work. A typical query I would have is: q=subjet:"Bridge the gap between your skills and your goals" Now, in this query, if I replace gap with gat, I was hoping I could do something such as: q=subjet:"Bridge the gat between your skills and your goals"~0.8 But this doesn't quite do what I am trying to achieve. Any suggestion?
Re: Exact match works only for some of the strings
Hello Gintas, >From the first letter I've got that you use colon to separate fieldname and text. But here it's =, which is never advised in lucence syntax. On Fri, Mar 17, 2017 at 2:37 PM, Gintautas Sulskus < gintautas.suls...@gmail.com> wrote: > Hi, > > Thank you for your replies. > Sorry, forgot to specify, I am using Solr 4.10.3 (from Cloudera CDH 5.9.0). > > When I search for name:Guardian I can see both "Guardian EU-referendum" > and "Guardian US" in the result set. > The debugQuery results for both queries are identical > http://pastebin.com/xr96EF0r > Reindexing did not help. > > > Cheers, > Gintas > > > > On Thu, Mar 16, 2017 at 8:18 PM, Alvaro Cabrerizo > wrote: > > > Hello, > > > > I've tested on an old solr 4.3 instance and the schema and the field > > definition are fine. I've also checked that only the > > query nameExact:"Guardian EU-referendum" gives the result, the other one > > you have commented (nameExact:"Guardian US") gives 0 hits. Maybe, you > > forgot to re-index after schema modification. I mean, you indexed your > > data, then changed the schema and then start querying using the new > schema > > that does not match your index. > > > > Hope it helps. > > > > On Thu, Mar 16, 2017 at 7:50 PM, Mikhail Khludnev > wrote: > > > > > You can try to check debugQuery to understand how this query is parsed: > > > double quotes hardly compatible with KeywordTokenizer. Also you can > check > > > which terms are indexed in SchemaBrowser. Also, there is Analysis page > at > > > Solr Admin. > > > > > > On Thu, Mar 16, 2017 at 8:55 PM, Gintautas Sulskus < > > > gintautas.suls...@gmail.com> wrote: > > > > > > > Hi All, > > > > > > > > I am trying to figure out why Solr returns an empty result when > > searching > > > > for the following query: > > > > > > > > nameExact:"Guardian EU-referendum" > > > > > > > > > > > > The field definition: > > > > > > > > stored="true" > > > /> > > > > > > > > > > > > The type definition: > > > > > > > > > > > sortMissingLast="true" omitNorms="true"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The analysis, as expected, matches the query parameter against the > > stored > > > > value. Please take a look at the attached image. I am using > > > > KeywordTokenizer and LowerCaseFilter. > > > > > > > > What is more strange, the query below works just fine: > > > > > > > > nameExact:"Guardian US" > > > > > > > > > > > > Could you please provide me with some clues on what could be wrong? > > > > > > > > Thanks, > > > > Gintas > > > > > > > > > > > > > > > > -- > > > Sincerely yours > > > Mikhail Khludnev > > > > > > -- Sincerely yours Mikhail Khludnev
Re: Exact match works only for some of the strings
Hi, Thank you for your replies. Sorry, forgot to specify, I am using Solr 4.10.3 (from Cloudera CDH 5.9.0). When I search for name:Guardian I can see both "Guardian EU-referendum" and "Guardian US" in the result set. The debugQuery results for both queries are identical http://pastebin.com/xr96EF0r Reindexing did not help. Cheers, Gintas On Thu, Mar 16, 2017 at 8:18 PM, Alvaro Cabrerizo wrote: > Hello, > > I've tested on an old solr 4.3 instance and the schema and the field > definition are fine. I've also checked that only the > query nameExact:"Guardian EU-referendum" gives the result, the other one > you have commented (nameExact:"Guardian US") gives 0 hits. Maybe, you > forgot to re-index after schema modification. I mean, you indexed your > data, then changed the schema and then start querying using the new schema > that does not match your index. > > Hope it helps. > > On Thu, Mar 16, 2017 at 7:50 PM, Mikhail Khludnev wrote: > > > You can try to check debugQuery to understand how this query is parsed: > > double quotes hardly compatible with KeywordTokenizer. Also you can check > > which terms are indexed in SchemaBrowser. Also, there is Analysis page at > > Solr Admin. > > > > On Thu, Mar 16, 2017 at 8:55 PM, Gintautas Sulskus < > > gintautas.suls...@gmail.com> wrote: > > > > > Hi All, > > > > > > I am trying to figure out why Solr returns an empty result when > searching > > > for the following query: > > > > > > nameExact:"Guardian EU-referendum" > > > > > > > > > The field definition: > > > > > > > /> > > > > > > > > > The type definition: > > > > > > > > sortMissingLast="true" omitNorms="true"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The analysis, as expected, matches the query parameter against the > stored > > > value. Please take a look at the attached image. I am using > > > KeywordTokenizer and LowerCaseFilter. > > > > > > What is more strange, the query below works just fine: > > > > > > nameExact:"Guardian US" > > > > > > > > > Could you please provide me with some clues on what could be wrong? > > > > > > Thanks, > > > Gintas > > > > > > > > > > > -- > > Sincerely yours > > Mikhail Khludnev > > >
Re: Exact match works only for some of the strings
Hello, I've tested on an old solr 4.3 instance and the schema and the field definition are fine. I've also checked that only the query nameExact:"Guardian EU-referendum" gives the result, the other one you have commented (nameExact:"Guardian US") gives 0 hits. Maybe, you forgot to re-index after schema modification. I mean, you indexed your data, then changed the schema and then start querying using the new schema that does not match your index. Hope it helps. On Thu, Mar 16, 2017 at 7:50 PM, Mikhail Khludnev wrote: > You can try to check debugQuery to understand how this query is parsed: > double quotes hardly compatible with KeywordTokenizer. Also you can check > which terms are indexed in SchemaBrowser. Also, there is Analysis page at > Solr Admin. > > On Thu, Mar 16, 2017 at 8:55 PM, Gintautas Sulskus < > gintautas.suls...@gmail.com> wrote: > > > Hi All, > > > > I am trying to figure out why Solr returns an empty result when searching > > for the following query: > > > > nameExact:"Guardian EU-referendum" > > > > > > The field definition: > > > > /> > > > > > > The type definition: > > > > > sortMissingLast="true" omitNorms="true"> > > > > > > > > > > > > > > > > > > > > > > > > The analysis, as expected, matches the query parameter against the stored > > value. Please take a look at the attached image. I am using > > KeywordTokenizer and LowerCaseFilter. > > > > What is more strange, the query below works just fine: > > > > nameExact:"Guardian US" > > > > > > Could you please provide me with some clues on what could be wrong? > > > > Thanks, > > Gintas > > > > > > -- > Sincerely yours > Mikhail Khludnev >
Re: Exact match works only for some of the strings
You can try to check debugQuery to understand how this query is parsed: double quotes hardly compatible with KeywordTokenizer. Also you can check which terms are indexed in SchemaBrowser. Also, there is Analysis page at Solr Admin. On Thu, Mar 16, 2017 at 8:55 PM, Gintautas Sulskus < gintautas.suls...@gmail.com> wrote: > Hi All, > > I am trying to figure out why Solr returns an empty result when searching > for the following query: > > nameExact:"Guardian EU-referendum" > > > The field definition: > > > > > The type definition: > > sortMissingLast="true" omitNorms="true"> > > > > > > > > > > > > The analysis, as expected, matches the query parameter against the stored > value. Please take a look at the attached image. I am using > KeywordTokenizer and LowerCaseFilter. > > What is more strange, the query below works just fine: > > nameExact:"Guardian US" > > > Could you please provide me with some clues on what could be wrong? > > Thanks, > Gintas > -- Sincerely yours Mikhail Khludnev
Exact match works only for some of the strings
Hi All, I am trying to figure out why Solr returns an empty result when searching for the following query: nameExact:"Guardian EU-referendum" The field definition: The type definition: The analysis, as expected, matches the query parameter against the stored value. Please take a look at the attached image. I am using KeywordTokenizer and LowerCaseFilter. What is more strange, the query below works just fine: nameExact:"Guardian US" Could you please provide me with some clues on what could be wrong? Thanks, Gintas
Re: Distinguish exact match from wildcard match
Hi,
how about q=code_text:bolt*&fq=code_text:bolt
Ahmet
On Thursday, March 2, 2017 4:41 PM, Сергей Твердохлеб
wrote:
Hi,
is there way to separate exact match from wildcard match in solr response?
e.g. there are two documents: {code_text:bolt} and {code_text:bolter}. When
I search for "bolt" I want to get both results, but somehow grouped, so I
can determine either it was found with exact or non-exact match.
Thanks.
--
Regards,
Sergey Tverdokhleb
Re: Distinguish exact match from wildcard match
Again, depending on your case, you can use functions in fl to return
additional indicator if doc is exact match or not:
q=code_text:bolt OR whatever&fl=*,isExact:tf('code_text_exact', 'bolt')
It will return isExact field with values >0 for any doc that has term
'bolt' in code_text_exact field. Note that I used different field to
make sure that term 'bolt' is not in field even document has 'bolter'
because of analysis chain.
Regards,
Emir
On 02.03.2017 15:51, Alexandre Rafalovitch wrote:
You could still use scoring with distinct bands of values and include
score field to see the assigned score. Then, on the client, you do
rough grouping.
You could try looking at highlighting, but that's probably
computationally irrational for this purpose.
You could try enabling debugging and see if information present in
there is sufficient.
All of these imply client-side post-processing.
Any other option would be possible only if that keyword was the only
content of the field. Then you could group or facet on it or
something. But not if it is one word of many.
Otherwise, you just do two queries and sort/merge yourself.
Regards,
Alex.
http://www.solr-start.com/ - Resources for Solr users, new and experienced
On 2 March 2017 at 09:09, Сергей Твердохлеб wrote:
Hi Emir,
Thanks for your answer.
However in my case I really need to separate results, because I need to
treat those resultsets differently.
Thanks.
2017-03-02 15:57 GMT+02:00 Emir Arnautovic :
Hi Sergei,
Usually you don't want to know which is which, but you do want to have
exact matches first. In case of simple queries and depending on your
usecase, you can use score to make distinction. If "bolter" matches "bolt"
because of some filters, you will need to index it in two fields and boost
fields differently to get different score for different matches:
code_text_exact:bolt^1 OR code_text:bolt
If you want to use wildcards, you can use similar approach:
code_text:bolt^1 OR code_text:bolt*
HTH,
Emir
On 02.03.2017 14:41, Сергей Твердохлеб wrote:
Hi,
is there way to separate exact match from wildcard match in solr response?
e.g. there are two documents: {code_text:bolt} and {code_text:bolter}.
When
I search for "bolt" I want to get both results, but somehow grouped, so I
can determine either it was found with exact or non-exact match.
Thanks.
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
--
Regards,
Sergey Tverdokhleb
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
Re: Distinguish exact match from wildcard match
You could still use scoring with distinct bands of values and include
score field to see the assigned score. Then, on the client, you do
rough grouping.
You could try looking at highlighting, but that's probably
computationally irrational for this purpose.
You could try enabling debugging and see if information present in
there is sufficient.
All of these imply client-side post-processing.
Any other option would be possible only if that keyword was the only
content of the field. Then you could group or facet on it or
something. But not if it is one word of many.
Otherwise, you just do two queries and sort/merge yourself.
Regards,
Alex.
http://www.solr-start.com/ - Resources for Solr users, new and experienced
On 2 March 2017 at 09:09, Сергей Твердохлеб wrote:
> Hi Emir,
>
> Thanks for your answer.
> However in my case I really need to separate results, because I need to
> treat those resultsets differently.
>
> Thanks.
>
> 2017-03-02 15:57 GMT+02:00 Emir Arnautovic :
>
>> Hi Sergei,
>>
>> Usually you don't want to know which is which, but you do want to have
>> exact matches first. In case of simple queries and depending on your
>> usecase, you can use score to make distinction. If "bolter" matches "bolt"
>> because of some filters, you will need to index it in two fields and boost
>> fields differently to get different score for different matches:
>>
>> code_text_exact:bolt^1 OR code_text:bolt
>>
>> If you want to use wildcards, you can use similar approach:
>>
>> code_text:bolt^1 OR code_text:bolt*
>>
>> HTH,
>> Emir
>>
>>
>> On 02.03.2017 14:41, Сергей Твердохлеб wrote:
>>
>>> Hi,
>>>
>>> is there way to separate exact match from wildcard match in solr response?
>>> e.g. there are two documents: {code_text:bolt} and {code_text:bolter}.
>>> When
>>> I search for "bolt" I want to get both results, but somehow grouped, so I
>>> can determine either it was found with exact or non-exact match.
>>>
>>> Thanks.
>>>
>>>
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>
>
>
> --
> Regards,
> Sergey Tverdokhleb
Re: Distinguish exact match from wildcard match
Hi Emir,
Thanks for your answer.
However in my case I really need to separate results, because I need to
treat those resultsets differently.
Thanks.
2017-03-02 15:57 GMT+02:00 Emir Arnautovic :
> Hi Sergei,
>
> Usually you don't want to know which is which, but you do want to have
> exact matches first. In case of simple queries and depending on your
> usecase, you can use score to make distinction. If "bolter" matches "bolt"
> because of some filters, you will need to index it in two fields and boost
> fields differently to get different score for different matches:
>
> code_text_exact:bolt^1 OR code_text:bolt
>
> If you want to use wildcards, you can use similar approach:
>
> code_text:bolt^1 OR code_text:bolt*
>
> HTH,
> Emir
>
>
> On 02.03.2017 14:41, Сергей Твердохлеб wrote:
>
>> Hi,
>>
>> is there way to separate exact match from wildcard match in solr response?
>> e.g. there are two documents: {code_text:bolt} and {code_text:bolter}.
>> When
>> I search for "bolt" I want to get both results, but somehow grouped, so I
>> can determine either it was found with exact or non-exact match.
>>
>> Thanks.
>>
>>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>
--
Regards,
Sergey Tverdokhleb
Re: Distinguish exact match from wildcard match
Hi Sergei,
Usually you don't want to know which is which, but you do want to have
exact matches first. In case of simple queries and depending on your
usecase, you can use score to make distinction. If "bolter" matches
"bolt" because of some filters, you will need to index it in two fields
and boost fields differently to get different score for different matches:
code_text_exact:bolt^1 OR code_text:bolt
If you want to use wildcards, you can use similar approach:
code_text:bolt^1 OR code_text:bolt*
HTH,
Emir
On 02.03.2017 14:41, Сергей Твердохлеб wrote:
Hi,
is there way to separate exact match from wildcard match in solr response?
e.g. there are two documents: {code_text:bolt} and {code_text:bolter}. When
I search for "bolt" I want to get both results, but somehow grouped, so I
can determine either it was found with exact or non-exact match.
Thanks.
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
Distinguish exact match from wildcard match
Hi,
is there way to separate exact match from wildcard match in solr response?
e.g. there are two documents: {code_text:bolt} and {code_text:bolter}. When
I search for "bolt" I want to get both results, but somehow grouped, so I
can determine either it was found with exact or non-exact match.
Thanks.
--
Regards,
Sergey Tverdokhleb
Re: Boosting exact match fields.
In addition to what was proposed We use the technic described here https://github.com/cominvent/exactmatch and it works quite well. Best regards Elisabeth 2016-06-15 16:32 GMT+02:00 Alessandro Benedetti : > In addition to what Erick correctly proposed, > are you storing norms for your field of interest ( to boost documents with > shorter field values )? > If you are, I find suspicious "Sony Ear Phones" to win over "Ear Phones" > for your "Ear Phones" query. > What are the other factors currently involved in your relevancy score > calculus ? > > Cheers > > On Tue, Jun 14, 2016 at 4:48 PM, Erick Erickson > wrote: > > > If these are the complete field, i.e. your document > > contains exactly "ear phones" and not "ear phones > > are great" use a copyField to put it into an "exact_match" > > field that uses a much simpler analysis chain based > > on KeywordTokenizer (plus, perhaps things like > > lowercaseFilter, maybe strip punctuation and the like". > > Then you add a clause on exact_match boosted > > really high. > > > > Best, > > Erick > > > > On Tue, Jun 14, 2016 at 1:01 AM, Naveen Pajjuri > > wrote: > > > Hi, > > > > > > I have documents with a field (data type definition for that field is > > > below) values as ear phones, sony ear phones, philips ear phones. when > i > > > query for earphones sony ear phones is the top result where as i want > ear > > > phones as top result. please suggest how to boost exact matches. PS: I > > have > > > earphones => ear phones in my synonyms.txt and the datatype definition > > for > > > that field keywords is > > positionIncrementGap="100"> > > "solr.WhitespaceTokenizerFactory"/> > class="solr.StopFilterFactory" > > > ignoreCase="true" words="stopwords.txt"/> > > "solr.LowerCaseFilterFactory"/> class="solr.SynonymFilterFactory" > > > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> class= > > > "solr.RemoveDuplicatesTokenFilterFactory"/> > > "query"> > class= > > > "solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/> > > > > class="solr.SynonymFilterFactory" synonyms="synonyms.txt" > > ignoreCase="true" > > > expand="true"/> > class= > > > "solr.RemoveDuplicatesTokenFilterFactory"/> > > REGARDS, > > > Naveen > > > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England >
Re: Boosting exact match fields.
In addition to what Erick correctly proposed, are you storing norms for your field of interest ( to boost documents with shorter field values )? If you are, I find suspicious "Sony Ear Phones" to win over "Ear Phones" for your "Ear Phones" query. What are the other factors currently involved in your relevancy score calculus ? Cheers On Tue, Jun 14, 2016 at 4:48 PM, Erick Erickson wrote: > If these are the complete field, i.e. your document > contains exactly "ear phones" and not "ear phones > are great" use a copyField to put it into an "exact_match" > field that uses a much simpler analysis chain based > on KeywordTokenizer (plus, perhaps things like > lowercaseFilter, maybe strip punctuation and the like". > Then you add a clause on exact_match boosted > really high. > > Best, > Erick > > On Tue, Jun 14, 2016 at 1:01 AM, Naveen Pajjuri > wrote: > > Hi, > > > > I have documents with a field (data type definition for that field is > > below) values as ear phones, sony ear phones, philips ear phones. when i > > query for earphones sony ear phones is the top result where as i want ear > > phones as top result. please suggest how to boost exact matches. PS: I > have > > earphones => ear phones in my synonyms.txt and the datatype definition > for > > that field keywords is > positionIncrementGap="100"> > "solr.WhitespaceTokenizerFactory"/> class="solr.StopFilterFactory" > > ignoreCase="true" words="stopwords.txt"/> > "solr.LowerCaseFilterFactory"/> > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > "solr.RemoveDuplicatesTokenFilterFactory"/> > "query"> class= > > "solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/> > > class="solr.SynonymFilterFactory" synonyms="synonyms.txt" > ignoreCase="true" > > expand="true"/> class= > > "solr.RemoveDuplicatesTokenFilterFactory"/> > REGARDS, > > Naveen > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Boosting exact match fields.
If these are the complete field, i.e. your document contains exactly "ear phones" and not "ear phones are great" use a copyField to put it into an "exact_match" field that uses a much simpler analysis chain based on KeywordTokenizer (plus, perhaps things like lowercaseFilter, maybe strip punctuation and the like". Then you add a clause on exact_match boosted really high. Best, Erick On Tue, Jun 14, 2016 at 1:01 AM, Naveen Pajjuri wrote: > Hi, > > I have documents with a field (data type definition for that field is > below) values as ear phones, sony ear phones, philips ear phones. when i > query for earphones sony ear phones is the top result where as i want ear > phones as top result. please suggest how to boost exact matches. PS: I have > earphones => ear phones in my synonyms.txt and the datatype definition for > that field keywords is positionIncrementGap="100"> "solr.WhitespaceTokenizerFactory"/> ignoreCase="true" words="stopwords.txt"/> "solr.LowerCaseFilterFactory"/> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> "solr.RemoveDuplicatesTokenFilterFactory"/> "query"> "solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/> class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" > expand="true"/> "solr.RemoveDuplicatesTokenFilterFactory"/> REGARDS, > Naveen
Boosting exact match fields.
Hi, I have documents with a field (data type definition for that field is below) values as ear phones, sony ear phones, philips ear phones. when i query for earphones sony ear phones is the top result where as i want ear phones as top result. please suggest how to boost exact matches. PS: I have earphones => ear phones in my synonyms.txt and the datatype definition for that field keywords is REGARDS, Naveen
Re: Query exact match with ASCIIFoldingFilterFactory
What query are you using? From what you've shown, the exact match should work. Perhaps use a phrase query? And while the analyzer is very cool, it has its limitations, particularly it doesn't show the interactions with the _parser_. So add &debug-query to the URL and look at the parsed_query bits of the output, that may show you that the query isn't quite being parsed the way you expect. Best, Erick On Wed, Jun 8, 2016 at 9:04 AM, marotosg wrote: > Hi all, > > I am trying to query and match on a collection of documents with a field > which is basically text coming from pdfs. It could contain any type of text. > > field type > positionIncrementGap="100"> > > > generateWordParts="1" generateNumberParts="1" catenateWords="1" > catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" > preserveOriginal="1"/> > preserveOriginal="false"/> > > > > > generateWordParts="0" generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0" splitOnCaseChange="0" > preserveOriginal="1"/> > preserveOriginal="false"/> > > > > > > It works well in general but i have one use case is not working and I don't > know how to solve it. > when I try to make an exact match like below. > q=docContent:"dq/ex report" > > It can't find the match because the worddelimiter is separating the > positions on the index but not in the query as I don't want to retrieve > false positives. > > Result from analyser > Index: dq/ex dq ex dqex report > Query: dq/exreport > > Is it possible to use the same functionality but make exact match. > > Thanks > Sergio > > > > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Query-exact-match-with-ASCIIFoldingFilterFactory-tp4281256.html > Sent from the Solr - User mailing list archive at Nabble.com.
Query exact match with ASCIIFoldingFilterFactory
Hi all, I am trying to query and match on a collection of documents with a field which is basically text coming from pdfs. It could contain any type of text. field type It works well in general but i have one use case is not working and I don't know how to solve it. when I try to make an exact match like below. q=docContent:"dq/ex report" It can't find the match because the worddelimiter is separating the positions on the index but not in the query as I don't want to retrieve false positives. Result from analyser Index: dq/ex dq ex dqex report Query: dq/exreport Is it possible to use the same functionality but make exact match. Thanks Sergio -- View this message in context: http://lucene.472066.n3.nabble.com/Query-exact-match-with-ASCIIFoldingFilterFactory-tp4281256.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: implement exact match for one of the search fields only?
The desired architecture is that you use a middle app layer that clients send queries to and that middle app layer then constructs the formal query and sends it on to Solr proper. This architecture also enables breaking a user query into multiple Solr queries and then aggregating the results. Besides, the general goal is to avoid app clients talking directly to Solr anyway. -- Jack Krupansky On Thu, Feb 4, 2016 at 2:57 AM, Derek Poh wrote: > Hi Erick > > << > The manual way of doing this would be to construct an elaborate query, > like q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) > OR NOTE: the parens are necessary or the last part of the above would > be parsed as P_ShortDescription:dvd default_searchfield:bracket > >> > > Your suggestion to construct the query like q=spp_keyword_exact:"dvd > bracket" OR P_ShortDescription:(dvd bracket) OR does not fit into our > current implementation. > The front-end pages will only pass the "q=search keywords" in the query to > solr. The list of search fields (qf) is pre-defined in solr. > > Do you have any alternatives to implement your suggestion without making > changes to the front-end? > > On 1/29/2016 1:49 AM, Erick Erickson wrote: > >> bq: if you are interested phrase query, you should use String field >> >> If you do this, you will NOT be able to search within the string. I.e. >> if the doc field is "my dog has fleas" you cannot match >> "dog has" with a string-based field. >> >> If you want to match the _entire_ string or you want prefix-only >> matching, then string might work, i.e. if you _only_ want to be able >> to match >> >> "my dog has fleas" >> "my dog*" >> but not >> "dog has fleas". >> >> On to the root question though. >> >> I really think you want to look at edismax. What you're trying to do >> is apply the same search term to individual fields. In particular, >> the pf parameter will automatically apply the search terms _as a phrase_ >> against the field specified, relieving you of having to enclose things >> in quotes. >> >> The manual way of doing this would be to construct an elaborate query, >> like >> q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) >> OR >> >> NOTE: the parens are necessary or the last part of the above would be >> parsed as >> P_ShortDescription:dvd default_searchfield:bracket >> >> And the &debug=query trick will show you exactly how things are actually >> searched, it's invaluable. >> >> Best, >> Erick >> >> On Thu, Jan 28, 2016 at 5:08 AM, Mugeesh Husain >> wrote: >> >>> Hi, >>> if you are interested phrase query, you should use String field instead >>> of >>> text field in schema like as >>> >>> >>> this will solved you problem. >>> >>> if you are missing anything else let share >>> >>> >>> >>> -- >>> View this message in context: >>> http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >> >> > -- > CONFIDENTIALITY NOTICE > This e-mail (including any attachments) may contain confidential and/or > privileged information. If you are not the intended recipient or have > received this e-mail in error, please inform the sender immediately and > delete this e-mail (including any attachments) from your computer, and you > must not use, disclose to anyone else or copy this e-mail (including any > attachments), whether in whole or in part. > This e-mail and any reply to it may be monitored for security, legal, > regulatory compliance and/or other appropriate reasons. > >
Re: implement exact match for one of the search fields only?
Hi Erick << The manual way of doing this would be to construct an elaborate query, like q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) OR NOTE: the parens are necessary or the last part of the above would be parsed as P_ShortDescription:dvd default_searchfield:bracket >> Your suggestion to construct the query like q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) OR does not fit into our current implementation. The front-end pages will only pass the "q=search keywords" in the query to solr. The list of search fields (qf) is pre-defined in solr. Do you have any alternatives to implement your suggestion without making changes to the front-end? On 1/29/2016 1:49 AM, Erick Erickson wrote: bq: if you are interested phrase query, you should use String field If you do this, you will NOT be able to search within the string. I.e. if the doc field is "my dog has fleas" you cannot match "dog has" with a string-based field. If you want to match the _entire_ string or you want prefix-only matching, then string might work, i.e. if you _only_ want to be able to match "my dog has fleas" "my dog*" but not "dog has fleas". On to the root question though. I really think you want to look at edismax. What you're trying to do is apply the same search term to individual fields. In particular, the pf parameter will automatically apply the search terms _as a phrase_ against the field specified, relieving you of having to enclose things in quotes. The manual way of doing this would be to construct an elaborate query, like q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) OR NOTE: the parens are necessary or the last part of the above would be parsed as P_ShortDescription:dvd default_searchfield:bracket And the &debug=query trick will show you exactly how things are actually searched, it's invaluable. Best, Erick On Thu, Jan 28, 2016 at 5:08 AM, Mugeesh Husain wrote: Hi, if you are interested phrase query, you should use String field instead of text field in schema like as this will solved you problem. if you are missing anything else let share -- View this message in context: http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html Sent from the Solr - User mailing list archive at Nabble.com. -- CONFIDENTIALITY NOTICE This e-mail (including any attachments) may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please inform the sender immediately and delete this e-mail (including any attachments) from your computer, and you must not use, disclose to anyone else or copy this e-mail (including any attachments), whether in whole or in part. This e-mail and any reply to it may be monitored for security, legal, regulatory compliance and/or other appropriate reasons.
Re: implement exact match for one of the search fields only?
> In our case we want to boost exact search ( not phrase search). > Simply adding the copy field not tokenised with the related boost to the > qf list should be fine. > Am I missing something ? Please try it out and report back. You’ll find that the “qf” param will only ever produce matches for single tokens, split on whitespace by the query parser. You are right that the pf solution is only good for boosting exact matches of a field which should also match partially, and Derek wants it to match phrase search but not partial search. This gets a bit trickier. You can fix it partially by rewriting the query into a normal dismax OR’ed with a phrase match on the exact field. But if you expect a sub-phrase to trigger the match, e.g. q=black dvd bracket, where black should match partially in the qfs and “dvd bracket” should match exact in the exact field, then it gets more tricky. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 29. jan. 2016 kl. 15.50 skrev Alessandro Benedetti : > > Still not following. > Why you would like to use pf for exact search ( which is different from > phrase search) ? > Let's summarise : > > pf > <https://cwiki.apache.org/confluence/display/solr/The+DisMax+Query+Parser#TheDisMaxQueryParser-Thepf(PhraseFields)Parameter> > > Phrase Fields: boosts the score of documents in cases where all of the > terms in the q parameter appear in close proximity. > > pf is for very specific use cases, when you want to boost phrase searches > in a set of fields, to boost those documents up. > In our case we want to boost exact search ( not phrase search). > Simply adding the copy field not tokenised with the related boost to the > qf list should be fine. > Am I missing something ? > > On 29 January 2016 at 10:33, Jan Høydahl wrote: > >> Hi >> >> Did you try? Sadly, the pf param will not apply to a non-analyzed field, >> and is not added if query consists of one term only. >> >> -- >> Jan Høydahl, search solution architect >> Cominvent AS - www.cominvent.com >> >>> 29. jan. 2016 kl. 11.14 skrev Alessandro Benedetti < >> abenede...@apache.org>: >>> >>> Jan sorry If i insist, but I really don't see the benefit. >>> If you use the edismax and a copy field not tokenised, you can apply the >>> boost you want to the exact match, and the query will simply be q=foo. >>> And you obtain exactly what you do without the markers. >>> >>> But please, explain me how your solution add some benefit , because I am >>> really curious and probably I missed some point ! :) >>> If i misunderstood the problem, sorry about that ! >>> >>> Cheers >>> >>> On 28 January 2016 at 21:55, Jan Høydahl wrote: >>> >>>> Depends on what exactly you try to do. I think the Github README >> explains >>>> in what situations my solution excels. >>>> Especially if you do not have control over the client application, you >>>> simply get a q=foo, then such a >>>> setup will allow you to boost exact matches very easily. >>>> >>>> -- >>>> Jan Høydahl, search solution architect >>>> Cominvent AS - www.cominvent.com >>>> >>>>> 28. jan. 2016 kl. 16.28 skrev Alessandro Benedetti < >>>> abenede...@apache.org>: >>>>> >>>>> Jan, >>>>> I admit I took a brief look, but what are the benefit of using your >>>>> strategy instead of an additional not tokenised ( keywordTokenized) >> copy >>>>> field ? >>>>> >>>>> Cheers >>>>> >>>>> On 28 January 2016 at 15:22, Jan Høydahl >> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> Please look at my github repo with a template for a field type >> allowing >>>>>> exact match. Typical use is with disMax query parser and the “pf” >> param. >>>>>> See https://github.com/cominvent/exactmatch >>>>>> >>>>>> -- >>>>>> Jan Høydahl, search solution architect >>>>>> Cominvent AS - www.cominvent.com >>>>>> >>>>>>> 28. jan. 2016 kl. 10.52 skrev Derek Poh : >>>>>>> >>>>>>> Hi >>>>>>> >>>>>>> First of all, sorry for the long post. >>>>>>> >>>>>>> How do I implement or structured the query such that one of the >> search >>>>
Re: implement exact match for one of the search fields only?
Still not following. Why you would like to use pf for exact search ( which is different from phrase search) ? Let's summarise : pf <https://cwiki.apache.org/confluence/display/solr/The+DisMax+Query+Parser#TheDisMaxQueryParser-Thepf(PhraseFields)Parameter> Phrase Fields: boosts the score of documents in cases where all of the terms in the q parameter appear in close proximity. pf is for very specific use cases, when you want to boost phrase searches in a set of fields, to boost those documents up. In our case we want to boost exact search ( not phrase search). Simply adding the copy field not tokenised with the related boost to the qf list should be fine. Am I missing something ? On 29 January 2016 at 10:33, Jan Høydahl wrote: > Hi > > Did you try? Sadly, the pf param will not apply to a non-analyzed field, > and is not added if query consists of one term only. > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > > 29. jan. 2016 kl. 11.14 skrev Alessandro Benedetti < > abenede...@apache.org>: > > > > Jan sorry If i insist, but I really don't see the benefit. > > If you use the edismax and a copy field not tokenised, you can apply the > > boost you want to the exact match, and the query will simply be q=foo. > > And you obtain exactly what you do without the markers. > > > > But please, explain me how your solution add some benefit , because I am > > really curious and probably I missed some point ! :) > > If i misunderstood the problem, sorry about that ! > > > > Cheers > > > > On 28 January 2016 at 21:55, Jan Høydahl wrote: > > > >> Depends on what exactly you try to do. I think the Github README > explains > >> in what situations my solution excels. > >> Especially if you do not have control over the client application, you > >> simply get a q=foo, then such a > >> setup will allow you to boost exact matches very easily. > >> > >> -- > >> Jan Høydahl, search solution architect > >> Cominvent AS - www.cominvent.com > >> > >>> 28. jan. 2016 kl. 16.28 skrev Alessandro Benedetti < > >> abenede...@apache.org>: > >>> > >>> Jan, > >>> I admit I took a brief look, but what are the benefit of using your > >>> strategy instead of an additional not tokenised ( keywordTokenized) > copy > >>> field ? > >>> > >>> Cheers > >>> > >>> On 28 January 2016 at 15:22, Jan Høydahl > wrote: > >>> > >>>> Hi > >>>> > >>>> Please look at my github repo with a template for a field type > allowing > >>>> exact match. Typical use is with disMax query parser and the “pf” > param. > >>>> See https://github.com/cominvent/exactmatch > >>>> > >>>> -- > >>>> Jan Høydahl, search solution architect > >>>> Cominvent AS - www.cominvent.com > >>>> > >>>>> 28. jan. 2016 kl. 10.52 skrev Derek Poh : > >>>>> > >>>>> Hi > >>>>> > >>>>> First of all, sorry for the long post. > >>>>> > >>>>> How do I implement or structured the query such that one of the > search > >>>> fields is an exact phrase match while the rest of the search fields > can > >> be > >>>> exact or partial matches? Is this possible? > >>>>> > >>>>> I have the following search fields > >>>>> - P_VeryShortDescription > >>>>> - P_ShortDescription > >>>>> - P_CatConcatKeyword > >>>>> - spp_keyword_exact > >>>>> > >>>>> For the spp_keyword_exact field, I want to apply an exact match to > it. > >>>>> > >>>>> I have a document with the following information. If I search for > >> 'dvd', > >>>> this document should not match. However if I search for 'dvd bracket', > >> this > >>>> document should match. > >>>>> Right now when I search for 'dvd', it is not return, which is > correct. > >>>>> I want it to be return when I search for 'dvd bracket' but it is not. > >>>>> I try enclosing it in double quotes "dvd bracket" but it is not > return. > >>>> Then again I can't enclosed the search terms in double quotes "dvd > >> bracket" > >>>> as those documents with the
Re: implement exact match for one of the search fields only?
Hi Derek, What if it does not match other fields but just exact match. From original question I assume it should return such results. It seems to me that you are AND-ing your fields and that is the reason why your query is not returning anything. Can you try just exact match field and see if it will return match. Maybe it is best if you explain us what you want with couple of documents/query/result and we can give you suggestions for config and query. Regards Emir -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On 29.01.2016 02:03, Derek Poh wrote: Hi Emir For the other search fields, if they have matches it should be return. On 1/28/2016 8:17 PM, Emir Arnautovic wrote: Hi Derek, It is not clear what you are trying to achieve: "one of the search fields is an exact phrase match while the rest of the search fields can be exact or partial matches". What does "while" mean - it has to match in other fields as well or result should be scored better if it does but not mandatory to match? For exact match you can use string type instead of text. For querying multiple fields you can take a look at (e)dismax query parser. Regards, Emir -- CONFIDENTIALITY NOTICE This e-mail (including any attachments) may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please inform the sender immediately and delete this e-mail (including any attachments) from your computer, and you must not use, disclose to anyone else or copy this e-mail (including any attachments), whether in whole or in part. This e-mail and any reply to it may be monitored for security, legal, regulatory compliance and/or other appropriate reasons.
Re: implement exact match for one of the search fields only?
Derek, the link is not available to the public. Did you restart and re-index after changing the field type? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 29. jan. 2016 kl. 04.21 skrev Derek Poh : > > Hi Erick and all > > Yes I am trying to apply the same search term to all the 4 search fieldsand 1 > of the search field must be an exact match. > > You mentioned "In particular, the pf parameter will automatically apply the > search terms _as a phrase_ against the field specified, relieving you of > having to enclose things in quotes." > I triedbut it is not returning the document. > > http://hkenedcdg1.globalsources.com:8983/solr/product/select?q=dvd%20bracket&qf=spp_keyword_exact&defType=edismax&debug=query&pf=spp_keyword_exact&fl=P_ProductId,spp_keyword_exact,P_SPPKW > > I may have misunderstood. > > > On 1/29/2016 1:49 AM, Erick Erickson wrote: >> bq: if you are interested phrase query, you should use String field >> >> If you do this, you will NOT be able to search within the string. I.e. >> if the doc field is "my dog has fleas" you cannot match >> "dog has" with a string-based field. >> >> If you want to match the _entire_ string or you want prefix-only >> matching, then string might work, i.e. if you _only_ want to be able >> to match >> >> "my dog has fleas" >> "my dog*" >> but not >> "dog has fleas". >> >> On to the root question though. >> >> I really think you want to look at edismax. What you're trying to do >> is apply the same search term to individual fields. In particular, >> the pf parameter will automatically apply the search terms _as a phrase_ >> against the field specified, relieving you of having to enclose things >> in quotes. >> >> The manual way of doing this would be to construct an elaborate query, like >> q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) OR >> >> NOTE: the parens are necessary or the last part of the above would be >> parsed as >> P_ShortDescription:dvd default_searchfield:bracket >> >> And the &debug=query trick will show you exactly how things are actually >> searched, it's invaluable. >> >> Best, >> Erick >> >> On Thu, Jan 28, 2016 at 5:08 AM, Mugeesh Husain wrote: >>> Hi, >>> if you are interested phrase query, you should use String field instead of >>> text field in schema like as >>> >>> >>> this will solved you problem. >>> >>> if you are missing anything else let share >>> >>> >>> >>> -- >>> View this message in context: >>> http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >> > > > -- > CONFIDENTIALITY NOTICE > This e-mail (including any attachments) may contain confidential and/or > privileged information. If you are not the intended recipient or have > received this e-mail in error, please inform the sender immediately and > delete this e-mail (including any attachments) from your computer, and you > must not use, disclose to anyone else or copy this e-mail (including any > attachments), whether in whole or in part. > This e-mail and any reply to it may be monitored for security, legal, > regulatory compliance and/or other appropriate reasons.
Re: implement exact match for one of the search fields only?
Hi Did you try? Sadly, the pf param will not apply to a non-analyzed field, and is not added if query consists of one term only. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 29. jan. 2016 kl. 11.14 skrev Alessandro Benedetti : > > Jan sorry If i insist, but I really don't see the benefit. > If you use the edismax and a copy field not tokenised, you can apply the > boost you want to the exact match, and the query will simply be q=foo. > And you obtain exactly what you do without the markers. > > But please, explain me how your solution add some benefit , because I am > really curious and probably I missed some point ! :) > If i misunderstood the problem, sorry about that ! > > Cheers > > On 28 January 2016 at 21:55, Jan Høydahl wrote: > >> Depends on what exactly you try to do. I think the Github README explains >> in what situations my solution excels. >> Especially if you do not have control over the client application, you >> simply get a q=foo, then such a >> setup will allow you to boost exact matches very easily. >> >> -- >> Jan Høydahl, search solution architect >> Cominvent AS - www.cominvent.com >> >>> 28. jan. 2016 kl. 16.28 skrev Alessandro Benedetti < >> abenede...@apache.org>: >>> >>> Jan, >>> I admit I took a brief look, but what are the benefit of using your >>> strategy instead of an additional not tokenised ( keywordTokenized) copy >>> field ? >>> >>> Cheers >>> >>> On 28 January 2016 at 15:22, Jan Høydahl wrote: >>> >>>> Hi >>>> >>>> Please look at my github repo with a template for a field type allowing >>>> exact match. Typical use is with disMax query parser and the “pf” param. >>>> See https://github.com/cominvent/exactmatch >>>> >>>> -- >>>> Jan Høydahl, search solution architect >>>> Cominvent AS - www.cominvent.com >>>> >>>>> 28. jan. 2016 kl. 10.52 skrev Derek Poh : >>>>> >>>>> Hi >>>>> >>>>> First of all, sorry for the long post. >>>>> >>>>> How do I implement or structured the query such that one of the search >>>> fields is an exact phrase match while the rest of the search fields can >> be >>>> exact or partial matches? Is this possible? >>>>> >>>>> I have the following search fields >>>>> - P_VeryShortDescription >>>>> - P_ShortDescription >>>>> - P_CatConcatKeyword >>>>> - spp_keyword_exact >>>>> >>>>> For the spp_keyword_exact field, I want to apply an exact match to it. >>>>> >>>>> I have a document with the following information. If I search for >> 'dvd', >>>> this document should not match. However if I search for 'dvd bracket', >> this >>>> document should match. >>>>> Right now when I search for 'dvd', it is not return, which is correct. >>>>> I want it to be return when I search for 'dvd bracket' but it is not. >>>>> I try enclosing it in double quotes "dvd bracket" but it is not return. >>>> Then again I can't enclosed the search terms in double quotes "dvd >> bracket" >>>> as those documents with the word 'dvd' and 'bracket' in the other fields >>>> will not be match, am I right? >>>>> >>>>> doc: >>>>> >>>>> TV Mounts >>>>> dvd bracket >>>>> >>>>> TV Mounts >>>>> Swivel TV Mounts, Suitable for 26-42 >>>> Inches Screen >>>>> Swivel TV mounts >>>>> >>>>> >>>>> Here are the fields definition: >>>>> >>>>> >>> type="gs_keyword_exact" multiValued="true"/> >>>>> >>>>> >>> positionIncrementGap="100"> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> The other search fields are defined as >>>>> >>>>> >>> positionIncrementGap="100"> >>&g
Re: implement exact match for one of the search fields only?
Jan sorry If i insist, but I really don't see the benefit. If you use the edismax and a copy field not tokenised, you can apply the boost you want to the exact match, and the query will simply be q=foo. And you obtain exactly what you do without the markers. But please, explain me how your solution add some benefit , because I am really curious and probably I missed some point ! :) If i misunderstood the problem, sorry about that ! Cheers On 28 January 2016 at 21:55, Jan Høydahl wrote: > Depends on what exactly you try to do. I think the Github README explains > in what situations my solution excels. > Especially if you do not have control over the client application, you > simply get a q=foo, then such a > setup will allow you to boost exact matches very easily. > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > > 28. jan. 2016 kl. 16.28 skrev Alessandro Benedetti < > abenede...@apache.org>: > > > > Jan, > > I admit I took a brief look, but what are the benefit of using your > > strategy instead of an additional not tokenised ( keywordTokenized) copy > > field ? > > > > Cheers > > > > On 28 January 2016 at 15:22, Jan Høydahl wrote: > > > >> Hi > >> > >> Please look at my github repo with a template for a field type allowing > >> exact match. Typical use is with disMax query parser and the “pf” param. > >> See https://github.com/cominvent/exactmatch > >> > >> -- > >> Jan Høydahl, search solution architect > >> Cominvent AS - www.cominvent.com > >> > >>> 28. jan. 2016 kl. 10.52 skrev Derek Poh : > >>> > >>> Hi > >>> > >>> First of all, sorry for the long post. > >>> > >>> How do I implement or structured the query such that one of the search > >> fields is an exact phrase match while the rest of the search fields can > be > >> exact or partial matches? Is this possible? > >>> > >>> I have the following search fields > >>> - P_VeryShortDescription > >>> - P_ShortDescription > >>> - P_CatConcatKeyword > >>> - spp_keyword_exact > >>> > >>> For the spp_keyword_exact field, I want to apply an exact match to it. > >>> > >>> I have a document with the following information. If I search for > 'dvd', > >> this document should not match. However if I search for 'dvd bracket', > this > >> document should match. > >>> Right now when I search for 'dvd', it is not return, which is correct. > >>> I want it to be return when I search for 'dvd bracket' but it is not. > >>> I try enclosing it in double quotes "dvd bracket" but it is not return. > >> Then again I can't enclosed the search terms in double quotes "dvd > bracket" > >> as those documents with the word 'dvd' and 'bracket' in the other fields > >> will not be match, am I right? > >>> > >>> doc: > >>> > >>> TV Mounts > >>> dvd bracket > >>> > >>> TV Mounts > >>> Swivel TV Mounts, Suitable for 26-42 > >> Inches Screen > >>> Swivel TV mounts > >>> > >>> > >>> Here are the fields definition: > >>> > >>> >> type="gs_keyword_exact" multiValued="true"/> > >>> > >>>>> positionIncrementGap="100"> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> The other search fields are defined as > >>> > >>> >> positionIncrementGap="100"> > >>> > >>> > >>> > >>> >> words="stopwords.txt" /> > >>> >> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > >>> > >>> > >>> > >>> > >>> > >>> >> words="stopwords.txt" /> > >>> >> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > >>> > >>> > >>> > >>> > >>> Derek > >>> > >>> -- > >>> CONFIDENTIALITY NOTICE > >>> This e-mail (including any attachments) may contain confidential and/or > >> privileged information. If you are not the intended recipient or have > >> received this e-mail in error, please inform the sender immediately and > >> delete this e-mail (including any attachments) from your computer, and > you > >> must not use, disclose to anyone else or copy this e-mail (including any > >> attachments), whether in whole or in part. > >>> This e-mail and any reply to it may be monitored for security, legal, > >> regulatory compliance and/or other appropriate reasons. > >> > >> > > > > > > -- > > -- > > > > Benedetti Alessandro > > Visiting card : http://about.me/alessandro_benedetti > > > > "Tyger, tyger burning bright > > In the forests of the night, > > What immortal hand or eye > > Could frame thy fearful symmetry?" > > > > William Blake - Songs of Experience -1794 England > > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: implement exact match for one of the search fields only?
Hi Erick and all Yes I am trying to apply the same search term to all the 4 search fieldsand 1 of the search field must be an exact match. You mentioned "In particular, the pf parameter will automatically apply the search terms _as a phrase_ against the field specified, relieving you of having to enclose things in quotes." I triedbut it is not returning the document. http://hkenedcdg1.globalsources.com:8983/solr/product/select?q=dvd%20bracket&qf=spp_keyword_exact&defType=edismax&debug=query&pf=spp_keyword_exact&fl=P_ProductId,spp_keyword_exact,P_SPPKW I may have misunderstood. On 1/29/2016 1:49 AM, Erick Erickson wrote: bq: if you are interested phrase query, you should use String field If you do this, you will NOT be able to search within the string. I.e. if the doc field is "my dog has fleas" you cannot match "dog has" with a string-based field. If you want to match the _entire_ string or you want prefix-only matching, then string might work, i.e. if you _only_ want to be able to match "my dog has fleas" "my dog*" but not "dog has fleas". On to the root question though. I really think you want to look at edismax. What you're trying to do is apply the same search term to individual fields. In particular, the pf parameter will automatically apply the search terms _as a phrase_ against the field specified, relieving you of having to enclose things in quotes. The manual way of doing this would be to construct an elaborate query, like q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) OR NOTE: the parens are necessary or the last part of the above would be parsed as P_ShortDescription:dvd default_searchfield:bracket And the &debug=query trick will show you exactly how things are actually searched, it's invaluable. Best, Erick On Thu, Jan 28, 2016 at 5:08 AM, Mugeesh Husain wrote: Hi, if you are interested phrase query, you should use String field instead of text field in schema like as this will solved you problem. if you are missing anything else let share -- View this message in context: http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html Sent from the Solr - User mailing list archive at Nabble.com. -- CONFIDENTIALITY NOTICE This e-mail (including any attachments) may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please inform the sender immediately and delete this e-mail (including any attachments) from your computer, and you must not use, disclose to anyone else or copy this e-mail (including any attachments), whether in whole or in part. This e-mail and any reply to it may be monitored for security, legal, regulatory compliance and/or other appropriate reasons.
Re: implement exact match for one of the search fields only?
Do you mean for the spp_keyword_exact field, I should use String field with keyword tokenised and lowercase token filtered? On 1/28/2016 10:54 PM, Alessandro Benedetti wrote: I think you are overthinking the problem : I agre the described one is the most obvious solution in your case. Only addition is to use a keyword tokenised field type, lowercase token filtered if you want to be case in-sensitive . Cheers On 28 January 2016 at 13:08, Mugeesh Husain wrote: Hi, if you are interested phrase query, you should use String field instead of text field in schema like as this will solved you problem. if you are missing anything else let share -- View this message in context: http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html Sent from the Solr - User mailing list archive at Nabble.com. -- CONFIDENTIALITY NOTICE This e-mail (including any attachments) may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please inform the sender immediately and delete this e-mail (including any attachments) from your computer, and you must not use, disclose to anyone else or copy this e-mail (including any attachments), whether in whole or in part. This e-mail and any reply to it may be monitored for security, legal, regulatory compliance and/or other appropriate reasons.
Re: implement exact match for one of the search fields only?
Hi Emir For the other search fields, if they have matches it should be return. On 1/28/2016 8:17 PM, Emir Arnautovic wrote: Hi Derek, It is not clear what you are trying to achieve: "one of the search fields is an exact phrase match while the rest of the search fields can be exact or partial matches". What does "while" mean - it has to match in other fields as well or result should be scored better if it does but not mandatory to match? For exact match you can use string type instead of text. For querying multiple fields you can take a look at (e)dismax query parser. Regards, Emir -- CONFIDENTIALITY NOTICE This e-mail (including any attachments) may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please inform the sender immediately and delete this e-mail (including any attachments) from your computer, and you must not use, disclose to anyone else or copy this e-mail (including any attachments), whether in whole or in part. This e-mail and any reply to it may be monitored for security, legal, regulatory compliance and/or other appropriate reasons.
Re: implement exact match for one of the search fields only?
Depends on what exactly you try to do. I think the Github README explains in what situations my solution excels. Especially if you do not have control over the client application, you simply get a q=foo, then such a setup will allow you to boost exact matches very easily. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 28. jan. 2016 kl. 16.28 skrev Alessandro Benedetti : > > Jan, > I admit I took a brief look, but what are the benefit of using your > strategy instead of an additional not tokenised ( keywordTokenized) copy > field ? > > Cheers > > On 28 January 2016 at 15:22, Jan Høydahl wrote: > >> Hi >> >> Please look at my github repo with a template for a field type allowing >> exact match. Typical use is with disMax query parser and the “pf” param. >> See https://github.com/cominvent/exactmatch >> >> -- >> Jan Høydahl, search solution architect >> Cominvent AS - www.cominvent.com >> >>> 28. jan. 2016 kl. 10.52 skrev Derek Poh : >>> >>> Hi >>> >>> First of all, sorry for the long post. >>> >>> How do I implement or structured the query such that one of the search >> fields is an exact phrase match while the rest of the search fields can be >> exact or partial matches? Is this possible? >>> >>> I have the following search fields >>> - P_VeryShortDescription >>> - P_ShortDescription >>> - P_CatConcatKeyword >>> - spp_keyword_exact >>> >>> For the spp_keyword_exact field, I want to apply an exact match to it. >>> >>> I have a document with the following information. If I search for 'dvd', >> this document should not match. However if I search for 'dvd bracket', this >> document should match. >>> Right now when I search for 'dvd', it is not return, which is correct. >>> I want it to be return when I search for 'dvd bracket' but it is not. >>> I try enclosing it in double quotes "dvd bracket" but it is not return. >> Then again I can't enclosed the search terms in double quotes "dvd bracket" >> as those documents with the word 'dvd' and 'bracket' in the other fields >> will not be match, am I right? >>> >>> doc: >>> >>> TV Mounts >>> dvd bracket >>> >>> TV Mounts >>> Swivel TV Mounts, Suitable for 26-42 >> Inches Screen >>> Swivel TV mounts >>> >>> >>> Here are the fields definition: >>> >>> > type="gs_keyword_exact" multiValued="true"/> >>> >>> > positionIncrementGap="100"> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> The other search fields are defined as >>> >>> > positionIncrementGap="100"> >>> >>> >>> >>> > words="stopwords.txt" /> >>> > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> >>> >>> >>> >>> >>> >>> > words="stopwords.txt" /> >>> > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> >>> >>> >>> >>> >>> Derek >>> >>> -- >>> CONFIDENTIALITY NOTICE >>> This e-mail (including any attachments) may contain confidential and/or >> privileged information. If you are not the intended recipient or have >> received this e-mail in error, please inform the sender immediately and >> delete this e-mail (including any attachments) from your computer, and you >> must not use, disclose to anyone else or copy this e-mail (including any >> attachments), whether in whole or in part. >>> This e-mail and any reply to it may be monitored for security, legal, >> regulatory compliance and/or other appropriate reasons. >> >> > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England
Re: implement exact match for one of the search fields only?
A simple boost query (bq) might do the trick, using edismax: q=dvd bracket bq=spp_keyword_exact:"dvd bracket"^100 qf=P_VeryShortDescription P_ShortDescription P_CatConcatKeyword -- Jack Krupansky On Thu, Jan 28, 2016 at 12:49 PM, Erick Erickson wrote: > bq: if you are interested phrase query, you should use String field > > If you do this, you will NOT be able to search within the string. I.e. > if the doc field is "my dog has fleas" you cannot match > "dog has" with a string-based field. > > If you want to match the _entire_ string or you want prefix-only > matching, then string might work, i.e. if you _only_ want to be able > to match > > "my dog has fleas" > "my dog*" > but not > "dog has fleas". > > On to the root question though. > > I really think you want to look at edismax. What you're trying to do > is apply the same search term to individual fields. In particular, > the pf parameter will automatically apply the search terms _as a phrase_ > against the field specified, relieving you of having to enclose things > in quotes. > > The manual way of doing this would be to construct an elaborate query, like > q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) > OR > > NOTE: the parens are necessary or the last part of the above would be > parsed as > P_ShortDescription:dvd default_searchfield:bracket > > And the &debug=query trick will show you exactly how things are actually > searched, it's invaluable. > > Best, > Erick > > On Thu, Jan 28, 2016 at 5:08 AM, Mugeesh Husain wrote: > > Hi, > > if you are interested phrase query, you should use String field instead > of > > text field in schema like as > > > > > > this will solved you problem. > > > > if you are missing anything else let share > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html > > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: implement exact match for one of the search fields only?
bq: if you are interested phrase query, you should use String field If you do this, you will NOT be able to search within the string. I.e. if the doc field is "my dog has fleas" you cannot match "dog has" with a string-based field. If you want to match the _entire_ string or you want prefix-only matching, then string might work, i.e. if you _only_ want to be able to match "my dog has fleas" "my dog*" but not "dog has fleas". On to the root question though. I really think you want to look at edismax. What you're trying to do is apply the same search term to individual fields. In particular, the pf parameter will automatically apply the search terms _as a phrase_ against the field specified, relieving you of having to enclose things in quotes. The manual way of doing this would be to construct an elaborate query, like q=spp_keyword_exact:"dvd bracket" OR P_ShortDescription:(dvd bracket) OR NOTE: the parens are necessary or the last part of the above would be parsed as P_ShortDescription:dvd default_searchfield:bracket And the &debug=query trick will show you exactly how things are actually searched, it's invaluable. Best, Erick On Thu, Jan 28, 2016 at 5:08 AM, Mugeesh Husain wrote: > Hi, > if you are interested phrase query, you should use String field instead of > text field in schema like as > > > this will solved you problem. > > if you are missing anything else let share > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: implement exact match for one of the search fields only?
Jan, I admit I took a brief look, but what are the benefit of using your strategy instead of an additional not tokenised ( keywordTokenized) copy field ? Cheers On 28 January 2016 at 15:22, Jan Høydahl wrote: > Hi > > Please look at my github repo with a template for a field type allowing > exact match. Typical use is with disMax query parser and the “pf” param. > See https://github.com/cominvent/exactmatch > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > > 28. jan. 2016 kl. 10.52 skrev Derek Poh : > > > > Hi > > > > First of all, sorry for the long post. > > > > How do I implement or structured the query such that one of the search > fields is an exact phrase match while the rest of the search fields can be > exact or partial matches? Is this possible? > > > > I have the following search fields > > - P_VeryShortDescription > > - P_ShortDescription > > - P_CatConcatKeyword > > - spp_keyword_exact > > > > For the spp_keyword_exact field, I want to apply an exact match to it. > > > > I have a document with the following information. If I search for 'dvd', > this document should not match. However if I search for 'dvd bracket', this > document should match. > > Right now when I search for 'dvd', it is not return, which is correct. > > I want it to be return when I search for 'dvd bracket' but it is not. > > I try enclosing it in double quotes "dvd bracket" but it is not return. > Then again I can't enclosed the search terms in double quotes "dvd bracket" > as those documents with the word 'dvd' and 'bracket' in the other fields > will not be match, am I right? > > > > doc: > > > > TV Mounts > > dvd bracket > > > > TV Mounts > > Swivel TV Mounts, Suitable for 26-42 > Inches Screen > > Swivel TV mounts > > > > > > Here are the fields definition: > > > > type="gs_keyword_exact" multiValued="true"/> > > > > positionIncrementGap="100"> > > > > > > > > > > > > > > > > > > > > > > > > > > > > The other search fields are defined as > > > > positionIncrementGap="100"> > > > > > > > > words="stopwords.txt" /> > > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > > > > > > > > > > > > words="stopwords.txt" /> > > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > > > > > > > > > > Derek > > > > -- > > CONFIDENTIALITY NOTICE > > This e-mail (including any attachments) may contain confidential and/or > privileged information. If you are not the intended recipient or have > received this e-mail in error, please inform the sender immediately and > delete this e-mail (including any attachments) from your computer, and you > must not use, disclose to anyone else or copy this e-mail (including any > attachments), whether in whole or in part. > > This e-mail and any reply to it may be monitored for security, legal, > regulatory compliance and/or other appropriate reasons. > > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: implement exact match for one of the search fields only?
Hi Please look at my github repo with a template for a field type allowing exact match. Typical use is with disMax query parser and the “pf” param. See https://github.com/cominvent/exactmatch -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 28. jan. 2016 kl. 10.52 skrev Derek Poh : > > Hi > > First of all, sorry for the long post. > > How do I implement or structured the query such that one of the search fields > is an exact phrase match while the rest of the search fields can be exact or > partial matches? Is this possible? > > I have the following search fields > - P_VeryShortDescription > - P_ShortDescription > - P_CatConcatKeyword > - spp_keyword_exact > > For the spp_keyword_exact field, I want to apply an exact match to it. > > I have a document with the following information. If I search for 'dvd', this > document should not match. However if I search for 'dvd bracket', this > document should match. > Right now when I search for 'dvd', it is not return, which is correct. > I want it to be return when I search for 'dvd bracket' but it is not. > I try enclosing it in double quotes "dvd bracket" but it is not return. Then > again I can't enclosed the search terms in double quotes "dvd bracket" as > those documents with the word 'dvd' and 'bracket' in the other fields will > not be match, am I right? > > doc: > > TV Mounts > dvd bracket > > TV Mounts > Swivel TV Mounts, Suitable for 26-42 Inches > Screen > Swivel TV mounts > > > Here are the fields definition: > > type="gs_keyword_exact" multiValued="true"/> > > positionIncrementGap="100"> > > > > > > > > > > > > > > The other search fields are defined as > > positionIncrementGap="100"> > > > > words="stopwords.txt" /> > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > > > > > > words="stopwords.txt" /> > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > > > > > Derek > > -- > CONFIDENTIALITY NOTICE > This e-mail (including any attachments) may contain confidential and/or > privileged information. If you are not the intended recipient or have > received this e-mail in error, please inform the sender immediately and > delete this e-mail (including any attachments) from your computer, and you > must not use, disclose to anyone else or copy this e-mail (including any > attachments), whether in whole or in part. > This e-mail and any reply to it may be monitored for security, legal, > regulatory compliance and/or other appropriate reasons.
Re: implement exact match for one of the search fields only?
I think you are overthinking the problem : I agre the described one is the most obvious solution in your case. Only addition is to use a keyword tokenised field type, lowercase token filtered if you want to be case in-sensitive . Cheers On 28 January 2016 at 13:08, Mugeesh Husain wrote: > Hi, > if you are interested phrase query, you should use String field instead of > text field in schema like as > > > this will solved you problem. > > if you are missing anything else let share > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: implement exact match for one of the search fields only?
Hi, if you are interested phrase query, you should use String field instead of text field in schema like as this will solved you problem. if you are missing anything else let share -- View this message in context: http://lucene.472066.n3.nabble.com/implement-exact-match-for-one-of-the-search-fields-only-tp4253786p4253827.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: implement exact match for one of the search fields only?
Hi Derek, It is not clear what you are trying to achieve: "one of the search fields is an exact phrase match while the rest of the search fields can be exact or partial matches". What does "while" mean - it has to match in other fields as well or result should be scored better if it does but not mandatory to match? For exact match you can use string type instead of text. For querying multiple fields you can take a look at (e)dismax query parser. Regards, Emir -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On 28.01.2016 10:52, Derek Poh wrote: Hi First of all, sorry for the long post. How do I implement or structured the query such that one of the search fields is an exact phrase match while the rest of the search fields can be exact or partial matches? Is this possible? I have the following search fields - P_VeryShortDescription - P_ShortDescription - P_CatConcatKeyword - spp_keyword_exact For the spp_keyword_exact field, I want to apply an exact match to it. I have a document with the following information. If I search for 'dvd', this document should not match. However if I search for 'dvd bracket', this document should match. Right now when I search for 'dvd', it is not return, which is correct. I want it to be return when I search for 'dvd bracket' but it is not. I try enclosing it in double quotes "dvd bracket" but it is not return. Then again I can't enclosed the search terms in double quotes "dvd bracket" as those documents with the word 'dvd' and 'bracket' in the other fields will not be match, am I right? doc: TV Mounts dvd bracket TV Mounts Swivel TV Mounts, Suitable for 26-42 Inches Screen Swivel TV mounts Here are the fields definition: type="gs_keyword_exact" multiValued="true"/> positionIncrementGap="100"> The other search fields are defined as positionIncrementGap="100"> words="stopwords.txt" /> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> words="stopwords.txt" /> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> Derek -- CONFIDENTIALITY NOTICE This e-mail (including any attachments) may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please inform the sender immediately and delete this e-mail (including any attachments) from your computer, and you must not use, disclose to anyone else or copy this e-mail (including any attachments), whether in whole or in part. This e-mail and any reply to it may be monitored for security, legal, regulatory compliance and/or other appropriate reasons.
implement exact match for one of the search fields only?
Hi First of all, sorry for the long post. How do I implement or structured the query such that one of the search fields is an exact phrase match while the rest of the search fields can be exact or partial matches? Is this possible? I have the following search fields - P_VeryShortDescription - P_ShortDescription - P_CatConcatKeyword - spp_keyword_exact For the spp_keyword_exact field, I want to apply an exact match to it. I have a document with the following information. If I search for 'dvd', this document should not match. However if I search for 'dvd bracket', this document should match. Right now when I search for 'dvd', it is not return, which is correct. I want it to be return when I search for 'dvd bracket' but it is not. I try enclosing it in double quotes "dvd bracket" but it is not return. Then again I can't enclosed the search terms in double quotes "dvd bracket" as those documents with the word 'dvd' and 'bracket' in the other fields will not be match, am I right? doc: TV Mounts dvd bracket TV Mounts Swivel TV Mounts, Suitable for 26-42 Inches Screen Swivel TV mounts Here are the fields definition: type="gs_keyword_exact" multiValued="true"/> positionIncrementGap="100"> The other search fields are defined as positionIncrementGap="100"> words="stopwords.txt" /> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> words="stopwords.txt" /> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> Derek -- CONFIDENTIALITY NOTICE This e-mail (including any attachments) may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please inform the sender immediately and delete this e-mail (including any attachments) from your computer, and you must not use, disclose to anyone else or copy this e-mail (including any attachments), whether in whole or in part. This e-mail and any reply to it may be monitored for security, legal, regulatory compliance and/or other appropriate reasons.
Re: Solr Exact match boost Reduce the results
Hi Alessandro Benedetti, Can you check what happens to the below given product_name , just check its score "product_name":"LAPTOP BATTERY DELL Inspiron 6400 1501 E1505 RD859 UD267 XU937" How is this product comes , ? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4213417.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
Hi Jack,
2015-06-23 6:27 GMT+01:00 JACK :
> Hi Alessandro Benedetti,
>
> I have changed the query like this.
>
>
> /select?q=("dell+laptop"~13+OR+"dell+laptop")&df=product_name&defType=edismax&qf=product_name^0.001+dummy_product_name^2&fl=product_name&wt=json&indent=true&debug=true
>
Pardon me, but I find this query awful. In my opinion is not necessary at
all.
I can't take a deep look to your debug right now, but I suggest you to use
this free tool ( http://splainer.io ) to read it easily .
My suggestions remain :
1) carefully take care of your analysis
2) carefully take care of the schema ( field attributes)
3) carefully take care of the handler config
Cheers
>
> The corresponding results also given in the below link.Now am getting exact
> match first.
>
> http://pastebin.com/rAYrFiB8
>
>
> Now the problem is Look at the 8th result
> "product_name":"Dell Inspiron 15R 15.6-inch Laptop without Laptop Bag by
> Dell", its not a relevant result
> can you check How this happens , check the scores.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4213382.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
--
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Solr Exact match boost Reduce the results
Hi Alessandro Benedetti,
I have changed the query like this.
/select?q=("dell+laptop"~13+OR+"dell+laptop")&df=product_name&defType=edismax&qf=product_name^0.001+dummy_product_name^2&fl=product_name&wt=json&indent=true&debug=true
The corresponding results also given in the below link.Now am getting exact
match first.
http://pastebin.com/rAYrFiB8
Now the problem is Look at the 8th result
"product_name":"Dell Inspiron 15R 15.6-inch Laptop without Laptop Bag by
Dell", its not a relevant result
can you check How this happens , check the scores.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4213382.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
Your posted terms doesn't make a lot of sense … I simply indexed your content in a text_general field. Without omitting norms or applying any deep analysis. And I got the expected results. You should use the analysis tool with your different inputs, taking a look to the token stream produced. Maybe a screenshot will be more readable. Cheers 2015-06-15 13:18 GMT+01:00 JACK : > Hi Alessandro Benedetti, > Its my Analysis value index > > > WT > > text > raw_bytes > start > end > positionLength > type > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > 2 > SF > > text > raw_bytes > start > end > positionLength > type > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > 2 > WDF > > text > raw_bytes > start > end > positionLength > type > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > 2 > LCF > > text > raw_bytes > start > end > positionLength > type > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > 2 > SF > > text > raw_bytes > start > end > positionLength > type > keyword > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > false > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > false > 2 > PSF > > text > raw_bytes > start > end > positionLength > type > keyword > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > false > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > false > 2 > KSF > > text > raw_bytes > start > end > positionLength > type > keyword > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > false > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > false > 2 > EMSF > > text > raw_bytes > start > end > positionLength > type > keyword > position > > laptop > [6c 61 70 74 6f 70] > 0 > 6 > 1 > word > false > 1 > > bag > [62 61 67] > 7 > 10 > 1 > word > false > 2 > > How do you get the result i can't understand > Can you help? > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211845.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Exact match boost Reduce the results
Hi Alessandro Benedetti, Its my Analysis value index WT text raw_bytes start end positionLength type position laptop [6c 61 70 74 6f 70] 0 6 1 word 1 bag [62 61 67] 7 10 1 word 2 SF text raw_bytes start end positionLength type position laptop [6c 61 70 74 6f 70] 0 6 1 word 1 bag [62 61 67] 7 10 1 word 2 WDF text raw_bytes start end positionLength type position laptop [6c 61 70 74 6f 70] 0 6 1 word 1 bag [62 61 67] 7 10 1 word 2 LCF text raw_bytes start end positionLength type position laptop [6c 61 70 74 6f 70] 0 6 1 word 1 bag [62 61 67] 7 10 1 word 2 SF text raw_bytes start end positionLength type keyword position laptop [6c 61 70 74 6f 70] 0 6 1 word false 1 bag [62 61 67] 7 10 1 word false 2 PSF text raw_bytes start end positionLength type keyword position laptop [6c 61 70 74 6f 70] 0 6 1 word false 1 bag [62 61 67] 7 10 1 word false 2 KSF text raw_bytes start end positionLength type keyword position laptop [6c 61 70 74 6f 70] 0 6 1 word false 1 bag [62 61 67] 7 10 1 word false 2 EMSF text raw_bytes start end positionLength type keyword position laptop [6c 61 70 74 6f 70] 0 6 1 word false 1 bag [62 61 67] 7 10 1 word false 2 How do you get the result i can't understand Can you help? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211845.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
Using your same data I get the expected ranking and not yours. I am using a simple analysis so I would invite you to analyse the analysis chain in first instance. as a second suggestion, are you omitting or not omitting norms ? By default they should not be omitted. 2015-06-15 11:13 GMT+01:00 JACK : > Hi Alessandro Benedetti, > The query is > > http://localhost:8983/solr/MYDBCORE/select?q=product_name:(laptop+bag)&wt=json&indent=true > > 1.Dell Inspiron 3542 Laptop (Black) without Laptop Bag > 2.Dell 3542 15-inch Laptop with Laptop Bag by Dell > 3.Dell Inspiron N3137 11-inch Laptop without Laptop Bag by Dell > 4.Dell Inspiron 3442 14-Inch Laptop (Black) without Laptop Bag by Dell > 5.Dell Inspiron 3442 Black 14 inch Laptop Without Laptop Bag by Dell > 6.Dell Alienware 13-inch Laptop without Laptop Bag by Dell > 7.Dell Vostro 3546 Laptop without Laptop Bag by Dell > 8.Laptop - BAG > 9.Laptop -BAG > 10.Laptop-BAG > > I need to get Last three results first, rest of the results can be any > order > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211826.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Exact match boost Reduce the results
Hi Alessandro Benedetti, The query is http://localhost:8983/solr/MYDBCORE/select?q=product_name:(laptop+bag)&wt=json&indent=true 1.Dell Inspiron 3542 Laptop (Black) without Laptop Bag 2.Dell 3542 15-inch Laptop with Laptop Bag by Dell 3.Dell Inspiron N3137 11-inch Laptop without Laptop Bag by Dell 4.Dell Inspiron 3442 14-Inch Laptop (Black) without Laptop Bag by Dell 5.Dell Inspiron 3442 Black 14 inch Laptop Without Laptop Bag by Dell 6.Dell Alienware 13-inch Laptop without Laptop Bag by Dell 7.Dell Vostro 3546 Laptop without Laptop Bag by Dell 8.Laptop - BAG 9.Laptop -BAG 10.Laptop-BAG I need to get Last three results first, rest of the results can be any order -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211826.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
I read you are using a really deep analysed field type. Let's forget about the dummy copy field. Can you show me based on your main field, an example of query, and an example of first 20 results ? Showing me the ranking you get and the ranking you expect ? Cheers 2015-06-15 10:39 GMT+01:00 Alessandro Benedetti : > > > 2015-06-12 17:53 GMT+01:00 JACK : > >> As explained above, actually I have around 10 lack data not 5 row. > > This is not changing how the edismax and norms in Solr work. So it's not a > point. > > >> It's not >> about synonyms . When I checked in the FAQ page of Solr wiki, it is found >> that if we need to get exact match results first, use a copy field with >> different configuration. That's why I followed this way. >> > > Can you show me this link, as I still find difficult to understand why you > are getting this not by default. > Have you tried the Analysis tool to verify which kind of terms you are > actually indexing ? > I will take a look better to you analysis configuration. > Before accusing Solr relevancy we should be sure what we index and what we > query ( after the analysis chain) > > Cheers > >> >> >> >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211434.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Exact match boost Reduce the results
2015-06-12 17:53 GMT+01:00 JACK : > As explained above, actually I have around 10 lack data not 5 row. This is not changing how the edismax and norms in Solr work. So it's not a point. > It's not > about synonyms . When I checked in the FAQ page of Solr wiki, it is found > that if we need to get exact match results first, use a copy field with > different configuration. That's why I followed this way. > Can you show me this link, as I still find difficult to understand why you are getting this not by default. Have you tried the Analysis tool to verify which kind of terms you are actually indexing ? I will take a look better to you analysis configuration. Before accusing Solr relevancy we should be sure what we index and what we query ( after the analysis chain) Cheers > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211434.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
RE: Solr Exact match boost Reduce the results
Hi chillra, I have changed the index and query filed configuration to But still my problem not solved , it won't resolve my problem. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211788.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Solr Exact match boost Reduce the results
If I understand you correctly you want to boost the score of documents where the contents of the product_name field match exactly (other than case) the query string. I think what you need is for the dummy_name field to be non-tokenized (indexed as a single string rather than parsed into individual words). The name of the field type you have configured the dummy_name field to use (string_ci) would seem to indicate this is your intent. However the definition of string_ci doesn't match the name. It is configured to use the WhitespaceTokenizerFactory tokenizer, which will break the contents of the field up into multiple tokens where ever white space occurs. Try defining string_ci using the (somewhat cryptically named) KeywordTokenizerFactory, which will index the entire contents of the field as a single token. Something like: - Andy - -Original Message- From: JACK [mailto:mfal...@gmail.com] Sent: Friday, June 12, 2015 12:54 PM To: solr-user@lucene.apache.org Subject: Re: Solr Exact match boost Reduce the results As explained above, actually I have around 10 lack data not 5 row. It's not about synonyms . When I checked in the FAQ page of Solr wiki, it is found that if we need to get exact match results first, use a copy field with different configuration. That's why I followed this way. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211434.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
As explained above, actually I have around 10 lack data not 5 row. It's not about synonyms . When I checked in the FAQ page of Solr wiki, it is found that if we need to get exact match results first, use a copy field with different configuration. That's why I followed this way. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211434.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
I did a simple test using out of the Box Edismax ( not even configuring specific params or the phrase field). As expected the exact match comes first. This is because of the norms and the natural way the Edismax boost exact matches. Are you sure you are using a proper query parser ? I did nothing analysis side, I simply used a standard field. Also I find a little bit suspicious your double field analysis approach and the field boost. Are you sure it's not related to the synonyms ? Cheers 2015-06-12 16:27 GMT+01:00 JACK : > The quoted search words will be different and it will be any word or more > than one word. In the query it's just example > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211410.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Exact match boost Reduce the results
The quoted search words will be different and it will be any word or more than one word. In the query it's just example -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211410.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
Hi, I have to search on the field product_name.It is found that in order to get exact matches first, I made one copy field named as dummy_name with the above field definition.And while query, just boost the copy field. I done this. So as to get exact matches I need to put quotes around the search words. When I do this my results is too less compared to search without quotes. But I need the same results without quotes along with exact matches should come first -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211409.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
Are we talking about a title field ? Querying only that title field ? And the texts you put are the titles ? If this is the case and you are not omitting norms ( because norms boost short field containing the query terms). The second results should be the ones expected. Can you describe a little bit more ? DO the field you are querying contain actually more text and not only the one you quoted ? Cheers 2015-06-12 14:25 GMT+01:00 JACK : > Hi Alessandro Benedetti , > > What i meant is that suppose if i have items like this > > dell laptop with bag > dell laptop > dell laptop without bag > dell inspiron laptop with bag > if i query for "dell laptop", the result should be like this > dell laptop > dell laptop with bag > dell laptop without bag > dell inspiron laptop with bag > Exact match should come first, rest of the things will be in the any order, > but should get the same number of results > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211377.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Exact match boost Reduce the results
Hi Alessandro Benedetti , What i meant is that suppose if i have items like this dell laptop with bag dell laptop dell laptop without bag dell inspiron laptop with bag if i query for "dell laptop", the result should be like this dell laptop dell laptop with bag dell laptop without bag dell inspiron laptop with bag Exact match should come first, rest of the things will be in the any order, but should get the same number of results -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352p4211377.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Exact match boost Reduce the results
Hi jack, do you mean exact match over the synonyms ? In that case, with your Analyzer you are not going to be able to see that. You apply index analysis synonym expansion. So for your Index there is no difference between the synonyms, there is no "exact match" . In this case I would suggest you to use a custom Synonym query parser that actually solve this problem boosting original terms and providing nice query time synonyms : http://nolanlawson.com/2012/10/31/better-synonym-handling-in-solr/ If it's not related the synonyms, you should take a look to the edismax query parser. I think he enhance exact phrase matches out of the box. Cheers 2015-06-12 10:38 GMT+01:00 JACK : > I have two fields, one is copy field. I have to get Exact match results > first > along with entire result of fuzzy search. > > stored="true" > required="true" multiValued="false"/> > required="true" /> > > > Its filed definition is given below > > positionIncrementGap="100"> > >class="solr.WhitespaceTokenizerFactory"/> > synonyms="synonyms.txt" > ignoreCase="true" expand="true"/> > class="solr.WordDelimiterFilterFactory" > generateWordParts="1" > generateNumberParts="1" > catenateWords="1" > catenateNumbers="1" > catenateAll="1" > preserveOriginal="1" > /> > class="solr.LowerCaseFilterFactory"/> > class="solr.SnowballPorterFilterFactory" language="English" /> > class="solr.PorterStemFilterFactory"/> > > class="solr.EnglishMinimalStemFilterFactory"/> > > > class="solr.WhitespaceTokenizerFactory"/> > synonyms="synonyms.txt" > ignoreCase="true" expand="true"/> > class="solr.WordDelimiterFilterFactory" > generateWordParts="1" > generateNumberParts="1" > catenateWords="1" > catenateNumbers="1" > catenateAll="1" > preserveOriginal="1" > /> > class="solr.LowerCaseFilterFactory"/> > class="solr.SnowballPorterFilterFactory" language="English" /> > class="solr.PorterStemFilterFactory"/> > > class="solr.EnglishMinimalStemFilterFactory"/> > > > > omitNorms="true"> > > class="solr.WhitespaceTokenizerFactory"/> > maxTokenCount="20"/> > > > > class="solr.WhitespaceTokenizerFactory"/> > maxTokenCount="20"/> > > language="English" /> > > class="solr.EnglishMinimalStemFilterFactory"/> > > > > The dummy filed is to get the exact match results. > 1. To get exact results first just use quotes around the search words. So i > am getting the exact results first. But the result is too less.Around > 8000.Query is given below > > q="laptop+bag"&df=product_name&defType=edismax&qf=product_name^0.01+dummy_name^200 > > 2. But for the query without quotes gives huge amount of results around > 2, but won't give exact one first. Its query is below > > q=laptop+bag&df=product_name&defType=edismax&qf=product_name^0.01+dummy_name^200 > > I have to get huge results like my second option with exact results first. > Is this the way to do it or any problem in my query? > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Solr Exact match boost Reduce the results
I have two fields, one is copy field. I have to get Exact match results first along with entire result of fuzzy search. Its filed definition is given below The dummy filed is to get the exact match results. 1. To get exact results first just use quotes around the search words. So i am getting the exact results first. But the result is too less.Around 8000.Query is given below q="laptop+bag"&df=product_name&defType=edismax&qf=product_name^0.01+dummy_name^200 2. But for the query without quotes gives huge amount of results around 2, but won't give exact one first. Its query is below q=laptop+bag&df=product_name&defType=edismax&qf=product_name^0.01+dummy_name^200 I have to get huge results like my second option with exact results first. Is this the way to do it or any problem in my query? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Exact-match-boost-Reduce-the-results-tp4211352.html Sent from the Solr - User mailing list archive at Nabble.com.
How to get exact match along with text edge_ngram
We have item_name indexed as text edge_ngram which returns like results...
Please suggest what will be the best approach (like "string" index (in
addition to "...edge_ngram"... or using copyField...) to search ALSO for
exact matches?
e.g. url should return item_name as "abc" entries only... I tried item_name
in quotes ("abc") but no luck...
http://localhost:8081/solr/item/select?q=item_name:abc&wt=json&indent=true
But, I get abc-1, abc-2... as result, however I only need "abc" entries.
RE: Exact match on string field with special characters
This should do what you want: String fq = "Field1" + "\"" + org.apache.solr.client.solrj.util.ClientUtils.escapeQueryChars(value) + "\""; -Michael -Original Message- From: tedsolr [mailto:tsm...@sciquest.com] Sent: Monday, October 06, 2014 10:49 AM To: solr-user@lucene.apache.org Subject: Re: Exact match on string field with special characters I may have provided too much background story for my question. What I am trying to do at the core here, is an exact match on a single field. I do this programmatically by reading the field value from the facet query and setting it equal to the field name for a subsequent search. if this is a sample facet query result ... (Field1 is defined as a string) [Field1:[HI! THIS IS A "VALUE" FOR \FIELD1\ (100)] Then I need to run a search for that exact value. The problem is the double quotes and slashes when I try to construct the facet query ... String fq = "Field1:" + "\"" + value + "\""; The quotes play havoc with the concatenation, as do backslashes. I was wondering if there's a way to build the search without having to manually construct it in code. The only thing I can come up with is to transform the field data at index time by replacing double quotes and backslashes. I don't strip special chars because I'm using the facet values for display. This problem may be specific to SolrJ. Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Exact-match-on-string-field-with-special-characters-tp4162209p4162907.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Exact match on string field with special characters
Shoot I just noticed the error in my original post which would certainly
cause confusion.
Instead of
query.addFacetField(fq);
I meant to write
query.setParam("fq", fg);
Sorry.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Exact-match-on-string-field-with-special-characters-tp4162209p4162908.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Exact match on string field with special characters
I may have provided too much background story for my question. What I am trying to do at the core here, is an exact match on a single field. I do this programmatically by reading the field value from the facet query and setting it equal to the field name for a subsequent search. if this is a sample facet query result ... (Field1 is defined as a string) [Field1:[HI! THIS IS A "VALUE" FOR \FIELD1\ (100)] Then I need to run a search for that exact value. The problem is the double quotes and slashes when I try to construct the facet query ... String fq = "Field1:" + "\"" + value + "\""; The quotes play havoc with the concatenation, as do backslashes. I was wondering if there's a way to build the search without having to manually construct it in code. The only thing I can come up with is to transform the field data at index time by replacing double quotes and backslashes. I don't strip special chars because I'm using the facet values for display. This problem may be specific to SolrJ. Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Exact-match-on-string-field-with-special-characters-tp4162209p4162907.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Exact match on string field with special characters
Hi, raw query parser or term query parser would be handy. https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermQueryParser Ahmet On Thursday, October 2, 2014 12:32 AM, tedsolr wrote: I am trying to do SQL like aggregation (GROUP BY) with solr faceting. So I use string fields for faceting - to try to get an exact match. However, it seems like to run a facet query I have to surround the value with double quotes. That poses issues when the field value is green "bath" towels -or- red \cars Those two special characters must be transformed somehow on indexing so I can create the query: (java) ... String fg = fieldName + ":\"" + fieldValue + "\""; query.addFacetField(fq); ... Is there a way to request an exact match search without having to resort to the quotes? I could possibly convert spaces to underscores at index time, but I'd like to avoid munging that data because I'm using the string field for display too! That saves time/searches when aggregating against 10 - 15 fields which takes a whole lot of facet searches to begin with. Using Solr 4.9 -- View this message in context: http://lucene.472066.n3.nabble.com/Exact-match-on-string-field-with-special-characters-tp4162209.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Exact match on string field with special characters
When you call addFacetField, the parameter you pass it should just be the fieldName. The fieldValue shouldn't come into play at all (unless I'm misunderstanding what you're trying to do). If you ever do need to escape a value for a query, you can use org.apache.solr.client.solrj.util.ClientUtils.escapeQueryChars(). -Michael -Original Message- From: tedsolr [mailto:tsm...@sciquest.com] Sent: Wednesday, October 01, 2014 5:33 PM To: solr-user@lucene.apache.org Subject: Exact match on string field with special characters I am trying to do SQL like aggregation (GROUP BY) with solr faceting. So I use string fields for faceting - to try to get an exact match. However, it seems like to run a facet query I have to surround the value with double quotes. That poses issues when the field value is green "bath" towels -or- red \cars Those two special characters must be transformed somehow on indexing so I can create the query: (java) ... String fg = fieldName + ":\"" + fieldValue + "\""; query.addFacetField(fq); ... Is there a way to request an exact match search without having to resort to the quotes? I could possibly convert spaces to underscores at index time, but I'd like to avoid munging that data because I'm using the string field for display too! That saves time/searches when aggregating against 10 - 15 fields which takes a whole lot of facet searches to begin with. Using Solr 4.9 -- View this message in context: http://lucene.472066.n3.nabble.com/Exact-match-on-string-field-with-special-characters-tp4162209.html Sent from the Solr - User mailing list archive at Nabble.com.
Exact match on string field with special characters
I am trying to do SQL like aggregation (GROUP BY) with solr faceting. So I use string fields for faceting - to try to get an exact match. However, it seems like to run a facet query I have to surround the value with double quotes. That poses issues when the field value is green "bath" towels -or- red \cars Those two special characters must be transformed somehow on indexing so I can create the query: (java) ... String fg = fieldName + ":\"" + fieldValue + "\""; query.addFacetField(fq); ... Is there a way to request an exact match search without having to resort to the quotes? I could possibly convert spaces to underscores at index time, but I'd like to avoid munging that data because I'm using the string field for display too! That saves time/searches when aggregating against 10 - 15 fields which takes a whole lot of facet searches to begin with. Using Solr 4.9 -- View this message in context: http://lucene.472066.n3.nabble.com/Exact-match-on-string-field-with-special-characters-tp4162209.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tricky exact match, unwanted search results
Seems now the problem is solved for me. I have already tried to re-index the documents, but after I have re-indexed the core once again, Solr starts returning the correct results. query: srcphrase:in AND srcphrase:case AND srcphrase:if the phrase found query: srcphrase:"case if in"~3 the phrase found query: srcphrase:"in case if" the phrase found -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Tricky-exact-match-unwanted-search-results-tp4158652p4158894.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tricky exact match, unwanted search results
I keep asking people this eternal question: What training or doc are you reading that is using this term "exact match"? Clearly the term is being used by a lot of people in a lot of ambiguous ways, when "exact" should be... "exact". I think we need to start using the term "exact match" ONLY for string field queries, and that don't use wildcard, fuzzy, or range queries. And maybe also keyword tokenizer text fields that don't have any filters, which might as well be string fields. -- Jack Krupansky -Original Message- From: FiMka Sent: Sunday, September 14, 2014 9:34 AM To: solr-user@lucene.apache.org Subject: Re: Solr: Tricky exact match, unwanted search results *Erick*, thank you for help! For exact match I still want: to use stemming (e.g. for "sleep" I want the word forms "slept", "sleeping", "sleeps" also to be used in searching) to disregard case sensitivity to disregard prepositions, conjunctions and other function words to match only docs having all of the query words and in the given order (except function words) to match only docs if there are no other words in the doc field besides the words in the query to use synonyms (e.g. "GB" == "gigabyte", "Television" == "TV") Erick Erickson wrote The easiest way to make your examples work wouldbe to use a copyField to an "exact match" field thatuses the KeywordTokenizer The KeywordTokenizer treats the entire field as a single token, regardless of its content. So this does not fit to my requirements. Erick Erickson wrote You'll have to be a little careful to escape spaces for muti-term bits, like exact_field:pussy\ cat. Hmm... I don't care about quoting right now at all. But should I? Erick Erickson wrote As far as your question about "if" and "in", what you're probably getting here is stopword removal, but that's a guess. I have the following document:After I disabled solr.StopFilterFactory for analyzer type="query" Solr stopped returning this document for the query: http://localhost:8983/solr/lexikos/select?q=phraseExact%3A%22on+a+case-by-case%22.Can I somehow implement the desired "exact match" behavior? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Tricky-exact-match-unwanted-search-results-tp4158652p4158745.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr: Tricky exact match, unwanted search results
FiMka wrote > After I disabled solr.StopFilterFactory for analyzer type="query" Solr > stopped returning this document for the query: > http://localhost:8983/solr/lexikos/select?q=phraseExact%3A%22on+a+case-by-case%22. Forgot to say, I have also disabled solr.StopFilterFactory for analyzer type="index", removed all the documents and then re-added them. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Tricky-exact-match-unwanted-search-results-tp4158652p4158748.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr: Tricky exact match, unwanted search results
*Erick*, thank you for help! For exact match I still want: to use stemming (e.g. for "sleep" I want the word forms "slept", "sleeping", "sleeps" also to be used in searching) to disregard case sensitivity to disregard prepositions, conjunctions and other function words to match only docs having all of the query words and in the given order (except function words) to match only docs if there are no other words in the doc field besides the words in the query to use synonyms (e.g. "GB" == "gigabyte", "Television" == "TV") Erick Erickson wrote > The easiest way to make your examples work wouldbe to use a copyField to > an "exact match" field thatuses the KeywordTokenizer The KeywordTokenizer treats the entire field as a single token, regardless of its content. So this does not fit to my requirements. Erick Erickson wrote > You'll have to be a little careful to escape spaces for muti-term bits, > like exact_field:pussy\ cat. Hmm... I don't care about quoting right now at all. But should I? Erick Erickson wrote > As far as your question about "if" and "in", what you're probably getting > here is stopword removal, but that's a guess. I have the following document:After I disabled solr.StopFilterFactory for analyzer type="query" Solr stopped returning this document for the query: http://localhost:8983/solr/lexikos/select?q=phraseExact%3A%22on+a+case-by-case%22.Can I somehow implement the desired "exact match" behavior? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Tricky-exact-match-unwanted-search-results-tp4158652p4158745.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr: Tricky exact match, unwanted search results
The easiest way to make your examples work would
be to use a copyField to an "exact match" field that
uses the KeywordTokenizer (and perhaps a lowercase filter).
Then your exact match would be satisfied by a simple
wildcard search for cat*.
You'll have to be a little careful to escape spaces for
muti-term bits, like
exact_field:pussy\ cat.
Quoting these is sometimes tricky.
Anyway, as the thread you linked to says, details matter
a lot in this case. The keyword tokenizer won't let you find
"dog has" in a field populated with "my dog has fleas" for
instance with this techinque.
As far as your question about "if" and "in", what you're probably
getting here is stopword removal, but that's a guess. Try it
again without including a stopword filter factory to test that
out.
Best,
Erick
On Sat, Sep 13, 2014 at 1:33 PM, FiMka wrote:
> Hi guys, could you help me with implementing exact match search in Solr.
> Say I have the following Solr documents: And my search query is:
> By default Solr for the given documents and the search query "cat" will give
> all the partially matched documents ("cat", "pussy cat" and "cats"), but
> only the "cat" and "cats" results are wanted by me.
> To enable the desired Sorl search behavior, as it was suggested here
> <http://lucene.472066.n3.nabble.com/How-to-do-exact-match-td4014264.html#a4014362>
> , I add prefixes and suffixes to phrases each time when adding new documents
> (much more details are given here
> <http://stackoverflow.com/questions/24344740/solr-exact-match-regarding-words-number>
> ).
> I can say in most of cases (like for the above documents) this solution is
> working (we have now only "cat" and "cats" as the results for the query
> "http://localhost:8983/solr/select?q=phraseExact%3A+%22_prefix_+cat+_suffix_%22";).
> But if we have the documents like:
> If submit the search query
> "http://localhost:8983/solr/select?q=phraseExact%3A+%22_prefix_+in+case+if++_suffix_%22";,
> then all three documents are returned ("in case if", "worst-case condition",
> "optimistic-case module").
> I can suppose that such behavior can be because of preposition and
> conjunction in the searched phrase "*in* case *if*". These words possibly
> are not treated by Solr? Also it is interesting that the number of words
> (three words) in the original phrase ("in case if") matches the resulting
> phrases ("worst-case condition" and "optimistic-case module").
> Do you have some ideas why I have such results and what can be the reasons?
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Tricky-exact-match-unwanted-search-results-tp4158652.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Solr: Tricky exact match, unwanted search results
Hi guys, could you help me with implementing exact match search in Solr.
Say I have the following Solr documents: And my search query is:
By default Solr for the given documents and the search query "cat" will give
all the partially matched documents ("cat", "pussy cat" and "cats"), but
only the "cat" and "cats" results are wanted by me.
To enable the desired Sorl search behavior, as it was suggested here
<http://lucene.472066.n3.nabble.com/How-to-do-exact-match-td4014264.html#a4014362>
, I add prefixes and suffixes to phrases each time when adding new documents
(much more details are given here
<http://stackoverflow.com/questions/24344740/solr-exact-match-regarding-words-number>
).
I can say in most of cases (like for the above documents) this solution is
working (we have now only "cat" and "cats" as the results for the query
"http://localhost:8983/solr/select?q=phraseExact%3A+%22_prefix_+cat+_suffix_%22";).
But if we have the documents like:
If submit the search query
"http://localhost:8983/solr/select?q=phraseExact%3A+%22_prefix_+in+case+if++_suffix_%22";,
then all three documents are returned ("in case if", "worst-case condition",
"optimistic-case module").
I can suppose that such behavior can be because of preposition and
conjunction in the searched phrase "*in* case *if*". These words possibly
are not treated by Solr? Also it is interesting that the number of words
(three words) in the original phrase ("in case if") matches the resulting
phrases ("worst-case condition" and "optimistic-case module").
Do you have some ideas why I have such results and what can be the reasons?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Tricky-exact-match-unwanted-search-results-tp4158652.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Exact match?
Maybe use dismax for this? Something like q={!dismax qf=field_name
mm=90%}query_string, or more verbosely and separately,
q=query_string&defType=dismax&mm=90%
Erik
On Aug 19, 2014, at 2:43 AM, William Bell wrote:
> If I have a long string, how do I match on 90% of the terms to see if there
> is a duplicate?
>
> If I add the field and index it, what is the best way to return 90%?
>
> # terms match
> # of terms in the field?
>
>
> --
> Bill Bell
> billnb...@gmail.com
> cell 720-256-8076
Exact match?
If I have a long string, how do I match on 90% of the terms to see if there is a duplicate? If I add the field and index it, what is the best way to return 90%? # terms match # of terms in the field? -- Bill Bell billnb...@gmail.com cell 720-256-8076
