Re: Solr Document expiration with TTL
Great, Thank you for confirming. This will help other people seeing similar kinds of issues. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 9 March 2015 at 05:02, Makailol Charls 4extrama...@gmail.com wrote: Hi, As suggested, we could manage to activate the document expiration using TTL by shifting the newly added URP setting in the starting of the add-unknown-fields-to-the-schema URP. That populates the expire_at_dt field and makes document to expire after defined TTL. Many Thanks, Makailol
RE: Check the return of suggestions
Hi Alex, It looks like your search term and index are both subject to a stem filter. Is that right? To avoid the default query parser for spellcheck purposes, you might try spellcheck.q=cartouche. But that may not be sufficient if the spellcheck field is also aggressively stemmed. I.e. try solr.EnglishMinimalStemFilterFactory vs. solr.PorterStemFilterFactory. Worst case, you may need to copy values to a separate spellcheck field with less aggressive stemming. It seems unlikely, to me, that touches and cartouche would have the same stem. But touches may or may not be an ok spellcheck correction for your app.You can tweak the accuracy parameter. Also, if using DirectSolrSpellChecker, check maxEdits. Just a couple thoughts ... hth, Charlie -Original Message- From: ale42 [mailto:alexandre.faye...@etu.esisar.grenoble-inp.fr] Sent: Friday, March 06, 2015 6:20 AM To: solr-user@lucene.apache.org Subject: Check the return of suggestions Hello everyone. I'm working with Solr 4.3. I use the Spellechecker component which gives me suggestions as i expect. I will explain my problem with an example : I am querying /cartouchhe/instead of /cartouche/. I obtain these suggestions array (size=5) 0 = array (size=2) 'word' = *string 'cartouche' (length=9)* 'freq' = *int 1519* 1 = array (size=2) 'word' = *string 'touches' (length=7)* 'freq' =* int 55* 2 = array (size=2) 'word' = *string 'cartouches' (length=10)* 'freq' =*int 32* 3 = array (size=2) 'word' =* string 'caoutchoucs' (length=11)* 'freq' =* int 16* 4 = array (size=2) 'word' = *string 'cartonnees' (length=10)* 'freq' =* int 15* This is what I want == OK. The problem is that when I query /cartouche/or /cartouches/, I exactly have the same results because for both query, the term that will be searching into my index is /cartouch/. Is there a way with Solr to fix this kind of problem ie check that 2 collations will not return exactly the same results? Thanks for your answers, Alex. -- View this message in context: http://lucene.472066.n3.nabble.com/Check-the-return-of-suggestions-tp4191383.html Sent from the Solr - User mailing list archive at Nabble.com. * This e-mail may contain confidential or privileged information. If you are not the intended recipient, please notify the sender immediately and then delete it. TIAA-CREF *
Re: Cores and and ranking (search quality)
(reposing this to see if anyone can help) Help me understand this better (regarding ranking). If I have two docs that are 100% identical with the exception of uid (which is stored but not indexed). In a single core setup, if I search xyz such that those 2 docs end up ranking as #1 and #2. When I switch over to two core setup, doc-A goes to core-A (which has 10 records) and doc-B goes to core-B (which has 100,000 records). Now, are you saying in 2 core setup if I search on xyz (just like in singe core setup) this time I will not see doc-A and doc-B as #1 and #2 in ranking? That is, are you saying doc-A may now be somewhere at the top / bottom far away from doc-B? If so, which will be #1: the doc off core-A (that has 10 records) or doc-B off core-B (that has 100,000 records)? If I got all this right, are you saying SOLR-1632 will fix this issue such that the end result will now be as if I had 1 core? - MJ -Original Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: Thursday, March 5, 2015 9:06 AM To: solr-user@lucene.apache.org Subject: Re: Cores and and ranking (search quality) On Thu, 2015-03-05 at 14:34 +0100, johnmu...@aol.com wrote: My question is this: if I put my data in multiple cores and use distributed search will the ranking be different if I had all my data in a single core? Yes, it will be different. The practical impact depends on how homogeneous your data are across the shards and how large your shards are. If you have small and dissimilar shards, your ranking will suffer a lot. Work is being done to remedy this: https://issues.apache.org/jira/browse/SOLR-1632 Also, will facet and more-like-this quality / result be the same? It is not formally guaranteed, but for most practical purposes, faceting on multi-shards will give you the same results as single-shards. I don't know about more-like-this. My guess is that it will be affected in the same way that standard searches are. Also, reading the distributed search wiki (http://wiki.apache.org/solr/DistributedSearch) it looks like Solr does the search and result merging (all I have to do is issue a search), is this correct? Yes. From a user-perspective, searches are no different. - Toke Eskildsen, State and University Library, Denmark
IllegalArgumentException: Unknown type of result: class javax.xml.transform.dom.DOMResult
I am seeing the following stacktrace(s): Caused by: java.lang.IllegalArgumentException: Unknown type of result: class javax.xml.transform.dom.DOMResult at net.sf.saxon.event.SerializerFactory.getReceiver(SerializerFactory.java:154) ~[netcdfAll.jar:4.5.4] at net.sf.saxon.IdentityTransformer.transform(IdentityTransformer.java:24) ~[netcdfAll.jar:4.5.4] at org.apache.solr.core.Config.copyDoc(Config.java:203) ~[solr-core.jar:5.0.0 1659987 - anshumgupta - 2015-02-15 12:26:10] at org.apache.solr.core.Config.init(Config.java:146) ~[solr-core.jar:5.0.0 1659987 - anshumgupta - 2015-02-15 12:26:10] at org.apache.solr.core.ConfigSolr.fromInputStream(ConfigSolr.java:68) ~[solr-core.jar:5.0.0 1659987 - anshumgupta - 2015-02-15 12:26:10] What jar's am I missing or what jar-version-conflicts am I confronted with? Unfortunately I don't see these errors when developing in eclipse, only when deploying onto our tomcat... Thanks for you hints Clemens
How to get MoreLikeThis result
I'm trying to understand how Solr MorelIkeThis works. Steps I've done -
1. In schema.xml I've written -
field name=path_exact type=string indexed=true stored=true
termVectors=true/
field name=title type=text_general indexed=true stored=true
multiValued=true termVectors=true/
2. Mentioned uniqueKeypath_exact/uniqueKey
3. Created index in solr by using below command -
{path_exact:id1,title:x1}
{path_exact:id2,title:x12}
4. Now when I'm trying to hit the below url then it return result but I'm
not able to understand what does it mean exactly? Is it not able to find
morelikethis item for id1 and id2? If, yes, then what I'm missing here?
http://:/solr/collection2/select?q=x1*mlt=truemlt.fl=titlewt=xml
Result -
lst name=moreLikeThis
result name=id1 numFound=0 start=0/
result name=id2 numFound=0 start=0/
Thanks for your help!
--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-get-MoreLikeThis-result-tp4191853.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: Check the return of suggestions
Thanks for your answer Charlie, Reitzel, Charles wrote / It looks like your search term and index are both subject to a stem filter. Is that right? / Yes, that is right and that is what I want ! / To avoid the default query parser for spellcheck purposes, you might try spellcheck.q=cartouche. But that may not be sufficient if the spellcheck field is also aggressively stemmed. I.e. try solr.EnglishMinimalStemFilterFactory vs. solr.PorterStemFilterFactory. Worst case, you may need to copy values to a separate spellcheck field with less aggressive stemming. / My spellcheck field is not agressive. It doesn't use stem, just * WhitespaceTokenizer * , * StopFilter * , * WordDelimiterFilter * , * LowerCaseFilter * and * ASCIIFoldingFilter * . I am using my website in France, so I don't think I can use * solr.EnglishMinimalStemFilterFactory * . / It seems unlikely, to me, that touches and cartouche would have the same stem. But touches may or may not be an ok spellcheck correction for your app.You can tweak the accuracy parameter. Also, if using DirectSolrSpellChecker, check maxEdits. / Yes, I success to get around the problem by using threshold and acuracy but it is not a perfect way for me, because it can miss few usefull suggestions if these one are not well represented in the corpus... Thanks for your help. -- View this message in context: http://lucene.472066.n3.nabble.com/Check-the-return-of-suggestions-tp4191383p4191856.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: SpellCheck component query
Hi Ashish, We are doing some very close to what you describe. As Aman says, it requires two solr queries to achieve that result. I.e. you need to build this logic into your application. Solr won't do it for you.In our case, for the second query, we use a faceted results against an ngram filtered field to get valid terms. hth, Charlie -Original Message- From: Ashish Mukherjee [mailto:ashish.mukher...@gmail.com] Sent: Monday, March 09, 2015 1:55 AM To: solr-user@lucene.apache.org Cc: amantandon...@gmail.com Subject: Re: SpellCheck component query Hi Aman, Thanks for your response. Taking your example further to elaborate what I am looking to do - if user types 'chai' and suggestion is 'chat' , then I would like to see all the values which 'chat' in them as suggestions, such as 'text chat', 'video chat', 'audio chat' etc. without making another search request for 'chat'. Can this be accomplished? Regards, Ashish On Mon, Mar 9, 2015 at 2:50 AM, Aman Tandon amantandon...@gmail.com wrote: Hi, AFAIK solr currently not providing this feature. Suppose a scenario, the user is trying to search for chai (hindi meaning of tea). And in your index you have more documents containing the keyword chat as compared to the the keyword chai. chat = 501 chai = 29 and the maxQueryFrequency is 25. So solr will suggest you chat as this term is present in more documents and if you want from solr to search internally for the suggestion then you will be displaying he results of chat to the user when the user intended to search for chai. So I suppose it is good to show wrong suggestion instead of providing the inappropriate results to the user. In this case you could the show the chat and chai as suggestion to the user and then he could select the appropriate suggestion. With Regards Aman Tandon On Sat, Mar 7, 2015 at 7:57 PM, Ashish Mukherjee ashish.mukher...@gmail.com wrote: Hello, I have enabled the Spellcheck component in Solr, which gives me spelling suggestions. However, I would like those suggestions to be applied in the same select request handler to retrieve additional results based on the suggestions. How can this be achieved with Solr? Regards, Ashish * This e-mail may contain confidential or privileged information. If you are not the intended recipient, please notify the sender immediately and then delete it. TIAA-CREF *
RE: Combine multiple SOLR Query Results
Hi AnilJayanti, You shouldn't need 2 separate solr queries. Just make sure both 'track name' and 'artist name' fields are queried. Solr will rank and sort the results for you. e.q. q=fooqf=trackName,artistName This is preferable for a number of reasons. I will be faster and simpler. But, also, highlight results should be better. hth, Charlie -Original Message- From: aniljayanti [mailto:aniljaya...@yahoo.co.in] Sent: Monday, March 09, 2015 6:20 AM To: solr-user@lucene.apache.org Subject: Combine multiple SOLR Query Results Hi, I am trying to work on combine multiple SOLR query results into single result. Below is my case. 1. Look up search term against ‘track name’, log results 2. Look up search term against ‘artist name’, log results of tracks by those artists 3. Combine results 4. results by score descending order. Using text_general fieldType for both track name and artist name. copy fields are trackname and artistname Plase suggest me how to write solr Query to combine two solr results into single result. Thanks in advance. AnilJayanti -- View this message in context: http://lucene.472066.n3.nabble.com/Combine-multiple-SOLR-Query-Results-tp4191816.html Sent from the Solr - User mailing list archive at Nabble.com. * This e-mail may contain confidential or privileged information. If you are not the intended recipient, please notify the sender immediately and then delete it. TIAA-CREF *
Re: SolrCloud: Chroot error
Hi Shawn, Thanks for replying, Just to send the mail, I replaced the IP addresses with the imaginary hostname, now the command is *./solr start -c -z localhost:2181,abc.com:2181 http://abc.com:2181,xyz.com:2181/home/aman/solrcloud/solr_zoo http://xyz.com:2181/home/aman/solrcloud/solr_zoo -p 4567* *ERROR - 2015-03-09 12:38:03.394; org.apache.solr.servlet.SolrDispatchFilter;* * Could not start Solr. Check solr/home property and the logs* *ERROR - 2015-03-09 12:38:03.416; org.apache.solr.common.SolrException; null:* *org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified in Z* *kHost but the znode doesn't exist. localhost:2181,abc.com:2181 http://abc.com:2181,xyz.com http://xyz.com* *:2181/data/solr/aman/solr_cloud/zoo_solr* *at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:1* *31)* With Regards Aman Tandon On Mon, Mar 9, 2015 at 9:26 PM, Aman Tandon amantandon...@gmail.com wrote: Hi Anshum, Thanks for your time. I am using the external zookeeper. I am unaware that where should i create the path in Zk. The directory structure for zookepeer is as follows: Zookeeper */ *instance / bin,etc Zookeeper / data Zookeeper / logs instance (contains bin,etc folders of zookeeper server) data (dataDir for zookeeper) logs (dataDirLog for zookeeper to keep the logs) With Regards Aman Tandon On Mon, Mar 9, 2015 at 9:02 PM, Anshum Gupta ans...@anshumgupta.net wrote: Looks like the zk you're pointing to doesn't contain the path 'data/solr/aman/solr_cloud' that you've specified as the chroot prefix. Can you confirm that your zk contains this path? If it doesn't, can you create this path and then try? On Mon, Mar 9, 2015 at 8:19 AM, Aman Tandon amantandon...@gmail.com wrote: Hi, I tried to start solr with chroot feature, i am using solr 5.0.0 on Centos 6 and i am getting this error in the logs. Please help. *./solr start -c -z localhost:2181,192.168.5.11:2181 http://192.168.5.11:2181, 192.168.3.13:2181/home/aman/solrcloud/solr_zoo http://192.168.3.13:2181/home/aman/solrcloud/solr_zoo -p 4567* *ERROR - 2015-03-09 12:38:03.394; org.apache.solr.servlet.SolrDispatchFilter;* * Could not start Solr. Check solr/home property and the logs* *ERROR - 2015-03-09 12:38:03.416; org.apache.solr.common.SolrException; null:* *org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified in Z* *kHost but the znode doesn't exist. 192.168.6.217:2181 http://192.168.6.217:2181/,192.168.5.81:2181 http://192.168.5.81:2181/,192.* *168.5.236:2181/data/solr/aman/solr_cloud/zoo_solr* *at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:1* *31)* With Regards Aman Tandon -- Anshum Gupta
Re: SolrCloud: Chroot error
If you 1. Don't know about chroot and 2. Don't have other apps using the same zk and no other requirement to change the default; don't change the defaults. The zk string then should just point to host1:port1,host2:port2... String without the /path/in/zk part. -Anshum On Mar 9, 2015, at 8:56 AM, Aman Tandon amantandon...@gmail.com wrote: Hi Anshum, Thanks for your time. I am using the external zookeeper. I am unaware that where should i create the path in Zk. The directory structure for zookepeer is as follows: Zookeeper */ *instance / bin,etc Zookeeper / data Zookeeper / logs instance (contains bin,etc folders of zookeeper server) data (dataDir for zookeeper) logs (dataDirLog for zookeeper to keep the logs) With Regards Aman Tandon On Mon, Mar 9, 2015 at 9:02 PM, Anshum Gupta ans...@anshumgupta.net wrote: Looks like the zk you're pointing to doesn't contain the path 'data/solr/aman/solr_cloud' that you've specified as the chroot prefix. Can you confirm that your zk contains this path? If it doesn't, can you create this path and then try? On Mon, Mar 9, 2015 at 8:19 AM, Aman Tandon amantandon...@gmail.com wrote: Hi, I tried to start solr with chroot feature, i am using solr 5.0.0 on Centos 6 and i am getting this error in the logs. Please help. *./solr start -c -z localhost:2181,192.168.5.11:2181 http://192.168.5.11:2181, 192.168.3.13:2181/home/aman/solrcloud/solr_zoo http://192.168.3.13:2181/home/aman/solrcloud/solr_zoo -p 4567* *ERROR - 2015-03-09 12:38:03.394; org.apache.solr.servlet.SolrDispatchFilter;* * Could not start Solr. Check solr/home property and the logs* *ERROR - 2015-03-09 12:38:03.416; org.apache.solr.common.SolrException; null:* *org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified in Z* *kHost but the znode doesn't exist. 192.168.6.217:2181 http://192.168.6.217:2181/,192.168.5.81:2181 http://192.168.5.81:2181/,192.* *168.5.236:2181/data/solr/aman/solr_cloud/zoo_solr* *at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:1* *31)* With Regards Aman Tandon -- Anshum Gupta
Re: Solr query to match document templates - sort of a reverse wildcard match
Are you looking for Luwak? https://github.com/flaxsearch/luwak Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 6 March 2015 at 09:19, Robert Stewart b...@kasisto.com wrote: If I have SOLR document with field value such as: a ? c ? e And I want a phrase query such as a b c d e to match that document. So: q:a b c d e -- return doc with a ? c ? e as field value for q field. Is this possible, or is there a way it can be done with a plug-in using lower level Lucene SDK? Maybe some custom implementation of TermQuery where value of ? always matches any term in the query? Thanks! Robert Stewart
SolrCloud: Chroot error
Hi, I tried to start solr with chroot feature, i am using solr 5.0.0 on Centos 6 and i am getting this error in the logs. Please help. *./solr start -c -z localhost:2181,192.168.5.11:2181 http://192.168.5.11:2181,192.168.3.13:2181/home/aman/solrcloud/solr_zoo http://192.168.3.13:2181/home/aman/solrcloud/solr_zoo -p 4567* *ERROR - 2015-03-09 12:38:03.394; org.apache.solr.servlet.SolrDispatchFilter;* * Could not start Solr. Check solr/home property and the logs* *ERROR - 2015-03-09 12:38:03.416; org.apache.solr.common.SolrException; null:* *org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified in Z* *kHost but the znode doesn't exist. 192.168.6.217:2181 http://192.168.6.217:2181/,192.168.5.81:2181 http://192.168.5.81:2181/,192.* *168.5.236:2181/data/solr/aman/solr_cloud/zoo_solr* *at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:1* *31)* With Regards Aman Tandon
Re: Solr packages in Apache BigTop.
Jay, This is music to my ears. I've used the bigtop packages and would love to see the Solr portion of them keep pace with releases. Let me know where to start! Thank you, Scott On Sat, Mar 7, 2015 at 5:03 PM, jay vyas jayunit100.apa...@gmail.com wrote: Hi Solr. I work on the apache bigtop project, and am interested in integrating it deeper with Solr, for example, for testing spark / solr integration cases. Is anyone in the Solr community interested in collborating on testing releases with us and maintaining Solr packagins in bigtop (with our help of course) ? The advantage here is that we can synergize efforts: When new SOLR releases come out, we can test them in bigtop to gaurantee that there are rpm/deb packages which work well with the hadoop ecosystem. For those that don't know, bigtop is the upstream apache bigdata packaging project, we build hadoop, spark, solr, hbase and so on in rpm/deb format, and supply puppet provisioners along with vagrant recipse for testing. -- jay vyas -- Scott Stults | Founder Solutions Architect | OpenSource Connections, LLC | 434.409.2780 http://www.opensourceconnections.com
RE: Solr query to match document templates - sort of a reverse wildcard match
Have a look at solr.StopFilterFactory. https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-StopFilter If your place holders (?) are works like and, the, is, to, etc (see lang/stopwords_??.txt), the stop filter is designed to do what you want. It leaves holes in the parsed text so that word distance is maintained in the output. -Original Message- From: Robert Stewart [mailto:b...@kasisto.com] Sent: Friday, March 06, 2015 9:19 AM To: solr-user@lucene.apache.org Subject: Solr query to match document templates - sort of a reverse wildcard match If I have SOLR document with field value such as: a ? c ? e And I want a phrase query such as a b c d e to match that document. So: q:a b c d e -- return doc with a ? c ? e as field value for q field. Is this possible, or is there a way it can be done with a plug-in using lower level Lucene SDK? Maybe some custom implementation of TermQuery where value of ? always matches any term in the query? Thanks! Robert Stewart * This e-mail may contain confidential or privileged information. If you are not the intended recipient, please notify the sender immediately and then delete it. TIAA-CREF *
RE: Solr TCP layer
A couple thoughts: 0. Interesting topic. 1. But perhaps better suited to the dev list. 2. Given the existing architecture, shouldn't we be looking to transport projects, e.g. Jetty, Apache HttpComponents, for support of new socket or even HTTP layer protocols? 3. To the extent such support exists, then integration work is still needed at the solr level. Shalin, is this your intention? Also, for those of us not tracking protocol standards in detail, can you describe the benefits to Solr users of http/2? Do you expect HTTP/2 to be transparent at the application layer? -Original Message- From: Shalin Shekhar Mangar [mailto:shalinman...@gmail.com] Sent: Monday, March 09, 2015 6:23 AM To: solr-user@lucene.apache.org Subject: Re: Solr TCP layer Hi Saumitra, I've been thinking of adding http/2 support for inter node communication initially and client server communication next in Solr. There's a patch for SPDY support but now that spdy is deprecated and http/2 is the new standard we need to wait for Jetty 9.3 to release. That will take care of many bottlenecks in solrcloud communication. The current trunk is already using jetty 9.2.x which has support for the draft http/2 spec. A brand new async TCP layer based on netty can be considered but that's a huge amount of work considering our need to still support simple http, SSL etc. Frankly for me that effort is better spent optimizing the routing layer. On 09-Mar-2015 1:37 am, Saumitra Srivastav saumitra.srivast...@gmail.com wrote: Dear Solr Contributors, I want to start working on adding a TCP layer for client to node and inter-node communication. I am not up to date on recent changes happening to Solr. So before I start looking into code, I would like to know if there is already some work done in this direction, which I can reuse. Are there any know challenges/complexities? I would appreciate any help to kick start this effort. Also, what would be the best way to discuss and get feedback on design from contributors? Open a JIRA?? Regards, Saumitra -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-TCP-layer-tp4191715.html Sent from the Solr - User mailing list archive at Nabble.com. * This e-mail may contain confidential or privileged information. If you are not the intended recipient, please notify the sender immediately and then delete it. TIAA-CREF *
Re: SolrCloud: Chroot error
On 3/9/2015 9:19 AM, Aman Tandon wrote: I tried to start solr with chroot feature, i am using solr 5.0.0 on Centos 6 and i am getting this error in the logs. Please help. *./solr start -c -z localhost:2181,192.168.5.11:2181 http://192.168.5.11:2181,192.168.3.13:2181/home/aman/solrcloud/solr_zoo http://192.168.3.13:2181/home/aman/solrcloud/solr_zoo -p 4567* *ERROR - 2015-03-09 12:38:03.394; org.apache.solr.servlet.SolrDispatchFilter;* * Could not start Solr. Check solr/home property and the logs* *ERROR - 2015-03-09 12:38:03.416; org.apache.solr.common.SolrException; null:* *org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified in Z* *kHost but the znode doesn't exist. 192.168.6.217:2181 As Anshum mentioned, it appears that the chroot you have specified doesn't exist. Note that this is a path within the zookeeper database ... you appear to have listed a filesystem path, which is NOT the same thing. Typically a chroot on zookeeper should be very simple, normally consisting of one forward slash followed by a short string, like /solr1 or /mysolr or similar. As for why it simply didn't create the path, I wonder if perhaps zookeeper (or the client code within Solr) is not capable of creating a multi-level path. You could try starting it four times, to see if it will create one path level on each run. I know from prior experience with SolrCloud on version 4.2.1 that a chroot with a single path element will work. Your mail client appears to have turned the text of your zkHost parameter into URLs, which makes it difficult to see what you are actually using. Can you make your reply plaintext and include it again so that doesn't happen? Thanks, Shawn
Re: SolrCloud: Chroot error
Looks like the zk you're pointing to doesn't contain the path 'data/solr/aman/solr_cloud' that you've specified as the chroot prefix. Can you confirm that your zk contains this path? If it doesn't, can you create this path and then try? On Mon, Mar 9, 2015 at 8:19 AM, Aman Tandon amantandon...@gmail.com wrote: Hi, I tried to start solr with chroot feature, i am using solr 5.0.0 on Centos 6 and i am getting this error in the logs. Please help. *./solr start -c -z localhost:2181,192.168.5.11:2181 http://192.168.5.11:2181,192.168.3.13:2181/home/aman/solrcloud/solr_zoo http://192.168.3.13:2181/home/aman/solrcloud/solr_zoo -p 4567* *ERROR - 2015-03-09 12:38:03.394; org.apache.solr.servlet.SolrDispatchFilter;* * Could not start Solr. Check solr/home property and the logs* *ERROR - 2015-03-09 12:38:03.416; org.apache.solr.common.SolrException; null:* *org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified in Z* *kHost but the znode doesn't exist. 192.168.6.217:2181 http://192.168.6.217:2181/,192.168.5.81:2181 http://192.168.5.81:2181/,192.* *168.5.236:2181/data/solr/aman/solr_cloud/zoo_solr* *at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:1* *31)* With Regards Aman Tandon -- Anshum Gupta
Re: Solr query to match document templates - sort of a reverse wildcard match
A Lucene MultiPhraseQuery would work, with each phrase position having two Terms -- the as-seen word in the query and the term ‘?’. Performance wouldn’t be so great given that ‘?’ would appear extremely frequently. A variation solving the performance is to use CommonGrams but it would add some complexity to building the query. ~ David Smiley Freelance Apache Lucene/Solr Search Consultant/Developer http://www.linkedin.com/in/davidwsmiley On Fri, Mar 6, 2015 at 9:19 AM, Robert Stewart b...@kasisto.com wrote: If I have SOLR document with field value such as: a ? c ? e And I want a phrase query such as a b c d e to match that document. So: q:a b c d e -- return doc with a ? c ? e as field value for q field. Is this possible, or is there a way it can be done with a plug-in using lower level Lucene SDK? Maybe some custom implementation of TermQuery where value of ? always matches any term in the query? Thanks! Robert Stewart
Re: SolrCloud: Chroot error
Hi Anshum, Thanks for your time. I am using the external zookeeper. I am unaware that where should i create the path in Zk. The directory structure for zookepeer is as follows: Zookeeper */ *instance / bin,etc Zookeeper / data Zookeeper / logs instance (contains bin,etc folders of zookeeper server) data (dataDir for zookeeper) logs (dataDirLog for zookeeper to keep the logs) With Regards Aman Tandon On Mon, Mar 9, 2015 at 9:02 PM, Anshum Gupta ans...@anshumgupta.net wrote: Looks like the zk you're pointing to doesn't contain the path 'data/solr/aman/solr_cloud' that you've specified as the chroot prefix. Can you confirm that your zk contains this path? If it doesn't, can you create this path and then try? On Mon, Mar 9, 2015 at 8:19 AM, Aman Tandon amantandon...@gmail.com wrote: Hi, I tried to start solr with chroot feature, i am using solr 5.0.0 on Centos 6 and i am getting this error in the logs. Please help. *./solr start -c -z localhost:2181,192.168.5.11:2181 http://192.168.5.11:2181, 192.168.3.13:2181/home/aman/solrcloud/solr_zoo http://192.168.3.13:2181/home/aman/solrcloud/solr_zoo -p 4567* *ERROR - 2015-03-09 12:38:03.394; org.apache.solr.servlet.SolrDispatchFilter;* * Could not start Solr. Check solr/home property and the logs* *ERROR - 2015-03-09 12:38:03.416; org.apache.solr.common.SolrException; null:* *org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified in Z* *kHost but the znode doesn't exist. 192.168.6.217:2181 http://192.168.6.217:2181/,192.168.5.81:2181 http://192.168.5.81:2181/,192.* *168.5.236:2181/data/solr/aman/solr_cloud/zoo_solr* *at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:1* *31)* With Regards Aman Tandon -- Anshum Gupta
Re: SolrCloud: Chroot error
On 3/9/2015 10:03 AM, Aman Tandon wrote: Thanks for replying, Just to send the mail, I replaced the IP addresses with the imaginary hostname, now the command is *./solr start -c -z localhost:2181,abc.com:2181 http://abc.com:2181,xyz.com:2181/home/aman/solrcloud/solr_zoo http://xyz.com:2181/home/aman/solrcloud/solr_zoo -p 4567* The same URL replacement is still happening. I think I know what you are doing, but I was hoping to have a clean string just to make sure. You should not be using localhost in the zkHost string unless there is only one zk server, or you are trying to start the entire cluster on one machine. All of your Solr machines should have identical zkHost parameters. That is not possible if they are separate machines and you use localhost. Your chroot should be very simple, as I mentioned in the other email. Using /solr is appropriate if you won't be sharing the zookeeper ensemble with multiple SolrCloud clusters. The filesystem layout of your zookeeper install (bin, data, logs, etc) is NOT relevant for this chroot. It exists only within the zookeeper database. Thanks, Shawn
Parsing cluster result's docs
Hi, I have a Solr instance using the clustering component (with the Lingo algorithm) working perfectly. However when I get back the cluster results only the ID's of these come back with it. What is the easiest way to retrieve full documents instead? Should I parse these IDs into a new query to Solr, or is there some configuration I am missing to return full docs instead of IDs? If it matters, I am using Solr 4.10. Thanks.
Re: Performance on faceting using docValues
Hi Lei, The mailing list doesn't allow attachments. Can you share these via a file sharing platform? On Mon, Mar 9, 2015 at 12:48 AM, lei simpl...@gmail.com wrote: The Solr instance is single-shard. Index size is around 20G and total doc # is about 12 million. Below are the histograms for the three facet fields in my query. Thanks. On Thu, Mar 5, 2015 at 11:57 PM, Toke Eskildsen t...@statsbiblioteket.dk wrote: On Thu, 2015-03-05 at 21:14 +0100, lei wrote: You present a very interesting observation. I have not noticed what you describe, but on the other hand we have not done comparative speed tests. q=*:*fq=country:USfq=category:112 First observation: Your query is '*:*, which is a magic query. Non-DV faceting has optimizations both for this query (although that ought to be disabled due to the fq) and for the inverse case where there are more hits than non-hits. Perhaps you could test with a handful of queries, which has different result sizes? facet=onfacet.sort=indexfacet.mincount=1facet.limit=2000 The combination of index order and a high limit might be an explanation: When resolving the Strings of the facet result, non-DV will perform ordinal-lookup, which is fast when done in monotonic rising order (sort=index) and if the values are close (limit=2000). I do not know if DV benefits the same way. On the other hand, your limit seems to apply only to material, so it could be that the real number of unique values is low and you just set the limit to 2000 to be sure you get everything? facet.field=manufacturerfacet.field=sellerfacet.field=material f.manufacturer.facet.mincount=1f.manufacturer.facet.sort=countf.manufacturer.facet.limit=100 f.seller.facet.mincount=1f.seller.facet.sort=countf.seller.facet.limit=100 f.material.facet.mincount=1sort=score+desc How large is your index in bytes, how many documents does it contain and is it single-shard or cloud? Could you paste the loglines containing UnInverted field, which describes the number of unique values and size of your facet fields? - Toke Eskildsen, State and University Library, Denmark -- Anshum Gupta
Re: Apache Solr Reference Guide 5.0
: The TOC I meant was as in a traditional print publication with section #s, I believe supressing the printed page TOC was a deliberate choice made in the PDF stylesheet ... IIRC it was about 10 (printed) pages long and didn't seem to add any value since hte PDF metadata seemed much more useful (and it wsa not anticipated that many people would actually print the guide out on physical paper) : This came up because a section said it would cover topics N - S and I could : not determine if all those topics fell in that section or not. Rather then using placeholders N and S can you please refer to the specific topics / sections you were looking at and what confused you about them so we can better understand the problem you ran into? -Hoss http://www.lucidworks.com/
Field Rename in SOLR
Hello, Can anyone know how to rename a field with the below field Name. When I try the below method it says undefined field Name_en fl=ProductName:Name_en-US It throws error saying undefined field 'Name_en', it is not recognizing the full field name.. 'Name_en-US' Is there any work around..? Thanks Ravi
Re: Performance on faceting using docValues
The term histograms are shared in this link. Sorry for the confusion. https://docs.google.com/presentation/d/1tma4hkYjxJfBTnMbO6Pq_dUHqZ0wI_UTlgoVqXtW4ZA/pub?start=falseloop=falsedelayms=3000slide=id.p On Mon, Mar 9, 2015 at 10:56 AM, Anshum Gupta ans...@anshumgupta.net wrote: Hi Lei, The mailing list doesn't allow attachments. Can you share these via a file sharing platform? On Mon, Mar 9, 2015 at 12:48 AM, lei simpl...@gmail.com wrote: The Solr instance is single-shard. Index size is around 20G and total doc # is about 12 million. Below are the histograms for the three facet fields in my query. Thanks. On Thu, Mar 5, 2015 at 11:57 PM, Toke Eskildsen t...@statsbiblioteket.dk wrote: On Thu, 2015-03-05 at 21:14 +0100, lei wrote: You present a very interesting observation. I have not noticed what you describe, but on the other hand we have not done comparative speed tests. q=*:*fq=country:USfq=category:112 First observation: Your query is '*:*, which is a magic query. Non-DV faceting has optimizations both for this query (although that ought to be disabled due to the fq) and for the inverse case where there are more hits than non-hits. Perhaps you could test with a handful of queries, which has different result sizes? facet=onfacet.sort=indexfacet.mincount=1facet.limit=2000 The combination of index order and a high limit might be an explanation: When resolving the Strings of the facet result, non-DV will perform ordinal-lookup, which is fast when done in monotonic rising order (sort=index) and if the values are close (limit=2000). I do not know if DV benefits the same way. On the other hand, your limit seems to apply only to material, so it could be that the real number of unique values is low and you just set the limit to 2000 to be sure you get everything? facet.field=manufacturerfacet.field=sellerfacet.field=material f.manufacturer.facet.mincount=1f.manufacturer.facet.sort=countf.manufacturer.facet.limit=100 f.seller.facet.mincount=1f.seller.facet.sort=countf.seller.facet.limit=100 f.material.facet.mincount=1sort=score+desc How large is your index in bytes, how many documents does it contain and is it single-shard or cloud? Could you paste the loglines containing UnInverted field, which describes the number of unique values and size of your facet fields? - Toke Eskildsen, State and University Library, Denmark -- Anshum Gupta
Re: Frequency of Suggestion are varying from original Frequency in index
Hi ale42, Yes. I am using the same field (gram_ci) to make a query and also using the same field(gram_ci) to build suggestion on it. Here is the explanation: I have a 2 fields - gram and gram_ci. where gram field sets to stored = true and index = true while gram_ci field sets to stored=false but index = true. and making copy field of gram into gram_ci. Both gram and gram_ci fields using same fieldType - StandardTokenizerFactory and ShingleFilterFactory for both index and query. Only the difference is that gram_ci is using lowercaseFilter and gram doesn't. And I am making query on gram_ci not on gram. On Mon, Mar 9, 2015 at 3:24 PM, ale42 alexandre.faye...@etu.esisar.grenoble-inp.fr wrote: When you make a query, does it use the same field type as the field that you are using to build suggestions? -- View this message in context: http://lucene.472066.n3.nabble.com/Frequency-of-Suggestion-are-varying-from-original-Frequency-in-index-tp4190927p4191813.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: backport Heliosearch features to Solr
Thanks everyone for voting! Result charts (note that these auto-generated charts don't show blanks as equivalent to 0) https://docs.google.com/forms/d/1gaMpNpHVdquA3q75yiFhqZhAWdWB-K6N8Jh3dBbWAU8/viewanalytics Raw results spreadsheet (correlations can be interesting), and percentages at the bottom. https://docs.google.com/spreadsheets/d/1uZ2qgOaKx1ZxJ_NKwj2zIAYFQ9fp8OrEPI5hqadcPeY/ -Yonik On Sun, Mar 1, 2015 at 4:50 PM, Yonik Seeley ysee...@gmail.com wrote: As many of you know, I've been doing some work in the experimental heliosearch fork of Solr over the past year. I think it's time to bring some more of those changes back. So here's a poll: Which Heliosearch features do you think should be brought back to Apache Solr? http://bit.ly/1E7wi1Q (link to google form) -Yonik
Re: Performance on faceting using docValues
Sure, here is the link to the image of term histograms. Thanks. https://docs.google.com/presentation/d/1tma4hkYjxJfBTnMbO6Pq_dUHqZ0wI_UTlgoVqXtW4ZA/edit?usp=sharing On Mon, Mar 9, 2015 at 10:56 AM, Anshum Gupta ans...@anshumgupta.net wrote: Hi Lei, The mailing list doesn't allow attachments. Can you share these via a file sharing platform? On Mon, Mar 9, 2015 at 12:48 AM, lei simpl...@gmail.com wrote: The Solr instance is single-shard. Index size is around 20G and total doc # is about 12 million. Below are the histograms for the three facet fields in my query. Thanks. On Thu, Mar 5, 2015 at 11:57 PM, Toke Eskildsen t...@statsbiblioteket.dk wrote: On Thu, 2015-03-05 at 21:14 +0100, lei wrote: You present a very interesting observation. I have not noticed what you describe, but on the other hand we have not done comparative speed tests. q=*:*fq=country:USfq=category:112 First observation: Your query is '*:*, which is a magic query. Non-DV faceting has optimizations both for this query (although that ought to be disabled due to the fq) and for the inverse case where there are more hits than non-hits. Perhaps you could test with a handful of queries, which has different result sizes? facet=onfacet.sort=indexfacet.mincount=1facet.limit=2000 The combination of index order and a high limit might be an explanation: When resolving the Strings of the facet result, non-DV will perform ordinal-lookup, which is fast when done in monotonic rising order (sort=index) and if the values are close (limit=2000). I do not know if DV benefits the same way. On the other hand, your limit seems to apply only to material, so it could be that the real number of unique values is low and you just set the limit to 2000 to be sure you get everything? facet.field=manufacturerfacet.field=sellerfacet.field=material f.manufacturer.facet.mincount=1f.manufacturer.facet.sort=countf.manufacturer.facet.limit=100 f.seller.facet.mincount=1f.seller.facet.sort=countf.seller.facet.limit=100 f.material.facet.mincount=1sort=score+desc How large is your index in bytes, how many documents does it contain and is it single-shard or cloud? Could you paste the loglines containing UnInverted field, which describes the number of unique values and size of your facet fields? - Toke Eskildsen, State and University Library, Denmark -- Anshum Gupta
RE: backport Heliosearch features to Solr
Ok, so what's next? Do you intend to open issues and send the links over here so interested persons can follow them? Clearly some would like to see features to merge. Let's see what the PMC thinks about it :) Cheers, M. -Original message- From:Yonik Seeley ysee...@gmail.com Sent: Monday 9th March 2015 19:53 To: solr-user@lucene.apache.org Subject: Re: backport Heliosearch features to Solr Thanks everyone for voting! Result charts (note that these auto-generated charts don't show blanks as equivalent to 0) https://docs.google.com/forms/d/1gaMpNpHVdquA3q75yiFhqZhAWdWB-K6N8Jh3dBbWAU8/viewanalytics Raw results spreadsheet (correlations can be interesting), and percentages at the bottom. https://docs.google.com/spreadsheets/d/1uZ2qgOaKx1ZxJ_NKwj2zIAYFQ9fp8OrEPI5hqadcPeY/ -Yonik On Sun, Mar 1, 2015 at 4:50 PM, Yonik Seeley ysee...@gmail.com wrote: As many of you know, I've been doing some work in the experimental heliosearch fork of Solr over the past year. I think it's time to bring some more of those changes back. So here's a poll: Which Heliosearch features do you think should be brought back to Apache Solr? http://bit.ly/1E7wi1Q (link to google form) -Yonik
Solr 5.0.0 - Multiple instances sharing Solr server *read-only* dir
Hi all, Quoted from https://cwiki.apache.org/confluence/display/solr/Solr+Start+Script+Reference When running multiple instances of Solr on the same host, it is more common to use the same server directory for each instance and use a unique Solr home directory using the -s option. Is there a way to achieve this without making *any* changes to the extracted content of solr-5.0.0.tgz and only use runtime parameters? I other words, make the extracted folder solr-5.0.0 strictly read-only? By default, the Solr web app is deployed under server/solr-webapp, as per solr-jetty-context.xml. So unless I change solr-jetty-context.xml, I cannot make folder sorl-5.0.0 read-only to my Solr instances. I've figured out how to make the log files and pid file to be located under the Solr data dir by doing: export SOLR_PID_DIR=mySolrDataDir/logs; \ export SOLR_LOGS_DIR=mySolrDataDir/logs; \ bin/solr start -c -z localhost:32101/solr \ -s mySolrDataDir \ -a -Dsolr.log=mySolrDataDir/logs \ -p 31100 -h localhost But if there was a way to not have to change solr-jetty-context.xml that would be awesome! Thoughts? Thanks, Damien
Re: backport Heliosearch features to Solr
Ask and you shall receive: SOLR-7210 Off-Heap filter cache SOLR-7211 Off-Heap field cache SOLR-7212 Parameter substitution SOLR-7214 JSON Facet API SOLR-7216 JSON Request API Regards, Alex. P.s. Oh, the power of GMail filters :-) Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 9 March 2015 at 18:59, Markus Jelsma markus.jel...@openindex.io wrote: Ok, so what's next? Do you intend to open issues and send the links over here so interested persons can follow them? Clearly some would like to see features to merge. Let's see what the PMC thinks about it :) Cheers, M. -Original message- From:Yonik Seeley ysee...@gmail.com Sent: Monday 9th March 2015 19:53 To: solr-user@lucene.apache.org Subject: Re: backport Heliosearch features to Solr Thanks everyone for voting! Result charts (note that these auto-generated charts don't show blanks as equivalent to 0) https://docs.google.com/forms/d/1gaMpNpHVdquA3q75yiFhqZhAWdWB-K6N8Jh3dBbWAU8/viewanalytics Raw results spreadsheet (correlations can be interesting), and percentages at the bottom. https://docs.google.com/spreadsheets/d/1uZ2qgOaKx1ZxJ_NKwj2zIAYFQ9fp8OrEPI5hqadcPeY/ -Yonik On Sun, Mar 1, 2015 at 4:50 PM, Yonik Seeley ysee...@gmail.com wrote: As many of you know, I've been doing some work in the experimental heliosearch fork of Solr over the past year. I think it's time to bring some more of those changes back. So here's a poll: Which Heliosearch features do you think should be brought back to Apache Solr? http://bit.ly/1E7wi1Q (link to google form) -Yonik
Re: Solr Document expiration with TTL
Hi, As suggested, we could manage to activate the document expiration using TTL by shifting the newly added URP setting in the starting of the add-unknown-fields-to-the-schema URP. That populates the expire_at_dt field and makes document to expire after defined TTL. Many Thanks, Makailol On Fri, Feb 27, 2015 at 7:41 PM, Alexandre Rafalovitch arafa...@gmail.com wrote: Yep, Your default URP chain is probably not being triggered due to the initParams. initParams are new in Solr 5, so this is still rough around the edges advice. But try giving your chain a name and adding explicit update.chain value to the requestHandler section (not initParams) section. Alternatively, since add-unknown-fields is already used, you could move your extra URPs to the start of that instead. In fact, if you are doing both timestamps and dynamically adding fields to the schema, you will need to do that anyway. Regards, Alex. On 27 February 2015 at 08:53, Makailol Charls 4extrama...@gmail.com wrote: initParams path=/update/** lst name=defaults str name=update.chainadd-unknown-fields-to-the-schema/str /lst /initParams Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/
Re: Frequency of Suggestion are varying from original Frequency in index
So, I think it's depend on the field that you are working on ?! -- View this message in context: http://lucene.472066.n3.nabble.com/Frequency-of-Suggestion-are-varying-from-original-Frequency-in-index-tp4190927p4191800.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Document expiration with TTL
Hi, As suggested, we could manage to activate the document expiration using TTL by shifting the newly added URP setting in the starting of the add-unknown-fields-to-the-schema URP. That populates the expire_at_dt field and makes document to expire after defined TTL. Many Thanks, Makailol On Fri, Feb 27, 2015 at 10:23 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : There were no changes made in the solrconfig.xml file except added that : updateRequestProcessorChain default=true/updateRequestProcessorChain : block. ok, first off: if you already *had* another updateRequestProcessorChain that said 'default=true' just adding a new one would be weird and would likely give you errors. you have t oconsider the whole context of the config and the other updateRequestProcessorChains when you make edits like that. : initParams path=/update/** : lst name=defaults : str name=update.chainadd-unknown-fields-to-the-schema/str : /lst : /initParams so that says whe nyou make any requests to a /update handler, it's going to use a default request param of update.chain=add-unknown-fields-to-the-schema. so your updates are not going to the default hanler (which you didn't give a name) they are going though the updateRequestProcessorChain/ with the name=add-unknown-fields-to-the-schema you should probably remove the chain you added, and instead put the new processors you want in the add-unknown-fields-to-the-schema chain. that's the simplest way to get what you want in place. -Hoss http://www.lucidworks.com/
Re: Performance on faceting using docValues
The Solr instance is single-shard. Index size is around 20G and total doc # is about 12 million. Below are the histograms for the three facet fields in my query. Thanks. On Thu, Mar 5, 2015 at 11:57 PM, Toke Eskildsen t...@statsbiblioteket.dk wrote: On Thu, 2015-03-05 at 21:14 +0100, lei wrote: You present a very interesting observation. I have not noticed what you describe, but on the other hand we have not done comparative speed tests. q=*:*fq=country:USfq=category:112 First observation: Your query is '*:*, which is a magic query. Non-DV faceting has optimizations both for this query (although that ought to be disabled due to the fq) and for the inverse case where there are more hits than non-hits. Perhaps you could test with a handful of queries, which has different result sizes? facet=onfacet.sort=indexfacet.mincount=1facet.limit=2000 The combination of index order and a high limit might be an explanation: When resolving the Strings of the facet result, non-DV will perform ordinal-lookup, which is fast when done in monotonic rising order (sort=index) and if the values are close (limit=2000). I do not know if DV benefits the same way. On the other hand, your limit seems to apply only to material, so it could be that the real number of unique values is low and you just set the limit to 2000 to be sure you get everything? facet.field=manufacturerfacet.field=sellerfacet.field=material f.manufacturer.facet.mincount=1f.manufacturer.facet.sort=countf.manufacturer.facet.limit=100 f.seller.facet.mincount=1f.seller.facet.sort=countf.seller.facet.limit=100 f.material.facet.mincount=1sort=score+desc How large is your index in bytes, how many documents does it contain and is it single-shard or cloud? Could you paste the loglines containing UnInverted field, which describes the number of unique values and size of your facet fields? - Toke Eskildsen, State and University Library, Denmark
Re: how to change configurations in solrcloud setup
Please help. With Regards Aman Tandon On Sat, Mar 7, 2015 at 9:58 PM, Aman Tandon amantandon...@gmail.com wrote: Hi, Please tell me what is best way to apply configuration changes in solr cloud and how to do that. Thanks in advance. With Regards Aman Tandon
Re: Frequency of Suggestion are varying from original Frequency in index
I am using field as standardTokenizerFactory with ShingleFilterFactory. Is it doing so? On 3/9/15, ale42 alexandre.faye...@etu.esisar.grenoble-inp.fr wrote: So, I think it's depend on the field that you are working on ?! -- View this message in context: http://lucene.472066.n3.nabble.com/Frequency-of-Suggestion-are-varying-from-original-Frequency-in-index-tp4190927p4191800.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Frequency of Suggestion are varying from original Frequency in index
When you make a query, does it use the same field type as the field that you are using to build suggestions? -- View this message in context: http://lucene.472066.n3.nabble.com/Frequency-of-Suggestion-are-varying-from-original-Frequency-in-index-tp4190927p4191813.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Parsing cluster result's docs
Hi, I have a Solr instance using the clustering component (with the Lingo algorithm) working perfectly. However when I get back the cluster results only the ID's of these come back with it. What is the easiest way to retrieve full documents instead? Should I parse these IDs into a new query to Solr, or is there some configuration I am missing to return full docs instead of IDs? If it matters, I am using Solr 4.10. Clustering results are attached to the regular Solr response (the text of the documents), much like shown in the docs: https://cwiki.apache.org/confluence/display/solr/Result+Clustering, so with the default configuration you should be getting both clusters and document content. If that's not the case, please post your solrconfig.xml and the URL you're using to initiate the search/clustering. Staszek
Re: 4.10.4 - nodes up, shard without leader
The update log replay issue looks like
https://issues.apache.org/jira/browse/SOLR-6583
On 9 Mar 2015 01:41, Mark Miller markrmil...@gmail.com wrote:
Interesting bug.
First there is the already closed transaction log. That by itself deserves
a look. I'm not even positive we should be replaying the log we
reconnecting from ZK disconnect, but even if we do, this should never
happen.
Beyond that there seems to be some race. Because of the log trouble, we try
and cancel the election - but we don't find the ephemeral election node yet
for some reason and so just assume it's fine, no node there to remove
(well, we WARN, because it is a little unexpected). Then that ephemeral
node materializes I guess, and the new leader doesn't register because the
old leader won't give up the thrown. We don't try and force the new leader
because that may just hide bugs and cause data loss, we no leader is
elected.
I'd guess there are two JIRA issues to resolve here.
- Mark
On Sun, Mar 8, 2015 at 8:37 AM Markus Jelsma markus.jel...@openindex.io
wrote:
Hello - i stumbled upon an issue i've never seen earlier, a shard with
all
nodes up and running but no leader. This is on 4.10.4. One of the two
nodes
emits the following error log entry:
2015-03-08 05:25:49,095 WARN [solr.cloud.ElectionContext] - [Thread-136]
-
: cancelElection did not find election node to remove
/overseer_elect/election/93434598784958483-178.21.116.
225:8080_solr-n_000246
2015-03-08 05:25:49,121 WARN [solr.cloud.ElectionContext] - [Thread-136]
-
: cancelElection did not find election node to remove
/collections/oi/leader_elect/shard3/election/93434598784958483-178.21.116.
225:8080_solr_oi_h-n_43
2015-03-08 05:25:49,220 ERROR [solr.update.UpdateLog] - [Thread-136] - :
Error inspecting tlog
tlog{file=/opt/solr/cores/oi_c/data/tlog/tlog.0001394
refcount=2}
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:99)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:679)
at org.apache.solr.update.ChannelFastInputStream.
readWrappedStream(TransactionLog.java:784)
at org.apache.solr.common.util.FastInputStream.refill(
FastInputStream.java:89)
at org.apache.solr.common.util.FastInputStream.read(
FastInputStream.java:125)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.solr.update.TransactionLog.endsWithCommit(
TransactionLog.java:218)
at org.apache.solr.update.UpdateLog.recoverFromLog(
UpdateLog.java:800)
at org.apache.solr.cloud.ZkController.register(
ZkController.java:841)
at org.apache.solr.cloud.ZkController$1.command(
ZkController.java:277)
at org.apache.solr.common.cloud.ConnectionManager$1$1.run(
ConnectionManager.java:166)
2015-03-08 05:25:49,225 ERROR [solr.update.UpdateLog] - [Thread-136] - :
Error inspecting tlog
tlog{file=/opt/solr/cores/oi_c/data/tlog/tlog.0001471
refcount=2}
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:99)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:679)
at org.apache.solr.update.ChannelFastInputStream.
readWrappedStream(TransactionLog.java:784)
at org.apache.solr.common.util.FastInputStream.refill(
FastInputStream.java:89)
at org.apache.solr.common.util.FastInputStream.read(
FastInputStream.java:125)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.solr.update.TransactionLog.endsWithCommit(
TransactionLog.java:218)
at org.apache.solr.update.UpdateLog.recoverFromLog(
UpdateLog.java:800)
at org.apache.solr.cloud.ZkController.register(
ZkController.java:841)
at org.apache.solr.cloud.ZkController$1.command(
ZkController.java:277)
at org.apache.solr.common.cloud.ConnectionManager$1$1.run(
ConnectionManager.java:166)
2015-03-08 12:21:04,438 WARN [solr.cloud.RecoveryStrategy] -
[zkCallback-2-thread-28] - : Stopping recovery for core=oi_h
coreNodeName=
178.21.116.225:8080_solr_oi_h
The other node makes a mess in the logs:
2015-03-08 05:25:46,020 WARN [solr.cloud.RecoveryStrategy] -
[zkCallback-2-thread-20] - : Stopping recovery for core=oi_c
coreNodeName=
194.145.201.190:
8080_solr_oi_c
2015-03-08 05:26:08,670 ERROR [solr.cloud.ShardLeaderElectionContext] -
[zkCallback-2-thread-19] - : There was a problem trying to register as
the
leader:org.
apache.solr.common.SolrException: Could not register as the leader
because creating the ephemeral registration node in ZooKeeper failed
at org.apache.solr.cloud.ShardLeaderElectionContextBase
.runLeaderProcess(ElectionContext.java:146)
at org.apache.solr.cloud.ShardLeaderElectionContext.
runLeaderProcess(ElectionContext.java:317)
at
Combine multiple SOLR Query Results
Hi, I am trying to work on combine multiple SOLR query results into single result. Below is my case. 1. Look up search term against ‘track name’, log results 2. Look up search term against ‘artist name’, log results of tracks by those artists 3. Combine results 4. results by score descending order. Using text_general fieldType for both track name and artist name. copy fields are trackname and artistname Plase suggest me how to write solr Query to combine two solr results into single result. Thanks in advance. AnilJayanti -- View this message in context: http://lucene.472066.n3.nabble.com/Combine-multiple-SOLR-Query-Results-tp4191816.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr TCP layer
Hi Saumitra, I've been thinking of adding http/2 support for inter node communication initially and client server communication next in Solr. There's a patch for SPDY support but now that spdy is deprecated and http/2 is the new standard we need to wait for Jetty 9.3 to release. That will take care of many bottlenecks in solrcloud communication. The current trunk is already using jetty 9.2.x which has support for the draft http/2 spec. A brand new async TCP layer based on netty can be considered but that's a huge amount of work considering our need to still support simple http, SSL etc. Frankly for me that effort is better spent optimizing the routing layer. On 09-Mar-2015 1:37 am, Saumitra Srivastav saumitra.srivast...@gmail.com wrote: Dear Solr Contributors, I want to start working on adding a TCP layer for client to node and inter-node communication. I am not up to date on recent changes happening to Solr. So before I start looking into code, I would like to know if there is already some work done in this direction, which I can reuse. Are there any know challenges/complexities? I would appreciate any help to kick start this effort. Also, what would be the best way to discuss and get feedback on design from contributors? Open a JIRA?? Regards, Saumitra -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-TCP-layer-tp4191715.html Sent from the Solr - User mailing list archive at Nabble.com.
