Re: distrib=false
On 12/28/2014 8:48 AM, S.L wrote: > I have attached the screen shot of the toplogy , as you can see I have > three nodes and no two replicas of the same shard reside on the same > node, this was made sure so as not affect the availability. > > The query that I use is a general get all query of type *:* to test . > > The behavior I notice is that even though when a particular replica of a > shard is queried using distrib=false , the request goes to the other > replica of the same shard. Attachments almost never make it through the mailing list processing. The screenshot you mentioned did not make it. You'll need to host the image somewhere and provide a URL. The dropbox service is a good way to do this, but it's not the only way. Just make sure you don't remove the image quickly. The message will live on for years in the archive ... it would be nice to have the image live on for years as well, though I know that is often not realistic. I do not know exactly how SolrCloud handles such requests, but it would not surprise me to learn that it forwards the request to another replica of the same shard on another server. An issue has been put forward to change the general load-balancing behavior of SolrCloud. There has been a fair amount of discussion on it: https://issues.apache.org/jira/browse/SOLR-6832 Thanks, Shawn
Re: distrib=false
Erik I have attached the screen shot of the toplogy , as you can see I have three nodes and no two replicas of the same shard reside on the same node, this was made sure so as not affect the availability. The query that I use is a general get all query of type *:* to test . The behavior I notice is that even though when a particular replica of a shard is queried using distrib=false , the request goes to the other replica of the same shard. Thanks. On Sat, Dec 27, 2014 at 2:10 PM, Erick Erickson wrote: > How are you sending the request? AFAIK, setting distrib=false > should should keep the query from being sent to any other node, > although I'm not quite sure what happens when you host multiple > replicas of the _same_ shard on the same node. > > So we need: > 1> your topology, How many nodes and what replicas on each? > 2> the actual query you send. > > Best, > Erick > > On Sat, Dec 27, 2014 at 8:14 AM, S.L wrote: > > Hi All, > > > > I have a question regarding distrib=false on the Solr query , it seems > that > > the distribution is restricted across only the shards when the parameter > > is set to false, meaning if I query a particular node with in a shard > with > > replication factor of more than one , the request could go to another > node > > with in the same shard which is a replica of the node that I made the > > initial request to, is my understanding correct ? > > > > If the answer to my question is yes, then how do we make sure that the > > request goes to only the node I intend to make the request to ? > > > > Thanks. >
Re: distrib=false
How are you sending the request? AFAIK, setting distrib=false should should keep the query from being sent to any other node, although I'm not quite sure what happens when you host multiple replicas of the _same_ shard on the same node. So we need: 1> your topology, How many nodes and what replicas on each? 2> the actual query you send. Best, Erick On Sat, Dec 27, 2014 at 8:14 AM, S.L wrote: > Hi All, > > I have a question regarding distrib=false on the Solr query , it seems that > the distribution is restricted across only the shards when the parameter > is set to false, meaning if I query a particular node with in a shard with > replication factor of more than one , the request could go to another node > with in the same shard which is a replica of the node that I made the > initial request to, is my understanding correct ? > > If the answer to my question is yes, then how do we make sure that the > request goes to only the node I intend to make the request to ? > > Thanks.
distrib=false
Hi All, I have a question regarding distrib=false on the Solr query , it seems that the distribution is restricted across only the shards when the parameter is set to false, meaning if I query a particular node with in a shard with replication factor of more than one , the request could go to another node with in the same shard which is a replica of the node that I made the initial request to, is my understanding correct ? If the answer to my question is yes, then how do we make sure that the request goes to only the node I intend to make the request to ? Thanks.
Re: distrib=false is not honoring
Thanks Jack i am using *q.alt* just for testing purpose only we uses *q=query* in our general production environment case and *mcat.intent* is our request handler to add extra number of rows and all. Here i was doing some mistake to properly explaining the situation, so i am sorry for that. *Requirement:* I want to test my sharded environment that a unique document should present in single shard not in both. *core name*: mcats *core.properties*: name=mcats, so default collection name would be mcats as well. And i was taking my non sharded index(mcats index) and copying it to node1 as well as node 2 and starting the first node as: *java -DzkRun -DnumShards=2 -Dbootstrap_confdir=./solr/mcats/conf -Dcollection.configName=myconf -jar start.jar* And second node as *java -Djetty.port=1983 -DzkHost=localhost:9983 -jar start.jar* So i guess it is taking the whole index as it is because when i run the query. http://localhost:8983/solr/mcats/select?q.alt=*:*<http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:8983/solr/mcats> it was giving me the sum of documents in both the shards which is 2*no of docs in mcats index. So the same document is present in both the shard at node1:8983 and node2:1983 To figure this out i indexed it with the another different doc now when i queried with http://localhost:8983/solr/mcats/select?q.alt=id:17406780&distrib=false<http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:8983/solr/mcats> the document is present. But in another query http://localhost:8983/solr/mcats/select?q.alt=id:17406780&distrib=flase<http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:8983/solr/mcats> it is not found. which fulfilled my test case. So i thought i have to do the full indexing of my core mcats to validate my test case for each ids. Please correct me if i am wrong. With Regards Aman Tandon On Wed, May 14, 2014 at 5:52 PM, Jack Krupansky wrote: > The q.alt param specifies only the parameter to use if the q parameter is > missing. Could you verify whether that is really the case? Typically > solrconfig gives a default of "*:*" for the q parameter. Specifying a query > via the q.alt parameter seems like a strange approach - what is your > rationale? > > What is this odd "mcat.intent" query response writer type that you are > specifying with the qt parameter? > > -- Jack Krupansky > > -Original Message- From: Aman Tandon > Sent: Wednesday, May 14, 2014 1:35 AM > To: solr-user@lucene.apache.org > Subject: distrib=false is not honoring > > > I am trying to use the solr cloud for solr 4.2.0 and solr 4.7.1. > Here mcats is our collection name. > > *With solr 4.2* > > shard1: localhost:8019 > shard1: localhost:6019 > > *With solr 4.7.1* > > shard1: localhost:8983 > shard1: localhost:1983 > > With both the server i make the copy of example directory as mentioned in > wiki, and queried over the both nodes > > *query 1*: > http://localhost:8983/solr/mcats/select?q.alt=id:69763&; > distrib=false&qt=mcat.intent > *query 2*: > > http://localhost:1983/solr/mcats/select?q.alt=id:69763&; > distrib=false&qt=mcat.intent > > total number of docs count become half but i search for specific id then > the result is same, if distrib=false this enables to search on particular > node then one of these node should not return the return the result. If i > am incorrect please help me out to test that one record is present in one > shard only. > > With Regards > Aman Tandon >
Re: distrib=false is not honoring
> The q.alt param specifies only the parameter to use if the q parameter is > missing. Could you verify whether that is really the case? Typically > solrconfig gives a default of "*:*" for the q parameter. Specifying a > query > via the q.alt parameter seems like a strange approach - what is your > rationale? As the author of a book about Solr, I'm sure you already know these things. For everyone else: The qt parameter only gets used if the handleSelect value on the request dispatcher configuration is true. When that is set, it chooses a request handler by name. This configuration value defaults to false since version 3.6, so that you can't use the /select handler to do other things, like /update. The q.alt parameter is applicable only to the dismax and edismax query parsers, and specifies the query to send to the standard query parser if the q parameter is blank or missing. It's most often defined as *:* to get all docs. The *:* value doesn't have any special meaning to the dismax parser. Thanks, Shawn
Re: distrib=false is not honoring
Hi,
There is one more problem today, i indexed the mcat core and again copied
the same and then starting the shard
(as decribed in above thread)
*And i was taking my non sharded index(mcats index) and copying it to node1
as well as node 2 and starting the first node as: *
I noticed that there is difference in the total count of docs, please see
these logs
query:
A) *localhost:1983*
*871949 [qtp27058272-19] INFO org.apache.solr.core.SolrCore – [mcats]
webapp=/solr path=/select
params={mm=%0a%09%0a2<-1+4<70%25%0a+&facet=true&tie=0.01&qf=%0a%09namex%0a+&distrib=false&wt=javabin&version=2&defType=edismax&rows=10&pf=%0a%09%0a+&NOW=1400148489992&shard.url=http://192.168.6.217:1983/solr/mcats/&fl=id,score&start=0&q=*:*&qs=50&isShard=true&fsv=true&ps=3
<http://192.168.6.217:1983/solr/mcats/&fl=id,score&start=0&q=*:*&qs=50&isShard=true&fsv=true&ps=3>}
hits=113573 status=0 QTime=1*
B) *localhost:8983* (running the embedded zookeper server)
*878735 [qtp27058272-15] INFO org.apache.solr.core.SolrCore – [mcats]
webapp=/solr path=/select
params={mm=%0a%09%0a2<-1+4<70%25%0a+&facet=true&tie=0.01&qf=%0a%09namex%0a+&distrib=false&wt=javabin&version=2&defType=edismax&rows=10&pf=%0a%09%0a+&NOW=1400148489992&shard.url=http://192.168.6.217:8983/solr/mcats/&fl=id,score&start=0&q=*:*&qs=50&isShard=true&fsv=true&ps=3
<http://192.168.6.217:8983/solr/mcats/&fl=id,score&start=0&q=*:*&qs=50&isShard=true&fsv=true&ps=3>}
hits=113573 status=0 QTime=1 878746 [qtp27058272-15] INFO
org.apache.solr.core.SolrCore – [mcats] webapp=/solr path=/select
params={mm=%0a%09%0a2<-1+4<70%25%0a+&facet=false&tie=0.01&ids=42663,26311,40545,4571,19114,26010,2716,38320,25724,29459&qf=%0a%09namex%0a+&distrib=false&wt=javabin&version=2&defType=edismax&pf=%0a%09%0a+&NOW=1400148489992&shard.url=http://192.168.6.217:8983/solr/mcats/&fl=%0a%0a%09*,+score%0a+&q=*:*&qs=50&isShard=true&ps=3
<http://192.168.6.217:8983/solr/mcats/&fl=%0a%0a%09*,+score%0a+&q=*:*&qs=50&isShard=true&ps=3>}
status=0 QTime=4 878750 [qtp27058272-13] INFO
org.apache.solr.core.SolrCore – [mcats] webapp=/solr path=/select
params={q=*:*&defType=edismax} hits=227136 status=0 QTime=19*
sum of docs should be: *113573 + *
*113573 = 227146 *
But it gives me back the total sum of 227136 as mentioned in logs. Can
anybody help that what's going on here
*.*
With Regards
Aman Tandon
On Thu, May 15, 2014 at 1:36 PM, Aman Tandon wrote:
> Thanks Jack i am using *q.alt* just for testing purpose only we uses
> *q=query* in our general production environment case and *mcat.intent* is
> our request handler to add extra number of rows and all.
>
> Here i was doing some mistake to properly explaining the situation, so i
> am sorry for that.
>
> *Requirement:* I want to test my sharded environment that a unique
> document should present in single shard not in both.
>
> *core name*: mcats
> *core.properties*: name=mcats, so default collection name would be mcats
> as well.
>
> And i was taking my non sharded index(mcats index) and copying it to node1
> as well as node 2 and starting the first node as:
>
> *java -DzkRun -DnumShards=2 -Dbootstrap_confdir=./solr/mcats/conf
> -Dcollection.configName=myconf -jar start.jar*
>
> And second node as
> *java -Djetty.port=1983 -DzkHost=localhost:9983 -jar start.jar*
>
> So i guess it is taking the whole index as it is because when i run the
> query.
>
> http://localhost:8983/solr/mcats/select?q.alt=*:*<http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:8983/solr/mcats>
> it was giving me the sum of documents in both the shards which is 2*no of
> docs in mcats index.
>
> So the same document is present in both the shard at node1:8983 and
> node2:1983
>
> To figure this out i indexed it with the another different doc now when i
> queried with
> http://localhost:8983/solr/mcats/select?q.alt=id:17406780&distrib=false<http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:8983/solr/mcats>
> the document is present.
>
> But in another query
> http://localhost:8983/solr/mcats/select?q.alt=id:17406780&distrib=flase<http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:8983/solr/mcats>
> it is not found.
>
> which fulfilled my test case.
>
> So i thought i have to do the full indexing of my core mcats to validate
> my test case for each ids. Please correct me if i am wrong.
>
>
> With Regards
> Aman Tandon
Re: distrib=false is not honoring
I also tried it to query specific shard to make sure that the unique record is present in different shards but still no success. http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:8983/solr/mcats http://localhost:8983/solr/mcats/select?q.alt=id:174060&shards=localhost:1983/solr/mcats My solr cloud setup configs are java -DzkRun -DnumShards=2 -Dbootstrap_confdir=./solr/mcats/conf -Dcollection.configName=myconf -jar start.jar java -Djetty.port=1983 -DzkHost=localhost:9983 -jar start.jar Please help. With Regards Aman Tandon On Wed, May 14, 2014 at 11:05 AM, Aman Tandon wrote: > I am trying to use the solr cloud for solr 4.2.0 and solr 4.7.1. > Here mcats is our collection name. > > *With solr 4.2* > shard1: localhost:8019 > shard1: localhost:6019 > > *With solr 4.7.1* > shard1: localhost:8983 > shard1: localhost:1983 > > With both the server i make the copy of example directory as mentioned in > wiki, and queried over the both nodes > > *query 1*: > http://localhost:8983/solr/mcats/select?q.alt=id:69763&distrib=false&qt=mcat.intent > *query 2*: > http://localhost:1983/solr/mcats/select?q.alt=id:69763&distrib=false&qt=mcat.intent > > total number of docs count become half but i search for specific id then > the result is same, if distrib=false this enables to search on particular > node then one of these node should not return the return the result. If i > am incorrect please help me out to test that one record is present in one > shard only. > > With Regards > Aman Tandon >
distrib=false is not honoring
I am trying to use the solr cloud for solr 4.2.0 and solr 4.7.1. Here mcats is our collection name. *With solr 4.2* shard1: localhost:8019 shard1: localhost:6019 *With solr 4.7.1* shard1: localhost:8983 shard1: localhost:1983 With both the server i make the copy of example directory as mentioned in wiki, and queried over the both nodes *query 1*: http://localhost:8983/solr/mcats/select?q.alt=id:69763&distrib=false&qt=mcat.intent *query 2*: http://localhost:1983/solr/mcats/select?q.alt=id:69763&distrib=false&qt=mcat.intent total number of docs count become half but i search for specific id then the result is same, if distrib=false this enables to search on particular node then one of these node should not return the return the result. If i am incorrect please help me out to test that one record is present in one shard only. With Regards Aman Tandon
Re: distrib=false is not honoring
The q.alt param specifies only the parameter to use if the q parameter is missing. Could you verify whether that is really the case? Typically solrconfig gives a default of "*:*" for the q parameter. Specifying a query via the q.alt parameter seems like a strange approach - what is your rationale? What is this odd "mcat.intent" query response writer type that you are specifying with the qt parameter? -- Jack Krupansky -Original Message- From: Aman Tandon Sent: Wednesday, May 14, 2014 1:35 AM To: solr-user@lucene.apache.org Subject: distrib=false is not honoring I am trying to use the solr cloud for solr 4.2.0 and solr 4.7.1. Here mcats is our collection name. *With solr 4.2* shard1: localhost:8019 shard1: localhost:6019 *With solr 4.7.1* shard1: localhost:8983 shard1: localhost:1983 With both the server i make the copy of example directory as mentioned in wiki, and queried over the both nodes *query 1*: http://localhost:8983/solr/mcats/select?q.alt=id:69763&distrib=false&qt=mcat.intent *query 2*: http://localhost:1983/solr/mcats/select?q.alt=id:69763&distrib=false&qt=mcat.intent total number of docs count become half but i search for specific id then the result is same, if distrib=false this enables to search on particular node then one of these node should not return the return the result. If i am incorrect please help me out to test that one record is present in one shard only. With Regards Aman Tandon
Re: Question on distrib=false
I think you misunderstand the concept of shards. If slave1 and slave2 are replicas of the master then neither "shards" param nor "distrib" parameter is required. On Fri, May 31, 2013 at 6:12 PM, sathish_ix wrote: > Hi, > > We have following setup , > > > Maseter Slave 1 > |_ Slave 2 > > we indexed few records in slave1 and few records in slave2, while querying > we need data only from slave 1. > So used below query > > http://xxx.com/solr/select?q=*:*&shards=slave1,slave2&distrib=false > > We are getting results from both slaves... Will this work in this setup ? > > Thanks, > Sathish > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Question-on-distrib-false-tp4067305.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Regards, Shalin Shekhar Mangar.
Question on distrib=false
Hi, We have following setup , Maseter Slave 1 |_ Slave 2 we indexed few records in slave1 and few records in slave2, while querying we need data only from slave 1. So used below query http://xxx.com/solr/select?q=*:*&shards=slave1,slave2&distrib=false We are getting results from both slaves... Will this work in this setup ? Thanks, Sathish -- View this message in context: http://lucene.472066.n3.nabble.com/Question-on-distrib-false-tp4067305.html Sent from the Solr - User mailing list archive at Nabble.com.
