[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979812#comment-16979812
]
David Smiley commented on SOLR-12217:
-
I understand you Tomas (and Houston).
> This patch doesn’t solve the case where a random node that doesn’t have the
>collection receives the request
That right there is my point. It's solveable. The Solr side needs to know
wether or not to behave like a SolrJ client itself (thus act like a
load-balancer) or to handle the request locally always. If CloudSolrClient
sent the request then the Solr side can ignore shards.preference. Any way,
again, I understand such a thing would best be its own issue.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979783#comment-16979783
]
Tomas Eduardo Fernandez Lobbe commented on SOLR-12217:
--
In SolrCloud, today (even before the {{shards.prefernce}} param was introduced)
if a node receives the request there are two options. If it doesn’t have a
replica of the target collection it’ll forward the request to a node that does.
If it has one of the replicas of the collection it will *handle* the query.
That query handling is different depending on the number of shards that the
collection has, if it has just one shard, then it will resolve the query
locally, if it has multiple shards, then it will act as coordinator, and send
shard requests to each shard. SolrJ attempts to always hit a node that does
host a replica of the collection to prevent an unnecessary hop.
The {{shards.preference}} as it is today in master will make the coordinator
choose the replicas based on some policy, however. The problem is that if the
collection has a single shard, then it won’t choose any replicas, it’ll just
process the query. This patch lets CloudSolrClient apply the same preference
logic on the client side, so that the replica that receives the request is not
selected randomly and instead is selected by the preference.
This patch doesn’t solve the case where a random node that doesn’t have the
collection receives the request (I.e if you are sending all queries to the same
Solr node that doesn’t have any collections). But does solve the SolrJ case.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979774#comment-16979774
]
Erick Erickson commented on SOLR-12217:
---
The question I have sparked by David's comment is in the TLOG/PULL case, and
assuming the prefer replica type isn't set, it's at least theoretically
possible for someone to send all requests to a single http end-point and have
_only_ that node handle all requests? Are they expected to put a load balancer
in front of all of them or exclusively use SolrJ?
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979768#comment-16979768
]
Tomas Eduardo Fernandez Lobbe commented on SOLR-12217:
--
Sorry, I don’t understand you question David.
bq. Isn't that a problem in its own right?
What do you mean? The fact that Solr handles the single shard request on the
landing node?
bq. If the current issue only proposes to solve it for CloudSolrClient but not
anything else
That’s because the multi-shard case already works, this is the missing part.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979409#comment-16979409
]
David Smiley commented on SOLR-12217:
-
{quote}So for single-sharded collections, you can't really do this over http.
Because whichever core receives the request will handle it.
{quote}
Isn't _that_ a problem in its own right? If the current issue only proposes to
solve it for CloudSolrClient but not anything else then the title should scope
it. Ideally we would solve everywhere but I understand you might want to
divide up the work.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969555#comment-16969555
]
Houston Putman commented on SOLR-12217:
---
So for single-sharded collections, you can't really do this over http. Because
whichever core receives the request will handle it. Therefore the SolrJ options
are needed to send the request to the correct core in the first place. All you
need to do is use the {{CloudSolrClient}} or {{CloudHttp2SolrClient}} and use
the [shards.preference
parameter|https://lucene.apache.org/solr/guide/7_4/distributed-requests.html#shards-preference-parameter]
in your request. The client will take your request params and route the
request accordingly.
For Streaming Expressions, you can add the {{shards.preference}} parameter in
the URL params with your streaming expression, or as an argument to the
expression itself, e.g. {{search(collection, field:a, ,
shards.preference=...)}}.
The PR has documentation added to the Ref Guide for both SolrJ and Streaming
Expressions.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969548#comment-16969548
]
Noble Paul commented on SOLR-12217:
---
Can you please give me examples of how a user would use it? Over http as well
as SolrJ?
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963183#comment-16963183
]
Houston Putman commented on SOLR-12217:
---
Did a first-pass for this, pretty straightforward. Still need to add some
routing-specific tests, and documentation to the ref-guide.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959157#comment-16959157
]
Tomas Eduardo Fernandez Lobbe commented on SOLR-12217:
--
It's about the no-leader change, but it included a refactor, which I thought
made the transition easier. Maybe not worth merging the efforts, lets just get
the move to SolrJ done if you are planing to work on that.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958462#comment-16958462
]
Houston Putman commented on SOLR-12217:
---
Thanks for the update Thomas. Does that patch include moving the logic to SolrJ
or is it just adding the no-leaders strategy? If it's the latter I can start
the work to migrate to SolrJ.
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958246#comment-16958246
]
Tomas Eduardo Fernandez Lobbe commented on SOLR-12217:
--
I don't think too much? As [~houstonputman] suggested in SOLR-13865, I think
the way is to move the logic to SolrJ. One of my coworkers did some work to
enable "selecting replica by leadership status (i.e. "don't query leaders"
strategy)" that I remember thinking it could be easy to move to SolrJ, and then
have support for the single shard case. I'll try go get that patch up to date
and post it here
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12217) Add support for shards.preference in single shard cases
[
https://issues.apache.org/jira/browse/SOLR-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943767#comment-16943767
]
Erick Erickson commented on SOLR-12217:
---
[~tflobbe] Do you have any wild guesses how much effort this would take?
Thanks,
Erick
> Add support for shards.preference in single shard cases
> ---
>
> Key: SOLR-12217
> URL: https://issues.apache.org/jira/browse/SOLR-12217
> Project: Solr
> Issue Type: New Feature
>Reporter: Tomas Eduardo Fernandez Lobbe
>Priority: Major
>
> SOLR-11982 Added support for {{shards.preference}}, a way to define the
> sorting of replicas within a shard by preference (replica types/location).
> This only works on multi-shard cases. We should add support for the case of
> single shards when using CloudSolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
