[
https://issues.apache.org/jira/browse/SOLR-18466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18148900#comment-18148900
]
ASF subversion and git services commented on SOLR-18466:
--------------------------------------------------------
Commit e8c74065586850774b12280b80f22b9849301062 in solr's branch
refs/heads/main from Mathew
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=e8c74065586 ]
SOLR-18466: Support _route_ in the streaming expression search function (#4921)
> Support _route_ in the streaming expression search function to prune shard
> fan-out
> ----------------------------------------------------------------------------------
>
> Key: SOLR-18466
> URL: https://issues.apache.org/jira/browse/SOLR-18466
> Project: Solr
> Issue Type: Improvement
> Components: SolrJ, streaming expressions
> Reporter: Mathew
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The {{search()}} function in streaming expressions currently ignores
> \_route\_ for shard selection and queries every active slice of the target
> collection. Users with known routing keys therefore incur unnecessary request
> and response overhead, which can be significant on large clusters. This
> change adds support for \_route\_ param in the streaming expression search
> function, reusing {{docCol.getRouter().getSearchSlices()}} which is used for
> shard selection in normal distributed search.
> *Changes:*
> Two files in {{{}solr/solrj-streaming{}}}:
> *1. the CloudSolrStream class:*
> the existing method:
> {code:java}
> List<Slice> getSlices(
> String collectionName,
> CloudSolrClient cloudSolrClient,
> boolean checkAlias)
> {code}
> is kept for existing calls from {{{}TopicStream{}}}, {{{}TextLogitStream{}}},
> {{{}FeaturesSelectionStream{}}}, and {{StatementImpl}} (all out of scope for
> this change); these are unaffected and continue to receive all active slices
> A new overloaded helper method is added:
> {code:java}
> List<Slice> getSlices(
> String collectionName,
> CloudSolrClient cloudSolrClient,
> boolean checkAlias,
> SolrParams solrParams)
> {code}
> When {{solrParams}} contains a nonempty \_route\_, each collection's slices
> are resolved via {{docCol.getRouter().getSearchSlices(routeKeys, params,
> docCol)}} and when {{_route_}} is absent (or empty), all active slices are
> returned exactly as before.
> *2. the TupleStream class:*
> {{CloudSolrStream.getSlices(collection, cloudSolrClient, true, solrParams)}}
> is called after the request params are assembled and passed to the route
> aware {{{}getSlices{}}}.
> *Example usages:*
> {noformat}
> expr=search(collection1,
> zkHost="localhost:9983",
> qt="/export",
> q="*:*",
> fl="id,a_s,a_i,a_f",
> sort="a_f asc, a_i asc", _route_="routeKeyA!")
> {noformat}
> {noformat}
> expr=search(collection1,
> zkHost="localhost:9983",
> qt="/export",
> q="*:*",
> fl="id,a_s,a_i,a_f",
> sort="a_f asc, a_i asc", _route_="routeKeyA!,routeKeyB!,routeKeyC!")
> {noformat}
> *One note on backward compatibility:*
> Previously, \_route\_ was ignored during shard selection for streaming
> expression {{{}search(){}}}. With this change, nonempty route values are
> passed to {{{}docCol.getRouter().getSearchSlices(routeKeys, params,
> docCol){}}}. For collections using the implicit router, an invalid \_route\_
> _value now causes the request to fail. Previously, the same request searched
> all active shards despite the invalid_ \_route\_. This should be documented
> in the release/ upgrade notes.
> Tests cover single and multiple route keys, composite id routing, missing and
> empty route values, and invalid implicit router shard names.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]