[
https://issues.apache.org/jira/browse/SOLR-16348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17609560#comment-17609560
]
Bruno Roustant edited comment on SOLR-16348 at 9/26/22 3:58 PM:
----------------------------------------------------------------
The first implementation with a skew factor seems good to decide if we need
more complexity. We would evaluate it with some synthetic or real cases.
With this URP approach, is it possible to trigger multiple times the same split
of the same shard?
Actually this is probably not an issue since there is an existing split lock
(SplitShardCmd.lockForSplit()/unlockForSplit()) that prevents a split on the
same shard.
Should we limit the number of concurrent splits per host on the SplitShardCmd?
For example could we have the static semaphore mentioned above inside the
lockForSplit()?
The skew scale probably depends on the load of the indexing flow, and maybe on
the number of shards? It might be hard to decide. But it's too early to guess,
we need some evaluation.
was (Author: broustant):
The first implementation with a skew factor seems good to decide if we need
more complexity. We would evaluate it with some synthetic or real cases.
With this URP approach, is it possible to trigger multiple times the same split
of the same shard?
Actually this is probably not an issue since there is an existing split lock
(SplitShardCmd.lockForSplit()/unlockForSplit()) that prevents a split on the
same shard.
Should we limit the number of concurrent splits per host on the SplitShardCmd
The skew scale probably depends on the load of the indexing flow, and maybe on
the number of shards? It might be hard to decide. But it's too early to guess,
we need some evaluation.
> New SplitShard UpdateRequestProcessor
> -------------------------------------
>
> Key: SOLR-16348
> URL: https://issues.apache.org/jira/browse/SOLR-16348
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
> Reporter: David Smiley
> Priority: Major
>
> The
> [SplitShard|https://solr.apache.org/guide/solr/latest/deployment-guide/shard-management.html#splitshard]
> command is used to split a shard into smaller shards to get better query
> scalability, especially across multiple machines. The most practical way to
> use it is to split shards larger than a configured size. Of course shards
> don't just grow by themselves; they grow when data is added. Here I propose
> a new UpdateRequestProcessor that splits based on the shard size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
