Re: maxShardsPerNode always over-riden by autoscaling?

2019-01-17 Thread Gus Heck
Sorry I meant still shows in /collections/myCollection/state.json (bleh
don't know why I always seem to say cluster state when I mean collection
sate)

I also can't account for why the ticket I linked says "Thirdly, the
maxShardsPerNode is not stored in collection state and therefore it is only
applicable during the time of collection creation."

Except perhaps that it shows up at the bottom of the state.json file

-Gus

On Thu, Jan 17, 2019 at 7:39 PM Gus Heck  wrote:

> I had a 8.0.0 cluster strangely put 2 shards on the same node and then 3
> on the same node the next collection I created. Both had maxShardsPerNode
> set to 1 (and it still shows in cluster_state.json).
>
> After some research I found https://jira.apache.org/jira/browse/SOLR-11239
>
> The 8x documentation says nothing about this in the vicinity of the
> maxShardsPerNode attribute, and when I checked I found that by default the
> cluster comes with 2 listeners and two triggers (.auto_add_replicas and
> .scheduled_maintenance). (they seem to be hard coded for automatic addition
> in OverSeerTriggerThread:
>
> @Override
> public void run() {
>   int lastZnodeVersion = znodeVersion;
>
>   // we automatically add a trigger for auto add replicas if it does not
> exists already
>   // we also automatically add a scheduled maintenance trigger
>   while (!isClosed)  {
> try {
>   if (Thread.currentThread().isInterrupted()) {
> log.warn("Interrupted");
> break;
>   }
>   AutoScalingConfig autoScalingConfig =
> cloudManager.getDistribStateManager().getAutoScalingConfig();
>   AutoScalingConfig updatedConfig =
> withAutoAddReplicasTrigger(autoScalingConfig);
>   updatedConfig = withScheduledMaintenanceTrigger(updatedConfig);
>
> So I think this means that it's not possible for maxShardsPerNode to ever
> have an effect?
>
> If so, we need to formally remove it and provide a clear example of how to
> achieve the same effect with the autoscaling framework referenced from the
> place where this attribute was documented.
>
> -Gus
>
>

-- 
http://www.the111shift.com


maxShardsPerNode always over-riden by autoscaling?

2019-01-17 Thread Gus Heck
I had a 8.0.0 cluster strangely put 2 shards on the same node and then 3 on
the same node the next collection I created. Both had maxShardsPerNode set
to 1 (and it still shows in cluster_state.json).

After some research I found https://jira.apache.org/jira/browse/SOLR-11239

The 8x documentation says nothing about this in the vicinity of the
maxShardsPerNode attribute, and when I checked I found that by default the
cluster comes with 2 listeners and two triggers (.auto_add_replicas and
.scheduled_maintenance). (they seem to be hard coded for automatic addition
in OverSeerTriggerThread:

@Override
public void run() {
  int lastZnodeVersion = znodeVersion;

  // we automatically add a trigger for auto add replicas if it does not
exists already
  // we also automatically add a scheduled maintenance trigger
  while (!isClosed)  {
try {
  if (Thread.currentThread().isInterrupted()) {
log.warn("Interrupted");
break;
  }
  AutoScalingConfig autoScalingConfig =
cloudManager.getDistribStateManager().getAutoScalingConfig();
  AutoScalingConfig updatedConfig =
withAutoAddReplicasTrigger(autoScalingConfig);
  updatedConfig = withScheduledMaintenanceTrigger(updatedConfig);

So I think this means that it's not possible for maxShardsPerNode to ever
have an effect?

If so, we need to formally remove it and provide a clear example of how to
achieve the same effect with the autoscaling framework referenced from the
place where this attribute was documented.

-Gus