[GitHub] [solr-operator] joshsouza commented on issue #471: How to prevent node rotation behavior from causing cluster instability

2022-09-16 Thread GitBox
joshsouza commented on issue #471: URL: https://github.com/apache/solr-operator/issues/471#issuecomment-1249574050 Thanks for all the thoughtful discussion. It hadn't even occurred to me to do a per-pod pdb, but that makes a ton of sense given the context, and I would say that's probabl

[GitHub] [solr-operator] joshsouza commented on issue #471: How to prevent node rotation behavior from causing cluster instability

2022-09-15 Thread GitBox
joshsouza commented on issue #471: URL: https://github.com/apache/solr-operator/issues/471#issuecomment-1248260669 It also occurred to me that if each SolrCloud had a PDB with a `maxUnavailable` of `0` _at all times_, the Solr Operator could monitor the cluster for node rotation behavior (n

[GitHub] [solr-operator] joshsouza commented on issue #471: How to prevent node rotation behavior from causing cluster instability

2022-09-14 Thread GitBox
joshsouza commented on issue #471: URL: https://github.com/apache/solr-operator/issues/471#issuecomment-1247042942 Just had a thought on this after perusing the docs further to see if there's anything I could find to support our end goals within current constraints: https://kubernetes.io/do

[GitHub] [solr-operator] joshsouza commented on issue #471: How to prevent node rotation behavior from causing cluster instability

2022-09-14 Thread GitBox
joshsouza commented on issue #471: URL: https://github.com/apache/solr-operator/issues/471#issuecomment-1247027292 (Read up more carefully on the docs, we can't use the sidecar idea, because it would indicate the whole pod isn't ready, and drop it from the service) -- This is an automated

[GitHub] [solr-operator] joshsouza commented on issue #471: How to prevent node rotation behavior from causing cluster instability

2022-09-14 Thread GitBox
joshsouza commented on issue #471: URL: https://github.com/apache/solr-operator/issues/471#issuecomment-1247002626 Ideas our team has been tossing around in discussions: `startupProbe` may also reduce risk (though still allows for some edge cases). If a newly starting pod had a startu

[GitHub] [solr-operator] joshsouza commented on issue #471: How to prevent node rotation behavior from causing cluster instability

2022-09-14 Thread GitBox
joshsouza commented on issue #471: URL: https://github.com/apache/solr-operator/issues/471#issuecomment-1246994429 I 100% support adding a cluster-level PDB here, as that's definitely a first step towards success. My concern is that the PDB will ensure we don't take a pod down if one is