Thanks Galo, this is useful information. When you say, “large” working sets, how large is large — just looking for order of magnitude (Gig, Tera, Peta….)?
Also, are you aware if any Mesos frameworks that offer similar capabilities as K8s stateful sets? Thanks again, -John > On Jun 23, 2017, at 6:37 PM, Galo Gimenez <[email protected]> wrote: > > One issue you will find on Jackrabbit is indexing, local storage is ephemeral > so new nodes need to re index and on large working sets this can take hours. > > Kubernetes introduced stateful sets, this allows you to have very stable > naming and storage inside the cluster, and a consistent ordering when nodes > are started > -https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ > <https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/>. > > — Galo > >> On Jun 23, 2017, at 11:03 PM, John Chilton <[email protected]> wrote: >> >> We are running in an orchestration environment — either >> Mesos/Chronos/Marathon or Kubernetes. >> >> Each docker container needs to join the Jackrabbit cluster for the lifetime >> of that container and then leave the Jackrabbit cluster when its work is >> complete. >> When each container joins the Jackrabbit cluster it is assigned a unique >> cluster node id (repository.xml). We also have no upper bound on the number >> of our containers that may join the cluster at any given time. >> >> Will this “dynamic” clustering work or will we encounter issues? Is this >> ill-advised? or are there things we need to do beyond uniquely identify each >> cluster node. >> I Am trying to get ahead of issues that may arise when exercising this. Any >> thoughts at all would be appreciated. >> >> Thanks, >> >> -John >> >
