Hi Bart, Let me first distinguish some words: - "Autofailover" will redistribute the workload of a failing instance to other previously existing instances ( https://en.wikipedia.org/wiki/Failover ) This can be done within an existing ArangoDB cluster and without the help of an orchestration framework. - "replacement of failing nodes by fresh instances" is more than Autofailover, and requires the control of an outside orcherstration framework. - "Autoscaling" means that the whole system can deploy more or less instances during runtime to scale according to the loadpattern.
We have chosen to start out to implement cluster management with the most complete open source framework on the market - mesosphere. The orchestration of the ArangoDB instances is controlled via the ArangoDB Mesos Framework (available at https://github.com/arangodb/arangodb-mesos-framework) from the DCOS. In the current implementation state the cluster orchestration framework controls the auto failover management and the replacement of broken nodes with good ones. The already available ArangoDB 2.8 comes with asynchronous replication, the soon to be released 3.0 will also bring synchronous replication. With ArangoDB 3.1 we plan to support failover for asynchronous replication without the aid of a cloud orchestration framework. Autoscaling and replacing of failed nodes by new nodes is and will remain under the control of a cloud orchestration framework. Once the work on the Mesosphere framework is complete, we plan to replicate these efforts to the other cloud orchestration frameworks. The architecture of the middleware was done in a modular way with these future enhancements in mind, 3.0 goes a big step into the direction of portabilizing the framework. Most probably we're going to start out next with porting the framework to Kubernetes. Others are going to follow one by one. As usual, we're always open for contributions from the community. In summary, the coupling with mesosphere is different in subsequent versions of ArangoDB and will become less tight with 3.0 However, a certain amount of work is always needed to do the integration with different Cloud orchestrating frameworks. In 3.0 automatic failover and rebalancing in the above sense is done completely inside of the ArangoDB cluster - as long as you use synchronous replication. This means it should be relatively straight forward to set up 3.0 with another Cloud orchestrating framework. Cheers, Willi On Tuesday, May 24, 2016 at 2:58:50 PM UTC+2, Bart DS wrote: > > Hi, > > Is it possible with ArangoDB 3 to set up such a cluster environment (i.e. > with automatic failover and rebalancing) without having to rely on mesos? > In other words, how tight is the coupling of ArangoDB 3 and mesos? > > The reason I'm asking this is that we are currently considering other > orchestration frameworks such as swarm or k8s. > > Bart > > On Friday, April 1, 2016 at 12:40:51 PM UTC+2, Claudius Weinberger wrote: >> >> >> >> Sébastien Médan schrieb: >> >> I have a few questions regarding the choice of ArangoDB to offer >> persistence in our cloud components. >> >> How well does ADB scale when adding nodes to cope with increasing loads: >> Can the `numberOfShards` attribute be altered on existing collections? >> >> Altering the numOfShards is not possible at the moment. It will come >> later this year. Nevertheless, that will be an expensive operation. But you >> can choose a numOfShards value easily by factor 100 of your current number >> of servers so that you can grow your cluster up to this size. You need at >> least one shard per machine. To take a numOfShards by factor 100 will not >> have a notable performance implication. >> >> Can we rebalance (even manually) after adding a node? >> >> At the moment this a manual process with, unfortunately, a downtime of >> your cluster. 3.0 will do this automatically . With 2.8, you have to >> dump your collection, add a node and restore the collection. Please keep in >> mind that the numOfShards could not be changed so that you have good value >> for that from beginning. >> >> Also, these roadmap features are important to us, and I would like to >> know if you have an estimate on when these features are expected to become >> available? >> >> - Transaction isolation within a cluster >> >> Yes, it will come later this year. Hopefully in the first half of the >> year. >> >> - Automatic failover >> >> This will come with 3.0. ETA is May. >> >> - Master / master replication >> >> This will come with 3.0. ETA is May. >> >> - Rebalance >> >> This will come with 3.0. ETA is May. >> >> From the FAQ: >> > Actually, ArangoDB doesn’t compete with massively distributed systems >> like Cassandra with thousands of nodes and many terabytes of data. >> >> While we're not aiming at such a massively distributed system, we do need >> a scalable db infrastructure. >> Do you consider your clustering to be production ready? >> >> Short answer: yes >> We have already happy customers who use an Arango Cluster in production. >> As you see above, some features for the cluster are missing. 3.0 will bring >> a lot of improvements and also the following releases. To give you a more >> detailed answer it would be great to hear more about your use-case. Please >> contact me at [email protected]. >> >> -- You received this message because you are subscribed to the Google Groups "ArangoDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
