Yes, I'm exploring many options at the moment but this would be an extremely small compute cluster (4 nodes max most likely to start). HA isn't required as it isn't serving up applications or the like. We would want to minimize downtime but if a node goes down for a few hours it isn't a big deal. So I'm just trying to figure everything out on the metal/architecture end of things. Setting up stuff on AWS and the like is fairly easy and straightforward as it is all virtualized, but physically organizing nodes and how to connect everything properly is a bit more challenging. Especially when you're not a networking guy.
________________________________________ From: rasput...@gmail.com <rasput...@gmail.com> on behalf of Dick Davies <d...@hellooperator.net> Sent: June-25-15 1:46 PM To: user@mesos.apache.org Subject: Re: Thoughts and opinions in physically building a cluster That doesn't sound too bad (it's a fairly typical setup e.g. on an Amazon VPC). You probably want to avoid NAT or similar things between master and slaves to avoid a lot of LIBPROCESS_IP tricks so same switch sounds good. Personally I quite like the master/slave distinction. I wouldn't want a runaway set of tasks to bog down the masters and operationally we'd alert if we're starting to lose masters whereas the slaves are 'cattle' and we can just spin up more as they die if need be (it's a little more tricky to scale out masters and zookeepers so they get treated as though they were a bit less expendable). I co-locate the zookeeper ensemble on the masters on smaller clusters to save VM count, but that's more personal taste than anything. On 25 June 2015 at 17:12, Daniel Gaston <daniel.gas...@dal.ca> wrote: > So this may be another relatively noob question, but when designing a mesos > cluster, is it basically as simple as the nodes connected by a switch? Since > any of the nodes can be "master nodes" or acting as both master and slave, I > am guessing there is no need for another head node as you would have with a > traditional cluster design. But would each of the nodes then have to be > connected to the external/institutional network? > > My rough idea was for this small cluster to not be connected to the main > institutional network but for my workstation to be connected to both the > cluster's network as well as to the institutional network > > > ________________________________________ > From: CCAAT <cc...@tampabay.rr.com> > Sent: June-19-15 4:57 PM > To: user@mesos.apache.org > Cc: cc...@tampabay.rr.com > Subject: Re: Thoughts and opinions in physically building a cluster > > On 06/19/2015 01:28 PM, Daniel Gaston wrote: >> >> On 19/06/2015 18:38, Oliver Nicholas wrote: >>> Unless you have some true HA requirements, it seems intuitively >>> wasteful to have 3 masters and 2 slaves (unless the cost of 5 nodes is >>> inconsequential to you and you hate the environment). >> Any particular reason not to have three nodes which are acting both as >> master and slaves? >> >> None at all. I'm not a cluster or networking guru, and have only played with >> mesos in >> cloud-based settings so I wasn't sure how this would work. But it makes >> sense, that way >> the 'standby' masters are still participating in the zookeeper quorum while >> still being >> available to do real work as slave nodes. > > Daniel. There is no such thing as a 'cluster guru'. It's all 'seat of > the pants' flying right now; so you are fine what you are doing and > propose. If codes do not exist to meet your specific needs and goals, > they can (should?) be created. > > > I'm working on an architectural expansion Where nodes (virtual, actual > or bare metal) migrate from master --> entrepreneur --> worker --> slave > --> embedded (bare metal or specially attached hardware. I'm proposing > to do all of this with the "Autonomy_Function" and decisions being made > bottom_up as opposed to the current top_down dichotomy. I'm prolly going > to have to 'fork codes' for a while to get things stable and then > hope they are included; when other minds see the validity of the ideas. > > > Surely one "box" can be set up as both master and slave. Moving slaves > to masters, should be an automatic function and will prolly will be > address in the future codes of mesos. > > > PS: Keep pushing your ideas and do not take no for an answer! > Mesos belongs to everybody..... > > hth, > James >
