Hi Itamar, there's definitely certain limit of machines which can Mesos master handle. This limit is between 10 000 - 20 000 (that's number reported by Twitter). This bottleneck is caused by event loop which handles communication at master.
With hundreds of machines you should be fine. Only in case that your framework scheduler would demand too many resources for computing allocations you might encounter some problems. How does the strength of the master & scheduler machines affect the overall > cluster performance? I would say that the network is usually the main bottleneck. Adding extra RAM won't improve mesos-master performance. Of course if there's high CPU load on master you might observe performance regression. Also this depends on granularity of your tasks, if you have few long running tasks or many short tasks (which runs just hundreds of ms). Tomas On 6 January 2015 at 10:12, Itamar Ostricher <ita...@yowza3d.com> wrote: > Are there recommendations regarding master / scheduler machines resources > as function of cluster size? > > Say I have a cluster with hundreds of slave machines and thousands of > CPUs, with a single framework that will schedule millions of tasks. > How does the strength of the master & scheduler machines affect the > overall cluster performance? > > Thanks, > - Itamar. >
