> > Does job map to a Marathon application?
I believe so, yes. An Aurora job is multiple replicas of a [group of] processes, usually (but not necessarily) homogeneous. If similar is there a known limitation to how many jobs one can have? This will depend on the hardware used for the scheduler machines, and the total number of instances. Hundreds of jobs totaling hundreds of thousands of tasks has been proven stable. What we discovered for Marathon more application+bigger env_vars = bigger > zknode size and we come dangerously close to hitting the 1 MB default of > the zk zknode size. I'm wondering if this type of a thing has potentially > been fixed already in Aurora. A key difference in implementation is that Aurora does not use ZK for storage, but instead uses the replicated log implementation provided by mesos. Aurora has no ceiling on transaction data size, however the storage must fit in JVM memory (which would be the same for equivalent data stored in ZooKeeper). I'm not sure what data marathon stores in a ZK node that you're hitting 1 MB. So long as we're not talking about ~1 MB of env var data for a single process, i doubt you would encounter a similar ceiling in Aurora. Happy to talk through this more, so feel free to ask more questions! On Fri, Mar 18, 2016 at 7:12 AM, Christopher M Luciano <cmluci...@us.ibm.com > wrote: > Hi all. It seems that we may be outgrowing Marathon. We have a problem > with the amount of application that we are using, causing us to not exactly > be "compliant" with Marathon goals. It seems that the unit for Aurora is a > job+instance of that job. Does job map to a Marathon application? If > similar is there a known limitation to how many jobs one can have? > > What we discovered for Marathon more application+bigger env_vars = bigger > zknode size and we come dangerously close to hitting the 1 MB default of > the zk zknode size. I'm wondering if this type of a thing has potentially > been fixed already in Aurora. > > > > > > > Christopher M Luciano > > Staff Software Engineer, Platform Services > > IBM Watson Core Technology > > > > >
