Yup, if that doesn't work for any reason, you can increase memory in 1G increments. Ensure that you have some head-room on top of your Xmx to account for native memory
On Fri, May 24, 2019 at 10:34 AM Malcolm McFarland <mmcfarl...@cavulus.com> wrote: > Thanks Jagadish, just wanted to verify that this wasn't an obvious thing I > was missing. I'm also setting the AM's heap size explicitly with the > yarn.am.opts configuration parameter; in this case, I'm allowing 512MB for > the JVM on top of my heap for both the AM and the container (ie, > yarn.am.container.memory.mb=1536, yarn.am.opts=-Xmx1024m); does that sound > reasonable? > > Cheers, > Malcolm McFarland > Cavulus > > > This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any > unauthorized or improper disclosure, copying, distribution, or use of the > contents of this message is prohibited. The information contained in this > message is intended only for the personal and confidential use of the > recipient(s) named above. If you have received this message in error, > please notify the sender immediately and delete the original message. > > Malcolm McFarland > Cavulus > > > This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any > unauthorized or improper disclosure, copying, distribution, or use of the > contents of this message is prohibited. The information contained in this > message is intended only for the personal and confidential use of the > recipient(s) named above. If you have received this message in error, > please notify the sender immediately and delete the original message. > > > On Thu, May 23, 2019 at 10:39 PM Jagadish Venkatraman < > jagadish1...@gmail.com> wrote: > > > No, the default setup should be sufficient - the number of tasks should > > have no significant impact on AM memory/resources. > > > > If you run out-of-memory, you can of course increase yarn.am.memory.mb. > > > > On Thu, May 23, 2019 at 10:45 AM Malcolm McFarland < > mmcfarl...@cavulus.com > > > > > wrote: > > > > > Hey folks, > > > > > > Are there any guidelines for how to provision an Application Master in > > > relation to the number of StreamTask instances it will be managing? Ie, > > are > > > there different memory, CPU, and thread-count figures for 100S > > StreamTasks > > > vs 1000, vs 10000? > > > > > > Cheers, > > > Malcolm McFarland > > > Cavulus > > > > > > > > > This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any > > > unauthorized or improper disclosure, copying, distribution, or use of > the > > > contents of this message is prohibited. The information contained in > this > > > message is intended only for the personal and confidential use of the > > > recipient(s) named above. If you have received this message in error, > > > please notify the sender immediately and delete the original message. > > > > > > > > > -- > > Jagadish V, > > Graduate Student, > > Department of Computer Science, > > Stanford University > > > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
