yeah, this is why i'm gonna keep a close eye on things this week... as for VMs vs containers, please do the latter more than the former. one of our longer-term plans here at the lab is to move most of our jenkins infra to VMs, and running tests w/nested VMs is Bad[tm].
On Mon, Sep 29, 2014 at 2:25 PM, Reynold Xin <r...@databricks.com> wrote: > Thanks. We might see more failures due to contention on resources. Fingers > acrossed ... At some point it might make sense to run the tests in a VM or > container. > > > On Mon, Sep 29, 2014 at 2:20 PM, shane knapp <skn...@berkeley.edu> wrote: > >> we were running at 8 executors per node, and BARELY even stressing the >> machines (32 cores, ~230G RAM). >> >> in the interest of actually using system resources, and giving ourselves >> some headroom, i upped the executors to 16 per node. i'll be keeping an >> eye on ganglia for the rest of the week to make sure everything's cool. >> >> i hope you all enjoy your freshly allocated capacity! :) >> >> shane >> > >
