On Fri, Feb 20, 2015 at 02:21PM, Evans Ye wrote: > I don't have spark expertise, but here're some points I'm thinking about. > IIRC, spark standalone do not support Kerberos. And the benefit of > deploying spark on yarn should be that you don't need to maintain packages > by your own on hundreds of node cluster.
Could you clarify what you mean by this? Are you referring that you won't need to install spark-worker on the cluster's nodes? Cos > Not sure if there're downsides. Just want to add some points:) > > 2015-02-20 9:49 GMT+08:00 Konstantin Boudnik <[email protected]>: > > > They way we're deploying spark is in the standalone mode - I never seen any > > value in using YARN for that, but I guess it's just me. > > > > HDFS use comes with no hassle, AFAIR, the way we setup it up. But my > > knowledge > > might be a bit outdated... > > > > Cos > > > > On Thu, Feb 19, 2015 at 08:45PM, jay vyas wrote: > > > hi folks. > > > > > > is anyone planning to use spark on yarn or spark w/ hdfs in bigtop? I > > > havent tried either... > > > > > > - anyone using spark <-> HDFS in bigtop ? Do we need to update any spark > > > configs to do so ? > > > - we want spark to run on yarn ? standalone ? > > > > > > im spinning some VMs up now, ill let folks know if it works. > > > -- > > > jay vyas > >
