Client versus cluster mode

2016-01-21 Thread Afshartous, Nick
Hi, In an AWS EMR/Spark 1.5 cluster we're launching a streaming job from the driver node. Would it make any sense in this case to use cluster mode ? More specifically would there be any benefit that YARN would provide when using cluster but not client mode ? Thanks, -- Nick

Re: Client versus cluster mode

2016-01-21 Thread Manoj Awasthi
The only difference is that in yarn-cluster mode your driver runs within a yarn container (called AM or application master). You would want to run your production jobs in yarn-cluster mode while for development environment may do with yarn-client mode. Again, I think this just a recommendation