Your mixing up different levels of scheduling. Spark's fair scheduler pools are about scheduling Jobs, not Applications; whereas YARN queues with Spark are about scheduling Applications, not Jobs.
On Wed, Apr 5, 2017 at 12:27 PM, Nick Chammas <nicholas.cham...@gmail.com> wrote: > I'm having trouble understanding the difference between Spark fair > scheduler pools > <https://spark.apache.org/docs/latest/job-scheduling.html#fair-scheduler-pools> > and YARN queues > <https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html>. > Do they conflict? Does one override the other? > > I posted a more detailed question about an issue I'm having with this on > Stack Overflow: http://stackoverflow.com/q/43239921/877069 > > Nick > > > ------------------------------ > View this message in context: Spark fair scheduler pools vs. YARN queues > <http://apache-spark-user-list.1001560.n3.nabble.com/Spark-fair-scheduler-pools-vs-YARN-queues-tp28572.html> > Sent from the Apache Spark User List mailing list archive > <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >
