depolying stage-level scheduling for Spark SQL and how to expose RDD code from Spark SQL?

Hi, I am trying to deploy the stage-level scheduling for Spark SQL. Since the current stage-level scheduling only supports the RDD-APIs, I want to expose the RDD transformation code from my Spark SQL code (with SQL syntax). Can you provide any pointers on how to do it? Stage level scheduling

Re: Scheduling in spark

Thank you so much for the link, Sujeet. regards Karthik -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Scheduling-in-spark-tp9035p9716.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Scheduling in spark

Thank you Andrew for the updated link. regards Karthik -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Scheduling-in-spark-tp9035p9717.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Scheduling in spark

Hi, I am a post graduate student, new to spark. I want to understand how Spark scheduler works. I just have theoretical understanding of DAG scheduler and the underlying task scheduler. I want to know, given a job to the framework, after the DAG scheduler phase, how the scheduling happens??

Re: Scheduling in spark

This is a good start: http://www.eecs.berkeley.edu/~tdas/spark_docs/job-scheduling.html On Tue, Jul 8, 2014 at 9:11 AM, rapelly kartheek kartheek.m...@gmail.com wrote: Hi, I am a post graduate student, new to spark. I want to understand how Spark scheduler works. I just have theoretical

Re: Scheduling in spark

Here's the most updated version of the same page: http://spark.apache.org/docs/latest/job-scheduling 2014-07-08 12:44 GMT-07:00 Sujeet Varakhedi svarakh...@gopivotal.com: This is a good start: http://www.eecs.berkeley.edu/~tdas/spark_docs/job-scheduling.html On Tue, Jul 8, 2014 at 9:11