Hi, Arijit, The code flow of spark-submit is simple.
Enter the main function of SparkSubmit.scala --> case SparkSubmitAction.SUBMIT => submit(appArgs) --> doRunMain() in function submit() in the same file --> runMain(childArgs,...) in the same file --> mainMethod.invoke(null, childArgs.toArray) in the same file Function Invoke() is provided by JAVA Reflection for invoking the main function of your JAR. Hopefully, it can help you understand the problem. Thanks, Xiao Li 2015-10-07 16:47 GMT-07:00 Arijit <arij...@live.com>: > Hi, > > I want to understand the code flow starting from the Spark jar that I > submit through spark-submit, how does Spark identify and extract the > closures, clean and serialize them and ship them to workers to execute as > tasks. Can someone point me to any documentation or a pointer to the source > code path to help me understand this. > > Thanks, Arijit >