I think PythonRunner is launched when executing python script.
PythonGatewayServer is entry point for python spark shell
if (args.isPython && deployMode == CLIENT) {
if (args.primaryResource == PYSPARK_SHELL) {
args.mainClass = "org.apache.spark.api.python.PythonGatewayServer"
} else {
// If a python file is provided, add it to the child arguments and
list of files to deploy.
// Usage: PythonAppRunner <main python file> <extra python files>
[app arguments]
args.mainClass = "org.apache.spark.deploy.PythonRunner"
args.childArgs = ArrayBuffer(args.primaryResource, args.pyFiles)
++ args.childArgs
if (clusterManager != YARN) {
// The YARN backend distributes the primary file differently, so
don't merge it.
args.files = mergeFileLists(args.files, args.primaryResource)
}
}
On Wed, Oct 14, 2015 at 12:46 PM, skaarthik oss <[email protected]>
wrote:
> See PythonRunner @
> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/PythonRunner.scala
>
> On Tue, Oct 13, 2015 at 7:50 PM, canan chen <[email protected]> wrote:
>
>> I look at the source code of spark, but didn't find where python program
>> is started in python.
>>
>> It seems spark-submit will call PythonGatewayServer, but where is python
>> program started ?
>>
>> Thanks
>>
>
>
