Xiangrui Meng created SPARK-1924: ------------------------------------ Summary: Make local:/ scheme work in more deploy modes Key: SPARK-1924 URL: https://issues.apache.org/jira/browse/SPARK-1924 Project: Spark Issue Type: Sub-task Components: Deploy Affects Versions: 1.0.0 Reporter: Xiangrui Meng Priority: Minor
A resource marked "local:/" is assumed to be available on the local file system of every node. In such case, no data copying over network should happen. I tested different deploy modes in v1.0, right now we only support local:/ scheme for the app jar and secondary jars in the following modes: 1) local (jars are copied to the working directory) 2) standalone client 3) yarn client It doesn’t work for --files and python apps (--py-files and app.py). For the next release, we could support more deploy modes. -- This message was sent by Atlassian JIRA (v6.2#6252)