[jira] [Updated] (SPARK-1924) Make local:/ scheme work in more deploy modes
[ https://issues.apache.org/jira/browse/SPARK-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-1924: Labels: bulk-closed (was: ) > Make local:/ scheme work in more deploy modes > - > > Key: SPARK-1924 > URL: https://issues.apache.org/jira/browse/SPARK-1924 > Project: Spark > Issue Type: Sub-task > Components: Deploy >Affects Versions: 1.0.0 >Reporter: Xiangrui Meng >Priority: Minor > Labels: bulk-closed > > A resource marked "local:/" is assumed to be available on the local file > system of every node. In such case, no data copying over network should > happen. I tested different deploy modes in v1.0, right now we only support > local:/ scheme for the app jar and secondary jars in the following modes: > 1) local (jars are copied to the working directory) > 2) standalone client > 3) yarn client > It doesn’t work for --files and python apps (--py-files and app.py). For the > next release, we could support more deploy modes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-1924) Make local:/ scheme work in more deploy modes
[ https://issues.apache.org/jira/browse/SPARK-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1924: --- Fix Version/s: (was: 1.2.0) 1.3.0 Make local:/ scheme work in more deploy modes - Key: SPARK-1924 URL: https://issues.apache.org/jira/browse/SPARK-1924 Project: Spark Issue Type: Sub-task Components: Deploy Affects Versions: 1.0.0 Reporter: Xiangrui Meng Priority: Minor Fix For: 1.3.0 A resource marked local:/ is assumed to be available on the local file system of every node. In such case, no data copying over network should happen. I tested different deploy modes in v1.0, right now we only support local:/ scheme for the app jar and secondary jars in the following modes: 1) local (jars are copied to the working directory) 2) standalone client 3) yarn client It doesn’t work for --files and python apps (--py-files and app.py). For the next release, we could support more deploy modes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-1924) Make local:/ scheme work in more deploy modes
[ https://issues.apache.org/jira/browse/SPARK-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1924: --- Fix Version/s: (was: 1.3.0) Make local:/ scheme work in more deploy modes - Key: SPARK-1924 URL: https://issues.apache.org/jira/browse/SPARK-1924 Project: Spark Issue Type: Sub-task Components: Deploy Affects Versions: 1.0.0 Reporter: Xiangrui Meng Priority: Minor A resource marked local:/ is assumed to be available on the local file system of every node. In such case, no data copying over network should happen. I tested different deploy modes in v1.0, right now we only support local:/ scheme for the app jar and secondary jars in the following modes: 1) local (jars are copied to the working directory) 2) standalone client 3) yarn client It doesn’t work for --files and python apps (--py-files and app.py). For the next release, we could support more deploy modes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-1924) Make local:/ scheme work in more deploy modes
[ https://issues.apache.org/jira/browse/SPARK-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1924: --- Fix Version/s: (was: 1.1.0) 1.2.0 Make local:/ scheme work in more deploy modes - Key: SPARK-1924 URL: https://issues.apache.org/jira/browse/SPARK-1924 Project: Spark Issue Type: Sub-task Components: Deploy Affects Versions: 1.0.0 Reporter: Xiangrui Meng Priority: Minor Fix For: 1.2.0 A resource marked local:/ is assumed to be available on the local file system of every node. In such case, no data copying over network should happen. I tested different deploy modes in v1.0, right now we only support local:/ scheme for the app jar and secondary jars in the following modes: 1) local (jars are copied to the working directory) 2) standalone client 3) yarn client It doesn’t work for --files and python apps (--py-files and app.py). For the next release, we could support more deploy modes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
