[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672312#comment-15672312 ] Jason Pan commented on SPARK-18353: --- Thanks sean. It works. Just for the doc: "spark.network.timeout, or 10s in standalone clusters" Actually the default is not 10s in standalone when using rest. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15656836#comment-15656836 ] Sean Owen commented on SPARK-18353: --- Any chance you can actually try a build with the patch? or comment on the PR? there's a question about whether this actually addresses your issue. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15656029#comment-15656029 ] Jason Pan commented on SPARK-18353: --- Yes, The pull requests make it can be set at least. Thanks. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653793#comment-15653793 ] Sean Owen commented on SPARK-18353: --- Let me know what you think of the pull requests at https://github.com/apache/spark/pull/15833 -- does it reflect your understanding? > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652585#comment-15652585 ] Jason Pan commented on SPARK-18353: --- No matter what the default value is at last, I think we need a way to configure it. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652576#comment-15652576 ] Jason Pan commented on SPARK-18353: --- append the summit command: spark-submit --class org.apache.spark.examples.SparkPi --master spark://9.111.159.127:7101 --conf spark.rpc.askTimeout=150 --deploy-mode cluster /opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar 10 the parameter doesn't work. Use rest: (6068 port is for rest) spark-submit --class org.apache.spark.examples.SparkPi --master spark://9.111.159.127:6068 --conf spark.rpc.askTimeout=150 --deploy-mode cluster /opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar 10 the parameter works. We now summit to rest url for a workaround, otherwise, the "spark.rpc.askTimeout" is always 10 due to the hardcode. Thanks. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652543#comment-15652543 ] Jason Pan commented on SPARK-18353: --- Hi Sean. I was using "--conf" to set the parameter when summit. It didn't work. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652537#comment-15652537 ] Jason Pan commented on SPARK-18353: --- Hi Sean. --conf also didn't make it work. Thanks. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651807#comment-15651807 ] Sean Owen commented on SPARK-18353: --- BTW [~JasonPan] it looks like you're setting JVM props, and not using --conf ? does --conf make it work? > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651584#comment-15651584 ] Apache Spark commented on SPARK-18353: -- User 'srowen' has created a pull request for this issue: https://github.com/apache/spark/pull/15833 > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15647623#comment-15647623 ] Jason Pan commented on SPARK-18353: --- Thanks. Yes, if spark.rpc.askTimeout is not configured, will use the value of spark.network.timeout. The hard code in Client make behavior inconsistent with document, in addition, when we use "--conf spark.rpc.askTimeout=" in spark-summit command or set it in spark-default.conf, it will not take effect due to it is configured in org.apache.spark.deploy.Client, when the network is busy, we can have no way to increase the rpc ascTimeout. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15647487#comment-15647487 ] Sean Owen commented on SPARK-18353: --- It does seem like Client should not set this value to 10, given the docs and the rest of the code. That could be removed. Also, spark.rpc.askTimeout doesn't actually default to 120s, but defaults to spark.network.timeout, which defaults to 120s. It could be worth updating the docs for the several properties that also have this default. > spark.rpc.askTimeout defalut value is not 120s > -- > > Key: SPARK-18353 > URL: https://issues.apache.org/jira/browse/SPARK-18353 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.6.1, 2.0.1 > Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 > 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jason Pan >Priority: Critical > > in http://spark.apache.org/docs/latest/configuration.html > spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait > before timing out > the defalut value is 120s as documented. > However when I run "spark-summit" with standalone cluster mode: > the cmd is: > Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp" > "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*" > "-Xmx1024M" "-Dspark.eventLog.enabled=true" > "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false" > "-Dspark.app.name=org.apache.spark.examples.SparkPi" > "-Dspark.submit.deployMode=cluster" > "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar" > "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10" > "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event" > "-Dspark.eventLog.dir=file:///opt/tmp/spark-event" > "org.apache.spark.deploy.worker.DriverWrapper" > "spark://Worker@9.111.159.127:7103" > "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar" > "org.apache.spark.examples.SparkPi" "1000" > Dspark.rpc.askTimeout=10 > the value is 10, it is not the same as document. > Note: when I summit to REST URL, it has no this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s
[
https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646795#comment-15646795
]
Jason Pan commented on SPARK-18353:
---
in org.apache.spark.deploy.Client,
there is one line:
conf.set("spark.rpc.askTimeout", "10")
should we remove this line?
when use rest:
in org.apache.spark.deploy.rest.RestSubmissionClient , there is no this line.
> spark.rpc.askTimeout defalut value is not 120s
> --
>
> Key: SPARK-18353
> URL: https://issues.apache.org/jira/browse/SPARK-18353
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.1, 2.0.1
> Environment: Linux zzz 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29
> 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Jason Pan
>Priority: Critical
>
> in http://spark.apache.org/docs/latest/configuration.html
> spark.rpc.askTimeout 120sDuration for an RPC ask operation to wait
> before timing out
> the defalut value is 120s as documented.
> However when I run "spark-summit":
> the cmd is:
> Launch Command: "/opt/jdk1.8.0_102/bin/java" "-cp"
> "/opt/spark-2.0.1-bin-hadoop2.7/conf/:/opt/spark-2.0.1-bin-hadoop2.7/jars/*"
> "-Xmx1024M" "-Dspark.eventLog.enabled=true"
> "-Dspark.master=spark://9.111.159.127:7101" "-Dspark.driver.supervise=false"
> "-Dspark.app.name=org.apache.spark.examples.SparkPi"
> "-Dspark.submit.deployMode=cluster"
> "-Dspark.jars=file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar"
> "-Dspark.history.ui.port=18087" "-Dspark.rpc.askTimeout=10"
> "-Dspark.history.fs.logDirectory=file:/opt/tmp/spark-event"
> "-Dspark.eventLog.dir=file:///opt/tmp/spark-event"
> "org.apache.spark.deploy.worker.DriverWrapper"
> "spark://Worker@9.111.159.127:7103"
> "/opt/spark-2.0.1-bin-hadoop2.7/work/driver-20161109031939-0002/spark-examples-1.6.1-hadoop2.6.0.jar"
> "org.apache.spark.examples.SparkPi" "1000"
> Dspark.rpc.askTimeout=10
> the value is 10, it is not the same as document.
> Note: when I summit to REST URL, it has no this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
