[ https://issues.apache.org/jira/browse/SPARK-11085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311906#comment-15311906 ]
Ion Alberdi commented on SPARK-11085: ------------------------------------- Hello to all, I reproduce the error using the docker in https://github.com/Yannael/kafka-sparkstreaming-cassandra. What I tried to communicate the -Dhttp.proxyHost, -Dhttp.proxyPort parameters: - the javaopts worarkound mentionned above - setting, spark.executor.extraJavaOptions. The launched command becomes % java org.apache.spark.deploy.SparkSubmit ... --conf spark.driver.extraJavaOptions=-Dhttp.proxyHost=<proxyHost> -Dhttp.proxyPort=<proxyPort> I wonder whether the shell is able to parse that line and thus transfer the two (-Dhttp.proxyHost=<proxyHost> and -Dhttp.proxyPort=<proxyPort>) parameters to org.apache.spark.deploy.SparkSubmit - setting "--driver-java-options" parameters that ends with % java org.apache.spark.deploy.SparkSubmit ... -Dhttp.proxyHost=<proxyHost> -Dhttp.proxyPort=<proxyPort> even if the shell seems more likely to parse the two informations, the packages are not downloaded as the http request do not go through the proxy > Add support for HTTP proxy > --------------------------- > > Key: SPARK-11085 > URL: https://issues.apache.org/jira/browse/SPARK-11085 > Project: Spark > Issue Type: Improvement > Components: Spark Shell, Spark Submit > Reporter: Dustin Cote > Priority: Minor > > Add a way to update ivysettings.xml for the spark-shell and spark-submit to > support proxy settings for clusters that need to access a remote repository > through an http proxy. Typically this would be done like: > JAVA_OPTS="$JAVA_OPTS -Dhttp.proxyHost=proxy.host -Dhttp.proxyPort=8080 > -Dhttps.proxyHost=proxy.host.secure -Dhttps.proxyPort=8080" > Directly in the ivysettings.xml would look like: > <ivysettings> > <setproxy proxyhost="proxy.host" > proxyport="8080" > nonproxyhosts="nonproxy.host"/> > </ivysettings> > Even better would be a way to customize the ivysettings.xml with command > options. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org