[jira] [Commented] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv
[
https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061130#comment-15061130
]
Shixiong Zhu commented on SPARK-12386:
--
So the consequence is, if the user removes `spark.executor.port`, he or she
cannot use Akka actorSystem if the firewall only allows a narrow range of ports
since actor system will listen to an arbitrary port.
> Setting "spark.executor.port" leads to NPE in SparkEnv
> --
>
> Key: SPARK-12386
> URL: https://issues.apache.org/jira/browse/SPARK-12386
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.0
>Reporter: Marcelo Vanzin
>Assignee: Apache Spark
>Priority: Critical
>
> From the list:
> {quote}
> when we set spark.executor.port in 1.6, we get thrown a NPE in
> SparkEnv$.create(SparkEnv.scala:259).
> {quote}
> Fix is simple; probably should make it to 1.6.0 since it will affect anyone
> using that config options, but I'll leave that to the release manager's
> discretion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv
[
https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061119#comment-15061119
]
Marcelo Vanzin commented on SPARK-12386:
The config option is not about assumptions; it's about controlling which ports
are used by Spark (so, for example, Spark listens on a port that is not
firewalled).
> Setting "spark.executor.port" leads to NPE in SparkEnv
> --
>
> Key: SPARK-12386
> URL: https://issues.apache.org/jira/browse/SPARK-12386
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.0
>Reporter: Marcelo Vanzin
>Assignee: Apache Spark
>Priority: Critical
>
> From the list:
> {quote}
> when we set spark.executor.port in 1.6, we get thrown a NPE in
> SparkEnv$.create(SparkEnv.scala:259).
> {quote}
> Fix is simple; probably should make it to 1.6.0 since it will affect anyone
> using that config options, but I'll leave that to the release manager's
> discretion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv
[
https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1506#comment-1506
]
Shixiong Zhu commented on SPARK-12386:
--
If the user doesn't depend on the assumption that `spark.executor.port` is the
port of Akka actor system in executor side, they can just remove the config.
Even in 1.5, the assumption is unreliable because multiple executors may run in
the same host.
> Setting "spark.executor.port" leads to NPE in SparkEnv
> --
>
> Key: SPARK-12386
> URL: https://issues.apache.org/jira/browse/SPARK-12386
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.0
>Reporter: Marcelo Vanzin
>Assignee: Apache Spark
>Priority: Critical
>
> From the list:
> {quote}
> when we set spark.executor.port in 1.6, we get thrown a NPE in
> SparkEnv$.create(SparkEnv.scala:259).
> {quote}
> Fix is simple; probably should make it to 1.6.0 since it will affect anyone
> using that config options, but I'll leave that to the release manager's
> discretion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv
[
https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061097#comment-15061097
]
Apache Spark commented on SPARK-12386:
--
User 'vanzin' has created a pull request for this issue:
https://github.com/apache/spark/pull/10339
> Setting "spark.executor.port" leads to NPE in SparkEnv
> --
>
> Key: SPARK-12386
> URL: https://issues.apache.org/jira/browse/SPARK-12386
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.0
>Reporter: Marcelo Vanzin
>Assignee: Apache Spark
>Priority: Critical
>
> From the list:
> {quote}
> when we set spark.executor.port in 1.6, we get thrown a NPE in
> SparkEnv$.create(SparkEnv.scala:259).
> {quote}
> Fix is simple; probably should make it to 1.6.0 since it will affect anyone
> using that config options, but I'll leave that to the release manager's
> discretion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv
[
https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061096#comment-15061096
]
Marcelo Vanzin commented on SPARK-12386:
BTW the bot seems to be lagging, but:
https://github.com/apache/spark/pull/10339
> Setting "spark.executor.port" leads to NPE in SparkEnv
> --
>
> Key: SPARK-12386
> URL: https://issues.apache.org/jira/browse/SPARK-12386
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.0
>Reporter: Marcelo Vanzin
>Assignee: Apache Spark
>Priority: Critical
>
> From the list:
> {quote}
> when we set spark.executor.port in 1.6, we get thrown a NPE in
> SparkEnv$.create(SparkEnv.scala:259).
> {quote}
> Fix is simple; probably should make it to 1.6.0 since it will affect anyone
> using that config options, but I'll leave that to the release manager's
> discretion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv
[
https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061084#comment-15061084
]
Marcelo Vanzin commented on SPARK-12386:
bq. The user can just remove that unused config
The problem is that it's not unused yet. It affects the port where the akka
actor system will be listening on.
> Setting "spark.executor.port" leads to NPE in SparkEnv
> --
>
> Key: SPARK-12386
> URL: https://issues.apache.org/jira/browse/SPARK-12386
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.0
>Reporter: Marcelo Vanzin
>Assignee: Apache Spark
>Priority: Critical
>
> From the list:
> {quote}
> when we set spark.executor.port in 1.6, we get thrown a NPE in
> SparkEnv$.create(SparkEnv.scala:259).
> {quote}
> Fix is simple; probably should make it to 1.6.0 since it will affect anyone
> using that config options, but I'll leave that to the release manager's
> discretion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv
[
https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061072#comment-15061072
]
Shixiong Zhu commented on SPARK-12386:
--
Yeah, I will submit a PR to fix it. However, I don't think it should be a
blocker for RC3. The user can just remove that unused config unless he or she
wants to use Akka in the executor side.
> Setting "spark.executor.port" leads to NPE in SparkEnv
> --
>
> Key: SPARK-12386
> URL: https://issues.apache.org/jira/browse/SPARK-12386
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.6.0
>Reporter: Marcelo Vanzin
>Priority: Critical
>
> From the list:
> {quote}
> when we set spark.executor.port in 1.6, we get thrown a NPE in
> SparkEnv$.create(SparkEnv.scala:259).
> {quote}
> Fix is simple; probably should make it to 1.6.0 since it will affect anyone
> using that config options, but I'll leave that to the release manager's
> discretion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
