[
https://issues.apache.org/jira/browse/SPARK-5691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312632#comment-14312632
]
Matt Cheah commented on SPARK-5691:
-----------------------------------
I've determined that this is a pretty simple bug in the Master code.
I'm on commit hash 0793ee1b4dea1f4b0df749e8ad7c1ab70b512faf. In Master.scala,
in the registerApplication method, it checks if the application is already
registered by checking the addressToWorker data structure. In reality, this is
wrong - it should examine the addressToApp data structure.
I'll submit a pull request shortly.
> Preventing duplicate registering of an application has incorrect logic
> ----------------------------------------------------------------------
>
> Key: SPARK-5691
> URL: https://issues.apache.org/jira/browse/SPARK-5691
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.1.1, 1.2.0
> Reporter: Matt Cheah
> Fix For: 1.3.0
>
>
> If an application registers twice with the Master, the Master accepts both
> copies and they both show up in the UI and consume resources. This is
> incorrect behavior.
> This happens inadvertently in regular usage when the Master is under high
> load, but it boils down to: when an application times out registering with
> the master and sends a second registration message, but the Master is still
> alive, it processes the first registration message for the application but
> also erroneously processes the second registration message instead of
> discarding it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
