[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-9500: -- Fix Version/s: (was: 2.9.0) 2.8.0 > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Erik Krogen > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: 9500-v1.patch, HDFS-9500.000.patch, HDFS-9500.001.patch, > HDFS-9500.002.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-9500: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.7.4 2.9.0 Status: Resolved (was: Patch Available) I just committed this to trunk, branch-2, -2.8 and -2.7. Thank you, Erik. > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Erik Krogen > Fix For: 2.9.0, 2.7.4, 3.0.0-alpha2 > > Attachments: 9500-v1.patch, HDFS-9500.000.patch, HDFS-9500.001.patch, > HDFS-9500.002.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-9500: -- Attachment: HDFS-9500.002.patch Good catch, [~raviprak]! Attaching v002 patch. Thanks! > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Erik Krogen > Attachments: 9500-v1.patch, HDFS-9500.000.patch, HDFS-9500.001.patch, > HDFS-9500.002.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[
https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Krogen updated HDFS-9500:
--
Attachment: HDFS-9500.001.patch
[~shv], thanks for the review. I have added Javadocs. For
{{DatanodeRegistration}} I had copied the style of
{{testNumVersionsReportedCorrect}} below but you're right that it can be done
with just the constructor. Attaching v002 patch with these changes.
> datanodesSoftwareVersions map may counting wrong when rolling upgrade
> -
>
> Key: HDFS-9500
> URL: https://issues.apache.org/jira/browse/HDFS-9500
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.7.1, 2.6.2
>Reporter: Phil Yang
>Assignee: Erik Krogen
> Attachments: 9500-v1.patch, HDFS-9500.000.patch, HDFS-9500.001.patch
>
>
> While rolling upgrading, namenode's website overview will report there are
> two versions datanodes in the cluster, for example, 2.6.0 has x nodes and
> 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version
> and start a new version one, namenode only increases the number of new
> version but not decreases the number of old version. So the total number x+y
> will be larger than the number of datanodes. Even all datanodes are upgraded,
> there will still have the messages that there are several datanode in old
> version. And I must run hdfs dfsadmin -refreshNodes to clear this message.
> I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in
> old version is not alive because of shutting down, it will not pass
> shouldCountVersion, so the number of old version won't be decreased. But this
> method only judges the status of heartbeat and isAlive at that moment, if
> namenode has not removed this node which will decrease the version map and
> this node restarts in the new version, the decrementVersionCount belongs to
> this node will never be executed.
> So the simplest way to fix this is that we always recounting the version map
> in registerDatanode since it is not a heavy operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[
https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Krogen updated HDFS-9500:
--
Attachment: HDFS-9500.000.patch
Attaching a patch with a unit test to reproduce the issue deterministically. It
also contains a fix for the problem by changing the condition for
{{shouldCountVersion}} as described in my previous comment. As far as I can
tell the {{node.isAlive()}} condition check is sufficient but I would
appreciate review to confirm. [~shv]?
> datanodesSoftwareVersions map may counting wrong when rolling upgrade
> -
>
> Key: HDFS-9500
> URL: https://issues.apache.org/jira/browse/HDFS-9500
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.7.1, 2.6.2
>Reporter: Phil Yang
>Assignee: Erik Krogen
> Attachments: 9500-v1.patch, HDFS-9500.000.patch
>
>
> While rolling upgrading, namenode's website overview will report there are
> two versions datanodes in the cluster, for example, 2.6.0 has x nodes and
> 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version
> and start a new version one, namenode only increases the number of new
> version but not decreases the number of old version. So the total number x+y
> will be larger than the number of datanodes. Even all datanodes are upgraded,
> there will still have the messages that there are several datanode in old
> version. And I must run hdfs dfsadmin -refreshNodes to clear this message.
> I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in
> old version is not alive because of shutting down, it will not pass
> shouldCountVersion, so the number of old version won't be decreased. But this
> method only judges the status of heartbeat and isAlive at that moment, if
> namenode has not removed this node which will decrease the version map and
> this node restarts in the new version, the decrementVersionCount belongs to
> this node will never be executed.
> So the simplest way to fix this is that we always recounting the version map
> in registerDatanode since it is not a heavy operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-9500: --- Target Version/s: 2.7.4 (was: 2.7.4, 2.6.6) > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangjin Lee updated HDFS-9500: -- Target Version/s: 2.7.4, 2.6.6 (was: 2.6.5, 2.7.4) Moving this issue to 2.6.6. Please move back if you feel otherwise. > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated HDFS-9500: -- Target Version/s: 2.6.5, 2.7.4 (was: 2.7.3, 2.6.5) 2.7.3 is under release process, changing target-version to 2.7.4. > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9500: - Target Version/s: 2.7.3, 2.6.5 (was: 2.7.3, 2.6.4) > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9500: - Target Version/s: 2.7.3, 2.6.4 (was: 2.7.2, 2.6.4) 2.7.2 is waiting for a couple of blockers. Targeting 2.7.3. > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9500: - Target Version/s: 2.7.2, 2.6.4 (was: 2.7.2, 2.6.3) > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Status: Patch Available (was: Open) > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.6.2, 2.7.1 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Description: While rolling upgrading, namenode's website overview will report there are two versions datanodes in the cluster, for example, 2.6.0 has x nodes and 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version and start a new version one, namenode only increases the number of new version but not decreases the number of old version. So the total number x+y will be larger than the number of datanodes. Even all datanodes are upgraded, there will still have the messages that there are several datanode in old version. And I must run hdfs dfsadmin -refreshNodes to clear this message. I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in old version is not alive because of shutting down, it will not pass shouldCountVersion, so the number of old version won't be decreased. But this method only judges the status of heartbeat and isAlive at that moment, if namenode has not removed this node which will decrease the version map and this node restarts in the new version, the decrementVersionCount belongs to this node will never be executed. So the simplest way to fix this is that we always recounting the version map in registerDatanode since it is not a heavy operation. was: While rolling upgrading, namenode's website overview will report there are two versions datanodes in the cluster, for example, 2.6.0 has x nodes and 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version and start a new version one, namenode only increases the number of new version but not decreases the number of old version. So the total number x+y will be larger than the number of datanodes. Even all datanodes are upgraded, there will still have the messages that there are several datanode in old version. And I must run hdfs dfsadmin -refreshNodes to clear this message. I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in old version is not alive because of shutting down, it will not pass shouldCountVersion, so the number of old version won't be decreased. But this method only judges the status of heartbeat and isAlive at that moment, if namenode has not been noticed and removed this node and this node restarts in the new version, the decrementVersionCount belongs to this node will never be executed. So the simplest way to fix this is that we always recounting the version map in registerDatanode since it is not a heavy operation. > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not removed this node which will decrease the version map and > this node restarts in the new version, the decrementVersionCount belongs to > this node will never be executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Description: While rolling upgrading, namenode's website overview will report there are two versions datanodes in the cluster, for example, 2.6.0 has x nodes and 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version and start a new version one, namenode only increases the number of new version but not decreases the number of old version. So the total number x+y will be larger than the number of datanodes. Even all datanodes are upgraded, there will still have the messages that there are several datanode in old version. And I must run hdfs dfsadmin -refreshNodes to clear this message. I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in old version is not alive because of shutting down, it will not pass shouldCountVersion, so the number of old version won't be decreased. But this method only judges the status of heartbeat and isAlive at that moment, if namenode has not been noticed and removed this node and this node restarts in the new version, the decrementVersionCount belongs to this node will never be executed. So the simplest way to fix this is that we always recounting the version map in registerDatanode since it is not a heavy operation. was: While rolling upgrading, namenode's website overview will report there are two versions datanodes in the cluster, for example, 2.6.0 has x nodes and 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version and start a new version one, namenode only increases the number of new version but not decreases the number of old version. So the total number x+y will be larger than the number of datanodes. Even all datanodes are upgraded, there will still have the messages that there are several datanode in old version. And I must run hdfs dfsadmin -refreshNodes to clear this message. I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in old version is not alive because of shutting down, it will not pass shouldCountVersion, so the number of old version won't be decreased. But this method only judges the status of heartbeat and isAlive on the moment, if namenode has not been noticed and removed this node and this node restarts in the new version, the decrementVersionCount belongs to this node will never be executed. So the simplest way to fix this is that we always recounting the version map in registerDatanode since it is not a heavy operation. > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive at that moment, if > namenode has not been noticed and removed this node and this node restarts in > the new version, the decrementVersionCount belongs to this node will never be > executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Attachment: 9500-v1.patch > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive on the moment, if > namenode has not been noticed and removed this node and this node restarts in > the new version, the decrementVersionCount belongs to this node will never be > executed. > So the simplest fix is that we always recounting the version map in > registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Description: While rolling upgrading, namenode's website overview will report there are two versions datanodes in the cluster, for example, 2.6.0 has x nodes and 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version and start a new version one, namenode only increases the number of new version but not decreases the number of old version. So the total number x+y will be larger than the number of datanodes. Even all datanodes are upgraded, there will still have the messages that there are several datanode in old version. And I must run hdfs dfsadmin -refreshNodes to clear this message. I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in old version is not alive because of shutting down, it will not pass shouldCountVersion, so the number of old version won't be decreased. But this method only judges the status of heartbeat and isAlive on the moment, if namenode has not been noticed and removed this node and this node restarts in the new version, the decrementVersionCount belongs to this node will never be executed. So the simplest way to fix this is that we always recounting the version map in registerDatanode since it is not a heavy operation. was: While rolling upgrading, namenode's website overview will report there are two versions datanodes in the cluster, for example, 2.6.0 has x nodes and 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version and start a new version one, namenode only increases the number of new version but not decreases the number of old version. So the total number x+y will be larger than the number of datanodes. Even all datanodes are upgraded, there will still have the messages that there are several datanode in old version. And I must run hdfs dfsadmin -refreshNodes to clear this message. I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in old version is not alive because of shutting down, it will not pass shouldCountVersion, so the number of old version won't be decreased. But this method only judges the status of heartbeat and isAlive on the moment, if namenode has not been noticed and removed this node and this node restarts in the new version, the decrementVersionCount belongs to this node will never be executed. So the simplest fix is that we always recounting the version map in registerDatanode since it is not a heavy operation. > datanodesSoftwareVersions map may counting wrong when rolling upgrade > - > > Key: HDFS-9500 > URL: https://issues.apache.org/jira/browse/HDFS-9500 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1, 2.6.2 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: 9500-v1.patch > > > While rolling upgrading, namenode's website overview will report there are > two versions datanodes in the cluster, for example, 2.6.0 has x nodes and > 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version > and start a new version one, namenode only increases the number of new > version but not decreases the number of old version. So the total number x+y > will be larger than the number of datanodes. Even all datanodes are upgraded, > there will still have the messages that there are several datanode in old > version. And I must run hdfs dfsadmin -refreshNodes to clear this message. > I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in > old version is not alive because of shutting down, it will not pass > shouldCountVersion, so the number of old version won't be decreased. But this > method only judges the status of heartbeat and isAlive on the moment, if > namenode has not been noticed and removed this node and this node restarts in > the new version, the decrementVersionCount belongs to this node will never be > executed. > So the simplest way to fix this is that we always recounting the version map > in registerDatanode since it is not a heavy operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
