[jira] [Created] (HDFS-15299) Add an option to enable reporting flaky disk

star created HDFS-15299: --- Summary: Add an option to enable reporting flaky disk Key: HDFS-15299 URL: https://issues.apache.org/jira/browse/HDFS-15299 Project: Hadoop HDFS Issue Type: Bug

[jira] [Comment Edited] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925622#comment-16925622 ] star edited comment on HDFS-14378 at 9/9/19 11:51 AM: -- Thanks [~jojochuang] for

[jira] [Comment Edited] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925622#comment-16925622 ] star edited comment on HDFS-14378 at 9/9/19 11:51 AM: -- Thanks [~jojochuang] for

[jira] [Commented] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925622#comment-16925622 ] star commented on HDFS-14378: - Thanks [~jojochuang] for reviewing and advise. Section 7.6 of HDFS-1073

[jira] [Commented] (HDFS-14361) SNN will always upload fsimage

[ https://issues.apache.org/jira/browse/HDFS-14361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879988#comment-16879988 ] star commented on HDFS-14361: - Right. isPrimaryCheckPointer will not be changed when any error/exception 

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877439#comment-16877439 ] star commented on HDFS-12914: - Yes, It is broken in branch-3.0 with 'private' indicator for 'processReport'.

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877439#comment-16877439 ] star edited comment on HDFS-12914 at 7/3/19 2:43 AM: - Yes, It is broken in branch-3.0

[jira] [Resolved] (HDFS-14424) NN failover failed beacuse of lossing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star resolved HDFS-14424. - Resolution: Duplicate > NN failover failed beacuse of lossing paxos directory >

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859347#comment-16859347 ] star commented on HDFS-12914: - lgtm. Thanks [~hexiaoqiao] for anwsering. Failing test maybe checked.  >

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859272#comment-16859272 ] star commented on HDFS-12914: - Yes client refers to datanode. DN get REGISTER command after the startup of NN

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859250#comment-16859250 ] star edited comment on HDFS-12914 at 6/8/19 4:10 PM: - Few comments about your unit

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859250#comment-16859250 ] star edited comment on HDFS-12914 at 6/8/19 4:09 PM: - Few comments about your unit

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859246#comment-16859246 ] star edited comment on HDFS-12914 at 6/8/19 3:56 PM: - [~hexiaoqiao], I also write a

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859250#comment-16859250 ] star commented on HDFS-12914: - Few comments about your unit tests.  Following codes bypass lease expiration

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859246#comment-16859246 ] star commented on HDFS-12914: - [~hexiaoqiao], I also write a unit test for this issue, mostly similar to

[jira] [Assigned] (HDFS-5853) Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml

[ https://issues.apache.org/jira/browse/HDFS-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star reassigned HDFS-5853: -- Assignee: (was: star) > Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml >

[jira] [Assigned] (HDFS-5853) Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml

[ https://issues.apache.org/jira/browse/HDFS-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star reassigned HDFS-5853: -- Assignee: star (was: Akira Ajisaka) > Add "hadoop.user.group.metrics.percentiles.intervals" to

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845414#comment-16845414 ] star edited comment on HDFS-12914 at 5/22/19 2:58 AM: -- Thanks [~smarella]. A few

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845414#comment-16845414 ] star commented on HDFS-12914: - Thanks [~smarella]. A few questioins for reference only. 1. Guess it's a

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844056#comment-16844056 ] star edited comment on HDFS-12914 at 5/20/19 3:47 PM: -- [~hexiaoqiao] proposed a good

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844056#comment-16844056 ] star commented on HDFS-12914: - [~hexiaoqiao] proposed a good solution to solve this issue. Except for rpc

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16843829#comment-16843829 ] star edited comment on HDFS-12914 at 5/20/19 10:08 AM: --- [~smarella] how many DNs do

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16843829#comment-16843829 ] star edited comment on HDFS-12914 at 5/20/19 10:07 AM: --- [~smarella] how many DNs do

[jira] [Comment Edited] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16843829#comment-16843829 ] star edited comment on HDFS-12914 at 5/20/19 10:07 AM: --- [~smarella] how many DNs do

[jira] [Commented] (HDFS-12914) Block report leases cause missing blocks until next report

[ https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16843829#comment-16843829 ] star commented on HDFS-12914: - [~smarella] how many DNs do you have?  According to the limited logs, I think

[jira] [Updated] (HDFS-14489) fix naming issue for ScmBlockLocationTestingClient

[ https://issues.apache.org/jira/browse/HDFS-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14489: Component/s: (was: fs/ozone) ozone > fix naming issue for ScmBlockLocationTestingClient >

[jira] [Updated] (HDFS-14489) fix naming issue for ScmBlockLocationTestingClient

[ https://issues.apache.org/jira/browse/HDFS-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14489: Attachment: HDFS-14489.patch > fix naming issue for ScmBlockLocationTestingClient >

[jira] [Created] (HDFS-14489) fix naming issue for ScmBlockLocationTestingClient

star created HDFS-14489: --- Summary: fix naming issue for ScmBlockLocationTestingClient Key: HDFS-14489 URL: https://issues.apache.org/jira/browse/HDFS-14489 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-14476) lock too long when fix inconsistent blocks between disk and in-memory

[ https://issues.apache.org/jira/browse/HDFS-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834675#comment-16834675 ] star commented on HDFS-14476: - HDFS-10477 may be a useful reference. Seems like a ubiquitous problem with

[jira] [Comment Edited] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

[ https://issues.apache.org/jira/browse/HDFS-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795553#comment-16795553 ] star edited comment on HDFS-14349 at 5/2/19 3:41 PM: - It has been verified that edit

[jira] [Commented] (HDFS-13189) Standby NameNode should roll active edit log when checkpointing

[ https://issues.apache.org/jira/browse/HDFS-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831694#comment-16831694 ] star commented on HDFS-13189: - I‘ve proposed a improvement issue HDFS-14378 to put things right once and for

[jira] [Comment Edited] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826604#comment-16826604 ] star edited comment on HDFS-14437 at 4/26/19 3:21 AM: -- As [~kihwal] said in

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826604#comment-16826604 ] star commented on HDFS-14437: - As [~kihwal] said in HDFS-10943 that '{{rollEditLog()}} does not and cannot

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823750#comment-16823750 ] star commented on HDFS-14437: - ok, Later I will take a view. > Exception happened when rollEditLog expects

[jira] [Comment Edited] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823714#comment-16823714 ] star edited comment on HDFS-14437 at 4/23/19 6:55 AM: -- [~angerszhuuu], I think I've

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823742#comment-16823742 ] star commented on HDFS-14437: - [~angerszhuuu] Right. lock state fixed. {quote}But logAppend also need lock.

[jira] [Comment Edited] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823714#comment-16823714 ] star edited comment on HDFS-14437 at 4/23/19 6:38 AM: -- [~angerszhuuu], I think I've

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823726#comment-16823726 ] star commented on HDFS-14437: - locked is object lock of FSEditLog. > Exception happened when rollEditLog

[jira] [Comment Edited] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823726#comment-16823726 ] star edited comment on HDFS-14437 at 4/23/19 6:24 AM: -- locked is the state of object

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823714#comment-16823714 ] star commented on HDFS-14437: - [~angerszhuuu], I think I've got that.  

[jira] [Commented] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822296#comment-16822296 ] star commented on HDFS-14378: - Failed test pass locally. > Simplify the design of multiple NN and both logic

[jira] [Resolved] (HDFS-14439) FileNotFoundException thrown in TestEditLogRace when just add a setPermission operation

[ https://issues.apache.org/jira/browse/HDFS-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star resolved HDFS-14439. - Resolution: Not A Problem > FileNotFoundException thrown in TestEditLogRace when just add a setPermission >

[jira] [Commented] (HDFS-14439) FileNotFoundException thrown in TestEditLogRace when just add a setPermission operation

[ https://issues.apache.org/jira/browse/HDFS-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822295#comment-16822295 ] star commented on HDFS-14439: - Not a issue. Function verifyEditlogs load single editlogs segment, which makes

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: HDFS-14378-trunk.006.patch > Simplify the design of multiple NN and both logic of edit log roll and >

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821862#comment-16821862 ] star commented on HDFS-14437: - [~angerszhuuu], better upload your unit test code for others to reproduce the

[jira] [Assigned] (HDFS-14439) FileNotFoundException thrown in TestEditLogRace when just add a setPermission operation

[ https://issues.apache.org/jira/browse/HDFS-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star reassigned HDFS-14439: --- Assignee: star > FileNotFoundException thrown in TestEditLogRace when just add a setPermission > operation

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: HDFS-14378-trunk.005.patch > Simplify the design of multiple NN and both logic of edit log roll and >

[jira] [Created] (HDFS-14439) FileNotFoundException thrown in TestEditLogRace when just add a setPermission operation

star created HDFS-14439: --- Summary: FileNotFoundException thrown in TestEditLogRace when just add a setPermission operation Key: HDFS-14439 URL: https://issues.apache.org/jira/browse/HDFS-14439 Project: Hadoop

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821092#comment-16821092 ] star commented on HDFS-14437: - [~angerszhuuu]. If fsLock ignored, the senario He Xiaoqiao listed could occur

[jira] [Commented] (HDFS-14436) Configuration#getTimeDuration is not consistent between default value and manual settings.

[ https://issues.apache.org/jira/browse/HDFS-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821045#comment-16821045 ] star commented on HDFS-14436: - Maybe I should move it to hadoop-common project. >

[jira] [Commented] (HDFS-14436) Configuration#getTimeDuration is not consistent between default value and manual settings.

[ https://issues.apache.org/jira/browse/HDFS-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821044#comment-16821044 ] star commented on HDFS-14436: - It is a little wired to return only the raw literal number when default time

[jira] [Updated] (HDFS-14436) Configuration#getTimeDuration is not consistent between default value and manual settings.

[ https://issues.apache.org/jira/browse/HDFS-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14436: Attachment: HDFS-14436.001.patch > Configuration#getTimeDuration is not consistent between default value and >

[jira] [Comment Edited] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820950#comment-16820950 ] star edited comment on HDFS-14437 at 4/18/19 10:43 AM: --- [~hexiaoqiao], agreed.

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820950#comment-16820950 ] star commented on HDFS-14437: - [~hexiaoqiao], agreed. {quote}In my opinion, The following code segment in

[jira] [Comment Edited] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820886#comment-16820886 ] star edited comment on HDFS-14437 at 4/18/19 9:29 AM: -- Thanks [~hexiaoqiao] . The

[jira] [Comment Edited] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820886#comment-16820886 ] star edited comment on HDFS-14437 at 4/18/19 9:25 AM: -- Thanks [~hexiaoqiao] . The

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820886#comment-16820886 ] star commented on HDFS-14437: - Thanks [~hexiaoqiao] . The table really make the issue clearer. I've tracking

[jira] [Updated] (HDFS-14436) Configuration#getTimeDuration is not consistent between default value and manual settings.

[ https://issues.apache.org/jira/browse/HDFS-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14436: Description: When call getTimeDuration like this: {quote}conf.getTimeDuration("nn.interval", 10,

[jira] [Updated] (HDFS-14436) Configuration#getTimeDuration is not consistent between default value and manual settings.

[ https://issues.apache.org/jira/browse/HDFS-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14436: Description: When call getTimeDuration like this: {quote}conf.getTimeDuration("nn.interval", 10,

[jira] [Updated] (HDFS-14436) Configuration#getTimeDuration is not consistent between default value and manual settings.

[ https://issues.apache.org/jira/browse/HDFS-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14436: Description: When call getTimeDuration like this: {quote}conf.getTimeDuration("nn.interval", 10,

[jira] [Created] (HDFS-14436) Configuration#getTimeDuration is not consistent between default value and manual settings.

star created HDFS-14436: --- Summary: Configuration#getTimeDuration is not consistent between default value and manual settings. Key: HDFS-14436 URL: https://issues.apache.org/jira/browse/HDFS-14436 Project:

[jira] [Commented] (HDFS-10659) Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820206#comment-16820206 ] star commented on HDFS-10659: - [~jojochuang], would you like to take a review for the patch? > Namenode

[jira] [Commented] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820196#comment-16820196 ] star commented on HDFS-14378: - Any one would like to make a review for the patch? Appreciated. > Simplify

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: HDFS-14378-trunk.004.patch > Simplify the design of multiple NN and both logic of edit log roll and >

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: HDFS-14378-trunk.003.patch > Simplify the design of multiple NN and both logic of edit log roll and >

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: (was: HDFS-14378-trunk.002.patch) > Simplify the design of multiple NN and both logic of edit log

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Labels: patch (was: ) Affects Version/s: 3.1.2 Attachment: HDFS-14378-trunk.002.patch

[jira] [Updated] (HDFS-14424) NN failover failed beacuse of lossing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14424: External issue ID: (was: HDFS-10659) > NN failover failed beacuse of lossing paxos directory >

[jira] [Updated] (HDFS-14424) NN failover failed beacuse of lossing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14424: External issue ID: HDFS-10659 > NN failover failed beacuse of lossing paxos directory >

[jira] [Commented] (HDFS-10659) Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818634#comment-16818634 ] star commented on HDFS-10659: - failed test unrelated. check style is not newly added for hidden variable 'sd'

[jira] [Commented] (HDFS-10659) Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818480#comment-16818480 ] star commented on HDFS-10659: - fix check style. Seems func JNStorage#findFinalizedEditsFile is never used.

[jira] [Updated] (HDFS-10659) Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-10659: Attachment: HDFS-10659.006.patch > Namenode crashes after Journalnode re-installation in an HA cluster due to >

[jira] [Commented] (HDFS-10659) Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16817936#comment-16817936 ] star commented on HDFS-10659: - [~hanishakoneru] thanks. [~jojochuang], patch for trunk uploaded. Mind make a

[jira] [Updated] (HDFS-10659) Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-10659: Attachment: HDFS-10659.005.patch > Namenode crashes after Journalnode re-installation in an HA cluster due to >

[jira] [Assigned] (HDFS-10659) Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star reassigned HDFS-10659: --- Assignee: star (was: Hanisha Koneru) > Namenode crashes after Journalnode re-installation in an HA cluster

[jira] [Commented] (HDFS-10477) Stop decommission a rack of DataNodes caused NameNode fail over to standby

[ https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16817908#comment-16817908 ] star commented on HDFS-10477: - [~jojochuang], [^HDFS-10477.branch-2.8.patch] is for branch 2.8. Anything

[jira] [Assigned] (HDFS-14424) NN failover failed beacuse of lossing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star reassigned HDFS-14424: --- Assignee: star > NN failover failed beacuse of lossing paxos directory >

[jira] [Updated] (HDFS-14424) NN failover failed beacuse of lossing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14424: Description: Recently, our hadoop namenode shutdown when switching active namenode, just because of missing paxos

[jira] [Created] (HDFS-14424) NN start beacuse of lossing paxos directory

star created HDFS-14424: --- Summary: NN start beacuse of lossing paxos directory Key: HDFS-14424 URL: https://issues.apache.org/jira/browse/HDFS-14424 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-14424) NN failover failed beacuse of lossing paxos directory

[ https://issues.apache.org/jira/browse/HDFS-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14424: Summary: NN failover failed beacuse of lossing paxos directory (was: NN start beacuse of lossing paxos directory)

[jira] [Commented] (HDFS-10477) Stop decommission a rack of DataNodes caused NameNode fail over to standby

[ https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815013#comment-16815013 ] star commented on HDFS-10477: - [~jojochuang], uploaded patch for branch-2.8.  > Stop decommission a rack of

[jira] [Updated] (HDFS-10477) Stop decommission a rack of DataNodes caused NameNode fail over to standby

[ https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-10477: Attachment: HDFS-10477.branch-2.8.patch > Stop decommission a rack of DataNodes caused NameNode fail over to

[jira] [Commented] (HDFS-13596) NN restart fails after RollingUpgrade from 2.x to 3.x

[ https://issues.apache.org/jira/browse/HDFS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814975#comment-16814975 ] star commented on HDFS-13596: - Sorry for my mistake. Indeed length field is not behaving the way as my

[jira] [Commented] (HDFS-13596) NN restart fails after RollingUpgrade from 2.x to 3.x

[ https://issues.apache.org/jira/browse/HDFS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813327#comment-16813327 ] star commented on HDFS-13596: - New properties will be ignored as I checked the code. There ls a length field

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811877#comment-16811877 ] star commented on HDFS-14403: - [~xkrogen], Impressive results. Can you give a detailed information about how

[jira] [Comment Edited] (HDFS-13596) NN restart fails after RollingUpgrade from 2.x to 3.x

[ https://issues.apache.org/jira/browse/HDFS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811584#comment-16811584 ] star edited comment on HDFS-13596 at 4/6/19 2:35 PM: - Maybe another option：we can

[jira] [Comment Edited] (HDFS-13596) NN restart fails after RollingUpgrade from 2.x to 3.x

[ https://issues.apache.org/jira/browse/HDFS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811584#comment-16811584 ] star edited comment on HDFS-13596 at 4/6/19 2:34 PM: - Maybe another option：we can

[jira] [Commented] (HDFS-13596) NN restart fails after RollingUpgrade from 2.x to 3.x

[ https://issues.apache.org/jira/browse/HDFS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811584#comment-16811584 ] star commented on HDFS-13596: - Maybe another option：we can write new properties after the older ones, so that

[jira] [Commented] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807719#comment-16807719 ] star commented on HDFS-14378: - Initial patch for review. [~xkrogen] would you like to make a review for the

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: HDFS-14378-trunk.002.patch > Simplify the design of multiple NN and both logic of edit log roll and >

[jira] [Commented] (HDFS-14370) Edit log tailing fast-path should allow for backoff

[ https://issues.apache.org/jira/browse/HDFS-14370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807341#comment-16807341 ] star commented on HDFS-14370: - [~xkrogen]， agree.  Server side handler as limited resource should not be

[jira] [Commented] (HDFS-14276) [SBN read] Reduce tailing overhead

[ https://issues.apache.org/jira/browse/HDFS-14276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806918#comment-16806918 ] star commented on HDFS-14276: - Maybe we can just sleep 10ms in rpc server side if there are no edit logs. As

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: (was: HDFS-14378-trunk.002.patch) > Simplify the design of multiple NN and both logic of edit log

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: HDFS-14378-trunk.002.patch > Simplify the design of multiple NN and both logic of edit log roll and >

[jira] [Commented] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806045#comment-16806045 ] star commented on HDFS-14378: - first step: make ANN rolling its own log. last step: make ANN download a

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Attachment: HDFS-14378-trunk.001.patch > Simplify the design of multiple NN and both logic of edit log roll and >

[jira] [Created] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

star created HDFS-14378: --- Summary: Simplify the design of multiple NN and both logic of edit log roll and checkpoint Key: HDFS-14378 URL: https://issues.apache.org/jira/browse/HDFS-14378 Project: Hadoop HDFS

[jira] [Updated] (HDFS-14378) Simplify the design of multiple NN and both logic of edit log roll and checkpoint

[ https://issues.apache.org/jira/browse/HDFS-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] star updated HDFS-14378: Description:       HDFS-6440 introduced a mechanism to support more than 2 NNs. It implements a first-writer-win

[jira] [Commented] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

[ https://issues.apache.org/jira/browse/HDFS-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795553#comment-16795553 ] star commented on HDFS-14349: - Yes, it seems that normal edit log roll will be triggered by multiple SNN. I

[jira] [Commented] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

[ https://issues.apache.org/jira/browse/HDFS-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794201#comment-16794201 ] star commented on HDFS-14349: - The autoroll operation is executed in ANN service, not triggered by SNN. So

  1   2   >