[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780193=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780193
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:46
Start Date: 10/Jun/22 03:46
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151901548
> When we set dfs.ha.tail-edits.in-progress=true, the edits can be read by
getJournaledEdits (there is no gap actually) . But there is an GAP exception
thrown.
I think there is a gap here because bootstrap expects to get 1050196644
txid, but can't find it in the result. So throwing GAP Exception is ok.
Issue Time Tracking
---
Worklog Id: (was: 780193)
Time Spent: 4h 10m (was: 4h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780191=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780191
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:28
Start Date: 10/Jun/22 03:28
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151890746
As I explained above, change to `if (next ==
HdfsServerConstants.INVALID_TXID || elis.isInProgress())` maybe change the
original semantics of the `checkgap` method.
About my explain, do you have any questions？ Discuss together and become
more familiar with the relevant logic.
Issue Time Tracking
---
Worklog Id: (was: 780191)
Time Spent: 4h (was: 3h 50m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780189
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:22
Start Date: 10/Jun/22 03:22
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151887486
So in this case, we should change bootstrap logic.
Solution one: set DFS_HA_TAILEDITS_INPROGRESS_KEY to false.
Solution two: call getJournaledEdits multiple times until get the latest
txid, and then go to checkgap
Issue Time Tracking
---
Worklog Id: (was: 780189)
Time Spent: 3h 50m (was: 3h 40m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780187=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780187
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:18
Start Date: 10/Jun/22 03:18
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151885068
Oh, i know, the root cause is that getJournaledEdits returns up to 5000
txids by default. And 1049842441 - 1049837441 = 5000.
I can't reached to 1050196644, so checkForGaps failed.
Issue Time Tracking
---
Worklog Id: (was: 780187)
Time Spent: 3h 40m (was: 3.5h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780186=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780186
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:14
Start Date: 10/Jun/22 03:14
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151882369
> OK, back to BootstrapStandby GAP. Form this stack information, I got that
it try to get streams from 1049842441 to 1050196644. But cannot get the txid
1049842441 from the result streams. So I think we should to trace the root
cause, why can't we find txid 1049842441 in the return result of
`selectInputStreams(streams, 1049842441, true, true)`?
>
> Please correct me if anything is wrong.
Please refer to the discussion with @xkrogen above.
The root cause is the` if` condition (`if(next ==
HdfsServerConstants.INVALID_TXID)`) that does not enter properly.
Issue Time Tracking
---
Worklog Id: (was: 780186)
Time Spent: 3.5h (was: 3h 20m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780183=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780183
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:07
Start Date: 10/Jun/22 03:07
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151877997
OK, back to BootstrapStandby GAP.
Form this stack information, I got that it try to get streams from
1049842441 to 1050196644. But cannot get the txid 1049842441 from the result
streams.
So I think we should to trace the root cause, why can't we find txid
1049842441 in the return result of `selectInputStreams(streams, 1049842441,
true, true)`?
Please correct me if anything is wrong.
Issue Time Tracking
---
Worklog Id: (was: 780183)
Time Spent: 3h 20m (was: 3h 10m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780179=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780179
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 02:29
Start Date: 10/Jun/22 02:29
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151852402
> Thanks @tomscut , after tracing the code, I think we cannot add
`elis.isInProgress()`.
>
> And I will explain my ideas trough questions and answers. **Question one:
Why was INVALID_TXID considered in the original code?**
>
> * CheckForGaps method is used to check whether streams contains continuous
TXids from fromTxId to toAtLeastTxid
> * LastTxId equals INVALID_TXID means the stream is in progress
> * toAtLeastTxid maybe abnormal value, like Long.MaxValue. So the
CheckForGaps method only need to cover the latest inprogress segment.
>
> **Question two: What is the difference between INVALID_TXID and is
InProgress()？**
>
> * Before introducing [SBN READ], LastTxId equals INVALID_TXID means the
stream is in progress. And stream is in progress means it's lastTxId is
INVALID_TXID.
> * But after introducing [SBN READ], LastTxId equals INVALID_TXID means the
stream is in progress. But stream is in progress cannot mean it's lastTxId is
INVALID_TXID. Because introducing getJournaledEdits.
> * So if we add `elis.isInProgress()` in CheckForGaps, it cannot cover the
last writing segments which actual contains latest edit.
>
> Please correct me if anything is wrong.
Thanks @ZanderXu for your comment. Please refer to the stack.
When we set `dfs.ha.tail-edits.in-progress=true`, the txID can be read by
getJournaledEdits (there is no gap actually) . But there is an GAP exception
thrown.
Issue Time Tracking
---
Worklog Id: (was: 780179)
Time Spent: 3h 10m (was: 3h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=779778=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779778
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 04:53
Start Date: 09/Jun/22 04:53
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1150668804
Thanks @tomscut , after tracing the code, I think we cannot add
`elis.isInProgress()`.
And I will explain my ideas trough questions and answers.
**Question one: Why was INVALID_TXID considered in the original code?**
- CheckForGaps method is used to check whether streams contains continuous
TXids from fromTxId to toAtLeastTxid
- LastTxId equals INVALID_TXID means the stream is in progress
- toAtLeastTxid maybe abnormal value, like Long.MaxValue. So the
CheckForGaps method only need to cover the latest inprogress segment.
**Question two: What is the difference between INVALID_TXID and is
InProgress()？**
- Before introducing [SBN READ], LastTxId equals INVALID_TXID means the
stream is in progress. And stream is in progress means it's lastTxId is
INVALID_TXID.
- But after introducing [SBN READ], LastTxId equals INVALID_TXID means the
stream is in progress. But stream is in progress cannot mean it's lastTxId is
INVALID_TXID. Because introducing getJournaledEdits.
- So if we add `elis.isInProgress()` in CheckForGaps, it cannot cover the
last writing segments which actual contains latest edit.
Please correct me if anything is wrong.
Issue Time Tracking
---
Worklog Id: (was: 779778)
Time Spent: 3h (was: 2h 50m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=779311=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779311
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 08/Jun/22 03:34
Start Date: 08/Jun/22 03:34
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1149418728
Hi @xkrogen , if you have enough bandwidth, please take a look. Thank you.
Issue Time Tracking
---
Worklog Id: (was: 779311)
Time Spent: 2h 50m (was: 2h 40m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=778816=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-778816
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 07/Jun/22 02:20
Start Date: 07/Jun/22 02:20
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1148120968
Hi @jojochuang @tasanuma @Hexiaoqiao , could you please also take a look.
Thanks.
Issue Time Tracking
---
Worklog Id: (was: 778816)
Time Spent: 2h 40m (was: 2.5h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=775636=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-775636
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 28/May/22 06:22
Start Date: 28/May/22 06:22
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1140184555
Hi @ayushtkn , could you please also take a look at this. Thanks.
Issue Time Tracking
---
Worklog Id: (was: 775636)
Time Spent: 2.5h (was: 2h 20m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=773413=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773413
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 23/May/22 10:25
Start Date: 23/May/22 10:25
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1134494753
> Thanks @tomscut for your report. Similar with
[HDFS-14806](https://issues.apache.org/jira/browse/HDFS-14806) ?
Thanks @ZanderXu for your comments. Setting DFS_HA_TAILEDITS_INPROGRESS_KEY
to false could solve the problem. But if we correct the logic of judging
in-progress EditLogInputStream, it seems more reasonable.
Issue Time Tracking
---
Worklog Id: (was: 773413)
Time Spent: 2h 20m (was: 2h 10m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=773269=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773269
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 23/May/22 02:20
Start Date: 23/May/22 02:20
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1134096700
Thanks @tomscut for your report. Similar with
[HDFS-14806](https://issues.apache.org/jira/browse/HDFS-14806) ?
Issue Time Tracking
---
Worklog Id: (was: 773269)
Time Spent: 2h 10m (was: 2h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=773104=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773104
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 21/May/22 08:29
Start Date: 21/May/22 08:29
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1133564345
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 58s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 20s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 10s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 256m 13s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4219/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not
generate ASF License warnings. |
| | | 365m 17s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.tools.TestDFSAdmin |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4219/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4219 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 99d71330e64c 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7a37d9572f827a1af00a75ac93c4874a40c3eb07 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=773091=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773091
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 21/May/22 02:23
Start Date: 21/May/22 02:23
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1133511045
Hi @xkrogen , please take a look if you have enough bandwidth. Thanks a lot.
Issue Time Tracking
---
Worklog Id: (was: 773091)
Time Spent: 1h 50m (was: 1h 40m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=764635=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764635
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 30/Apr/22 08:33
Start Date: 30/Apr/22 08:33
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1113949271
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 12m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 45s | | trunk passed |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 16s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 247m 28s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 368m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4219/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4219 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e1b95ddc93cd 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6e602e9e05a17e8ee69124dec08a81a96eab627e |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4219/2/testReport/ |
| Max. process+thread count | 3067 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=764596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764596
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 30/Apr/22 02:29
Start Date: 30/Apr/22 02:29
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1113895505
Hi @xkrogen , to make the change safe, we can change the condition from:
`if(next == HdfsServerConstants.INVALID_TXID)`
to
`if(next == HdfsServerConstants.INVALID_TXID || elis.isInProgress())`
Do you think it's necessary?
Issue Time Tracking
---
Worklog Id: (was: 764596)
Time Spent: 1.5h (was: 1h 20m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=764592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764592
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 30/Apr/22 02:24
Start Date: 30/Apr/22 02:24
Worklog Time Spent: 10m
Work Description: tomscut commented on code in PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#discussion_r862281008
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java:
##
@@ -1792,7 +1792,7 @@ private void checkForGaps(List
streams, long fromTxId,
EditLogInputStream elis = iter.next();
if (elis.getFirstTxId() > txId) break;
long next = elis.getLastTxId();
Review Comment:
Thanks @xkrogen for you suggestion.
Issue Time Tracking
---
Worklog Id: (was: 764592)
Time Spent: 1h 20m (was: 1h 10m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=764587=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764587
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 30/Apr/22 02:12
Start Date: 30/Apr/22 02:12
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1113892494
> This seems right to me, but I don't fully understand what went wrong to
cause the error. Can you explain more fully? Why did we previously make the
assumption that `INVALID_TXID` meant in-progress, and what has changed to make
that not true / what happened in your specific scenario to cause that not to be
true?
Thank you @xkrogen very much for your review.
After introducing [SBN READ], we updated the configuration:
`dfs.ha.tail-edits.in-progress=true`.
Then when we `bootstrapStandby`, we will encounter something like this:
1. We need to start an Observer Namenode, so we execute bootstrapStandby
before start it. This will automatically pull the latest FSImage from the
Active Namenode and check whether the edits in the journals has a gap based on
the `lastTxid` of the FSImage.
2. Assume that the txid of the latest FSImage is x, and editslogs from x in
journals is in `InProgress` state, `FSEditLog#checkForGaps` will be skipped.
Because the `lastTxid` of the InProgress EditLogInputStream is not
`HdfsServerConstants.INVALID_TXID`, but a specific number.
3. However, between x and txID currently being written, there is finalize
Edit log, and `bootstrapStandby` can execute normally.
The `lastTxId` of an InProgress EditLogInputStream isn't always as
`HdfsServerConstants.INVALID_TXID`, could also be a specific number.
Issue Time Tracking
---
Worklog Id: (was: 764587)
Time Spent: 1h 10m (was: 1h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=764519=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764519
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 29/Apr/22 21:59
Start Date: 29/Apr/22 21:59
Worklog Time Spent: 10m
Work Description: xkrogen commented on code in PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#discussion_r862193291
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java:
##
@@ -1792,7 +1792,7 @@ private void checkForGaps(List
streams, long fromTxId,
EditLogInputStream elis = iter.next();
if (elis.getFirstTxId() > txId) break;
long next = elis.getLastTxId();
Review Comment:
The local variable is redundant now, we can just update L1805 to be:
```java
txId = elis.getLastTxId() + 1;
```
Issue Time Tracking
---
Worklog Id: (was: 764519)
Time Spent: 1h (was: 50m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=762723=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762723
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 27/Apr/22 06:20
Start Date: 27/Apr/22 06:20
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1110588939
Hi @ayushtkn @Hexiaoqiao @ferhui , could you please also take a look? Thanks
a lot.
Issue Time Tracking
---
Worklog Id: (was: 762723)
Time Spent: 50m (was: 40m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=761367=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-761367
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 24/Apr/22 01:19
Start Date: 24/Apr/22 01:19
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1107679485
Hi @tasanuma @xkrogen @sunchao , could you please have a look? Thanks a lot.
Issue Time Tracking
---
Worklog Id: (was: 761367)
Time Spent: 40m (was: 0.5h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> For example, set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=761366=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-761366
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 24/Apr/22 01:18
Start Date: 24/Apr/22 01:18
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1107679378
Thanks @ashutoshcipher for your review.
Issue Time Tracking
---
Worklog Id: (was: 761366)
Time Spent: 0.5h (was: 20m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> For example, set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=760879=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760879
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 22/Apr/22 15:33
Start Date: 22/Apr/22 15:33
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1106630776
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 12m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 52s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 34s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 19s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 248m 17s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 13s | | The patch does not
generate ASF License warnings. |
| | | 371m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4219/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4219 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 762330a41f0c 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / abd27315b22d09607203cbe3ff1fbb9ed8b47ca2 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4219/1/testReport/ |
| Max. process+thread count | 3553 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4219/1/console |
| versions |
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking Gap for inprogress EditLogInputStream
[ https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=760706=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760706 ] ASF GitHub Bot logged work on HDFS-16557: - Author: ASF GitHub Bot Created on: 22/Apr/22 09:21 Start Date: 22/Apr/22 09:21 Worklog Time Spent: 10m Work Description: tomscut opened a new pull request, #4219: URL: https://github.com/apache/hadoop/pull/4219 JIRA: HDFS-16557. The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily HdfsServerConstants.INVALID_TXID. We can determine its status directly by EditLogInputStream#isInProgress. For example, when bootstrapStandby, the EditLogInputStream of inProgress is misjudged, resulting in a gap check failure, which causes bootstrapStandby to fail.   Issue Time Tracking --- Worklog Id: (was: 760706) Remaining Estimate: 0h Time Spent: 10m > BootstrapStandby failed because of checking Gap for inprogress > EditLogInputStream > - > > Key: HDFS-16557 > URL: https://issues.apache.org/jira/browse/HDFS-16557 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: tomscut >Assignee: tomscut >Priority: Major > Attachments: image-2022-04-22-17-17-14-577.png, > image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png, > image-2022-04-22-17-17-32-487.png > > Time Spent: 10m > Remaining Estimate: 0h > > The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily > HdfsServerConstants.INVALID_TXID. We can determine its status directly by > EditLogInputStream#isInProgress. > For example, when bootstrapStandby, the EditLogInputStream of inProgress is > misjudged, resulting in a gap check failure, which causes bootstrapStandby to > fail. > !image-2022-04-22-17-17-32-487.png|width=766,height=161! > !image-2022-04-22-17-17-14-577.png|width=598,height=187! -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
