Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-05-11 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2105622801 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | -1 :x: | patch

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-05-11 Thread via GitHub
ZanderXu commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2105617264 Merged. Thanks @kokonguyen191 for your contribution and thanks @Hexiaoqiao @haiyang1987 @hfutatzhanghb for your review. -- This is an automated message from the Apache Git Service. To

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-05-11 Thread via GitHub
ZanderXu merged PR #6759: URL: https://github.com/apache/hadoop/pull/6759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-05-08 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2101828246 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-05-06 Thread via GitHub
haiyang1987 commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2097227087 LGTM. Hi @Hexiaoqiao @hfutatzhanghb any other comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-05-05 Thread via GitHub
kokonguyen191 commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1590504507 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java: ## @@ -185,6 +185,8 @@ public class

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-05-05 Thread via GitHub
haiyang1987 commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1590485852 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetrics.java: ## @@ -185,6 +185,8 @@ public class

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-28 Thread via GitHub
kokonguyen191 commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1582488078 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java: ## @@ -324,6 +325,10 @@ private void

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-28 Thread via GitHub
hfutatzhanghb commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2081770902 > @kokonguyen191 Thanks for your report and contributions. Sorry didn't get this issue completely. As description, you mentioned that `storage == null` as following. I wonder why

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-28 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2081754060 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-28 Thread via GitHub
kokonguyen191 commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1582197257 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java: ## @@ -324,6 +325,10 @@ private void

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-28 Thread via GitHub
kokonguyen191 commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2081512148 Hi @Hexiaoqiao, thanks for the review. For your first question, `getPerStorageIBR(storage)` doesn't do a null check on `storage` and assigns a new `PerStorageIBG` object to `null`.

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-28 Thread via GitHub
Hexiaoqiao commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2081497703 @kokonguyen191 Thanks for your report and contributions. Sorry didn't get this issue completely. As description, you mentioned that `storage == null` as following. I wonder why NPE

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-28 Thread via GitHub
haiyang1987 commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1582071262 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java: ## @@ -324,6 +325,10 @@ private void

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-24 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2074474125 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-23 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2072325438 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-23 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2072209318 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-23 Thread via GitHub
kokonguyen191 commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1575868360 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/IncrementalBlockReportManager.java: ## @@ -304,4 +310,4 @@ void

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-23 Thread via GitHub
kokonguyen191 commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1575867553 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/IncrementalBlockReportManager.java: ## @@ -169,11 +169,17 @@ private

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-23 Thread via GitHub
ZanderXu commented on code in PR #6759: URL: https://github.com/apache/hadoop/pull/6759#discussion_r1575829376 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/IncrementalBlockReportManager.java: ## @@ -169,11 +169,17 @@ private

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-23 Thread via GitHub
kokonguyen191 commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2071495428 Hi @ZanderXu @Hexiaoqiao, can you help me take a look whenever you're free? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-22 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2071375244 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-22 Thread via GitHub
hadoop-yetus commented on PR #6759: URL: https://github.com/apache/hadoop/pull/6759#issuecomment-2069730206 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] HDFS-17488. DN can fail IBRs with NPE when a volume is removed [hadoop]

2024-04-22 Thread via GitHub
kokonguyen191 opened a new pull request, #6759: URL: https://github.com/apache/hadoop/pull/6759 ### Description of PR Error logs ``` 2024-04-22 15:46:33,422 [BP-1842952724-10.22.68.249-1713771988830 heartbeating to localhost/127.0.0.1:64977] ERROR datanode.DataNode