[jira] [Updated] (HDFS-16828) Fsck doesn't count orphaned missing blocks.

2022-10-28 Thread Lei Yang (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Yang updated HDFS-16828:

Description: 
Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() to 
get missing blocks. This creates inconsistency with metasave.

In the case where orphaned block is present, metasave and fsck show different 
result:

metasave shows:

 
{code:java}
Metasave: Blocks currently missing: 1
[orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 e: 
0){code}
 

but fsck -list-corruptfileblocks  shows different count:
{noformat}
The filesystem under path '/' has 0 CORRUPT files
{noformat}
This also created inconsistency in dfshealth ui. In dfshealth UI:
 * Missing blocks count comes from blockManager.getMissinngBlockCount() which 
is 1
 * Corrupted file comes from fsck: NamenodeFsck.listCorruptFileBlocks() which 
doesn't count orphaned blocks so it is empty.

!image-2022-10-28-15-18-24-944.png!

  was:
Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() to 
get missing blocks. This creates inconsistency with metasave.

In the case where orphaned block is present, metasave and fsck show different 
result:

metasave shows:

 
{code:java}
Metasave: Blocks currently missing: 1
[orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 e: 
0){code}
 

but fsck -list-corruptfileblocks  shows different count:
{noformat}
The filesystem under path '/' has 0 CORRUPT files
{noformat}
This also created inconsistency between dfshealth ui. In dfshealth UI:
 * Missing blocks count comes from blockManager.getMissinngBlockCount()
 * Corrupted file comes from fsck: NamenodeFsck.listCorruptFileBlocks() which 
doesn't count orphaned blocks.

!image-2022-10-28-15-18-24-944.png!


> Fsck doesn't count orphaned missing blocks.
> ---
>
> Key: HDFS-16828
> URL: https://issues.apache.org/jira/browse/HDFS-16828
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Priority: Minor
> Attachments: image-2022-10-28-15-18-24-944.png
>
>
> Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() 
> to get missing blocks. This creates inconsistency with metasave.
> In the case where orphaned block is present, metasave and fsck show different 
> result:
> metasave shows:
>  
> {code:java}
> Metasave: Blocks currently missing: 1
> [orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 
> e: 0){code}
>  
> but fsck -list-corruptfileblocks  shows different count:
> {noformat}
> The filesystem under path '/' has 0 CORRUPT files
> {noformat}
> This also created inconsistency in dfshealth ui. In dfshealth UI:
>  * Missing blocks count comes from blockManager.getMissinngBlockCount() which 
> is 1
>  * Corrupted file comes from fsck: NamenodeFsck.listCorruptFileBlocks() which 
> doesn't count orphaned blocks so it is empty.
> !image-2022-10-28-15-18-24-944.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-16828) Fsck doesn't count orphaned missing blocks.

2022-10-28 Thread Lei Yang (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Yang updated HDFS-16828:

Description: 
Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() to 
get missing blocks. This creates inconsistency with metasave.

In the case where orphaned block is present, metasave and fsck show different 
result:

metasave shows:

 
{code:java}
Metasave: Blocks currently missing: 1
[orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 e: 
0){code}
 

but fsck -list-corruptfileblocks  shows different count:
{noformat}
The filesystem under path '/' has 0 CORRUPT files
{noformat}
This also created inconsistency between dfshealth ui. In dfshealth UI:
 * Missing blocks count comes from blockManager.getMissinngBlockCount()
 * Corrupted file comes from fsck: NamenodeFsck.listCorruptFileBlocks() which 
doesn't count orphaned blocks.

!image-2022-10-28-15-18-24-944.png!

  was:
Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() to 
get missing blocks. This creates inconsistency with metasave.

In the case where orphaned block is present, metasave and fsck show different 
result:

metasave shows:

 
{code:java}
Metasave: Blocks currently missing: 1
[orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 e: 
0){code}
 

but fsck -list-corruptfileblocks  shows different count:
{noformat}
The filesystem under path '/' has 0 CORRUPT files
{noformat}
 


> Fsck doesn't count orphaned missing blocks.
> ---
>
> Key: HDFS-16828
> URL: https://issues.apache.org/jira/browse/HDFS-16828
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Priority: Minor
> Attachments: image-2022-10-28-15-18-24-944.png
>
>
> Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() 
> to get missing blocks. This creates inconsistency with metasave.
> In the case where orphaned block is present, metasave and fsck show different 
> result:
> metasave shows:
>  
> {code:java}
> Metasave: Blocks currently missing: 1
> [orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 
> e: 0){code}
>  
> but fsck -list-corruptfileblocks  shows different count:
> {noformat}
> The filesystem under path '/' has 0 CORRUPT files
> {noformat}
> This also created inconsistency between dfshealth ui. In dfshealth UI:
>  * Missing blocks count comes from blockManager.getMissinngBlockCount()
>  * Corrupted file comes from fsck: NamenodeFsck.listCorruptFileBlocks() which 
> doesn't count orphaned blocks.
> !image-2022-10-28-15-18-24-944.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-16828) Fsck doesn't count orphaned missing blocks.

2022-10-28 Thread Lei Yang (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Yang updated HDFS-16828:

Attachment: image-2022-10-28-15-18-24-944.png

> Fsck doesn't count orphaned missing blocks.
> ---
>
> Key: HDFS-16828
> URL: https://issues.apache.org/jira/browse/HDFS-16828
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Priority: Minor
> Attachments: image-2022-10-28-15-18-24-944.png
>
>
> Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() 
> to get missing blocks. This creates inconsistency with metasave.
> In the case where orphaned block is present, metasave and fsck show different 
> result:
> metasave shows:
>  
> {code:java}
> Metasave: Blocks currently missing: 1
> [orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 
> e: 0){code}
>  
> but fsck -list-corruptfileblocks  shows different count:
> {noformat}
> The filesystem under path '/' has 0 CORRUPT files
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-16828) Fsck doesn't count orphaned missing blocks.

2022-10-28 Thread Lei Yang (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Yang updated HDFS-16828:

Affects Version/s: 2.10.0

> Fsck doesn't count orphaned missing blocks.
> ---
>
> Key: HDFS-16828
> URL: https://issues.apache.org/jira/browse/HDFS-16828
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.10.0
>Reporter: Lei Yang
>Priority: Minor
>
> Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() 
> to get missing blocks. This creates inconsistency with metasave.
> In the case where orphaned block is present, metasave and fsck show different 
> result:
> metasave shows:
>  
> {code:java}
> Metasave: Blocks currently missing: 1
> [orphaned]: blk_106452613228_105447711565 MISSING (replicas: l: 0 d: 0 c: 0 
> e: 0){code}
>  
> but fsck -list-corruptfileblocks  shows different count:
> {noformat}
> The filesystem under path '/' has 0 CORRUPT files
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org