[jira] [Commented] (HDFS-17523) Add fine-grained locks metrics in DataSetLockManager

2024-05-15 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846659#comment-17846659 ] lei w commented on HDFS-17523: -- [~hexiaoqiao]Thank you for your comment, PR will be submitted later > Add

[jira] [Created] (HDFS-17523) Add fine-grained locks metrics in DataSetLockManager

2024-05-12 Thread lei w (Jira)
lei w created HDFS-17523: Summary: Add fine-grained locks metrics in DataSetLockManager Key: HDFS-17523 URL: https://issues.apache.org/jira/browse/HDFS-17523 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-17519) Reduce lease checks when the last block is in the complete state and the penultimate block is in the committed state

2024-05-10 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17519: - Summary: Reduce lease checks when the last block is in the complete state and the penultimate block is in the

[jira] [Commented] (HDFS-17519) Reduce lease checks when the last block is in the complete state and the penultimate block is in the committed state in ones file.

2024-05-10 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845401#comment-17845401 ] lei w commented on HDFS-17519: -- [~ayushsaxena] Can you give me some suggestions? > Reduce lease checks when

[jira] [Updated] (HDFS-17519) Reduce lease checks when the last block is in the complete state and the penultimate block is in the committed state in ones file.

2024-05-10 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17519: - Summary: Reduce lease checks when the last block is in the complete state and the penultimate block is in the

[jira] [Created] (HDFS-17519) LeaseMonitor renew the lease when the last block is in the complete state and the penultimate block is in the committed state in ones file.

2024-05-10 Thread lei w (Jira)
lei w created HDFS-17519: Summary: LeaseMonitor renew the lease when the last block is in the complete state and the penultimate block is in the committed state in ones file. Key: HDFS-17519 URL:

[jira] [Created] (HDFS-17518) In the lease monitor, if a file is closed, we should sync the editslog

2024-05-10 Thread lei w (Jira)
lei w created HDFS-17518: Summary: In the lease monitor, if a file is closed, we should sync the editslog Key: HDFS-17518 URL: https://issues.apache.org/jira/browse/HDFS-17518 Project: Hadoop HDFS

[jira] [Updated] (HDFS-17408) Reduce the number of quota calculations in FSDirRenameOp

2024-03-12 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17408: - Description: During the execution of the rename operation, we first calculate the quota for the source INode

[jira] [Created] (HDFS-17422) Enhance the stability of the unit test TestDFSAdmin

2024-03-11 Thread lei w (Jira)
lei w created HDFS-17422: Summary: Enhance the stability of the unit test TestDFSAdmin Key: HDFS-17422 URL: https://issues.apache.org/jira/browse/HDFS-17422 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-17408) Reduce the number of quota calculations in FSDirRenameOp

2024-03-04 Thread lei w (Jira)
lei w created HDFS-17408: Summary: Reduce the number of quota calculations in FSDirRenameOp Key: HDFS-17408 URL: https://issues.apache.org/jira/browse/HDFS-17408 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-17391) Adjust the checkpoint io buffer size to the chunk size

2024-02-20 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17391: - Description: Adjust the checkpoint io buffer size to the chunk size to reduce checkpoint time. Before change:

[jira] [Created] (HDFS-17391) Adjust the checkpoint io buffer size to the chunk size

2024-02-20 Thread lei w (Jira)
lei w created HDFS-17391: Summary: Adjust the checkpoint io buffer size to the chunk size Key: HDFS-17391 URL: https://issues.apache.org/jira/browse/HDFS-17391 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-17383) Datanode current block token should come from active NameNode in HA mode

2024-02-19 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17383: - Summary: Datanode current block token should come from active NameNode in HA mode (was: Use the block token

[jira] [Updated] (HDFS-17383) Use the block token from active NameNode to transfer block in DataNode

2024-02-18 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17383: - Description: We found that transfer block failed during the namenode upgrade. The specific error reported was

[jira] [Updated] (HDFS-17383) Use the block token from active NameNode to transfer block in DataNode

2024-02-18 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17383: - Attachment: reproduce.diff > Use the block token from active NameNode to transfer block in DataNode >

[jira] [Updated] (HDFS-17383) Use the block token from active NameNode to transfer block in DataNode

2024-02-18 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17383: - Description: We found that transfer block failed during the namenode upgrade. The specific error reported was

[jira] [Created] (HDFS-17383) Use the block token from active NameNode to transfer block in DataNode

2024-02-18 Thread lei w (Jira)
lei w created HDFS-17383: Summary: Use the block token from active NameNode to transfer block in DataNode Key: HDFS-17383 URL: https://issues.apache.org/jira/browse/HDFS-17383 Project: Hadoop HDFS

[jira] [Updated] (HDFS-17354) Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up

2024-01-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17354: - Description: We should start clear expired namespace thread at RouterRpcServer RUNNING phase because

[jira] [Updated] (HDFS-17354) Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up

2024-01-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17354: - Description: We should start clear expired namespace thread at RouterRpcServer initialization phase because

[jira] [Created] (HDFS-17354) Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up

2024-01-24 Thread lei w (Jira)
lei w created HDFS-17354: Summary: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up Key: HDFS-17354 URL: https://issues.apache.org/jira/browse/HDFS-17354 Project: Hadoop HDFS

[jira] [Created] (HDFS-17339) BPServiceActor should skip cacheReport when one blockPool does not have CacheBlock on this DataNode

2024-01-15 Thread lei w (Jira)
lei w created HDFS-17339: Summary: BPServiceActor should skip cacheReport when one blockPool does not have CacheBlock on this DataNode Key: HDFS-17339 URL: https://issues.apache.org/jira/browse/HDFS-17339

[jira] [Updated] (HDFS-17331) Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in federationhealth.html

2024-01-14 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17331: - Attachment: Before fix.png > Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in >

[jira] [Updated] (HDFS-17331) Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in federationhealth.html

2024-01-14 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17331: - Attachment: After fix.png > Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in >

[jira] [Updated] (HDFS-17331) Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in federationhealth.html

2024-01-14 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17331: - Attachment: (was: 截屏2024-01-08 下午8.17.07.png) > Fix Blocks are always -1 and DataNode`s version are always

[jira] [Created] (HDFS-17331) Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in federationhealth.html

2024-01-09 Thread lei w (Jira)
lei w created HDFS-17331: Summary: Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in federationhealth.html Key: HDFS-17331 URL: https://issues.apache.org/jira/browse/HDFS-17331 Project:

[jira] [Updated] (HDFS-17331) Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in federationhealth.html

2024-01-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-17331: - Attachment: 截屏2024-01-08 下午8.17.07.png > Fix Blocks are always -1 and DataNode`s version are always UNKNOWN in

[jira] [Commented] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2023-12-11 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795580#comment-17795580 ] lei w commented on HDFS-16083: -- [~tomscut] Is this reasonable? > Forbid Observer NameNode trigger active

[jira] [Resolved] (HDFS-16102) Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to save time

2023-12-11 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w resolved HDFS-16102. -- Resolution: Invalid > Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to > save time

[jira] [Resolved] (HDFS-16020) DatanodeReportType should add LIVE_NOT_DECOMMISSIONING type

2023-12-11 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w resolved HDFS-16020. -- Resolution: Invalid > DatanodeReportType should add LIVE_NOT_DECOMMISSIONING type >

[jira] [Updated] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2023-12-11 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16196: - Resolution: Fixed Status: Resolved (was: Patch Available) > Namesystem#completeFile method will log

[jira] [Created] (HDFS-17275) We should determine whether the block has been deleted in the block report

2023-12-05 Thread lei w (Jira)
lei w created HDFS-17275: Summary: We should determine whether the block has been deleted in the block report Key: HDFS-17275 URL: https://issues.apache.org/jira/browse/HDFS-17275 Project: Hadoop HDFS

[jira] [Assigned] (HDFS-17270) Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper Client to get token in some case

2023-12-01 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w reassigned HDFS-17270: Assignee: lei w > Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper Client to get > token in

[jira] [Created] (HDFS-17270) Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper Client to get token in some case

2023-12-01 Thread lei w (Jira)
lei w created HDFS-17270: Summary: Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper Client to get token in some case Key: HDFS-17270 URL: https://issues.apache.org/jira/browse/HDFS-17270

[jira] [Commented] (HDFS-16038) DataNode Unrecognized Observer Node when cluster add an observer node

2022-04-28 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529764#comment-17529764 ] lei w commented on HDFS-16038: -- Is it possible to add a switch on the namenode node, if namenode state is

[jira] [Commented] (HDFS-16428) Source path setted storagePolicy will cause wrong typeConsumed in rename operation

2022-01-17 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477588#comment-17477588 ] lei w commented on HDFS-16428: -- [~LiJinglun] [~ayushtkn] [~xiaoyuyao] Looking forward to your comments! >

[jira] [Updated] (HDFS-16428) Source path setted storagePolicy will cause wrong typeConsumed in rename operation

2022-01-17 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16428: - Summary: Source path setted storagePolicy will cause wrong typeConsumed in rename operation (was: When source

[jira] [Updated] (HDFS-16428) When source path setted storagePolicy in rename operation will cause wrong typeConsumed

2022-01-17 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16428: - Attachment: example.txt > When source path setted storagePolicy in rename operation will cause wrong >

[jira] [Created] (HDFS-16428) When source path setted storagePolicy in rename operation will cause wrong typeConsumed

2022-01-17 Thread lei w (Jira)
lei w created HDFS-16428: Summary: When source path setted storagePolicy in rename operation will cause wrong typeConsumed Key: HDFS-16428 URL: https://issues.apache.org/jira/browse/HDFS-16428 Project:

[jira] [Assigned] (HDFS-15068) DataNode could meet deadlock if invoke refreshVolumes when register

2021-11-29 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w reassigned HDFS-15068: Assignee: Aiphago (was: lei w) > DataNode could meet deadlock if invoke refreshVolumes when register >

[jira] [Assigned] (HDFS-15068) DataNode could meet deadlock if invoke refreshVolumes when register

2021-11-29 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w reassigned HDFS-15068: Assignee: lei w (was: Aiphago) > DataNode could meet deadlock if invoke refreshVolumes when register >

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2021-09-16 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415995#comment-17415995 ] lei w commented on HDFS-14768: -- hi [~gjhkael] Please provide patch for branch 3.1 and 3.2, We need it.

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2021-09-13 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-15240: - Description: # When read some lzo files we found some blocks were broken. I read back all internal

[jira] [Commented] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-06 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410605#comment-17410605 ] lei w commented on HDFS-16196: -- Hi [~hexiaoqiao] . I saw other requests on NameNode, those do not have the

[jira] [Assigned] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-06 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w reassigned HDFS-16196: Assignee: lei w > Namesystem#completeFile method will log incorrect path information when > router to

[jira] [Updated] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-06 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16196: - Attachment: HDFS-16196.001.patch Status: Patch Available (was: Open) > Namesystem#completeFile method

[jira] [Updated] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-06 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16196: - Attachment: HDFS-16196.001.patch > Namesystem#completeFile method will log incorrect path information when >

[jira] [Updated] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-06 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16196: - Attachment: (was: HDFS-16196.001.patch) > Namesystem#completeFile method will log incorrect path information

[jira] [Commented] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-06 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410501#comment-17410501 ] lei w commented on HDFS-16196: -- OK,Thanks for [~hexiaoqiao] comment. > Namesystem#completeFile method will

[jira] [Comment Edited] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-05 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410334#comment-17410334 ] lei w edited comment on HDFS-16196 at 9/6/21, 4:17 AM: --- Thanks [~hexiaoqiao]

[jira] [Comment Edited] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-05 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410334#comment-17410334 ] lei w edited comment on HDFS-16196 at 9/6/21, 3:28 AM: --- Thanks [~hexiaoqiao]

[jira] [Commented] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-05 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410334#comment-17410334 ] lei w commented on HDFS-16196: -- Thanks [~hexiaoqiao] comment. I checked the logic of trunk was same like

[jira] [Commented] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-09-03 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409393#comment-17409393 ] lei w commented on HDFS-16196: -- Thanks [~ayushtkn] comment. I may not have made it clear . The issue is

[jira] [Created] (HDFS-16196) Namesystem#completeFile method will log incorrect path information when router to access

2021-08-30 Thread lei w (Jira)
lei w created HDFS-16196: Summary: Namesystem#completeFile method will log incorrect path information when router to access Key: HDFS-16196 URL: https://issues.apache.org/jira/browse/HDFS-16196 Project:

[jira] [Resolved] (HDFS-16126) VolumePair should override hashcode() method

2021-07-13 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w resolved HDFS-16126. -- Resolution: Invalid > VolumePair should override hashcode() method >

[jira] [Created] (HDFS-16126) VolumePair should override hashcode() method

2021-07-13 Thread lei w (Jira)
lei w created HDFS-16126: Summary: VolumePair should override hashcode() method Key: HDFS-16126 URL: https://issues.apache.org/jira/browse/HDFS-16126 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-07-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: HDFS-16083.005.patch Status: Patch Available (was: Open) > Forbid Observer NameNode trigger

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-07-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Status: Open (was: Patch Available) > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-07-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: (was: HDFS-16083.005.patch) > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-07-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: HDFS-16083.005.patch > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-07-09 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: (was: HDFS-16083.005.patch) > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Updated] (HDFS-16097) Datanode receives ipc requests will throw NPE when datanode quickly restart

2021-07-07 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16097: - Description: Datanode receives ipc requests will throw NPE when datanode quickly restart. This is because when

[jira] [Commented] (HDFS-16097) Datanode receives ipc requests will throw NPE when datanode quickly restart

2021-07-02 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373287#comment-17373287 ] lei w commented on HDFS-16097: -- Thanks [~hexiaoqiao] for your comment. I did not look for the consequences

[jira] [Commented] (HDFS-16101) Remove unuse variable and IoException in ProvidedStorageMap

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372310#comment-17372310 ] lei w commented on HDFS-16101: -- Thanks [~ayushtkn] for your reply. > Remove unuse variable and IoException

[jira] [Commented] (HDFS-16102) Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to save time

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372307#comment-17372307 ] lei w commented on HDFS-16102: -- Thanks [~hexiaoqiao] for your reply. I will update it. > Remove redundant

[jira] [Commented] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372039#comment-17372039 ] lei w commented on HDFS-16083: -- Thanks [~LiJinglun] reply. Take your suggestion and make some changes in v05

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: HDFS-16083.005.patch > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Commented] (HDFS-16101) Remove unuse variable and IoException in ProvidedStorageMap

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371998#comment-17371998 ] lei w commented on HDFS-16101: -- [~ayushsaxena] Could you give me some advice? > Remove unuse variable and

[jira] [Updated] (HDFS-16102) Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to save time

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16102: - Attachment: HDFS-16102.001.patch > Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to

[jira] [Updated] (HDFS-16102) Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to save time

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16102: - Description: The current logic in removeBlocksAssociatedTo(...) is as follows: {code:java} void

[jira] [Created] (HDFS-16102) Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to save time

2021-06-30 Thread lei w (Jira)
lei w created HDFS-16102: Summary: Remove redundant iteration in BlockManager#removeBlocksAssociatedTo(...) to save time Key: HDFS-16102 URL: https://issues.apache.org/jira/browse/HDFS-16102 Project: Hadoop

[jira] [Updated] (HDFS-16101) Remove unuse variable and IoException in ProvidedStorageMap

2021-06-30 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16101: - Attachment: HDFS-16101.001.patch > Remove unuse variable and IoException in ProvidedStorageMap >

[jira] [Created] (HDFS-16101) Remove unuse variable and IoException in ProvidedStorageMap

2021-06-30 Thread lei w (Jira)
lei w created HDFS-16101: Summary: Remove unuse variable and IoException in ProvidedStorageMap Key: HDFS-16101 URL: https://issues.apache.org/jira/browse/HDFS-16101 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-29 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371403#comment-17371403 ] lei w commented on HDFS-16083: -- Add test in HDFS-16083.003.patch > Forbid Observer NameNode trigger

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-29 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: HDFS-16083.003.patch > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Updated] (HDFS-16097) Datanode receives ipc requests will throw NPE when datanode quickly restart

2021-06-29 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16097: - Attachment: HDFS-16097.001.patch > Datanode receives ipc requests will throw NPE when datanode quickly restart

[jira] [Created] (HDFS-16097) Datanode receives ipc requests will throw NPE when datanode quickly restart

2021-06-29 Thread lei w (Jira)
lei w created HDFS-16097: Summary: Datanode receives ipc requests will throw NPE when datanode quickly restart Key: HDFS-16097 URL: https://issues.apache.org/jira/browse/HDFS-16097 Project: Hadoop HDFS

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Description: When the Observer NameNode is turned on in the cluster, the Active NameNode will receive

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Description: When the Observer NameNode is turned on in the cluster, the Active NameNode will receive

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Description: When the Observer NameNode is turned on in the cluster, the Active NameNode will receive

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Description: When the Observer NameNode is turned on in the cluster, the Active NameNode will receive

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Description: When the Observer NameNode is turned on in the cluster, the Active NameNode will receive

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: activeRollEdits.png > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-24 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Description: When the Observer NameNode is turned on in the cluster, the Active NameNode will receive

[jira] [Commented] (HDFS-16081) List a large directory, the client waits for a long time

2021-06-23 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368075#comment-17368075 ] lei w commented on HDFS-16081: -- OK,Thanks [~ste...@apache.org] reply. > List a large directory, the client

[jira] [Commented] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-23 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368017#comment-17368017 ] lei w commented on HDFS-16083: -- Thanks [~LiJinglun] reply ,I will add more information and add unit tests

[jira] [Commented] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-23 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367906#comment-17367906 ] lei w commented on HDFS-16083: -- [~ayushsaxena] , [~LiJinglun] anyone have any suggestions? > Forbid

[jira] [Commented] (HDFS-16081) List a large directory, the client waits for a long time

2021-06-23 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367889#comment-17367889 ] lei w commented on HDFS-16081: -- Thanks [~ste...@apache.org] comment.  We use original DFSClient API , not

[jira] [Updated] (HDFS-16081) List a large directory, the client waits for a long time

2021-06-23 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16081: - Description: When we list a large directory, we need to wait a lot of time. This is because the NameNode only

[jira] [Commented] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-22 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367858#comment-17367858 ] lei w commented on HDFS-16083: -- Thanks [~tomscut] comment. Changed and add new patch HDFS-16083.002.patch >

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-22 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: HDFS-16083.002.patch > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Updated] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16083: - Attachment: HDFS-16083.001.patch > Forbid Observer NameNode trigger active namenode log roll >

[jira] [Created] (HDFS-16083) Forbid Observer NameNode trigger active namenode log roll

2021-06-21 Thread lei w (Jira)
lei w created HDFS-16083: Summary: Forbid Observer NameNode trigger active namenode log roll Key: HDFS-16083 URL: https://issues.apache.org/jira/browse/HDFS-16083 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-16081) List a large directory, the client waits for a long time

2021-06-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16081: - Description: When we list a large directory, we need to wait a lot of time. This is because the NameNode only

[jira] [Comment Edited] (HDFS-16081) List a large directory, the client waits for a long time

2021-06-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17366440#comment-17366440 ] lei w edited comment on HDFS-16081 at 6/21/21, 12:46 PM: - Our cluster

[jira] [Commented] (HDFS-16081) List a large directory, the client waits for a long time

2021-06-21 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17366440#comment-17366440 ] lei w commented on HDFS-16081: -- Our cluster dfs.ls.limit is not configured. In many cases, there will be

[jira] [Created] (HDFS-16081) List a large directory, the client waits for a long time

2021-06-21 Thread lei w (Jira)
lei w created HDFS-16081: Summary: List a large directory, the client waits for a long time Key: HDFS-16081 URL: https://issues.apache.org/jira/browse/HDFS-16081 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-16073) Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB

2021-06-16 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364098#comment-17364098 ] lei w commented on HDFS-16073: -- [~ayushtkn] thanks for your comment . Add HDFS-16073.001.patch to adjust it

[jira] [Updated] (HDFS-16073) Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB

2021-06-16 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16073: - Attachment: HDFS-16073.001.patch > Remove redundant RPC requests for getFileLinkInfo in >

[jira] [Commented] (HDFS-16073) Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB

2021-06-15 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364049#comment-17364049 ] lei w commented on HDFS-16073: --  [~hexiaoqiao],  [~ayushsaxena], [~zhuqi] anyone have any suggestions? >

[jira] [Updated] (HDFS-16073) Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB

2021-06-15 Thread lei w (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16073: - Attachment: HDFS-16073.patch > Remove redundant RPC requests for getFileLinkInfo in >

  1   2   >