[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1700275531 I submit the pr many times and [hadoop-yetus] always report the same file hadoop-hdfs-project/hadoop-hdfs/hs_err_pid52701.log. Does the hs_err_pid52701.log not delete after new submit？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1683622535 > If true, is it enough to make IBR and FBR asynchronously together and use only one thread? Thanks. I tried and I've given up. The reason are as follows: IBR and FBR are not on the same dispatching frequency. i)FBR should be sent immediately when heartbeat gets variable fullBlockReportLeaseId ii) IBR should be scheduled with waitTillNextIBR(heartbeatTime) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1663510246 Revert the previous commit because of problems. Can we review the ibr and fbr in different thread first？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1657869992 Apache Yetus(jenkins) error: mvninstallCould not transfer artifact org.codehaus.mojo:extra-enforcer-rules:pom:1.5.1 from/to central (https://repo.maven.apache.org/maven2): Transfer failed for https://repo.maven.apache.org/maven2/org/codehaus/mojo/extra-enforcer-rules/1.5.1/extra-enforcer-rules-1.5.1.pom: Connection reset -> [Help 1] It could be a network connection problem. How can I trigger compilation without change the code ？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1655422522 > Suggest to submit at one PR (rather than different PR) , thus we could focus and discuss at the same place. OK , done. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1655248903 > If true, is it enough to make IBR and FBR asynchronously together and use only one thread? Thanks. Please review https://github.com/apache/hadoop/pull/5898 , make IBR and FBR into one thread. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652883289 > If true, is it enough to make IBR and FBR asynchronously together and use only one thread? Variable fullBlockReportLeaseId updates from heartbeat and used by fbr and then update to 0 . It is difficulty to do all in one thread. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652806518 @Hexiaoqiao ， well, I user original sendIBRLock to make right order for ibr and fbr. The FBR will execute ibr first, so the sendIBRLock will be successful for it. Please review for it, Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1651805316 Yes，I know it. It is a problem with mis-order. a. heartbeat got a delete cmd on replication named r b. generate fbr c. datanode delete repliction r and send ibr d. send fbr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1651437873 And maybe it not a problem. The FBR will only process the reported block list，the replication r will not discard. FBR finally execute BlockManager.processReportedBlock method , just as ibr addblock does. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1651382745 @Hexiaoqiao ，if the order of ibr and fbr is must be guaranteed at datanode side， [HDFS-16016](https://issues.apache.org/jira/browse/HDFS-16016) also can not guarantee it. Thus if it is import the order of br rpc , a idea is that I put the ibr ,fbr into a queue with the right order. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1651228974 > Good improvement. Not think carefully but my first feeling, IBR - FBR mis-order could trigger some issues, such as miss some block report? One case, a. generate full block report, b. received one replica named `r` and send IBR, c. send full block report; And NameNode process FBF and IBR as the above order, then NameNode will consider that this DataNode does not include replica `r`, right? > > Thanks. In the NamenodeRpcServer side，the ibr and fbr will both into BlockManager.BlockReportProcessingThread queue . I think the order of ibr and fbr cannot guarantee in the previous hadoop version.  blockreport will be processed with one after one. But the ibrs order is not guaranteed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
