[ https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14575058#comment-14575058 ]
Colin Patrick McCabe commented on HDFS-7923: -------------------------------------------- bq. Should the checkLease logs be done to the blockLog? We log the startup error log there in processReport I like having the ability to turn on and off TRACE logging for various subsystems. Putting everything in the blockLog would make that harder, right? bq. Update javadoc in BlockReportContext with what leaseID is for. added bq. Add something to the log message about overwriting the old leaseID in offerService. Agree that this shouldn't really trigger, but good defensive coding practice ok bq. DatanodeManager, there's still a register/unregister in registerDatanode I think we could skip. This is the node restart case where it's registered previously. Good catch. The calls in {{removeDatanode}} and {{addDatanode}} should take care of this, so there's no need to have it here. bq. BRLManager requestLease, we auto-register the node on requestLease. This shouldn't happen since DNs need to register before doing anything else. We can keep this here I added a warn message since this shouldn't happen. bq. Still need documentation of new config keys in hdfs-default.xml added bq. Extra import in TestBPSAScheduler and BPSA removed bq. If you want to pursue \[the block report timing\] logic change more, let's split it out into a follow-on JIRA. The rest LGTM, +1 pending above comments. OK, I will restore the old behavior for now, and we can do this in a follow-on change. > The DataNodes should rate-limit their full block reports by asking the NN on > heartbeat messages > ----------------------------------------------------------------------------------------------- > > Key: HDFS-7923 > URL: https://issues.apache.org/jira/browse/HDFS-7923 > Project: Hadoop HDFS > Issue Type: Sub-task > Affects Versions: 2.8.0 > Reporter: Colin Patrick McCabe > Assignee: Colin Patrick McCabe > Attachments: HDFS-7923.000.patch, HDFS-7923.001.patch, > HDFS-7923.002.patch, HDFS-7923.003.patch, HDFS-7923.004.patch > > > The DataNodes should rate-limit their full block reports. They can do this > by first sending a heartbeat message to the NN with an optional boolean set > which requests permission to send a full block report. If the NN responds > with another optional boolean set, the DN will send an FBR... if not, it will > wait until later. This can be done compatibly with optional fields. -- This message was sent by Atlassian JIRA (v6.3.4#6332)