[jira] [Updated] (HDFS-11492) Ozone: Add the ability to handle sendContainerReport Command
[ https://issues.apache.org/jira/browse/HDFS-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11492: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Target Version/s: HDFS-7240 Status: Resolved (was: Patch Available) [~yuanbo] and [~xyao] Thank you for the reviews. I have committed this to the feature branch. > Ozone: Add the ability to handle sendContainerReport Command > > > Key: HDFS-11492 > URL: https://issues.apache.org/jira/browse/HDFS-11492 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Fix For: HDFS-7240 > > Attachments: HDFS-11492-HDFS-7240.001.patch, > HDFS-11492-HDFS-7240.002.patch, HDFS-11492-HDFS-7240.003.patch > > > Once a container report is ready datanodes sends that information to SCM. SCM > returns a command asking the data node to send container report. Add the > ability to handle this command on datanode side and send the actual container > Report. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11492) Ozone: Add the ability to handle sendContainerReport Command
[ https://issues.apache.org/jira/browse/HDFS-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11492: Attachment: HDFS-11492-HDFS-7240.003.patch Thank you for your comments [~xyao]. The new patch addresses all comments except one. bq. Line 622-632: NIT: a good candidate to use JDK8 stream/filter/map? fixed. bq. Line 276: NIT: typo in commandProcessThead fixed. bq. Line 280: should we limit the maximum restart attempts? I am very reluctant to do this, because we have no safe way of only disabling this thread. That is if statemachine is running, we must have command processor also running. The hope is that if we get errors in some handler, that handler will handle it correctly and set the shutdown flag for errors which are not recoverable. if the unhandledException handler is getting run, that is probably a Bug. We can add a counter to monitor this condition in data nodes via JMX later. > Ozone: Add the ability to handle sendContainerReport Command > > > Key: HDFS-11492 > URL: https://issues.apache.org/jira/browse/HDFS-11492 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11492-HDFS-7240.001.patch, > HDFS-11492-HDFS-7240.002.patch, HDFS-11492-HDFS-7240.003.patch > > > Once a container report is ready datanodes sends that information to SCM. SCM > returns a command asking the data node to send container report. Add the > ability to handle this command on datanode side and send the actual container > Report. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11492) Ozone: Add the ability to handle sendContainerReport Command
[ https://issues.apache.org/jira/browse/HDFS-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11492: Attachment: HDFS-11492-HDFS-7240.002.patch > Ozone: Add the ability to handle sendContainerReport Command > > > Key: HDFS-11492 > URL: https://issues.apache.org/jira/browse/HDFS-11492 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11492-HDFS-7240.001.patch, > HDFS-11492-HDFS-7240.002.patch > > > Once a container report is ready datanodes sends that information to SCM. SCM > returns a command asking the data node to send container report. Add the > ability to handle this command on datanode side and send the actual container > Report. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11492) Ozone: Add the ability to handle sendContainerReport Command
[ https://issues.apache.org/jira/browse/HDFS-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11492: Attachment: (was: HDFS-11492-HDFS-7240.002.patch) > Ozone: Add the ability to handle sendContainerReport Command > > > Key: HDFS-11492 > URL: https://issues.apache.org/jira/browse/HDFS-11492 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11492-HDFS-7240.001.patch > > > Once a container report is ready datanodes sends that information to SCM. SCM > returns a command asking the data node to send container report. Add the > ability to handle this command on datanode side and send the actual container > Report. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11492) Ozone: Add the ability to handle sendContainerReport Command
[ https://issues.apache.org/jira/browse/HDFS-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11492: Attachment: HDFS-11492-HDFS-7240.002.patch [~yuanbo] Thank you for your kind comments. This patch fixes all issues mentioned by you. bq. Would it be better to init command handler thread in start method, after container.start() Fixed. bq. Suggest to add Thread.sleep here. Thank you, good catch. Done. bq. Currently I don't foresee the purpose of counting command number here, would you please elaborate a bit. Eventually exposed as a counter. Right now plan is to use in tests in the next few patches. bq. Would you mind renaming it to CommandHandler.java or something else. There is already another Handler.java in ozone world. Fixed. bq. We have filtered the opened container data in container.getContainerReports();, so I think there is no need to check whether it's opened in the loop, right? But I'm ok if you wanna keep it. fixed, it was meant to be invariant to help people who are reading code in future. Removed. bq.'return' is unnecessary here. Thank you, fixed. bq. should be handlerMap.containsKey(h.getCommandType()) Thanks, fixed. bq. What's your plan of calculating the size of a container? Count the keys and add up the size of chunks which make up the container. bq. Please use new Handler[handlerList.size()] instead. Fixed. > Ozone: Add the ability to handle sendContainerReport Command > > > Key: HDFS-11492 > URL: https://issues.apache.org/jira/browse/HDFS-11492 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11492-HDFS-7240.001.patch, > HDFS-11492-HDFS-7240.002.patch > > > Once a container report is ready datanodes sends that information to SCM. SCM > returns a command asking the data node to send container report. Add the > ability to handle this command on datanode side and send the actual container > Report. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11492) Ozone: Add the ability to handle sendContainerReport Command
[ https://issues.apache.org/jira/browse/HDFS-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11492: Status: Patch Available (was: Open) > Ozone: Add the ability to handle sendContainerReport Command > > > Key: HDFS-11492 > URL: https://issues.apache.org/jira/browse/HDFS-11492 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11492-HDFS-7240.001.patch > > > Once a container report is ready datanodes sends that information to SCM. SCM > returns a command asking the data node to send container report. Add the > ability to handle this command on datanode side and send the actual container > Report. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11492) Ozone: Add the ability to handle sendContainerReport Command
[ https://issues.apache.org/jira/browse/HDFS-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11492: Attachment: HDFS-11492-HDFS-7240.001.patch > Ozone: Add the ability to handle sendContainerReport Command > > > Key: HDFS-11492 > URL: https://issues.apache.org/jira/browse/HDFS-11492 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11492-HDFS-7240.001.patch > > > Once a container report is ready datanodes sends that information to SCM. SCM > returns a command asking the data node to send container report. Add the > ability to handle this command on datanode side and send the actual container > Report. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
