[jira] [Updated] (HDFS-15627) Audit log deletes before collecting blocks
[ https://issues.apache.org/jira/browse/HDFS-15627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-15627: --- Fix Version/s: 3.2.2 > Audit log deletes before collecting blocks > -- > > Key: HDFS-15627 > URL: https://issues.apache.org/jira/browse/HDFS-15627 > Project: Hadoop HDFS > Issue Type: Bug > Components: logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Fix For: 3.2.2, 3.3.1, 3.4.0, 3.1.5, 3.2.3 > > Attachments: HDFS-15627.001.patch > > > Deletes currently collect blocks in the write lock, write the edit, > incrementally block delete, finally +audit log+. It should be collect blocks, > edit log, +audit log+, incremental delete. Once the edit is durable it's > consistent to audit log the delete. There is no sense in deferring the audit > into the indeterminate future. > The problem occurs when thereto server hung due to large deletes but it won't > be easy to identify the problem. That should have been easily identified as > the first delete logged after the hang. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15627) Audit log deletes before collecting blocks
[ https://issues.apache.org/jira/browse/HDFS-15627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15627: -- Fix Version/s: 3.2.3 3.1.5 3.4.0 3.3.1 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) I've committed this to trunk, branch-3.3, branch-3.2 and branch-3.1. Thanks for working on this [~ahussein]. > Audit log deletes before collecting blocks > -- > > Key: HDFS-15627 > URL: https://issues.apache.org/jira/browse/HDFS-15627 > Project: Hadoop HDFS > Issue Type: Bug > Components: logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Fix For: 3.3.1, 3.4.0, 3.1.5, 3.2.3 > > Attachments: HDFS-15627.001.patch > > > Deletes currently collect blocks in the write lock, write the edit, > incrementally block delete, finally +audit log+. It should be collect blocks, > edit log, +audit log+, incremental delete. Once the edit is durable it's > consistent to audit log the delete. There is no sense in deferring the audit > into the indeterminate future. > The problem occurs when thereto server hung due to large deletes but it won't > be easy to identify the problem. That should have been easily identified as > the first delete logged after the hang. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15627) Audit log deletes before collecting blocks
[ https://issues.apache.org/jira/browse/HDFS-15627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15627: -- Summary: Audit log deletes before collecting blocks (was: Audit log deletes after edit is written) > Audit log deletes before collecting blocks > -- > > Key: HDFS-15627 > URL: https://issues.apache.org/jira/browse/HDFS-15627 > Project: Hadoop HDFS > Issue Type: Bug > Components: logging, namenode >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Attachments: HDFS-15627.001.patch > > > Deletes currently collect blocks in the write lock, write the edit, > incrementally block delete, finally +audit log+. It should be collect blocks, > edit log, +audit log+, incremental delete. Once the edit is durable it's > consistent to audit log the delete. There is no sense in deferring the audit > into the indeterminate future. > The problem occurs when thereto server hung due to large deletes but it won't > be easy to identify the problem. That should have been easily identified as > the first delete logged after the hang. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
