[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11915: --- Fix Version/s: 2.9.1 > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.1 > > Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch, > HDFS-11915-branch-2-02.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11915: --- Fix Version/s: 2.10.0 > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Fix For: 3.1.0, 2.10.0, 3.0.1 > > Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch, > HDFS-11915-branch-2-02.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11915: --- Fix Version/s: (was: 2.9.1) > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Fix For: 3.1.0, 2.10.0, 3.0.1 > > Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch, > HDFS-11915-branch-2-02.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11915: --- Fix Version/s: 3.0.1 > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Fix For: 3.1.0, 2.9.1, 3.0.1 > > Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch, > HDFS-11915-branch-2-02.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11915: --- Resolution: Fixed Fix Version/s: 2.9.1 Status: Resolved (was: Patch Available) Committed the patch to branch-2 for 2.9.1 release. Thanks [~vinayrpet]! > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch, > HDFS-11915-branch-2-02.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[
https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated HDFS-11915:
-
Attachment: HDFS-11915-branch-2-02.patch
Attached the updated patch for branch-2.
Moved {{FsDatasetImpl#fsyncDirectory()}} to {{DatanodeUtil}} and re-used same.
Please review
> Sync rbw dir on the first hsync() to avoid file lost on power failure
> -
>
> Key: HDFS-11915
> URL: https://issues.apache.org/jira/browse/HDFS-11915
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Kanaka Kumar Avvaru
>Assignee: Vinayakumar B
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch,
> HDFS-11915-branch-2-02.patch
>
>
> As discussed in HDFS-5042, there is a chance to lose blocks on power failure
> if rbw file creation entry is not yet sync to device. Then the block created
> is nowhere exists on disk. Neither in rbw nor in finalized.
> As suggested by [~kihwal], will discuss and track it in this JIRA.
> As suggested by [~vinayrpet], May be first hsync() request on block file can
> call fsync on its parent directory (rbw) directory.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11915: --- Fix Version/s: 3.1.0 > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Fix For: 3.1.0 > > Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11915: - Attachment: HDFS-11915-branch-2-01.patch > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Attachments: HDFS-11915-01.patch, HDFS-11915-branch-2-01.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11915: -- Description: As discussed in HDFS-5042, there is a chance to lose blocks on power failure if rbw file creation entry is not yet sync to device. Then the block created is nowhere exists on disk. Neither in rbw nor in finalized. As suggested by [~kihwal], will discuss and track it in this JIRA. As suggested by [~vinayrpet], May be first hsync() request on block file can call fsync on its parent directory (rbw) directory. was: As discussed in HDFS-5042, there is a chance to loose blocks on power failure if rbw file creation entry is not yet sync to device. Then the block created is nowhere exists on disk. Neither in rbw nor in finalized. As suggested by [~kihwal], will discuss and track it in this JIRA. As suggested by [~vinayrpet], May be first hsync() request on block file can call fsync on its parent directory (rbw) directory. > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Attachments: HDFS-11915-01.patch > > > As discussed in HDFS-5042, there is a chance to lose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11915: - Status: Patch Available (was: Open) > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Attachments: HDFS-11915-01.patch > > > As discussed in HDFS-5042, there is a chance to loose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11915) Sync rbw dir on the first hsync() to avoid file lost on power failure
[ https://issues.apache.org/jira/browse/HDFS-11915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-11915: - Attachment: HDFS-11915-01.patch Attached the patch for trunk. Please review. If this is fine, I will prepare patches for branch-2* > Sync rbw dir on the first hsync() to avoid file lost on power failure > - > > Key: HDFS-11915 > URL: https://issues.apache.org/jira/browse/HDFS-11915 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Kanaka Kumar Avvaru >Assignee: Vinayakumar B >Priority: Critical > Attachments: HDFS-11915-01.patch > > > As discussed in HDFS-5042, there is a chance to loose blocks on power failure > if rbw file creation entry is not yet sync to device. Then the block created > is nowhere exists on disk. Neither in rbw nor in finalized. > As suggested by [~kihwal], will discuss and track it in this JIRA. > As suggested by [~vinayrpet], May be first hsync() request on block file can > call fsync on its parent directory (rbw) directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
