[jira] [Comment Edited] (HDFS-11448) JN log segment syncing should support HA upgrade

2022-06-10 Thread JiangHua Zhu (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545848#comment-17545848
 ] 

JiangHua Zhu edited comment on HDFS-11448 at 6/10/22 7:15 AM:
--

Hi [~hanishakoneru], nice to communicate with you.
In JNStorage, getCurrentDir() is not used anywhere.
If you don't mind, I'll remove JNStorage#getCurrentDir() which is not used.


was (Author: jianghuazhu):
Hi [~hanishakoneru], nice to communicate with you.
I found the new addition of JNStorage#getCurrentDir() here, and yes, that's 
good because sd.getCurrentDir() is used in multiple places in the context, but 
there is no use of it anywhere.
If you don't mind, I'll modify this to replace sd.getCurrentDir() with 
JNStorage#getCurrentDir().


> JN log segment syncing should support HA upgrade
> 
>
> Key: HDFS-11448
> URL: https://issues.apache.org/jira/browse/HDFS-11448
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Hanisha Koneru
>Assignee: Hanisha Koneru
>Priority: Major
> Fix For: 3.0.0-alpha4
>
> Attachments: HDFS-11448.001.patch, HDFS-11448.002.patch, 
> HDFS-11448.003.patch
>
>
> HDFS-4025 adds support for sychronizing past log segments to JNs that missed 
> them. But, as pointed out by [~jingzhao], if the segment download happens 
> when an admin tries to rollback, it might fail ([see 
> comment|https://issues.apache.org/jira/browse/HDFS-4025?focusedCommentId=15850633&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15850633]).



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Arpit Agarwal (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995890#comment-15995890
 ] 

Arpit Agarwal edited comment on HDFS-11448 at 5/3/17 11:20 PM:
---

Thanks for updating the patch [~hanishakoneru]. The v2 patch looks good. A few 
comments:
# {{editsSyncDir.delete();}} should probably use {{FileUtil#fullyDelete}} 
instead.
# The loop in {{createEditsSyncDir}} should be deleted.
{code}
// Wait for journal node to get formatted
while (!journal.getStorage().isFormatted()) {

}
{code}
# The syncer thread should not swallow InterruptedException. At least after 
catching InterruptedException it should invoke 
{{Thread.currentThread.interrupt()}} and return immediately. This needs to be 
fixed on lines 161, 181 and 189. Also there is a missing return after line 161.
# A minor comment: STORAGE_EDITS_SYNC looks out of place in the class 
StorageDirectory. Instead you can rewrite JNStorage#getEditsSyncDir as:
{code}
  File getEditsSyncDir() {
return new File(sd.getRoot(), STORAGE_EDITS_SYNC);
  }
{code}


was (Author: arpitagarwal):
Thanks for updating the patch [~hanishakoneru]. The v2 patch looks good. A few 
comments:
# {{editsSyncDir.delete();}} should probably use {{FileUtil#fullyDelete}} 
instead.
# The loop in {{createEditsSyncDir}} should be deleted.
{code}
// Wait for journal node to get formatted
while (!journal.getStorage().isFormatted()) {

}
{code}
# The syncer thread should not swallow InterruptedException. At least after 
catching InterruptedException it should invoke 
{{Thread.currentThread.interrupt()}} and return immediately. This needs to be 
fixed on lines 161, 181 and 189. Also there is a missing return after line 161.
# A minor comment: STORAGE_EDITS_SYNC looks out of place in the class 
StorageDirectory. Instead you can rewrite JNStorage#getEditsSyncDir as:
{code}
  File getEditsSyncDir() {
return sd.getRoot() + STORAGE_EDITS_SYNC;
  }
{code}

> JN log segment syncing should support HA upgrade
> 
>
> Key: HDFS-11448
> URL: https://issues.apache.org/jira/browse/HDFS-11448
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Hanisha Koneru
>Assignee: Hanisha Koneru
> Attachments: HDFS-11448.001.patch, HDFS-11448.002.patch
>
>
> HDFS-4025 adds support for sychronizing past log segments to JNs that missed 
> them. But, as pointed out by [~jingzhao], if the segment download happens 
> when an admin tries to rollback, it might fail ([see 
> comment|https://issues.apache.org/jira/browse/HDFS-4025?focusedCommentId=15850633&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15850633]).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-04-20 Thread Hanisha Koneru (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977854#comment-15977854
 ] 

Hanisha Koneru edited comment on HDFS-11448 at 4/21/17 12:25 AM:
-

Patch v01 adds support for JN sync during upgrade/ rollback.
Instead of downloading the log segments to _current_ directory, we download 
them to a temporary directory (_current.tmp_). Once the download is complete, 
we atomically move the log segment to current directory. 

cc. [~jingzhao]


was (Author: hanishakoneru):
Patch v01 adds support for JN sync during upgrade/ rollback.
Instead of downloading the log segments to _current_ directory, we download 
them to a temporary directory (_current.tmp_). Once the download is complete, 
we atomically move the log segment to current directory. 

> JN log segment syncing should support HA upgrade
> 
>
> Key: HDFS-11448
> URL: https://issues.apache.org/jira/browse/HDFS-11448
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Hanisha Koneru
>Assignee: Hanisha Koneru
> Attachments: HDFS-11448.001.patch
>
>
> HDFS-4025 adds support for sychronizing past log segments to JNs that missed 
> them. But, as pointed out by [~jingzhao], if the segment download happens 
> when an admin tries to rollback, it might fail ([see 
> comment|https://issues.apache.org/jira/browse/HDFS-4025?focusedCommentId=15850633&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15850633]).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org