[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201568#comment-13201568 ] Todd Lipcon commented on HDFS-2752: --- The other issue is that, in order to support multiple shared edits, we'd need a quorum-like behavior rather than the current "at least one" behavior. Otherwise you could imagine that, with two shared dirs (SD1 and SD2) and two NNs, you might have the case where NN1 is writing to only SD1 and NN2 is reading from only SD2. Let's continue this discussion on HDFS-2782 > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > Attachments: hdfs-2752.txt > > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201552#comment-13201552 ] Jitendra Nath Pandey commented on HDFS-2752: bq. I agree with Eli - we don't currently use the JournalSet abstraction in EditLogTailer, so it can only use a single shared dir. Of course in the future we should support using multiple, but it adds some complexity to the initial release. EditLogTailer uses FSEditLog which uses JournalSet. I think it should be able to handle multiple shared edits, unless there is another bug. > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > Attachments: hdfs-2752.txt > > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201259#comment-13201259 ] Hudson commented on HDFS-2752: -- Integrated in Hadoop-Hdfs-HAbranch-build #70 (See [https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/70/]) HDFS-2752. HA: exit if multiple shared dirs are configured. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240916 Files : * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-1623.txt * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystem.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestFailureOfSharedDir.java * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHAConfiguration.java > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > Attachments: hdfs-2752.txt > > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[
https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201091#comment-13201091
]
Todd Lipcon commented on HDFS-2752:
---
Just one nit: you can remove the following empty javadoc annotation:
{code}
+ * @param conf
{code}
Aside from that, +1.
> HA: exit if multiple shared dirs are configured
> ---
>
> Key: HDFS-2752
> URL: https://issues.apache.org/jira/browse/HDFS-2752
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
>Affects Versions: HA branch (HDFS-1623)
>Reporter: Eli Collins
>Assignee: Eli Collins
> Attachments: hdfs-2752.txt
>
>
> We don't support multiple shared edits dirs, we should fail to start with an
> error in this case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183052#comment-13183052 ] Todd Lipcon commented on HDFS-2752: --- I agree with Eli - we don't currently use the JournalSet abstraction in EditLogTailer, so it can only use a single shared dir. Of course in the future we should support using multiple, but it adds some complexity to the initial release. > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180953#comment-13180953 ] Eli Collins commented on HDFS-2752: --- Sorry for the lack of context, this came up in HDFS-2709. With multiple shared edits dirs a failure to read from one of them will prevent the edit log tailer from catching up, ie users are currently *less reliable* with multiple shared dirs. Until we know we've got it working reasonably well (ie have tests for the common scenarios) it doesn't seem like we should let people shoot themselves in the foot. And while nice to have, it doesn't seem like multiple shared dir support should block an initial release. Agree? Perhaps warn loudly instead of exit? > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180894#comment-13180894 ] Suresh Srinivas commented on HDFS-2752: --- Eli, I am confused a bit. What is the special required in supporting multiple shared directories that is not handled currently? Why turn it off because tests are not there? > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180859#comment-13180859 ] Eli Collins commented on HDFS-2752: --- It's not currently, but it should be until we support it (see HDFS-2735, needs tests and fixes). > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2752) HA: exit if multiple shared dirs are configured
[ https://issues.apache.org/jira/browse/HDFS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180853#comment-13180853 ] Suresh Srinivas commented on HDFS-2752: --- Why is this an error? > HA: exit if multiple shared dirs are configured > --- > > Key: HDFS-2752 > URL: https://issues.apache.org/jira/browse/HDFS-2752 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Eli Collins >Assignee: Eli Collins > > We don't support multiple shared edits dirs, we should fail to start with an > error in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
