[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Attachment: HDFS-1973-HDFS-1623.patch Trivial change from the last patch to get rid of the *new* findbugs warning. Third time's a charm? :) HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Suresh Srinivas Assignee: Aaron T. Myers Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Resolution: Fixed Fix Version/s: HA branch (HDFS-1623) Target Version/s: HA branch (HDFS-1623) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks a lot for the several reviews, Todd. I've just committed this to the HA branch. HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: HA branch (HDFS-1623) Reporter: Suresh Srinivas Assignee: Aaron T. Myers Fix For: HA branch (HDFS-1623) Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Affects Version/s: HA branch (HDFS-1623) HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: HA branch (HDFS-1623) Reporter: Suresh Srinivas Assignee: Aaron T. Myers Fix For: HA branch (HDFS-1623) Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Attachment: HDFS-1973-HDFS-1623.patch Here's an updated patch which takes advantage of HADOOP-7717 by no longer needing to check for concurrent fail-overs in the event of concurrent method invocation failures. HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Suresh Srinivas Assignee: Aaron T. Myers Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[
https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-1973:
-
Attachment: HDFS-1973-HDFS-1623.patch
Here's an updated patch which reuses {{AppendTestUtil.(write|check)}}. Note
that I also had to add the {{@Idempotent}} annotation to
{{ClientProtocol.getFileInfo}} since {{AppendTestUtil.check}} calls
{{FileSystem.getFileStatus}}.
HA: HDFS clients must handle namenode failover and switch over to the new
active namenode.
--
Key: HDFS-1973
URL: https://issues.apache.org/jira/browse/HDFS-1973
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Suresh Srinivas
Assignee: Aaron T. Myers
Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch,
HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch,
HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch
During failover, a client must detect the current active namenode failure and
switch over to the new active namenode. The switch over might make use of IP
failover or some thing more elaborate such as zookeeper to discover the new
active.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Attachment: HDFS-1973-HDFS-1623.patch Trivial change from the last patch to remove the findbugs warning. HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Suresh Srinivas Assignee: Aaron T. Myers Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Attachment: HDFS-1973-HDFS-1623.patch Thanks a lot for the review, Todd. Here's a patch which addresses your latest comments. bq. Yes, because unfortunately the Private annotation was added after the 0.20 release. If it's problematic to keep around, we don't have to, but it seemed easy enough to maintain for now. That's fair. I've put it back. bq. This code confuses me – isn't currentProxy a proxy object, whereas proxies.get(...) is an AddressRpcProxyPair? Which is to say they're always un-equal? Yep, good catch. Fixed. HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Suresh Srinivas Assignee: Aaron T. Myers Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Status: Patch Available (was: Open) HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Suresh Srinivas Assignee: Aaron T. Myers Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Attachment: HDFS-1973-HDFS-1623.patch HA: HDFS clients must handle namenode failover and switch over to the new active namenode. -- Key: HDFS-1973 URL: https://issues.apache.org/jira/browse/HDFS-1973 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Suresh Srinivas Assignee: Aaron T. Myers Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch During failover, a client must detect the current active namenode failure and switch over to the new active namenode. The switch over might make use of IP failover or some thing more elaborate such as zookeeper to discover the new active. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[
https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-1973:
-
Attachment: HDFS-1973-HDFS-1623.patch
Here's an updated patch which builds upon the previous patch and addresses all
of the issues I mentioned above.
The key differences between this patch and the previous:
# Take advantage of the changes in HADOOP-7635 and HDFS-2337 to enable cleanup
of RPC connection resources.
# Add the @Idempotent annotation to more methods of {{ClientProtocol}}.
# Add some appropriate synchronization to {{ConfiguredFailoverProxyProvider}}.
Please review this patch for potential inclusion in the HDFS-1623 branch.
HA: HDFS clients must handle namenode failover and switch over to the new
active namenode.
--
Key: HDFS-1973
URL: https://issues.apache.org/jira/browse/HDFS-1973
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Suresh Srinivas
Assignee: Aaron T. Myers
Attachments: HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch
During failover, a client must detect the current active namenode failure and
switch over to the new active namenode. The switch over might make use of IP
failover or some thing more elaborate such as zookeeper to discover the new
active.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[
https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-1973:
-
Attachment: hdfs-1973.0.patch
Here's a preliminary patch (not intended for commit) to give people an idea how
this will work.
The main things this is missing before it could reasonably be committed are:
# It currently doesn't handle clean-up of fail-over client resources at all.
The way RPC resource cleanup currently works is by looking up the appropriate
RPCEngine given a protocol class, and leaving it up to that class's
InvocationHandler. This implicitly assumes that there is a one-to-one mapping
from protocol class - invocation handler, which is no longer true. It's not
obvious to me at the moment what's the best way to deal with this.
# Currently only one of the {{ClientProtocol}} methods is annotated with the
@Idempotent annotation.
# It currently doesn't handle concurrent connections at all.
HA: HDFS clients must handle namenode failover and switch over to the new
active namenode.
--
Key: HDFS-1973
URL: https://issues.apache.org/jira/browse/HDFS-1973
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Suresh Srinivas
Assignee: Aaron T. Myers
Attachments: hdfs-1973.0.patch
During failover, a client must detect the current active namenode failure and
switch over to the new active namenode. The switch over might make use of IP
failover or some thing more elaborate such as zookeeper to discover the new
active.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
