[jira] Created: (HDFS-1379) Multihoming brokenness in HDFS
Multihoming brokenness in HDFS -- Key: HDFS-1379 URL: https://issues.apache.org/jira/browse/HDFS-1379 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client, name-node Affects Versions: 0.20.1 Environment: Multi-homed namenode and datanodes. hadoop-0.20.1 (cloudera distribution on linux) Reporter: Matthew Byng-Maddick We have a setup where - because we only have a very few machines (4 x 16 core) we're looking at co-locating namenodes and datanodes. We also have front-end and back-end networks. Set-up is something like: * machine1 ** front-end 10.18.80.80 ** back-end 192.168.24.40 * machine2 ** front-end 10.18.80.82 ** back-end 192.168.24.41 * machine3 ** front-end 10.18.80.84 ** back-end 192.168.24.42 * machine4 ** front-end 10.18.80.86 ** back-end 192.168.24.43 On each, the property *slave.host.name* is configured with the 192. address, (the *.dns.interface settings don't actually seem to help, but that's a separate problem), and the *dfs.datanode.address* is bound to the 192.168.24.x address on :50010, similarly the *dfs.datanode.ipc.address* is bound there. In order to get efficient use of our machines, we bring up a namenode on one of them (this then rsyncs the latest namenode fsimage etc) by bringing up a VIP on each side (we use the 10.18.80.x side for monitoring, rather than actual hadoop comms), and binding the namenode to that - on the inside this is 192.168.24.19. The namenode now knows about 4 datanodes - 192.168.24.40/1/2/3. These datanodes know how they're bound. However, when the datanode is telling an external hdfs client where to store the blocks, it gives out 192.168.24.19:50010 as one of the addresses (despite the datanode not being bound there) - because that's where the datanode->namenode RPC comes from. This is wrong because if you've bound the datanode explicitly (using *dfs.datanode.address*) then that's should be the only address the namenode can give out (it's reasonable, given your comms model not to support NAT between clients and data slaves). If you bind it to * then your normal rules for slave.host.name, dfs.datanode.dns.interface etc should take precedence. This may already be fixed in later releases than 0.20.1 - but if it isn't it should probably be - you explicitly allow binding of the datanode addresses - it's unreasonable to expect that comms to the datanode will always come from those addresses - especially in multi-homed environments (and separating traffic out by network - especially when dealing with large volumes of data) is useful. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1379) Multihoming brokenness in HDFS
[ https://issues.apache.org/jira/browse/HDFS-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906841#action_12906841 ] Allen Wittenauer commented on HDFS-1379: Some of the issues here are also covered in HADOOP-6364 . But yes, multi-homing is a known brokenness. It is probably worth pointing out that a) the bang-for-buck by having a separate network for IPC/RPC communications isn't very good, so pretty much no one does it b) monitoring a private interface instead of the public one leaves you exposed to failures on the network side > Multihoming brokenness in HDFS > -- > > Key: HDFS-1379 > URL: https://issues.apache.org/jira/browse/HDFS-1379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs client, name-node >Affects Versions: 0.20.1 > Environment: Multi-homed namenode and datanodes. hadoop-0.20.1 > (cloudera distribution on linux) >Reporter: Matthew Byng-Maddick > > We have a setup where - because we only have a very few machines (4 x 16 > core) we're looking at co-locating namenodes and datanodes. We also have > front-end and back-end networks. Set-up is something like: > * machine1 > ** front-end 10.18.80.80 > ** back-end 192.168.24.40 > * machine2 > ** front-end 10.18.80.82 > ** back-end 192.168.24.41 > * machine3 > ** front-end 10.18.80.84 > ** back-end 192.168.24.42 > * machine4 > ** front-end 10.18.80.86 > ** back-end 192.168.24.43 > On each, the property *slave.host.name* is configured with the 192. address, > (the *.dns.interface settings don't actually seem to help, but that's a > separate problem), and the *dfs.datanode.address* is bound to the > 192.168.24.x address on :50010, similarly the *dfs.datanode.ipc.address* is > bound there. > In order to get efficient use of our machines, we bring up a namenode on one > of them (this then rsyncs the latest namenode fsimage etc) by bringing up a > VIP on each side (we use the 10.18.80.x side for monitoring, rather than > actual hadoop comms), and binding the namenode to that - on the inside this > is 192.168.24.19. > The namenode now knows about 4 datanodes - 192.168.24.40/1/2/3. These > datanodes know how they're bound. However, when the datanode is telling an > external hdfs client where to store the blocks, it gives out > 192.168.24.19:50010 as one of the addresses (despite the datanode not being > bound there) - because that's where the datanode->namenode RPC comes from. > This is wrong because if you've bound the datanode explicitly (using > *dfs.datanode.address*) then that's should be the only address the namenode > can give out (it's reasonable, given your comms model not to support NAT > between clients and data slaves). If you bind it to * then your normal rules > for slave.host.name, dfs.datanode.dns.interface etc should take precedence. > This may already be fixed in later releases than 0.20.1 - but if it isn't it > should probably be - you explicitly allow binding of the datanode addresses - > it's unreasonable to expect that comms to the datanode will always come from > those addresses - especially in multi-homed environments (and separating > traffic out by network - especially when dealing with large volumes of data) > is useful. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1379) Multihoming brokenness in HDFS
[ https://issues.apache.org/jira/browse/HDFS-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906880#action_12906880 ] Matthew Byng-Maddick commented on HDFS-1379: Hi Allen, Thanks for your comments - I looked for other things first, and I didn't notice HADOOP-6364 - thanks for linking it! I'm aware that multi-homing is known not to work, but it seems to me that being able to define the bind address and then having the namenode give out something that it has constructed using incomplete information to hdfs clients is more than just a simple "multi-homing is broken" bug. In response to the points: a) I'm well aware of that - segregating the traffic with our other network is important as a lot of people have access to that network, but keeping a cluster network which is separate is sensible - it means it can be provisioned differently with potentially different QoS at switches etc. - We're not just doing IPC/RPC on that - but the entire datanode<->datanode and client<->datanode comms too - as our clients run within that network - ideally I'd have the jobtracker bind to INADDR_ANY (IN6ADDR_ANY) so you could submit a job from anywhere, but all the datanode comms work - but that isn't possible because of the dual use of mapred.job.tracker. b) correct, though what's also exposed on the (more) public side is bits of JMX, rather than just trivial availability monitoring, and that does help to point out what hadoop is seeing. At some stage I'd like to have availability monitoring on both sides, but that's not possible right now. > Multihoming brokenness in HDFS > -- > > Key: HDFS-1379 > URL: https://issues.apache.org/jira/browse/HDFS-1379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs client, name-node >Affects Versions: 0.20.1 > Environment: Multi-homed namenode and datanodes. hadoop-0.20.1 > (cloudera distribution on linux) >Reporter: Matthew Byng-Maddick > > We have a setup where - because we only have a very few machines (4 x 16 > core) we're looking at co-locating namenodes and datanodes. We also have > front-end and back-end networks. Set-up is something like: > * machine1 > ** front-end 10.18.80.80 > ** back-end 192.168.24.40 > * machine2 > ** front-end 10.18.80.82 > ** back-end 192.168.24.41 > * machine3 > ** front-end 10.18.80.84 > ** back-end 192.168.24.42 > * machine4 > ** front-end 10.18.80.86 > ** back-end 192.168.24.43 > On each, the property *slave.host.name* is configured with the 192. address, > (the *.dns.interface settings don't actually seem to help, but that's a > separate problem), and the *dfs.datanode.address* is bound to the > 192.168.24.x address on :50010, similarly the *dfs.datanode.ipc.address* is > bound there. > In order to get efficient use of our machines, we bring up a namenode on one > of them (this then rsyncs the latest namenode fsimage etc) by bringing up a > VIP on each side (we use the 10.18.80.x side for monitoring, rather than > actual hadoop comms), and binding the namenode to that - on the inside this > is 192.168.24.19. > The namenode now knows about 4 datanodes - 192.168.24.40/1/2/3. These > datanodes know how they're bound. However, when the datanode is telling an > external hdfs client where to store the blocks, it gives out > 192.168.24.19:50010 as one of the addresses (despite the datanode not being > bound there) - because that's where the datanode->namenode RPC comes from. > This is wrong because if you've bound the datanode explicitly (using > *dfs.datanode.address*) then that's should be the only address the namenode > can give out (it's reasonable, given your comms model not to support NAT > between clients and data slaves). If you bind it to * then your normal rules > for slave.host.name, dfs.datanode.dns.interface etc should take precedence. > This may already be fixed in later releases than 0.20.1 - but if it isn't it > should probably be - you explicitly allow binding of the datanode addresses - > it's unreasonable to expect that comms to the datanode will always come from > those addresses - especially in multi-homed environments (and separating > traffic out by network - especially when dealing with large volumes of data) > is useful. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1365) HDFS federation: propose ClusterID and BlockPoolID format
[ https://issues.apache.org/jira/browse/HDFS-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906894#action_12906894 ] Tanping Wang commented on HDFS-1365: ClusterID is the birthmark of cluster. It is a globally unique ID created when a cluster is created. HDFS cluster is initialized when the very first namespace volume of the cluster is created. As part of formatting a NN with the "-newCluster" option, it will generate a unique ClusterID and a unique BlockPoolID, which are persisted on the namenode.Subsequent NN must be given the same ClusterID during its format to be in the same cluster. Each DN discovers the ClusterID when it registers and from then on "sticks" to this cluster. If at any point a NN or a DN tries to join another cluster, the DNs or the NNs in that cluster will reject registration. Why do we need ClusterID? We cannot solve it merely with global BlockPoolIDs or NamespaceIDs - In federation setup, a DN talks to multiple NNs. If a DN is accidentally moved to another cluster, the DN continues to keep its old blocks and creates new block pools for the NNs in the new cluster. The NamespaceID that previously prevented such moves will not work in this case. The above content can be also found in the design doc, high-level-design.pdf of HDFS-1052. > HDFS federation: propose ClusterID and BlockPoolID format > - > > Key: HDFS-1365 > URL: https://issues.apache.org/jira/browse/HDFS-1365 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Tanping Wang >Assignee: Tanping Wang > Fix For: Federation Branch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-881) Refactor DataNode Packet header into DataTransferProtocol
[ https://issues.apache.org/jira/browse/HDFS-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-881: - Status: Resolved (was: Patch Available) Fix Version/s: 0.22.0 Resolution: Fixed I've committed this. Resolving as fixed. Thanks, Todd. > Refactor DataNode Packet header into DataTransferProtocol > - > > Key: HDFS-881 > URL: https://issues.apache.org/jira/browse/HDFS-881 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.22.0 > > Attachments: hdfs-881.txt, hdfs-881.txt > > > The Packet Header format is used ad-hoc in various places. This JIRA is to > refactor it into a class inside DataTransferProtocol (like was done with > PipelineAck) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1365) HDFS federation: propose ClusterID and BlockPoolID format
[ https://issues.apache.org/jira/browse/HDFS-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906900#action_12906900 ] Tanping Wang commented on HDFS-1365: Regarding to the VERSION files, we are thinking of the following, with federation, the VERSION file is split into the two files, 1./ID ClusterID NamespaceID BlockPoolID Type of node (NAME_NODE) 2./VERSION ctime LayoutVersion each storage directory will store these two files. The above content can be also found in the design doc, high-level-design.pdf of HDFS-1052. > HDFS federation: propose ClusterID and BlockPoolID format > - > > Key: HDFS-1365 > URL: https://issues.apache.org/jira/browse/HDFS-1365 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Tanping Wang >Assignee: Tanping Wang > Fix For: Federation Branch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1310) TestFileConcurrentReader fails
[ https://issues.apache.org/jira/browse/HDFS-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906912#action_12906912 ] sam rash commented on HDFS-1310: my apologies for the delay--i came down with a cold right before the long weekend results of test-patch: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile. > TestFileConcurrentReader fails > -- > > Key: HDFS-1310 > URL: https://issues.apache.org/jira/browse/HDFS-1310 > Project: Hadoop HDFS > Issue Type: Test >Affects Versions: 0.22.0 >Reporter: Suresh Srinivas >Assignee: sam rash > Attachments: hdfs-1310-1.txt, hdfs-1310-2.txt > > > For details of test failure see > http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/218/testReport/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1310) TestFileConcurrentReader fails
[ https://issues.apache.org/jira/browse/HDFS-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906927#action_12906927 ] Tsz Wo (Nicholas), SZE commented on HDFS-1310: -- No problem and thanks for running test-patch. Have you also run the unit tests? > TestFileConcurrentReader fails > -- > > Key: HDFS-1310 > URL: https://issues.apache.org/jira/browse/HDFS-1310 > Project: Hadoop HDFS > Issue Type: Test >Affects Versions: 0.22.0 >Reporter: Suresh Srinivas >Assignee: sam rash > Attachments: hdfs-1310-1.txt, hdfs-1310-2.txt > > > For details of test failure see > http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/218/testReport/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-829) hdfsJniHelper.c: #include is not portable
[ https://issues.apache.org/jira/browse/HDFS-829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-829: - Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Assignee: Allen Wittenauer Resolution: Fixed +1. I verified that this patch allows libhdfs to compile on OSX (although build was still not successful, I had to manually chmod +x on ./src/c++/libhdfs/install-sh to get a full build. We should open a JIRA for this). I've committed this. Resolving as fixed. Thanks, Allen. > hdfsJniHelper.c: #include is not portable > --- > > Key: HDFS-829 > URL: https://issues.apache.org/jira/browse/HDFS-829 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.21.0, 0.22.0 >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Fix For: 0.22.0 > > Attachments: HDFS-632.patch, hdfs-829.patch > > > hdfsJniHelper.c includes but this appears to be unnecessary, since > even under Linux none of the routines that are prototyped are used. Worse > yet, error.h doesn't appear to be a standard header file so this breaks on > Mac OS X and Solaris and prevents libhdfs from being built. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1310) TestFileConcurrentReader fails
[ https://issues.apache.org/jira/browse/HDFS-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906964#action_12906964 ] sam rash commented on HDFS-1310: I have not run it all the way through yet. Is it 'test' or 'test-core' these days? > TestFileConcurrentReader fails > -- > > Key: HDFS-1310 > URL: https://issues.apache.org/jira/browse/HDFS-1310 > Project: Hadoop HDFS > Issue Type: Test >Affects Versions: 0.22.0 >Reporter: Suresh Srinivas >Assignee: sam rash > Attachments: hdfs-1310-1.txt, hdfs-1310-2.txt > > > For details of test failure see > http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/218/testReport/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1310) TestFileConcurrentReader fails
[ https://issues.apache.org/jira/browse/HDFS-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906980#action_12906980 ] Tsz Wo (Nicholas), SZE commented on HDFS-1310: -- It is "ant test". > TestFileConcurrentReader fails > -- > > Key: HDFS-1310 > URL: https://issues.apache.org/jira/browse/HDFS-1310 > Project: Hadoop HDFS > Issue Type: Test >Affects Versions: 0.22.0 >Reporter: Suresh Srinivas >Assignee: sam rash > Attachments: hdfs-1310-1.txt, hdfs-1310-2.txt > > > For details of test failure see > http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/218/testReport/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1380) The append pipeline does not followed TSP principal
The append pipeline does not followed TSP principal --- Key: HDFS-1380 URL: https://issues.apache.org/jira/browse/HDFS-1380 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.20-append Reporter: Thanh Do 1. Say we have 2 racks: rack-0 and rack-1. Rack-0 has dn1, dn2, dn3. Rack-0 has dn4, dn5, dn6. 2. Suppose client is in rack-0, and the write pipeline is: client --> localnode --> other rack --> other rack In this example we have the pipeline client-dn1-dn4-dn6. That is rack0-rack0-rack1-rack1. So far so good. 3. Now other client comes, and append to file. This client is also in rack-0. Interestingly, the append pipeline is client-dn6-dn4-dn1. That is the new client (from rack0) sends packet to the first node in pipeline (dn6) which belongs to rack1. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1361) Add -fileStatus operation to NNThroughputBenchmark
[
https://issues.apache.org/jira/browse/HDFS-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906983#action_12906983
]
Konstantin Shvachko commented on HDFS-1361:
---
Tests pass.
test-patch results:
{code}
.[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] +1 tests included. The patch appears to include 6 new or
modified tests.
[exec] -1 javadoc. The javadoc tool appears to have generated 1
warning messages.
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
[exec] +1 system tests framework. The patch passed system tests
framework compile.
[exec]
==
[exec]
==
[exec] Finished build.
[exec]
==
[exec]
==
{code}
The javadoc warning is fixed by HDFS-1369.
> Add -fileStatus operation to NNThroughputBenchmark
> --
>
> Key: HDFS-1361
> URL: https://issues.apache.org/jira/browse/HDFS-1361
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 0.22.0
>Reporter: Konstantin Shvachko
>Assignee: Konstantin Shvachko
> Fix For: 0.22.0
>
> Attachments: NNThroughput-fileStatus.patch
>
>
> getFileStatus() is a frequently used operation in HDFS. It important to
> benchmark the name-node throughput on it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1380) The append pipeline does not followed TSP principal
[ https://issues.apache.org/jira/browse/HDFS-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thanh Do updated HDFS-1380: --- Description: 1. Say we have 2 racks: rack-0 and rack-1. Rack-0 has dn1, dn2, dn3. Rack-0 has dn4, dn5, dn6. 2. Suppose client is in rack-0, and the write pipeline is: client --> localnode --> other rack --> other rack In this example we have the pipeline client-dn1-dn4-dn6. That is rack0-rack0-rack1-rack1. So far so good. 3. Now other client comes, and append to file. This client is also in rack-0. Interestingly, the append pipeline is client-dn6-dn4-dn1. That is the new client (from rack0) sends packet to the first node in pipeline (dn6) which belongs to rack1. This bug was found by our Failure Testing Service framework: http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html For questions, please email us: Thanh Do (than...@cs.wisc.edu) and Haryadi Gunawi (hary...@eecs.berkeley.edu) was: 1. Say we have 2 racks: rack-0 and rack-1. Rack-0 has dn1, dn2, dn3. Rack-0 has dn4, dn5, dn6. 2. Suppose client is in rack-0, and the write pipeline is: client --> localnode --> other rack --> other rack In this example we have the pipeline client-dn1-dn4-dn6. That is rack0-rack0-rack1-rack1. So far so good. 3. Now other client comes, and append to file. This client is also in rack-0. Interestingly, the append pipeline is client-dn6-dn4-dn1. That is the new client (from rack0) sends packet to the first node in pipeline (dn6) which belongs to rack1. > The append pipeline does not followed TSP principal > --- > > Key: HDFS-1380 > URL: https://issues.apache.org/jira/browse/HDFS-1380 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs client >Affects Versions: 0.20-append >Reporter: Thanh Do > > 1. Say we have 2 racks: rack-0 and rack-1. > Rack-0 has dn1, dn2, dn3. Rack-0 has dn4, dn5, dn6. > > 2. Suppose client is in rack-0, and the write pipeline is: > client --> localnode --> other rack --> other rack > In this example we have the pipeline client-dn1-dn4-dn6. > That is rack0-rack0-rack1-rack1. So far so good. > > 3. Now other client comes, and append to file. > This client is also in rack-0. Interestingly, > the append pipeline is client-dn6-dn4-dn1. > That is the new client (from rack0) sends packet > to the first node in pipeline (dn6) which belongs to rack1. > This bug was found by our Failure Testing Service framework: > http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html > For questions, please email us: Thanh Do (than...@cs.wisc.edu) and > Haryadi Gunawi (hary...@eecs.berkeley.edu) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1381) MiniDFSCluster documentation refers to out-of-date configuration parameters
MiniDFSCluster documentation refers to out-of-date configuration parameters --- Key: HDFS-1381 URL: https://issues.apache.org/jira/browse/HDFS-1381 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.20.1 Reporter: Jakob Homan Fix For: 0.22.0 The javadoc for MiniDFSCluster makes repeated references to setting dfs.name.dir and dfs.data.dir. These should be replaced with references to DFSConfigKeys' DFS_NAMENODE_NAME_DIR_KEY and DFS_DATANODE_DATA_DIR_KEY, respectively. The old values are deprecated in DFSConfigKeys, but we should switch to the new values where ever we can. Also, a quick search the code shows that TestDFSStorageStateRecovery.java and UpgradeUtilities.java should be updated as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1382) A transient failure with edits log and a corrupted fstime together could lead to a data loss
A transient failure with edits log and a corrupted fstime together could lead to a data loss Key: HDFS-1382 URL: https://issues.apache.org/jira/browse/HDFS-1382 Project: Hadoop HDFS Issue Type: Bug Components: name-node Reporter: Thanh Do We experienced a data loss situation that due to double failures. One is transient disk failure with edits logs and the other is corrupted fstime. Here is the detail: 1. NameNode has 2 edits directory (say edit0 and edit1) 2. During an update to edit0, there is a transient disk failure, making NameNode bump the fstime and mark edit0 as stale and continue working with edit1. 3. NameNode is shut down. Now, and unluckily fstime in edit0 is corrupted. Hence during NameNode startup, the log in edit0 is replayed, hence data loss. This bug was found by our Failure Testing Service framework: http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html For questions, please email us: Thanh Do (than...@cs.wisc.edu) and Haryadi Gunawi (hary...@eecs.berkeley.edu) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1382) A transient failure with edits log and a corrupted fstime together could lead to a data loss
[ https://issues.apache.org/jira/browse/HDFS-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907003#action_12907003 ] Todd Lipcon commented on HDFS-1382: --- Is the bug here that we don't properly decide which edit to replay? Shouldn't it have looked at fstime across the directories, seen dir2's higher fstime, and used that one? > A transient failure with edits log and a corrupted fstime together could lead > to a data loss > > > Key: HDFS-1382 > URL: https://issues.apache.org/jira/browse/HDFS-1382 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Reporter: Thanh Do > > We experienced a data loss situation that due to double failures. > One is transient disk failure with edits logs and the other is corrupted > fstime. > > Here is the detail: > > 1. NameNode has 2 edits directory (say edit0 and edit1) > > 2. During an update to edit0, there is a transient disk failure, > making NameNode bump the fstime and mark edit0 as stale > and continue working with edit1. > > 3. NameNode is shut down. Now, and unluckily fstime in edit0 > is corrupted. Hence during NameNode startup, the log in edit0 > is replayed, hence data loss. > This bug was found by our Failure Testing Service framework: > http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html > For questions, please email us: Thanh Do (than...@cs.wisc.edu) and > Haryadi Gunawi (hary...@eecs.berkeley.edu) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-718) configuration parameter to prevent accidental formatting of HDFS filesystem
[ https://issues.apache.org/jira/browse/HDFS-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-718: - Status: Open (was: Patch Available) > configuration parameter to prevent accidental formatting of HDFS filesystem > --- > > Key: HDFS-718 > URL: https://issues.apache.org/jira/browse/HDFS-718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.22.0 > Environment: Any >Reporter: Andrew Ryan >Assignee: Andrew Ryan >Priority: Minor > Attachments: HDFS-718.patch-2.txt, HDFS-718.patch.txt > > > Currently, any time the NameNode is not running, an HDFS filesystem will > accept the 'format' command, and will duly format itself. There are those of > us who have multi-PB HDFS filesystems who are really quite uncomfortable with > this behavior. There is "Y/N" confirmation in the format command, but if the > formatter genuinely believes themselves to be doing the right thing, the > filesystem will be formatted. > This patch adds a configuration parameter to the namenode, > dfs.namenode.support.allowformat, which defaults to "true," the current > behavior: always allow formatting if the NameNode is down or some other > process is not holding the namenode lock. But if > dfs.namenode.support.allowformat is set to "false," the NameNode will not > allow itself to be formatted until this config parameter is changed to "true". > The general idea is that for production HDFS filesystems, the user would > format the HDFS once, then set dfs.namenode.support.allowformat to "false" > for all time. > The attached patch was generated against trunk and +1's on my test machine. > We have a 0.20 version that we are using in our cluster as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-718) configuration parameter to prevent accidental formatting of HDFS filesystem
[ https://issues.apache.org/jira/browse/HDFS-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-718: - Attachment: HDFS-718-3.patch Went to review the patch and found it no longer applied. Sync'ed with trunk and did some clean up. Mainly, switched to using DFSConfigKeys, fixed some values in the MiniDFSCluster setup, cleaned up logging, changed config value to not run together allowformat. If a committer wants to +1 these changes, I feel this patch is ready to go, barring further objections. > configuration parameter to prevent accidental formatting of HDFS filesystem > --- > > Key: HDFS-718 > URL: https://issues.apache.org/jira/browse/HDFS-718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.22.0 > Environment: Any >Reporter: Andrew Ryan >Assignee: Andrew Ryan >Priority: Minor > Attachments: HDFS-718-3.patch, HDFS-718.patch-2.txt, > HDFS-718.patch.txt > > > Currently, any time the NameNode is not running, an HDFS filesystem will > accept the 'format' command, and will duly format itself. There are those of > us who have multi-PB HDFS filesystems who are really quite uncomfortable with > this behavior. There is "Y/N" confirmation in the format command, but if the > formatter genuinely believes themselves to be doing the right thing, the > filesystem will be formatted. > This patch adds a configuration parameter to the namenode, > dfs.namenode.support.allowformat, which defaults to "true," the current > behavior: always allow formatting if the NameNode is down or some other > process is not holding the namenode lock. But if > dfs.namenode.support.allowformat is set to "false," the NameNode will not > allow itself to be formatted until this config parameter is changed to "true". > The general idea is that for production HDFS filesystems, the user would > format the HDFS once, then set dfs.namenode.support.allowformat to "false" > for all time. > The attached patch was generated against trunk and +1's on my test machine. > We have a 0.20 version that we are using in our cluster as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-718) configuration parameter to prevent accidental formatting of HDFS filesystem
[ https://issues.apache.org/jira/browse/HDFS-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-718: - Status: Patch Available (was: Open) submitting v3 of patch to Hudson. > configuration parameter to prevent accidental formatting of HDFS filesystem > --- > > Key: HDFS-718 > URL: https://issues.apache.org/jira/browse/HDFS-718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.22.0 > Environment: Any >Reporter: Andrew Ryan >Assignee: Andrew Ryan >Priority: Minor > Attachments: HDFS-718-3.patch, HDFS-718.patch-2.txt, > HDFS-718.patch.txt > > > Currently, any time the NameNode is not running, an HDFS filesystem will > accept the 'format' command, and will duly format itself. There are those of > us who have multi-PB HDFS filesystems who are really quite uncomfortable with > this behavior. There is "Y/N" confirmation in the format command, but if the > formatter genuinely believes themselves to be doing the right thing, the > filesystem will be formatted. > This patch adds a configuration parameter to the namenode, > dfs.namenode.support.allowformat, which defaults to "true," the current > behavior: always allow formatting if the NameNode is down or some other > process is not holding the namenode lock. But if > dfs.namenode.support.allowformat is set to "false," the NameNode will not > allow itself to be formatted until this config parameter is changed to "true". > The general idea is that for production HDFS filesystems, the user would > format the HDFS once, then set dfs.namenode.support.allowformat to "false" > for all time. > The attached patch was generated against trunk and +1's on my test machine. > We have a 0.20 version that we are using in our cluster as well. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1382) A transient failure with edits log and a corrupted fstime together could lead to a data loss
[ https://issues.apache.org/jira/browse/HDFS-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907015#action_12907015 ] Eli Collins commented on HDFS-1382: --- Per HDFS-903 I think it uses the first image image and edit log regardless of the date. > A transient failure with edits log and a corrupted fstime together could lead > to a data loss > > > Key: HDFS-1382 > URL: https://issues.apache.org/jira/browse/HDFS-1382 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Reporter: Thanh Do > > We experienced a data loss situation that due to double failures. > One is transient disk failure with edits logs and the other is corrupted > fstime. > > Here is the detail: > > 1. NameNode has 2 edits directory (say edit0 and edit1) > > 2. During an update to edit0, there is a transient disk failure, > making NameNode bump the fstime and mark edit0 as stale > and continue working with edit1. > > 3. NameNode is shut down. Now, and unluckily fstime in edit0 > is corrupted. Hence during NameNode startup, the log in edit0 > is replayed, hence data loss. > This bug was found by our Failure Testing Service framework: > http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html > For questions, please email us: Thanh Do (than...@cs.wisc.edu) and > Haryadi Gunawi (hary...@eecs.berkeley.edu) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1361) Add -fileStatus operation to NNThroughputBenchmark
[ https://issues.apache.org/jira/browse/HDFS-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-1361: -- Status: Resolved (was: Patch Available) Resolution: Fixed I just committed this. > Add -fileStatus operation to NNThroughputBenchmark > -- > > Key: HDFS-1361 > URL: https://issues.apache.org/jira/browse/HDFS-1361 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.22.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.22.0 > > Attachments: NNThroughput-fileStatus.patch > > > getFileStatus() is a frequently used operation in HDFS. It important to > benchmark the name-node throughput on it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-31) Hadoop distcp tool fails if file path contains special characters + & !
[ https://issues.apache.org/jira/browse/HDFS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HDFS-31. Fix Version/s: (was: 0.22.0) Resolution: Invalid Closing as invalid. Please feel free to reopen if the problem still exists. > Hadoop distcp tool fails if file path contains special characters + & ! > --- > > Key: HDFS-31 > URL: https://issues.apache.org/jira/browse/HDFS-31 > Project: Hadoop HDFS > Issue Type: Bug > Components: tools >Affects Versions: 0.20.2, 0.21.0, 0.22.0 >Reporter: Viraj Bhat > > Copying folders containing + & ! characters between hdfs (using hftp) does > not work in distcp > For example: > Copying folder "string1+string2" at "namenode.address.com", hftp port > myport to "/myotherhome/folder" on "myothermachine" does not work > myothermachine prompt>>> hadoop --config ~/mycluster/ distcp > "hftp://namenode.address.com:myport/myhome/dir/string1+string2"; > /myotherhome/folder/ > > Error results for hadoop job1: > > 08/07/16 00:27:39 INFO tools.DistCp: > srcPaths=[hftp://namenode.address.com:myport/myhome/dir/string1+string2] > 08/07/16 00:27:39 INFO tools.DistCp: destPath=/myotherhome/folder/ > 08/07/16 00:27:41 INFO tools.DistCp: srcCount=2 > 08/07/16 00:27:42 INFO mapred.JobClient: Running job: job1 > 08/07/16 00:27:43 INFO mapred.JobClient: map 0% reduce 0% > 08/07/16 00:27:58 INFO mapred.JobClient: Task Id : attempt_1_m_00_0, > Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) > at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) > 08/07/16 00:28:14 INFO mapred.JobClient: Task Id : attempt_1_m_00_1, > Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) > at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) > 08/07/16 00:28:28 INFO mapred.JobClient: Task Id : attempt_1_m_00_2, > Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:538) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:226) > at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2208) > With failures, global counters are inaccurate; consider running with -i > Copy failed: java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1053) > at org.apache.hadoop.tools.DistCp.copy(DistCp.java:615) > at org.apache.hadoop.tools.DistCp.run(DistCp.java:764) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > at org.apache.hadoop.tools.DistCp.main(DistCp.java:784) > > Error log for the map task which failed > > INFO org.apache.hadoop.tools.DistCp: FAIL > string1+string2/myjobtrackermachine.com-joblog.tar.gz : java.io.IOException: > Server returned HTTP response code: 500 for URL: > http://mymachine.com:myport/streamFile?filename=/myhome/dir/string1+string2/myjobtrackermachine.com-joblog.tar.gz&ugi=myid,mygroup > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1241) > at org.apache.hadoop.dfs.HftpFileSystem.open(HftpFileSystem.java:117) > at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:371) > at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:377) > at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:504) > at org.apache.hadoo
[jira] Updated: (HDFS-908) TestDistributedFileSystem fails with Wrong FS on weird hosts
[ https://issues.apache.org/jira/browse/HDFS-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-908: Status: Patch Available (was: Open) Try Hudson. > TestDistributedFileSystem fails with Wrong FS on weird hosts > > > Key: HDFS-908 > URL: https://issues.apache.org/jira/browse/HDFS-908 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 0.20.1 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Attachments: hdfs-908.txt, hdfs-908.txt > > > On the same host where I experienced HDFS-874, I also experience this failure > for TestDistributedFileSystem: > Testcase: testFileChecksum took 0.492 sec > Caused an ERROR > Wrong FS: hftp://localhost.localdomain:59782/filechecksum/foo0, expected: > hftp://127.0.0.1:59782 > java.lang.IllegalArgumentException: Wrong FS: > hftp://localhost.localdomain:59782/filechecksum/foo0, expected: > hftp://127.0.0.1:59782 > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310) > at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:222) > at > org.apache.hadoop.hdfs.HftpFileSystem.getFileChecksum(HftpFileSystem.java:318) > at > org.apache.hadoop.hdfs.TestDistributedFileSystem.testFileChecksum(TestDistributedFileSystem.java:166) > Doesn't appear to occur on trunk or branch-0.21. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-908) TestDistributedFileSystem fails with Wrong FS on weird hosts
[ https://issues.apache.org/jira/browse/HDFS-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-908: Hadoop Flags: [Reviewed] Assignee: Todd Lipcon +1 patch looks good. > TestDistributedFileSystem fails with Wrong FS on weird hosts > > > Key: HDFS-908 > URL: https://issues.apache.org/jira/browse/HDFS-908 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 0.20.1 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Attachments: hdfs-908.txt, hdfs-908.txt > > > On the same host where I experienced HDFS-874, I also experience this failure > for TestDistributedFileSystem: > Testcase: testFileChecksum took 0.492 sec > Caused an ERROR > Wrong FS: hftp://localhost.localdomain:59782/filechecksum/foo0, expected: > hftp://127.0.0.1:59782 > java.lang.IllegalArgumentException: Wrong FS: > hftp://localhost.localdomain:59782/filechecksum/foo0, expected: > hftp://127.0.0.1:59782 > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310) > at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:222) > at > org.apache.hadoop.hdfs.HftpFileSystem.getFileChecksum(HftpFileSystem.java:318) > at > org.apache.hadoop.hdfs.TestDistributedFileSystem.testFileChecksum(TestDistributedFileSystem.java:166) > Doesn't appear to occur on trunk or branch-0.21. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1383) Better error messages on hftp
Better error messages on hftp
--
Key: HDFS-1383
URL: https://issues.apache.org/jira/browse/HDFS-1383
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
If the file is not accessible, HftpFileSystem returns only a HTTP response code.
{noformat}
2010-08-27 20:57:48,091 INFO org.apache.hadoop.tools.DistCp: FAIL README.txt :
java.io.IOException:
Server returned HTTP response code: 400 for URL:
http:/namenode:50070/data/user/tsz/README.txt?ugi=tsz,users
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1290)
at org.apache.hadoop.hdfs.HftpFileSystem.open(HftpFileSystem.java:143)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:356)
...
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1382) A transient failure with edits log and a corrupted fstime together could lead to a data loss
[ https://issues.apache.org/jira/browse/HDFS-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907054#action_12907054 ] Thanh Do commented on HDFS-1382: >From my understanding, fstime in very dir is scan for the latest one. In this case, fstime in edit0 is corrupted to a later time than that of edit1, hence the lost. The bug here is that metadata (fstime) has no checksum, hence a corruption can lead to a data loss. > A transient failure with edits log and a corrupted fstime together could lead > to a data loss > > > Key: HDFS-1382 > URL: https://issues.apache.org/jira/browse/HDFS-1382 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Reporter: Thanh Do > > We experienced a data loss situation that due to double failures. > One is transient disk failure with edits logs and the other is corrupted > fstime. > > Here is the detail: > > 1. NameNode has 2 edits directory (say edit0 and edit1) > > 2. During an update to edit0, there is a transient disk failure, > making NameNode bump the fstime and mark edit0 as stale > and continue working with edit1. > > 3. NameNode is shut down. Now, and unluckily fstime in edit0 > is corrupted. Hence during NameNode startup, the log in edit0 > is replayed, hence data loss. > This bug was found by our Failure Testing Service framework: > http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html > For questions, please email us: Thanh Do (than...@cs.wisc.edu) and > Haryadi Gunawi (hary...@eecs.berkeley.edu) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1384) NameNode should give client the first node in the pipeline from different rack other than that of excludedNodes list in the same rack.
NameNode should give client the first node in the pipeline from different rack other than that of excludedNodes list in the same rack. --- Key: HDFS-1384 URL: https://issues.apache.org/jira/browse/HDFS-1384 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.1 Reporter: Thanh Do We saw a case that NN keeps giving client nodes from the same rack, hence an exception from client when try to setup the pipeline. Client retries 5 times and fails. Here is more details. Support we have 2 rack - Rack 0: from dn1 to dn7 - Rack 1: from dn8 to dn14 Client asks for 3 dns and NN replies with dn1, dn8 and dn9, for example. Because there is network partition, so client doesn't see any node in Rack 0. Hence, client add dn1 to excludedNodes list, and ask NN again. Interestingly, NN picks a different node (from those in excludedNodes) in Rack 0, and gives back to client, and so on. Client keeps retrying and after 5 times of retrials, write fails. This bug was found by our Failure Testing Service framework: http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html For questions, please email us: Thanh Do (than...@cs.wisc.edu) and Haryadi Gunawi (hary...@eecs.berkeley.edu) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1384) NameNode should give client the first node in the pipeline from different rack other than that of excludedNodes list in the same rack.
[ https://issues.apache.org/jira/browse/HDFS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thanh Do updated HDFS-1384: --- Affects Version/s: 0.20-append > NameNode should give client the first node in the pipeline from different > rack other than that of excludedNodes list in the same rack. > --- > > Key: HDFS-1384 > URL: https://issues.apache.org/jira/browse/HDFS-1384 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20-append, 0.20.1 >Reporter: Thanh Do > > We saw a case that NN keeps giving client nodes from the same rack, hence an > exception > from client when try to setup the pipeline. Client retries 5 times and fails. > > Here is more details. Support we have 2 rack > - Rack 0: from dn1 to dn7 > - Rack 1: from dn8 to dn14 > Client asks for 3 dns and NN replies with dn1, dn8 and dn9, for example. > Because there is network partition, so client doesn't see any node in Rack 0. > Hence, client add dn1 to excludedNodes list, and ask NN again. > Interestingly, NN picks a different node (from those in excludedNodes) in > Rack 0, > and gives back to client, and so on. Client keeps retrying and after 5 times > of retrials, > write fails. > This bug was found by our Failure Testing Service framework: > http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html > For questions, please email us: Thanh Do (than...@cs.wisc.edu) and > Haryadi Gunawi (hary...@eecs.berkeley.edu) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1363) startFileInternal should return the last block of the file opened for append as an under-construction block
[
https://issues.apache.org/jira/browse/HDFS-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko updated HDFS-1363:
--
Attachment: appendFileSync.patch
This is a patch for trunk.
- It eliminates the second synchronized section in appendFile().
- I moved all this logic related to generating LocatedBlock inside
startFileInternal(),
- which now can return a LocatedBlock if called from appendFile().
- I reintroduce setBlockTokens() in FSNamesystem, otherwise I could not reuse
getBlockLocation() in BlockManager.
- The patch removes the part of the code responsible for cleaning up replicas
of the last block from respective DataNodeDescriptors. This logic was
introduced in HADOOP-5134 for append-20 in order to avoid replication of blocks
under construction. We do not need this anymore as blocks under-construction
are not replicated in current impl, and getBlockLocations() always returns
expected locations for such blocks, ignoring the triplets.
> startFileInternal should return the last block of the file opened for append
> as an under-construction block
> ---
>
> Key: HDFS-1363
> URL: https://issues.apache.org/jira/browse/HDFS-1363
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 0.21.0
>Reporter: Konstantin Shvachko
> Fix For: 0.21.1
>
> Attachments: appendFileSync.patch
>
>
> {{FSNamesystem.startFileInternal}} should convert the last block of the file
> opened for append to an under-construction block and return it. This will let
> remove the second synchronized section in {{FSNamesystem.appendFile()}} and
> avoid redundant computations and potential inconsistencies as stated in
> HDFS-1152.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
