[jira] Commented: (HDFS-1295) Improve namenode restart times by short-circuiting the first block reports from datanodes
[ https://issues.apache.org/jira/browse/HDFS-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887673#action_12887673 ] dhruba borthakur commented on HDFS-1295: > what happens with blocks in the first block report from a DN that do not > belong to any file They are discarded as usual inside FSnamesystem.addStoredBlock(). This method checks if the block belongs to any inode, and if so, only then insert it into the blocksmap (this is existing code and is not modified by this patch). > Improve namenode restart times by short-circuiting the first block reports > from datanodes > - > > Key: HDFS-1295 > URL: https://issues.apache.org/jira/browse/HDFS-1295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.22.0 > > Attachments: shortCircuitBlockReport_1.txt > > > The namenode restart is dominated by the performance of processing block > reports. On a 2000 node cluster with 90 million blocks, block report > processing takes 30 to 40 minutes. The namenode "diffs" the contents of the > incoming block report with the contents of the blocks map, and then applies > these diffs to the blocksMap, but in reality there is no need to compute the > "diff" because this is the first block report from the datanode. > This code change improves block report processing time by 300%. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1297) Fix some comments
[ https://issues.apache.org/jira/browse/HDFS-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Ames updated HDFS-1297: Status: Patch Available (was: Open) > Fix some comments > - > > Key: HDFS-1297 > URL: https://issues.apache.org/jira/browse/HDFS-1297 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Jeff Ames >Priority: Trivial > Attachments: HDFS-1297.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1297) Fix some comments
Fix some comments - Key: HDFS-1297 URL: https://issues.apache.org/jira/browse/HDFS-1297 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jeff Ames Priority: Trivial -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1297) Fix some comments
[ https://issues.apache.org/jira/browse/HDFS-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Ames updated HDFS-1297: Attachment: HDFS-1297.patch > Fix some comments > - > > Key: HDFS-1297 > URL: https://issues.apache.org/jira/browse/HDFS-1297 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Jeff Ames >Priority: Trivial > Attachments: HDFS-1297.patch > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1295) Improve namenode restart times by short-circuiting the first block reports from datanodes
[ https://issues.apache.org/jira/browse/HDFS-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887618#action_12887618 ] Konstantin Shvachko commented on HDFS-1295: --- Could you please explain what happens with blocks in the first block report from a DN that do not belong to any file. Are they also inserted to the blocksMap? > Improve namenode restart times by short-circuiting the first block reports > from datanodes > - > > Key: HDFS-1295 > URL: https://issues.apache.org/jira/browse/HDFS-1295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.22.0 > > Attachments: shortCircuitBlockReport_1.txt > > > The namenode restart is dominated by the performance of processing block > reports. On a 2000 node cluster with 90 million blocks, block report > processing takes 30 to 40 minutes. The namenode "diffs" the contents of the > incoming block report with the contents of the blocks map, and then applies > these diffs to the blocksMap, but in reality there is no need to compute the > "diff" because this is the first block report from the datanode. > This code change improves block report processing time by 300%. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1130) Pass Administrator acl to HTTPServer for common servlet access.
[ https://issues.apache.org/jira/browse/HDFS-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HDFS-1130: -- Attachment: hdfs-1130-trunk.patch Patch for trunk. > Pass Administrator acl to HTTPServer for common servlet access. > --- > > Key: HDFS-1130 > URL: https://issues.apache.org/jira/browse/HDFS-1130 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Reporter: Amareshwari Sriramadasu > Fix For: 0.22.0 > > Attachments: hdfs-1130-trunk.patch, hdfs-1130.3.patch, hdfs-1130.patch > > > Once HADOOP-6748 is done, HDFS should pass administrator acl when HTTPServer > is constructed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1130) Pass Administrator acl to HTTPServer for common servlet access.
[ https://issues.apache.org/jira/browse/HDFS-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HDFS-1130: -- Status: Patch Available (was: Open) Assignee: Devaraj Das > Pass Administrator acl to HTTPServer for common servlet access. > --- > > Key: HDFS-1130 > URL: https://issues.apache.org/jira/browse/HDFS-1130 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Reporter: Amareshwari Sriramadasu >Assignee: Devaraj Das > Fix For: 0.22.0 > > Attachments: hdfs-1130-trunk.patch, hdfs-1130.3.patch, hdfs-1130.patch > > > Once HADOOP-6748 is done, HDFS should pass administrator acl when HTTPServer > is constructed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-202) Add a bulk FIleSystem.getFileBlockLocations
[
https://issues.apache.org/jira/browse/HDFS-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887600#action_12887600
]
Hairong Kuang commented on HDFS-202:
I read FileInputFormat and understand the usecase much better. So the client
needs to know FileStatus for filtering and there is a configuration parameter
to specify whether the input paths need to be traversed recursively. In this
case, how about the following revised API?
{code}
class FileStatusAndBlockLocations {
FileStatus fileStatus;
BlockLocation [] blocks;
}
Iterator getBlockLocations(Path[] paths, boolean
isRecursive);
{code}
> Add a bulk FIleSystem.getFileBlockLocations
> ---
>
> Key: HDFS-202
> URL: https://issues.apache.org/jira/browse/HDFS-202
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Arun C Murthy
>Assignee: Hairong Kuang
> Fix For: 0.22.0
>
>
> Currently map-reduce applications (specifically file-based input-formats) use
> FileSystem.getFileBlockLocations to compute splits. However they are forced
> to call it once per file.
> The downsides are multiple:
># Even with a few thousand files to process the number of RPCs quickly
> starts getting noticeable
># The current implementation of getFileBlockLocations is too slow since
> each call results in 'search' in the namesystem. Assuming a few thousand
> input files it results in that many RPCs and 'searches'.
> It would be nice to have a FileSystem.getFileBlockLocations which can take in
> a directory, and return the block-locations for all files in that directory.
> We could eliminate both the per-file RPC and also the 'search' by a 'scan'.
> When I tested this for terasort, a moderate job with 8000 input files the
> runtime halved from the current 8s to 4s. Clearly this is much more important
> for latency-sensitive applications...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-132) Namenode in Safemode reports to Simon non-zero number of deleted files during startup
[ https://issues.apache.org/jira/browse/HDFS-132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887598#action_12887598 ] Suresh Srinivas commented on HDFS-132: -- +1 for the 20 and y20 version of the patch. > Namenode in Safemode reports to Simon non-zero number of deleted files during > startup > - > > Key: HDFS-132 > URL: https://issues.apache.org/jira/browse/HDFS-132 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20.3, 0.21.0, 0.22.0 >Reporter: Hairong Kuang >Assignee: Suresh Srinivas >Priority: Minor > Fix For: 0.21.0 > > Attachments: HDFS-132-20.patch, HDFS-132.1.patch, HDFS-132.2.patch, > HDFS-132.patch, HDFS-132.y20.patch > > > Currently when a namenode starts up it may call "delete" when loading edits. > But metrics file_deleted is collected and reported in FSDirectory where it > does not and is not able to check if namenode is in safemode. So from the > monitoring UI, we see non-zero number of files deleted. > A possible fix is to have delete to return the number of files deleted and > then the metrics is collected and reported at NameNode the same as other > namenode metrics. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-599) Improve Namenode robustness by prioritizing datanode heartbeats over client requests
[
https://issues.apache.org/jira/browse/HDFS-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887593#action_12887593
]
Konstantin Shvachko commented on HDFS-599:
--
y20 looks good.
One comment, although this is related more to the original patch.
The setter and getter methods should be named symmetrically, otherwise it gets
confusing. So in the NameNode class instead of the pair of
{code}
getServiceRpcServerAddress()
setRpcServiceServerAddress()
{code}
we should have
{code}
getServiceRpcServerAddress()
setServiceRpcServerAddress()
{code}
Dmytro, could you please rename this in your next patch.
> Improve Namenode robustness by prioritizing datanode heartbeats over client
> requests
>
>
> Key: HDFS-599
> URL: https://issues.apache.org/jira/browse/HDFS-599
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
>Reporter: dhruba borthakur
>Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HDFS-599.3.patch, HDFS-599.patch, HDFS-599.y20.patch
>
>
> The namenode processes RPC requests from clients that are reading/writing to
> files as well as heartbeats/block reports from datanodes.
> Sometime, because of various reasons (Java GC runs, inconsistent performance
> of NFS filer that stores HDFS transacttion logs, etc), the namenode
> encounters transient slowness. For example, if the device that stores the
> HDFS transaction logs becomes sluggish, the Namenode's ability to process
> RPCs slows down to a certain extent. During this time, the RPCs from clients
> as well as the RPCs from datanodes suffer in similar fashion. If the
> underlying problem becomes worse, the NN's ability to process a heartbeat
> from a DN is severly impacted, thus causing the NN to declare that the DN is
> dead. Then the NN starts replicating blocks that used to reside on the
> now-declared-dead datanode. This adds extra load to the NN. Then the
> now-declared-datanode finally re-establishes contact with the NN, and sends a
> block report. The block report processing on the NN is another heavyweight
> activity, thus casing more load to the already overloaded namenode.
> My proposal is tha the NN should try its best to continue processing RPCs
> from datanodes and give lesser priority to serving client requests. The
> Datanode RPCs are integral to the consistency and performance of the Hadoop
> file system, and it is better to protect it at all costs. This will ensure
> that NN recovers from the hiccup much faster than what it does now.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-202) Add a bulk FIleSystem.getFileBlockLocations
[
https://issues.apache.org/jira/browse/HDFS-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887592#action_12887592
]
Hairong Kuang commented on HDFS-202:
I am quite bothered that the proposed API returns a map. Is the reason for
returning a map because the API does one-level listPath? Is there a use case
that needs only one-level expansion?
If we eventually need to get the block locations of all files recursively under
the input paths, is the following API a better choice?
{code}
/**
* @return the block locations of all files recursively under the input paths
*/
Iterator getBlockLocations(Path[] paths)
{code}
When implementing this in HDFS, we might need to issue multiple RPCs and be
very careful to limit the size of each RPC request and response.
> Add a bulk FIleSystem.getFileBlockLocations
> ---
>
> Key: HDFS-202
> URL: https://issues.apache.org/jira/browse/HDFS-202
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Arun C Murthy
>Assignee: Hairong Kuang
> Fix For: 0.22.0
>
>
> Currently map-reduce applications (specifically file-based input-formats) use
> FileSystem.getFileBlockLocations to compute splits. However they are forced
> to call it once per file.
> The downsides are multiple:
># Even with a few thousand files to process the number of RPCs quickly
> starts getting noticeable
># The current implementation of getFileBlockLocations is too slow since
> each call results in 'search' in the namesystem. Assuming a few thousand
> input files it results in that many RPCs and 'searches'.
> It would be nice to have a FileSystem.getFileBlockLocations which can take in
> a directory, and return the block-locations for all files in that directory.
> We could eliminate both the per-file RPC and also the 'search' by a 'scan'.
> When I tested this for terasort, a moderate job with 8000 input files the
> runtime halved from the current 8s to 4s. Clearly this is much more important
> for latency-sensitive applications...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1295) Improve namenode restart times by short-circuiting the first block reports from datanodes
[ https://issues.apache.org/jira/browse/HDFS-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HDFS-1295: --- Attachment: shortCircuitBlockReport_1.txt This patch applied only to 0.20 and does two things: 1. The first block report from a datanode is directly inserted into the blocksMap 2. Optimize the countLiveNodes methods to be lightweight I extended the attached unit test to gather the average time to process the first block report from a datanode: ||Configuration| without patch | with patch|| |40 nodes, 200K blocks each | 232 sec| 80 sec | |80 nodes, 200K blocks each| 240 sec| 82 sec | > Improve namenode restart times by short-circuiting the first block reports > from datanodes > - > > Key: HDFS-1295 > URL: https://issues.apache.org/jira/browse/HDFS-1295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.22.0 > > Attachments: shortCircuitBlockReport_1.txt > > > The namenode restart is dominated by the performance of processing block > reports. On a 2000 node cluster with 90 million blocks, block report > processing takes 30 to 40 minutes. The namenode "diffs" the contents of the > incoming block report with the contents of the blocks map, and then applies > these diffs to the blocksMap, but in reality there is no need to compute the > "diff" because this is the first block report from the datanode. > This code change improves block report processing time by 300%. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1007) HFTP needs to be updated to use delegation tokens
[ https://issues.apache.org/jira/browse/HDFS-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boris Shkolnik updated HDFS-1007: - Attachment: HDFS-1007-2.patch fixed javadoc and findbug warnings Ran all the tests - all passed (except TestHdfsTrash). There are not tests in this patch. It will be tested by the test in TestTokenCache (in MR). > HFTP needs to be updated to use delegation tokens > - > > Key: HDFS-1007 > URL: https://issues.apache.org/jira/browse/HDFS-1007 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 0.22.0 >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.22.0 > > Attachments: 1007-bugfix.patch, distcp-hftp-2.1.1.patch, > distcp-hftp.1.patch, distcp-hftp.2.1.patch, distcp-hftp.2.patch, > distcp-hftp.patch, HDFS-1007-1.patch, HDFS-1007-2.patch, > HDFS-1007-BP20-fix-1.patch, HDFS-1007-BP20-fix-2.patch, > HDFS-1007-BP20-fix-3.patch, HDFS-1007-BP20.patch, > hdfs-1007-long-running-hftp-client.patch, hdfs-1007-securityutil-fix.patch > > > HFTPFileSystem should be updated to use the delegation tokens so that it can > talk to the secure namenodes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-202) Add a bulk FIleSystem.getFileBlockLocations
[ https://issues.apache.org/jira/browse/HDFS-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HDFS-202: --- Assignee: Hairong Kuang (was: Jakob Homan) Fix Version/s: 0.22.0 > Add a bulk FIleSystem.getFileBlockLocations > --- > > Key: HDFS-202 > URL: https://issues.apache.org/jira/browse/HDFS-202 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Arun C Murthy >Assignee: Hairong Kuang > Fix For: 0.22.0 > > > Currently map-reduce applications (specifically file-based input-formats) use > FileSystem.getFileBlockLocations to compute splits. However they are forced > to call it once per file. > The downsides are multiple: ># Even with a few thousand files to process the number of RPCs quickly > starts getting noticeable ># The current implementation of getFileBlockLocations is too slow since > each call results in 'search' in the namesystem. Assuming a few thousand > input files it results in that many RPCs and 'searches'. > It would be nice to have a FileSystem.getFileBlockLocations which can take in > a directory, and return the block-locations for all files in that directory. > We could eliminate both the per-file RPC and also the 'search' by a 'scan'. > When I tested this for terasort, a moderate job with 8000 input files the > runtime halved from the current 8s to 4s. Clearly this is much more important > for latency-sensitive applications... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1296) using delegation token over hftp for long running clients (spawn from hdfs-1007).
[ https://issues.apache.org/jira/browse/HDFS-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boris Shkolnik updated HDFS-1296: - Attachment: hdfs-1007-long-running-hftp-client.patch hdfs-1007-securityutil-fix.patch previous version patches. Not for commit. For forward porting. > using delegation token over hftp for long running clients (spawn from > hdfs-1007). > - > > Key: HDFS-1296 > URL: https://issues.apache.org/jira/browse/HDFS-1296 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Boris Shkolnik > Attachments: hdfs-1007-long-running-hftp-client.patch, > hdfs-1007-securityutil-fix.patch > > > this patch incorporates patches > https://issues.apache.org/jira/secure/attachment/12446280/hdfs-1007-long-running-hftp-client.patch > and > https://issues.apache.org/jira/secure/attachment/12446362/hdfs-1007-securityutil-fix.patch > patches are attached. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1296) using delegation token over hftp for long running clients (spawn from hdfs-1007).
using delegation token over hftp for long running clients (spawn from hdfs-1007). - Key: HDFS-1296 URL: https://issues.apache.org/jira/browse/HDFS-1296 Project: Hadoop HDFS Issue Type: Bug Reporter: Boris Shkolnik this patch incorporates patches https://issues.apache.org/jira/secure/attachment/12446280/hdfs-1007-long-running-hftp-client.patch and https://issues.apache.org/jira/secure/attachment/12446362/hdfs-1007-securityutil-fix.patch patches are attached. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1257) Race condition introduced by HADOOP-5124
[ https://issues.apache.org/jira/browse/HDFS-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar Vadali updated HDFS-1257: -- Attachment: HDFS-1257.patch Use protected access as suggested by Hairong. > Race condition introduced by HADOOP-5124 > > > Key: HDFS-1257 > URL: https://issues.apache.org/jira/browse/HDFS-1257 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Reporter: Ramkumar Vadali > Attachments: HDFS-1257.patch > > > HADOOP-5124 provided some improvements to FSNamesystem#recentInvalidateSets. > But it introduced unprotected access to the data structure > recentInvalidateSets. Specifically, FSNamesystem.computeInvalidateWork > accesses recentInvalidateSets without read-lock protection. If there is > concurrent activity (like reducing replication on a file) that adds to > recentInvalidateSets, the name-node crashes with a > ConcurrentModificationException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1109) HFTP and URL Encoding
[ https://issues.apache.org/jira/browse/HDFS-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-1109: - Component/s: contrib/hdfsproxy data-node > HFTP and URL Encoding > - > > Key: HDFS-1109 > URL: https://issues.apache.org/jira/browse/HDFS-1109 > Project: Hadoop HDFS > Issue Type: Bug > Components: contrib/hdfsproxy, data-node >Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 >Reporter: Dmytro Molkov >Assignee: Dmytro Molkov > Fix For: 0.22.0 > > Attachments: HDFS-1109.2.patch, HDFS-1109.2_y0.20.1xx.patch, > HDFS-1109.patch > > > We just saw this error happen in our cluster. If there is a file that has a > "+" sign in the name it is not readable through HFTP protocol. > The problem is when we are reading a file with HFTP we are passing a name of > the file as a parameter in request and + gets undecoded into space on the > server side. So the datanode receiving the streamFile request tries to access > a file with space instead of + in the name and doesn't find that file. > The proposed solution is to pass the filename as a part of URL as with all > the other HFTP commands, since this is the only place where it is not being > treated this way. Are there any objections to this? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1109) HFTP and URL Encoding
[ https://issues.apache.org/jira/browse/HDFS-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-1109: - Attachment: HDFS-1109.2_y0.20.1xx.patch HDFS-1109.2_y0.20.1xx.patch: for y0.20.1xx > HFTP and URL Encoding > - > > Key: HDFS-1109 > URL: https://issues.apache.org/jira/browse/HDFS-1109 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 >Reporter: Dmytro Molkov >Assignee: Dmytro Molkov > Fix For: 0.22.0 > > Attachments: HDFS-1109.2.patch, HDFS-1109.2_y0.20.1xx.patch, > HDFS-1109.patch > > > We just saw this error happen in our cluster. If there is a file that has a > "+" sign in the name it is not readable through HFTP protocol. > The problem is when we are reading a file with HFTP we are passing a name of > the file as a parameter in request and + gets undecoded into space on the > server side. So the datanode receiving the streamFile request tries to access > a file with space instead of + in the name and doesn't find that file. > The proposed solution is to pass the filename as a part of URL as with all > the other HFTP commands, since this is the only place where it is not being > treated this way. Are there any objections to this? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-132) Namenode in Safemode reports to Simon non-zero number of deleted files during startup
[ https://issues.apache.org/jira/browse/HDFS-132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-132: - Attachment: HDFS-132.y20.patch HDFS-132-20.patch This proting to 0.20 and y0.20 > Namenode in Safemode reports to Simon non-zero number of deleted files during > startup > - > > Key: HDFS-132 > URL: https://issues.apache.org/jira/browse/HDFS-132 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20.3, 0.21.0, 0.22.0 >Reporter: Hairong Kuang >Assignee: Suresh Srinivas >Priority: Minor > Fix For: 0.21.0 > > Attachments: HDFS-132-20.patch, HDFS-132.1.patch, HDFS-132.2.patch, > HDFS-132.patch, HDFS-132.y20.patch > > > Currently when a namenode starts up it may call "delete" when loading edits. > But metrics file_deleted is collected and reported in FSDirectory where it > does not and is not able to check if namenode is in safemode. So from the > monitoring UI, we see non-zero number of files deleted. > A possible fix is to have delete to return the number of files deleted and > then the metrics is collected and reported at NameNode the same as other > namenode metrics. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1295) Improve namenode restart times by short-circuiting the first block reports from datanodes
Improve namenode restart times by short-circuiting the first block reports from datanodes - Key: HDFS-1295 URL: https://issues.apache.org/jira/browse/HDFS-1295 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.22.0 The namenode restart is dominated by the performance of processing block reports. On a 2000 node cluster with 90 million blocks, block report processing takes 30 to 40 minutes. The namenode "diffs" the contents of the incoming block report with the contents of the blocks map, and then applies these diffs to the blocksMap, but in reality there is no need to compute the "diff" because this is the first block report from the datanode. This code change improves block report processing time by 300%. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1257) Race condition introduced by HADOOP-5124
[
https://issues.apache.org/jira/browse/HDFS-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887506#action_12887506
]
Hairong Kuang commented on HDFS-1257:
-
RecentInvalidateSets were protected by FSNamesystem lock. It seems to me that
the ConcurrentModificationException might be caused by computeInvalidateWork
accessing recentInvalidateSets without holding FsNamesystem lock. The following
code should be surrounded by "synchronized (this)".
{code}
int numOfNodes = recentInvalidateSets.size();
nodesToProcess = Math.min(numOfNodes, nodesToProcess);
// get an array of the keys
ArrayList keyArray =
new ArrayList(recentInvalidateSets.keySet());
{code}
> Race condition introduced by HADOOP-5124
>
>
> Key: HDFS-1257
> URL: https://issues.apache.org/jira/browse/HDFS-1257
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
>Reporter: Ramkumar Vadali
>
> HADOOP-5124 provided some improvements to FSNamesystem#recentInvalidateSets.
> But it introduced unprotected access to the data structure
> recentInvalidateSets. Specifically, FSNamesystem.computeInvalidateWork
> accesses recentInvalidateSets without read-lock protection. If there is
> concurrent activity (like reducing replication on a file) that adds to
> recentInvalidateSets, the name-node crashes with a
> ConcurrentModificationException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1174) New properties for suspend and resume process.
[ https://issues.apache.org/jira/browse/HDFS-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik updated HDFS-1174: - Status: Patch Available (was: Open) Hadoop Flags: [Reviewed] Affects Version/s: 0.21.0 +1 patch looks good. > New properties for suspend and resume process. > -- > > Key: HDFS-1174 > URL: https://issues.apache.org/jira/browse/HDFS-1174 > Project: Hadoop HDFS > Issue Type: Task > Components: test >Affects Versions: 0.21.0 >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Attachments: HDFS-1174.patch, HDFS-1174.patch > > > Adding new properties in system-test-hdfs.xml file for suspend and resume > process. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1254) 0.20: mark dfs.support.append to be true by default for the 0.20-append branch
[ https://issues.apache.org/jira/browse/HDFS-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HDFS-1254: --- Summary: 0.20: mark dfs.support.append to be true by default for the 0.20-append branch (was: 0.20: mark dfs.supprt.append to be true by default for the 0.20-append branch) > 0.20: mark dfs.support.append to be true by default for the 0.20-append branch > -- > > Key: HDFS-1254 > URL: https://issues.apache.org/jira/browse/HDFS-1254 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.20-append > > Attachments: append.txt > > > The 0.20-append branch supports append/sync for HDFS. Change the default > configuration to enable append. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1158) HDFS-457 increases the chances of losing blocks
[
https://issues.apache.org/jira/browse/HDFS-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887444#action_12887444
]
dhruba borthakur commented on HDFS-1158:
One idea that came up could be a policy that makes the datanode automatically
start to decommission itself when it finds a bad disk. This is what we do with
our cluster nodes. This had caused our block-loss problems to go away.
what do people think of this idea? Of course it wil be a configurable policy
over-and-above the protection provided via HDFS-1161?
> HDFS-457 increases the chances of losing blocks
>
>
> Key: HDFS-1158
> URL: https://issues.apache.org/jira/browse/HDFS-1158
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
>Affects Versions: 0.20.2
>Reporter: Koji Noguchi
> Fix For: 0.20.3
>
> Attachments: rev-HDFS-457.patch
>
>
> Whenever we restart a cluster, there's a chance of losing some blocks if more
> than three datanodes don't come up.
> HDFS-457 increases this chance by keeping the datanodes up even when
># /tmp disk goes read-only
># /disk0 that is used for storing PID goes read-only
> and probably more.
> In our environment, /tmp and /disk0 are from the same device.
> When trying to restart a datanode, it would fail with
> 1)
> {noformat}
> 2010-05-15 05:45:45,575 WARN org.mortbay.log: tmpdir
> java.io.IOException: Read-only file system
> at java.io.UnixFileSystem.createFileExclusively(Native Method)
> at java.io.File.checkAndCreate(File.java:1704)
> at java.io.File.createTempFile(File.java:1792)
> at java.io.File.createTempFile(File.java:1828)
> at
> org.mortbay.jetty.webapp.WebAppContext.getTempDirectory(WebAppContext.java:745)
> {noformat}
> or
> 2)
> {noformat}
> hadoop-daemon.sh: line 117: /disk/0/hadoop-datanodecom.out: Read-only
> file system
> hadoop-daemon.sh: line 118: /disk/0/hadoop-datanode.pid: Read-only file system
> {noformat}
> I can recover the missing blocks but it takes some time.
> Also, we are losing track of block movements since log directory can also go
> to read-only but datanode would continue running.
> For 0.21 release, can we revert HDFS-457 or make it configurable?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1254) 0.20: mark dfs.supprt.append to be true by default for the 0.20-append branch
[ https://issues.apache.org/jira/browse/HDFS-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887399#action_12887399 ] M. C. Srivas commented on HDFS-1254: Can we fix the spelling? dfs.supprt.append --> dfs.support.append > 0.20: mark dfs.supprt.append to be true by default for the 0.20-append branch > - > > Key: HDFS-1254 > URL: https://issues.apache.org/jira/browse/HDFS-1254 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.20-append > > Attachments: append.txt > > > The 0.20-append branch supports append/sync for HDFS. Change the default > configuration to enable append. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1093) Improve namenode scalability by splitting the FSNamesystem synchronized section in a read/write lock
[ https://issues.apache.org/jira/browse/HDFS-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HDFS-1093: --- Attachment: NNreadwriteLock_trunk_3.txt I have purposely kept the try-catch block un-indented so that it is easier for you to review. Once the review is complete, I will upload a final patch with proper indentation. FSNamesystem.java - I removed the import of the ReadWriteLock. However, I kept the fsLock initializing inside the initialize method rather than moving it to the declaration. Most variables seem to be getting initialized inside the initialize() method, isn't it? getBlockLocations() - I now acquire the readLock and attempt to proceed ahead. If the access-time has to be set, then I release the readLock, acquire the writeLock and start all over again. Check for safemode should be inside the FSnamesystem lock, in fact more of it is coming in HDFS-988. setTimes() move isAccessTimeSupported() call out side the lock : done. CommitBlockSynchronization(): the key is to call the logSync() outside the FSnamesystem lock. And there is a LOG statement that prints out "src" after the call to logSync(). That is the reason why i had to declare "String src" right at the beginning of the method. I agree with you that there are code portions in processReport, createSymLinkInternal, startFileInternal() that can move outside the FSNamesystem lock. However, I would like to avoid doing this code reorganizatin as part of this JIRA, especially because it makes the code difficult to review. Also, this is not a regression because the original code has all these code inside the synchronized section anyway. Please let me know if you agree on this one. getDatanodeListForReport - move boolean and HashMap out of read lock: done. getSafeModeTip - readLock instead of writeLock : done BlockManager.java chooseUnderReplicatedBlocks - should we grab readLock instead of writeLock? I wold prefer to keep the writeLock because this method updates member variables of the class, e.g. missingBlocksInPrevIter removeStoredBlock - assert to be replaced by grab writeLock: I have the impression that all calls to removeStoredBlock already has the writeLock, that is the reason for the assert. Do you know of a code path via which this is not the case? DecommissionManager.java - readLock instead of writeLock; The DecommissionManager.check() calls FSNamesystem.checkDecommissionStateInterna() which, in turn, invokes node.setDecommissioned(). This updates the state of the node, hence I was confortable keeping the writeLock in this code path. FSDirectory.java - BackupStorage.loadCheckPoint() is now fixed. Do you have a suggestion on how I can fix the code in FSPermissionChecker.checkPermission()? > Improve namenode scalability by splitting the FSNamesystem synchronized > section in a read/write lock > > > Key: HDFS-1093 > URL: https://issues.apache.org/jira/browse/HDFS-1093 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: NNreadwriteLock.txt, NNreadwriteLock_trunk_1.txt, > NNreadwriteLock_trunk_2.txt, NNreadwriteLock_trunk_3.txt > > > Most critical data structures in the NameNode (NN) are protected by a > syncronized methods in the FSNamesystem class. This essentially makes > critical code paths in the NN single-threaded. However, a large percentage of > the NN calls are listStatus, getBlockLocations, etc which do not change > internal data structures at all, these are read-only calls. If we change the > FSNamesystem lock to a read/write lock, many of the above operations can > occur in parallel, thus improving the scalability of the NN. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1093) Improve namenode scalability by splitting the FSNamesystem synchronized section in a read/write lock
[ https://issues.apache.org/jira/browse/HDFS-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HDFS-1093: --- Status: Open (was: Patch Available) Incorporate review comments. > Improve namenode scalability by splitting the FSNamesystem synchronized > section in a read/write lock > > > Key: HDFS-1093 > URL: https://issues.apache.org/jira/browse/HDFS-1093 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: NNreadwriteLock.txt, NNreadwriteLock_trunk_1.txt, > NNreadwriteLock_trunk_2.txt > > > Most critical data structures in the NameNode (NN) are protected by a > syncronized methods in the FSNamesystem class. This essentially makes > critical code paths in the NN single-threaded. However, a large percentage of > the NN calls are listStatus, getBlockLocations, etc which do not change > internal data structures at all, these are read-only calls. If we change the > FSNamesystem lock to a read/write lock, many of the above operations can > occur in parallel, thus improving the scalability of the NN. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
