[jira] [Created] (HDFS-7359) NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address.

2014-11-04 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-7359: --- Summary: NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address. Key: HDFS-7359 URL: https://issues.apache.o

Re: Why do reads take as long as replicated writes?

2014-11-04 Thread daemeon reiydelle
Reads can be faster than writes for smaller bursts of IO in part due to disk and memory caching of reads (if you turn on write back (not recommended!) your numbers above are likely to get closer together). As your volume of IO increases, you tend to reach a point where you are bound (more or less)

Re: Why do reads take as long as replicated writes?

2014-11-04 Thread Andrew Wang
I would advise against using TestDFSIO, instead trying TeraGen and TeraValidate. IIRC TestDFSIO doesn't actually schedule for task locality, so it's not very good if you have a cluster bigger than your replication factor. You might be network bound as you try to read more files. Best, Andrew On T

[jira] [Created] (HDFS-7358) Clients may get stuck waiting when using ByteArrayManager

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-7358: - Summary: Clients may get stuck waiting when using ByteArrayManager Key: HDFS-7358 URL: https://issues.apache.org/jira/browse/HDFS-7358 Project: Hadoop HDFS

[jira] [Created] (HDFS-7357) FSNamesystem.checkFileProgress should log file path

2014-11-04 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-7357: - Summary: FSNamesystem.checkFileProgress should log file path Key: HDFS-7357 URL: https://issues.apache.org/jira/browse/HDFS-7357 Project: Hadoop HDFS

[jira] [Created] (HDFS-7356) Use DirectoryListing.hasMore() directly in nfs

2014-11-04 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-7356: Summary: Use DirectoryListing.hasMore() directly in nfs Key: HDFS-7356 URL: https://issues.apache.org/jira/browse/HDFS-7356 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-7355) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.

2014-11-04 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-7355: --- Summary: TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. Key: HDFS-7355 URL: https://issues.apache.

[jira] [Created] (HDFS-7354) Support parity blocks in block management

2014-11-04 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-7354: --- Summary: Support parity blocks in block management Key: HDFS-7354 URL: https://issues.apache.org/jira/browse/HDFS-7354 Project: Hadoop HDFS Issue Type: Sub-task

Why do reads take as long as replicated writes?

2014-11-04 Thread Eitan Rosenfeld
I am benchmarking my cluster of 16 nodes (all in one rack) with TestDFSIO on Hadoop 1.0.4. For simplicity, I turned off speculative task execution and set the max map and reduce tasks to 1. With a replication factor of 2, writing 1 file of 5GB takes twice as long as reading 1 file. This result se

[jira] [Created] (HDFS-7353) Common Erasure Codec API and plugin support

2014-11-04 Thread Kai Zheng (JIRA)
Kai Zheng created HDFS-7353: --- Summary: Common Erasure Codec API and plugin support Key: HDFS-7353 URL: https://issues.apache.org/jira/browse/HDFS-7353 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-7352) Common side changes for HDFS Erasure coding support

2014-11-04 Thread Uma Maheswara Rao G (JIRA)
Uma Maheswara Rao G created HDFS-7352: - Summary: Common side changes for HDFS Erasure coding support Key: HDFS-7352 URL: https://issues.apache.org/jira/browse/HDFS-7352 Project: Hadoop HDFS

[jira] [Created] (HDFS-7351) Document the HDFS Erasure Coding feature

2014-11-04 Thread Uma Maheswara Rao G (JIRA)
Uma Maheswara Rao G created HDFS-7351: - Summary: Document the HDFS Erasure Coding feature Key: HDFS-7351 URL: https://issues.apache.org/jira/browse/HDFS-7351 Project: Hadoop HDFS Issue Ty

[jira] [Created] (HDFS-7350) WebHDFS: Support EC commands through webhdfs

2014-11-04 Thread Uma Maheswara Rao G (JIRA)
Uma Maheswara Rao G created HDFS-7350: - Summary: WebHDFS: Support EC commands through webhdfs Key: HDFS-7350 URL: https://issues.apache.org/jira/browse/HDFS-7350 Project: Hadoop HDFS Issu

[jira] [Created] (HDFS-7349) Support DFS command for the EC encoding

2014-11-04 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-7349: --- Summary: Support DFS command for the EC encoding Key: HDFS-7349 URL: https://issues.apache.org/jira/browse/HDFS-7349 Project: Hadoop HDFS Issue Type: Sub-task