[jira] [Created] (HDFS-7592) A bug in BlocksMap that cause NameNode memory leak.

2015-01-07 Thread JichengSong (JIRA)
JichengSong created HDFS-7592: - Summary: A bug in BlocksMap that cause NameNode memory leak. Key: HDFS-7592 URL: https://issues.apache.org/jira/browse/HDFS-7592 Project: Hadoop HDFS Issue Type:

[jira] [Created] (HDFS-7591) hdfs classpath command should support same options as hadoop classpath.

2015-01-07 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-7591: --- Summary: hdfs classpath command should support same options as hadoop classpath. Key: HDFS-7591 URL: https://issues.apache.org/jira/browse/HDFS-7591 Project: Hadoop HDF

[jira] [Resolved] (HDFS-1213) Implement an Apache Commons VFS Driver for HDFS

2015-01-07 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth resolved HDFS-1213. - Resolution: Not a Problem There is now an HDFS provider implemented in the Apache Commons VFS tree:

[jira] [Created] (HDFS-7590) Stabilize and document getBlockLocation API in WebHDFS

2015-01-07 Thread Jakob Homan (JIRA)
Jakob Homan created HDFS-7590: - Summary: Stabilize and document getBlockLocation API in WebHDFS Key: HDFS-7590 URL: https://issues.apache.org/jira/browse/HDFS-7590 Project: Hadoop HDFS Issue Type

Re: Getting a list of all StorageInfo

2015-01-07 Thread Chris Nauroth
Hi Youssef, You might want to review the code for BlockPlacementPolicyDefault to get a sense for how an implementation works. Perhaps you could even start your own work by copying and modifying that class. This code would show you how to get ahold of a DatanodeDescriptor and then use it to call

Re: HDFS 2.6.0 upgrade ends with missing blocks

2015-01-07 Thread dlmarion
I went back and read through the comments in HDFS-6482 and found that the restrictions (question #2 below) have already been documented at https://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F I will update HDFS-1312 with this information

Hadoop-Hdfs-trunk - Build # 1998 - Still Failing

2015-01-07 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1998/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 6190 lines...] [INFO] Executing tasks main: [mkdir]

Build failed in Jenkins: Hadoop-Hdfs-trunk #1998

2015-01-07 Thread Apache Jenkins Server
See Changes: [aw] HADOOP-11058. Missing HADOOP_CONF_DIR generates strange results (Masatake Iwasaki via aw) [jlowe] HADOOP-11445. Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split. C

HDFS 2.6.0 upgrade ends with missing blocks

2015-01-07 Thread dlmarion
I recently upgraded from CDH 5.1.2 to CDH 5.3.0. I know, contact Cloudera, but this is actually a generic issue. After the upgrade I brought up the DNs and after all of them had checked in I ended up with missing blocks. I tracked this down in the DN logs to an error at startup where the DN is