Re: problem with HDFS caching in Hadoop 2.3

How was this cluster configured? Run dfsadmin -report to see the aggregate configured cache capacity as seen by the NN. You need to configure some GBs of cache on each DN and also raise the ulimit for max locked memory. I'll also note that you are unlikely to see speedups with most mapreduce jobs,

[jira] [Created] (HDFS-6083) TestQuorumJournalManager#testChangeWritersLogsOutOfSync2 occasionally fails

Ted Yu created HDFS-6083: Summary: TestQuorumJournalManager#testChangeWritersLogsOutOfSync2 occasionally fails Key: HDFS-6083 URL: https://issues.apache.org/jira/browse/HDFS-6083 Project: Hadoop HDFS

[jira] [Created] (HDFS-6082) List all NNs with state

Rafal Wojdyla created HDFS-6082: --- Summary: List all NNs with state Key: HDFS-6082 URL: https://issues.apache.org/jira/browse/HDFS-6082 Project: Hadoop HDFS Issue Type: Improvement Com

[jira] [Created] (HDFS-6081) TestRetryCacheWithHA#testCreateSymlink occasionally fails in trunk

Ted Yu created HDFS-6081: Summary: TestRetryCacheWithHA#testCreateSymlink occasionally fails in trunk Key: HDFS-6081 URL: https://issues.apache.org/jira/browse/HDFS-6081 Project: Hadoop HDFS Issue T

Hadoop-Hdfs-trunk - Build # 1696 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1696/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 12959 lines...] [INFO] [INFO] --- maven-antrun-plugin:1.

Build failed in Jenkins: Hadoop-Hdfs-trunk #1696

See Changes: [wang] HDFS-3405. Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages. Contributed by Vinayakumar B. [wang] Revert HDFS-3405 for recommit with correct renamed files [wang] Remove extra

Fwd: problem with HDFS caching in Hadoop 2.3

add dev. -- Forwarded message -- From: "hwpstorage" Date: Mar 7, 2014 11:38 PM Subject: problem with HDFS caching in Hadoop 2.3 To: Cc: Hello, It looks like the HDFS caching does not work well. The cached log file is around 200MB. The hadoop cluster has 3 nodes, each has 4GB mem