NullPointerException in offLineImageViewer

2014-03-19 Thread Chetan Agrawal 
I want to access and study the hadoop cluster's metadata which is stored in 
fsimage file on namenode machine. i came to know that offLineImageViewer is 
used to do so. But when i am trying doing it i am getting an exception.
/usr/hadoop/hadoop-1.2.1# bin/hadoop oiv -i fsimage -o fsimage.txt Warning: 
$HADOOP_HOME is deprecated.
Exception in thread main java.lang.NullPointerException   at 
org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.go(OfflineImageViewer.java:141)
   at 
org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.main(OfflineImageViewer.java:261)
i am not able to solve this error. Is it due to that warning (Warning: 
$HADOOP_HOME is deprecated.) or there is something else?

NullpointerException in OffLineImageViewer

2014-03-19 Thread Chetan Agrawal 
I want to access and study the hadoop cluster's metadata which is stored in 
fsimage file on namenode machine. i came to know that offLineImageViewer is 
used to do so. But when i am trying doing it i am getting an exception.
/usr/hadoop/hadoop-1.2.1# bin/hadoop oiv -i fsimage -o fsimage.txt Warning: 
$HADOOP_HOME is deprecated.
Exception in thread main java.lang.NullPointerException   at 
org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.go(OfflineImageViewer.java:141)
   at 
org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.main(OfflineImageViewer.java:261)
i am not able to solve this error. Is it due to that warning (Warning: 
$HADOOP_HOME is deprecated.) or there is something else?I am having 
hadoop-1.2.1 version

memory management module of Namenode

2014-01-28 Thread Chetan Agrawal 
I read that the namenode keeps all the metadata record in main memory right for 
fast access.
I want to study the code where the namenode is instructed to do so i.e. to keep 
metadata record in main memory. where can i find the source code file for this 
namenode memory management? i am using github and have 'hadoop-common' 
repository.

RE: Datanode Shutting down automatically

2014-01-25 Thread Chetan Agrawal 
thanx a lot...is working fine now

 From: ha...@cloudera.com
 Date: Sat, 25 Jan 2014 12:15:08 +0530
 Subject: Re: Datanode Shutting down automatically
 To: user@hadoop.apache.org
 
 You reformatted your NameNode at some point, but likely failed to also
 clear out the DN data directories, which would not auto-wipe
 themselves.
 
 Clear the contents of /app/hadoop/tmp/dfs/data at the DN and it should
 start up the next time you invoke it.
 
 P.s. Please do not email gene...@hadoop.apache.org with any user
 questions. It exists for project level discussions and announcements.
 
 On Sat, Jan 25, 2014 at 11:15 AM, Pranav Gadekar ppgadekar...@gmail.com 
 wrote:
  This is my log file.
 
  014-01-24 17:24:58,238 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
  STARTUP_MSG:
  /
  STARTUP_MSG: Starting DataNode
  STARTUP_MSG:   host = user/127.0.1.1
  STARTUP_MSG:   args = []
  STARTUP_MSG:   version = 1.2.1
  STARTUP_MSG:   build =
  https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r
  1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
  STARTUP_MSG:   java = 1.6.0_27
  /
  2014-01-24 17:24:58,622 INFO org.apache.hadoop.metrics2.impl.MetricsConfig:
  loaded properties from hadoop-metrics2.properties
  2014-01-24 17:24:58,669 INFO
  org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
  MetricsSystem,sub=Stats registered.
  2014-01-24 17:24:58,670 INFO
  org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period
  at 10 second(s).
  2014-01-24 17:24:58,670 INFO
  org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system
  started
  2014-01-24 17:24:58,877 INFO
  org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
  registered.
  2014-01-24 17:24:58,880 WARN
  org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
  exists!
  2014-01-24 17:25:10,778 ERROR
  org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
  Incompatible namespaceIDs in /app/hadoop/tmp/dfs/data: namenode namespaceID
  = 102782159; datanode namespaceID = 1227483104
  at
  org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232)
  at
  org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:147)
  at
  org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:414)
  at
  org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:321)
  at
  org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1712)
  at
  org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1651)
  at
  org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1669)
  at
  org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1795)
  at
  org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1812)
 
  2014-01-24 17:25:10,779 INFO
  org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
  /
  SHUTDOWN_MSG: Shutting down DataNode at user/127.0.1.1
  /
  2014-01-24 17:26:13,413 INFO
  org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
  /
  STARTUP_MSG: Starting DataNode
  STARTUP_MSG:   host = user/127.0.1.1
  STARTUP_MSG:   args = []
  STARTUP_MSG:   version = 1.2.1
  STARTUP_MSG:   build =
  https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r
  1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
  STARTUP_MSG:   java = 1.6.0_27
  /
  2014-01-24 17:26:13,510 INFO org.apache.hadoop.metrics2.impl.MetricsConfig:
  loaded properties from hadoop-metrics2.properties
  2014-01-24 17:26:13,518 INFO
  org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
  MetricsSystem,sub=Stats registered.
  2014-01-24 17:26:13,518 INFO
  org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period
  at 10 second(s).
  2014-01-24 17:26:13,518 INFO
  org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system
  started
  2014-01-24 17:26:13,626 INFO
  org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
  registered.
  2014-01-24 17:26:13,628 WARN
  org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
  exists!
  2014-01-24 17:26:28,860 ERROR
  org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
  Incompatible namespaceIDs in /app/hadoop/tmp/dfs/data: namenode namespaceID
  = 102782159; datanode namespaceID = 1227483104
  at
  org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232)
  at