Re: Please explain FSNamesystemState TotalLoad

2013-06-07 Thread Nick Niemeyer 
Regarding TotalLoad, what would be normal operating tolerances per node for 
this metric?  When should one become concerned?  Thanks again to everyone 
participating in this community.  :)

Nick



From: Suresh Srinivas sur...@hortonworks.commailto:sur...@hortonworks.com
Reply-To: user@hadoop.apache.orgmailto:user@hadoop.apache.org 
user@hadoop.apache.orgmailto:user@hadoop.apache.org
Date: Thursday, June 6, 2013 4:14 PM
To: hdfs-u...@hadoop.apache.orgmailto:hdfs-u...@hadoop.apache.org 
user@hadoop.apache.orgmailto:user@hadoop.apache.org
Subject: Re: Please explain FSNamesystemState TotalLoad

It is the total number of transceivers (readers and writers) reported by all 
the datanodes. Datanode reports this count in periodic heartbeat to the 
namenode.


On Thu, Jun 6, 2013 at 1:48 PM, Nick Niemeyer 
nnieme...@riotgames.commailto:nnieme...@riotgames.com wrote:
Can someone please explain what TotalLoad represents below?  Thanks for your 
response in advance!

Version: hadoop-0.20-namenode-0.20.2+923.197-1

Example pulled from the output of via the name node:
  # curl -i http://localhost:50070/jmx

{
name : hadoop:service=NameNode,name=FSNamesystemState,
modelerType : org.apache.hadoop.hdfs.server.namenode.FSNamesystem,
CapacityTotal : #,
CapacityUsed : #,
CapacityRemaining : #,
TotalLoad : #,
BlocksTotal : #,
FilesTotal : #,
PendingReplicationBlocks : 0,
UnderReplicatedBlocks : 0,
ScheduledReplicationBlocks : 0,
FSState : Operational
  }


Thanks,
Nick



--
http://hortonworks.com/download/

Re: Please explain FSNamesystemState TotalLoad

2013-06-07 Thread Suresh Srinivas 
On Fri, Jun 7, 2013 at 9:10 AM, Nick Niemeyer nnieme...@riotgames.comwrote:

  Regarding TotalLoad, what would be normal operating tolerances per node
 for this metric?  When should one become concerned?  Thanks again to
 everyone participating in this community.  :)


Why do you want to be concered :) I have not seen many issues related to
high TotalLoad.

This is mainly useful in terms of understanding how many concurrent
jobs/file accesses are happening and how busy datanodes are. When you are
debugging issues where cluster slow down due to overload, or correlating a
run of big jobs, this is useful. Knowing what it represent, you would find
many other uses as well.


   From: Suresh Srinivas sur...@hortonworks.com
 Reply-To: user@hadoop.apache.org user@hadoop.apache.org
 Date: Thursday, June 6, 2013 4:14 PM
 To: hdfs-u...@hadoop.apache.org user@hadoop.apache.org
 Subject: Re: Please explain FSNamesystemState TotalLoad

   It is the total number of transceivers (readers and writers) reported
 by all the datanodes. Datanode reports this count in periodic heartbeat to
 the namenode.


 On Thu, Jun 6, 2013 at 1:48 PM, Nick Niemeyer nnieme...@riotgames.comwrote:

   Can someone please explain what TotalLoad represents below?  Thanks
 for your response in advance!

  Version: hadoop-0.20-namenode-0.20.2+923.197-1

  Example pulled from the output of via the name node:
   # curl -i http://localhost:50070/jmx

  {
 name : hadoop:service=NameNode,name=FSNamesystemState,
 modelerType : org.apache.hadoop.hdfs.server.namenode.FSNamesystem,
 CapacityTotal : #,
 CapacityUsed : #,
 CapacityRemaining : #,
* TotalLoad : #,*
 BlocksTotal : #,
 FilesTotal : #,
 PendingReplicationBlocks : 0,
 UnderReplicatedBlocks : 0,
 ScheduledReplicationBlocks : 0,
 FSState : Operational
   }


  Thanks,
 Nick




  --
 http://hortonworks.com/download/




-- 
http://hortonworks.com/download/

Please explain FSNamesystemState TotalLoad

2013-06-06 Thread Nick Niemeyer 
Can someone please explain what TotalLoad represents below?  Thanks for your 
response in advance!

Version: hadoop-0.20-namenode-0.20.2+923.197-1

Example pulled from the output of via the name node:
  # curl -i http://localhost:50070/jmx

{
name : hadoop:service=NameNode,name=FSNamesystemState,
modelerType : org.apache.hadoop.hdfs.server.namenode.FSNamesystem,
CapacityTotal : #,
CapacityUsed : #,
CapacityRemaining : #,
TotalLoad : #,
BlocksTotal : #,
FilesTotal : #,
PendingReplicationBlocks : 0,
UnderReplicatedBlocks : 0,
ScheduledReplicationBlocks : 0,
FSState : Operational
  }


Thanks,
Nick

Re: Please explain FSNamesystemState TotalLoad

2013-06-06 Thread Suresh Srinivas 
It is the total number of transceivers (readers and writers) reported by
all the datanodes. Datanode reports this count in periodic heartbeat to the
namenode.


On Thu, Jun 6, 2013 at 1:48 PM, Nick Niemeyer nnieme...@riotgames.comwrote:

   Can someone please explain what TotalLoad represents below?  Thanks for
 your response in advance!

  Version: hadoop-0.20-namenode-0.20.2+923.197-1

  Example pulled from the output of via the name node:
   # curl -i http://localhost:50070/jmx

  {
 name : hadoop:service=NameNode,name=FSNamesystemState,
 modelerType : org.apache.hadoop.hdfs.server.namenode.FSNamesystem,
 CapacityTotal : #,
 CapacityUsed : #,
 CapacityRemaining : #,
* TotalLoad : #,*
 BlocksTotal : #,
 FilesTotal : #,
 PendingReplicationBlocks : 0,
 UnderReplicatedBlocks : 0,
 ScheduledReplicationBlocks : 0,
 FSState : Operational
   }


  Thanks,
 Nick




-- 
http://hortonworks.com/download/