Hadoop trunk will require a different configuration than the one described in the Quick_Start guide.
1. Apply this patch: HADOOP-7436, and rebuild hadoop. 2. Copy hadoop-metrics2.properties enclosed in this message to HADOOP_CONF_DIR Restart hadoop. You might need to match the HBase table schema with the metrics emitted by Hadoop metrics 2 framework. Hope this helps.
hadoop-metrics2.properties.template
Description: Binary data
regards, Eric On Oct 25, 2011, at 1:55 PM, SALAMI Patrick wrote: > Hi Eric, > There was actually no hadoop-metrics.properties file in trunk/conf; there was > only a hadoop-metrics2.properties file. My HADOOP_CONF_DIR has a > hadoop-metrics.properties file already. I copied hadoop-metrics2.properties > to HADOOP_CONF_DIR but I am not sure if I should keep the existing > hadoop-metrics.properties or delete it. > I also do see a hadoop-log4j.properties file in /conf. Should I copy this > file to my HADOOP_CONF_DIR as log4j.properties? > > Best, > > Patrick > > -----Original Message----- > From: Eric Yang [mailto:[email protected]] On Behalf Of Eric Yang > Sent: Tuesday, October 25, 2011 1:13 PM > To: [email protected] > Subject: Re: Chukwa trunk does not write to HBase > > Hi Patrick, > > Do you have hadoop-metrics.properties copied to HADOOP_CONF_DIR, and place > chukwa-hadoop-0.5.0-client.jar and json.jar in Hadoop classpath and restart > Hadoop? > > regards, > Eric > > On Oct 25, 2011, at 1:02 PM, SALAMI Patrick wrote: > >> Hi Eric, >> Thanks for your quick reply. I actually just fixed the problem: as per the >> instructions at http://wiki.apache.org/hadoop/Chukwa_Quick_Start I had >> commented out both chukwaCollector.writerClass and chukwaCollector.pipeline. >> This did not work; there were no exceptions in the logs but they were >> showing references to SeqFileWriter and no mention of HBaseWriter. I >> un-commented chukwaCollector.writerClass (see attached >> chukwa-collector-conf.xml) and now it is writing to HBase properly. >> >> There are some other issues though: >> - the hbase.schema file was missing some column families and one table >> (I fixed this locally, see attached updated hbase.schema). >> - However, it only seems to be writing SystemMetrics and no Hadoop or >> HDFS logs. I updated the log4j.properties file for Hadoop with >> log4j.appender.DRFA=org.apache.log4j.net.SocketAppender. I will maybe tinker >> with this some more and see if I can figure out what the issue is. >> >> Best, >> >> Patrick >> >> -----Original Message----- >> From: Eric Yang [mailto:[email protected]] On Behalf Of Eric Yang >> Sent: Tuesday, October 25, 2011 11:16 AM >> To: [email protected] >> Subject: Re: Chukwa trunk does not write to HBase >> >> Hi, >> >> What does the chukwa-collector-conf.xml looks like? >> Do you have hbase-site.xml and core-site.xml on the classpath? >> Is there any exception in the collector log file? >> >> regards, >> Eric >> >> On Oct 25, 2011, at 11:04 AM, SALAMI Patrick wrote: >> >>> I set up Chukwa trunk and have a local HBase cluster running in distributed >>> mode. Chukwa logs to HDFS but does not write anything to HBase. I followed >>> the instructions at http://wiki.apache.org/hadoop/Chukwa_Quick_Start. Has >>> anyone else noticed this issue? >>> >>> Thanks! >>> >>> Patrick >>> >>> >>> This message contains confidential information and is intended only for the >>> individual(s) named. If you are not the named addressee you should not >>> disseminate, distribute or copy this e-mail. Please notify the sender >>> immediately by e-mail if you have received this e-mail by mistake and >>> delete this e-mail from your system. >> >> >> This message contains confidential information and is intended only for the >> individual(s) named. If you are not the named addressee you should not >> disseminate, distribute or copy this e-mail. Please notify the sender >> immediately by e-mail if you have received this e-mail by mistake and delete >> this e-mail from your system. >> <chukwa-collector-conf.xml><hbase.schema> > > > This message contains confidential information and is intended only for the > individual(s) named. If you are not the named addressee you should not > disseminate, distribute or copy this e-mail. Please notify the sender > immediately by e-mail if you have received this e-mail by mistake and delete > this e-mail from your system.
