Hey Eric,
So as a test, i copied TsProcessor.java to TsProcessor2.java. I changed
the references for TsProcessor to be TsProcessor2 and updated my
chukwa-demux.conf to be
<property>
<name>TsProcessor</name>
<value>org.apache.hadoop.chukwa.extraction.demux.processor.mapper.TsProcessor2</value>
<description>Parser class for </description>
</property>
I then ran ant in the root and copied build/collector-0.4.0.war and
build/chukwa-core.jar to the root and started collector and demux.
I am now getting the following errors, any ideas ?
org.apache.hadoop.chukwa.extraction.demux.processor.mapper.UnknownRecordTypeException:
Unknown
parserClass:org.apache.hadoop.chukwa.extraction.demux.processor.mapper.TsProcessor2
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.chukwa.extraction.demux.processor.mapper.TsProcessor2
On Wed, Oct 26, 2011 at 1:00 PM, Eric Yang <[email protected]> wrote:
> Yes, write a mapper class which extends
> org/apache/hadoop/chukwa/extraction/demux/processor/mapper/AbstractProcessor.java.
> There are several extraction class in
> org/apache/hadoop/chukwa/extraction/demux/processor/mapper as examples.
> Once the extraction class is written, configure chukwa-demux-conf.xml to
> map the data type to the new extraction class.
>
> regards,
> Eric
>
> On Oct 26, 2011, at 5:48 AM, AD wrote:
>
> > interesting. So demux is now part of the collector and does not need to
> be run as another job?
> >
> > Since demux is doing basic ETL into hbase, is there a way to actually
> parse the fields of the log record and insert those into sep fields into
> Hbase to run mapreduce there (instead of the whole body as a field is what I
> think is happening)
> >
> > On Wed, Oct 26, 2011 at 12:59 AM, SALAMI Patrick <[email protected]>
> wrote:
> > Ok, thanks!
> >
> > -----Original Message-----
> > From: Eric Yang [mailto:[email protected]] On Behalf Of Eric Yang
> > Sent: Tuesday, October 25, 2011 6:19 PM
> > To: [email protected]
> > Subject: Re: Chukwa trunk does not write to HBase
> >
> > Demuxer for Chukwa 0.4 is a mapreduce job acting as ETL process to
> convert the data into semi structure format for further processing. For
> Chukwa trunk with HBase, demux is running as part of collector for ETL.
> Hence, there is no need to run demux process.
> >
> > regards,
> > Eric
> >
> > On Oct 25, 2011, at 3:49 PM, SALAMI Patrick wrote:
> >
> > > Also, while we are chatting, I was hoping to understand the role of the
> demuxer. I am assuming that HICC pulls all of its data from HBase. If using
> HBase, is it still necessary to run the demuxer? I didn't see any mention of
> it in the latest quick start guide.
> > >
> > > Thanks!
> > >
> > > Patrick
> > >
> > > -----Original Message-----
> > > From: Eric Yang [mailto:[email protected]] On Behalf Of Eric Yang
> > > Sent: Tuesday, October 25, 2011 2:45 PM
> > > To: [email protected]
> > > Subject: Re: Chukwa trunk does not write to HBase
> > >
> > > Hadoop trunk will require a different configuration than the one
> described in the Quick_Start guide.
> > >
> > > 1. Apply this patch: HADOOP-7436, and rebuild hadoop.
> > > 2. Copy hadoop-metrics2.properties enclosed in this message to
> HADOOP_CONF_DIR
> > >
> > > Restart hadoop. You might need to match the HBase table schema with
> the metrics emitted by Hadoop metrics 2 framework.
> > > Hope this helps.
> > >
> > >
> > > This message contains confidential information and is intended only for
> the individual(s) named. If you are not the named addressee you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately by e-mail if you have received this e-mail by mistake and delete
> this e-mail from your system.
> >
> >
> > This message contains confidential information and is intended only for
> the individual(s) named. If you are not the named addressee you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately by e-mail if you have received this e-mail by mistake and delete
> this e-mail from your system.
> >
>
>
