Hi all,
I have applied the patch and set the property in chukwa-demux-conf.xml file as
follows :
<property>
<name>chukwa.demux.mapper.default.processor</name>
<value>com.test.genericdemuxer.xyz</value>
</property>
<property>
<name>chukwa.demux.reducer.default.processor</name>
<value>com.test.genericdemuxer.xyz</value>
</property>
I tested the whole process from log collection to creation of .evt files in
repos, that worked fine.
Please note I have not added any datatype for my logtype in demux-conf file so
that it can go to my processor that I have set in the property file.
I also tested the whole process again without these entries in the demux-conf
file to check the default behavior of chukwa, but surprisingly that also worked
fine without any error.
There was a difference in the sizes of the .evt file that generated from both
the process. I am not able to figure out that whether my processor is getting
used or not.
How can I get to know that it is using my processor?
I also have a query that in demux.java , we have code where DefaultProcessor
is hardcoded, and in documentation it is written that TSProcessor will the
default processor will be used if we do not specify the datatype for particular
log type in demux-con.xml file.
String defaultProcessor = Demux.jobConf.get(
"chukwa.demux.mapper.default.processor",
"org.apache.hadoop.chukwa.extraction.demux.processor.mapper.DefaultProcessor");
String processorClass = Demux.jobConf.get(chunk.getDataType(),
defaultProcessor);
I think that either I am missing something or not able to clearly understand
the flow.
Please suggest.
Regards
Stuti Awasthi
-----Original Message-----
From: Bill Graham [mailto:[email protected]]
Sent: Tuesday, September 28, 2010 9:32 AM
To: [email protected]
Cc: Jaydeep Ayachit
Subject: Re: Changing default demuxer
This can be changed with the following configs:
chukwa.demux.mapper.default.processor
chukwa.demux.reducer.default.processor
Reference JIRA:
https://issues.apache.org/jira/browse/CHUKWA-473
This is part from Chukwa 0.4.0.
On Mon, Sep 27, 2010 at 5:47 AM, Jaydeep Ayachit
<[email protected]> wrote:
> Hello,
>
>
>
> Chukwa by default uses TSProcessor demuxer for data types that are not
> defined in chukwa-demux-config file.
>
> Can this be changed through some config file?
>
>
>
> I looked through the code and saw that TSProcessor is hard coded as default
> demuxer.
>
> I am using chukwa 0.4
>
>
>
> Regards
>
> Jaydeep
>
>
>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the
property of Persistent Systems Ltd. It is intended only for the use of the
individual or entity to which it is addressed. If you are not the intended
recipient, you are not authorized to read, retain, copy, print, distribute or
use this message. If you have received this communication in error, please
notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.
