How I was sure data getting collected at collector, is by seeing the below statistics
Date:1357649516006 Now:1357649518803 numberHTTPConnection in time window:2 numberchunks in time window:7 lifetimechunks:2612 On Tue, Jan 8, 2013 at 6:20 PM, Shouvanik Haldar <[email protected] > wrote: > Hi, > > I was following the post -> http://www.undercloud.org/?p=429. I did the > steps as was mentioned here. But, when I was going to check in HDFS whether > data has come from chukwa, I cannot see the page. I get the exception as > > > > Also, I want to know what is the default location of chukwa logs. I found > out a location -> > "/home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/logs". Is this the > location? Previously, I had some logs inside this directory. I have deleted > these logs. Now, no logs are getting created. > > Previously, when logs were existing, I checked one log entry. It says > "nice: /home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/bin/chukwa: No > such file or directory". > > > I have edited bash_profile to give the path of CHUKWA_HOME. Is this > required? > > I am using "chukwa-incubating-0.5.0" and "hadoop-1.0.4". I can see data > getting collected at the collector by browsing the url => > http://ec2-54-234-57-44.compute-1.amazonaws.com:8080/chukwa?ping=true > > > > > > But somehow, data not getting inside hdfs. Please help. > > > Regards, > Shouvanik > > > On Tue, Jan 8, 2013 at 10:22 AM, Eric Yang <[email protected]> wrote: > >> The quick start guide may help: >> >> http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html >> >> regards, >> Eric >> >> >> On Mon, Jan 7, 2013 at 1:17 AM, Shouvanik Haldar < >> [email protected]> wrote: >> >>> Hi, >>> I want to install chukwa on a single node and test it. Where can I find >>> a suitable document? >>> >>> >>> On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang <[email protected]> wrote: >>> >>>> There are a few things that you need to do: >>>> >>>> 1. Configure the collector to use HBaseWriter. >>>> >>>> http://incubator.apache.org/chukwa/docs/r0.5.0/collector.html >>>> >>>> Make sure that hadoop and hbase configuration are in the classpath of >>>> the collector. >>>> >>>> 2. Write a demux parser for your log file to store on HBase, and this >>>> runs in chukwa collector. >>>> >>>> http://incubator.apache.org/chukwa/docs/r0.5.0/programming.html >>>> >>>> 3. What log4j adaptors and appenders are you using to write the log >>>> files? >>>> I would recommend to use socket appender from Log4j and use socket >>>> adaptor to listen for the incoming data stream. This will ensure the >>>> records are delimited properly. >>>> >>>> Hope this helps. >>>> >>>> regards, >>>> Eric >>>> >>>> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh < >>>> [email protected]> wrote: >>>> >>>>> I am using chukwa 0.5. My probleam is want to store the logs from >>>>> log4j or other party which is not hadoop.Also i want to write the logs to >>>>> hbase. >>>>> But i am not ale to do so. I have configuerd log4j.properties file so >>>>> i am able to see th logs in hdfs. >>>>> what are the configuration or steps using which i can write the logs >>>>> to hbase. >>>>> Please help. >>>>> ________________________________________ >>>>> From: Eric Yang [[email protected]] >>>>> Sent: Saturday, January 05, 2013 11:55 PM >>>>> To: [email protected] >>>>> Subject: Re: Writing logs collected by chukwa to hbase >>>>> >>>>> What version of Chukwa are you running? >>>>> >>>>> regards, >>>>> Eric >>>>> >>>>> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh < >>>>> [email protected]<mailto:[email protected]>> >>>>> wrote: >>>>> Hi, >>>>> I am using Chukwa to collect log4j logs which I can >>>>> see in HDFS. But I am not able to write it to HBASE. Can anyone help ? >>>>> >>>>> Regards, >>>>> Neha Singh >>>>> >>>>> >>>>> ________________________________ >>>>> The contents of this e-mail and any attachment(s) may contain >>>>> confidential or privileged information for the intended recipient(s). >>>>> Unintended recipients are prohibited from taking action on the basis of >>>>> information in this e-mail and using or disseminating the information, and >>>>> must notify the sender and delete it from their system. L&T Infotech will >>>>> not accept responsibility or liability for the accuracy or completeness >>>>> of, >>>>> or the presence of any virus or disabling code in this e-mail" >>>>> >>>>> >>>> >>> >>> >>> -- >>> Thanks, >>> *Shouvanik* >>> >> >> > > > -- > Thanks, > *Shouvanik* > -- Thanks, *Shouvanik*
