In some implementations of Swimlanes, it doesn't work because of incompatibilities with Hadoop >= 0.21 due to a change in the JobHistory log format. The parser may need to be updated for this to work with newer version of Hadoop.
regards, Eric On Sun, Oct 7, 2012 at 3:07 PM, Jesse Chang <[email protected]> wrote: > I'm getting the following error message from postprocess.log after I > switched to Chukwa 0.4.0. > > 2012-10-07 14:17:41,166 ERROR main FSMDataLoader - > java.io.FileNotFoundException: File > /chukwa/temp/fsm_1349644589617_org.apache.hadoop.chukwa.analysis.salsa.fsm.JobHistoryTaskDataMapper > does not exist. > > I got the same kind of error messages when I used Chukwa 0.5.0. What am I > missing? > > Thanks, > Jesse > > On Wed, Oct 3, 2012 at 2:05 PM, Jesse Chang <[email protected]> wrote: >> >> OK. I added org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter >> to chukwaCollector.pipeline, and I am getting some data such as >> /chukwa/postProcess/demuxOutputDir_1349295946720/MyCluster/mapred_tasktracker. >> >> But it cannot process data further because of the following errors. >> >> 2012-10-03 13:10:45,023 ERROR pool-1-thread-1 MetricDataLoader - >> Initialization failed for: MyCluster. Please check jdbc configuration. >> >> I guess I need MySQL and follow the guide for the version 0.4. >> >> Jesse >> >> >> On Wed, Oct 3, 2012 at 1:24 AM, Jesse Chang <[email protected]> >> wrote: >>> >>> Hi, >>> >>> I am using Chukwa version 0.5.0 and the CDH4 package - "Version: 4.0.4 >>> (#79 built by jenkins on 20120821-1039 git: >>> 119fe7452f0c356dcde763a1d22eda9363d24ad4)". I re-built Chukwa 0.5.0 with >>> cdh4 jar files - hbase-0.92.1-cdh4.0.0.jar, >>> hadoop-common-2.0.0-cdh4.0.1.jar, hadoop-core-2.0.0-mr1-cdh4.0.1.jar, and >>> hadoop-hdfs-2.0.0-cdh4.0.1.jar. >>> >>> I could see some data such as shuffle_output_bytes was collected at >>> http://myhost:4080/hicc/jsp/graph_explorer.jsp. I ran the aggregation script >>> - share/chukwa/script/pig/ClusterSummary.pig - successfully to see Cluster >>> Status graphs and MapReduce graphs at http://myhost:4080/hicc/#. There are >>> no HDFS metrics graphs at the HDFS tab though. >>> >>> First of all, is the Swimlanes visualization available at Chukwa 0.5.0? >>> If not, what is the current status? >>> >>> How can I check if Demux is happening? I can see >>> org.apache.hadoop.chukwa.extraction.demux.DemuxManager is running, but I >>> don't see anything is really happening. What would be the cause? How can I >>> debug it? Is Demux still necessary to see a swimlanes visualization of >>> Mapreduce stages? >>> >>> Basically I followed the guide at >>> http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html and >>> http://incubator.apache.org/chukwa/docs/r0.5.0/admin.html. I just found that >>> Chukwa Demux was removed from the picture when I compared it to the 0.4.0 >>> guide at http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html. Still >>> does Chukwa 0.5.0 support Demux? >>> >>> What would be the best way to see swimlanes of mapreduce stages? >>> >>> Thanks, >>> Jesse >>> >>> >>> >>> >>> >> >
