MapReduce Traces

2015-04-25 Thread Lixiang Ao
Hi all, I'm looking for some real-world Mapreduce traces (jobhistory) to analyze the characteristics. But I couldn't found any except for SWIM https://github.com/SWIMProjectUCB/SWIM/wiki/Workloads-repository that contains several traces from Facebook. However, they're incomplete and lack job

Re: MapReduce Traces

2015-04-25 Thread Lixiang Ao
, Lixiang Ao aolixi...@gmail.com wrote: Hi all, I'm looking for some real-world Mapreduce traces (jobhistory) to analyze the characteristics. But I couldn't found any except for SWIM https://github.com/SWIMProjectUCB/SWIM/wiki/Workloads-repository that contains several traces from Facebook

Re: Namenode HA failover time

2014-12-01 Thread Lixiang Ao
I am curious about this, too. On Sat, Nov 29, 2014 at 2:35 PM, Alice 6900848...@gmail.com wrote: Hi,all: Namenode HA (NFS, QJM) is available in hadoop 2.x (HDFS-1623). It provides fast failover for Namenode, but I can't find any description on how long does it take to recover from failure.

Re: Benchmark Failure

2014-03-22 Thread Lixiang Ao
following jirafor sameHope you also facing same issue... https://issues.apache.org/jira/browse/HDFS-4929 Thanks Regards Brahma Reddy Battula -- *From:* Lixiang Ao [aolixi...@gmail.com] *Sent:* Tuesday, March 18, 2014 10:34 AM *To:* user

Benchmark Failure

2014-03-17 Thread Lixiang Ao
Hi all, I'm running jobclient tests(on single node), other tests like TestDFSIO, mrbench succeed except nnbench. I got a lot of Exceptions but without any explanation(see below). Could anyone tell me what might went wrong? Thanks! 14/03/17 23:54:22 INFO hdfs.NNBench: Waiting in barrier for:

Run multiple HDFS instances

2013-04-18 Thread Lixiang Ao
] localhost: namenode running as process 20544. Stop it first. localhost: datanode running as process 20786. Stop it first. Starting secondary namenodes [0.0.0.0] 0.0.0.0: secondarynamenode running as process 21074. Stop it first. Is there a way to solve this? Thank you in advance, Lixiang Ao

Run multiple HDFS instances

2013-04-18 Thread Lixiang Ao
a different PID directory (I think its called HADOOP_PID_DIR) every time you invoke them. I instead prefer to start the daemons up via their direct command such as hdfs namenode and so and move them to the background, with a redirect for logging. On Thu, Apr 18, 2013 at 2:34 PM, Lixiang Ao aolixi

Re: Run multiple HDFS instances

2013-04-18 Thread Lixiang Ao
trying to understand why you might want to do that. On Thu, Apr 18, 2013 at 11:29 AM, Lixiang Ao aolixi...@gmail.comjavascript:_e({}, 'cvml', 'aolixi...@gmail.com'); wrote: I modified sbin/hadoop-daemon.sh, where HADOOP_PID_DIR is set. It works! Everything looks fine now. Seems direct

Re: Run multiple HDFS instances

2013-04-18 Thread Lixiang Ao
whether it works. 在 2013年4月19日星期五,Hemanth Yamijala 写道: Are you trying to implement something like namespace federation, that's a part of Hadoop 2.0 - http://hadoop.apache.org/docs/r2.0.3-alpha/hadoop-project-dist/hadoop-hdfs/Federation.html On Thu, Apr 18, 2013 at 10:02 PM, Lixiang Ao aolixi