Re: HttpConfig API changes in hadoop 2.4.0

2014-04-30 Thread Haohui Mai
Hi, Can you describe your use cases, that is, how the prefix is used? Usually you can get around with it by generating relative URLs, which starts at //. ~Haohui On Wed, Apr 30, 2014 at 2:31 PM, Gaurav Gupta gaurav.gopi...@gmail.comwrote: Hi, I was using hadoop 2.2.0 to build my

Re: datanode is slow

2014-02-20 Thread Haohui Mai
It looks like your datanode is overloaded. You can scale your system by adding more datanodes. You can also try tighten the admission control to recover. You can lower the number of dfs.datanode.max.transfer.threads so that the datanode accepts fewer concurrent requests (but which also means that

Re: Is it possible to access a hadoop 1 file system (hdfs) via the hadoop 2.2.0 command line tools?

2014-02-01 Thread Haohui Mai
You can use webhdfs / hftp to access the data from the Hadoop 1 cluster. Haohui On Sat, Feb 1, 2014 at 2:39 AM, Jitendra Yadav jeetuyadav200...@gmail.comwrote: In this case I believe Hadoop client version should be same as Hadoop cluster version. Thanks Jitendra On Sat, Feb 1, 2014 at

Re: Processing steps of NameNode Secondary NameNode

2014-01-27 Thread Haohui Mai
Conceptually you can think of the namenode is similar to a journal file system. For each write, it updates the in-memory data structure, persists the operations on the stable storage (i.e., calling sync to flush the buffer of the edit logs), then responds to the client. Note that all writes are

Re: kerberos for outside threads

2014-01-21 Thread Haohui Mai
Hi Koert, I'm wondering what is the end-to-end goal you want to achieve. You can disable security in Hadoop, where the cluster does not perform additional authentication. Obviously you can go without kerberos in this case and protect your clusters with other measures you've mentioned.

Re: Admin Scripts

2014-01-20 Thread Haohui Mai
Hi, pssh should be able do to the job. ~Haohui On Mon, Jan 20, 2014 at 10:34 AM, hadoop hive hadooph...@gmail.com wrote: Hi Folks, Does anyone have idea regrading, start and stop services on all the nodes in parallel. i am facing an issue i have a big cluster like 1000 nodes and i want

Re: JAVA cannot execute binary file

2014-01-07 Thread Haohui Mai
Seems like that you have a wrong JAVA_HOME. You can check whether the directory exists, or search around a little bit to find the right configuration with respect to your distribution. ~Haohui On Tue, Jan 7, 2014 at 2:07 PM, navaz navaz@gmail.com wrote: Hadoop env export