Can't browse the filesystem By Internet Explorer

2013-01-20 Thread kira . wang
Hi, I have installed a cluster with hadoop2.0.0-alpha, totally 4 pc works, 1 Namenode, 3 Datanodes. I opened the http://master:50070/dfshealth.jps page by Chrome from a remote pc, it's all right, However, when I clicked the "browse the filesystem", the Chrome redirect to http://slave16:50075/brow

Re: Time taken for launching Application Master

2013-01-20 Thread Rahul Jain
Check your node manager logs to understand the bottleneck first. When we had a similar issue on recent version of hadoop, which includes fix for MAPREDUCE-4068: we rearranged our job jar file to reduce time spent on 'expanding' the job jar file by the node manager(s). -Rahul On Sun, Jan 20, 2013

Re: How to copy log files from remote windows machine to Hadoop cluster

2013-01-20 Thread Mahesh Balija
Hi Mirko, Thanks for your reply. It works for me as well. Now I was able to mount the folder on the master node and configured Flume such that it can either poll for logs in real time or even for periodic retrieval. Thanks, Mahesh Balija. Calsof Labs. On Thu, Jan 17, 2013 a

Re: Fair Scheduler of Hadoop

2013-01-20 Thread Joep Rottinghuis
Lin, The article you are reading us old. Fair scheduler does have preemption. Tasks get killed and rerun later, potentially on a different node. You can set a minimum / guaranteed capacity. The sum of those across pools would typically equal the total capacity of your cluster or less. Then you c

Fwd: new join algorithm using mapreduce

2013-01-20 Thread Vikas Jadhav
-- Forwarded message -- From: Vikas Jadhav Date: Sat, Jan 19, 2013 at 10:58 PM Subject: new join algorithm using mapreduce To: user@hadoop.apache.org I am writing new join algorithm using hadoop and want to do multi way join in single mapreduce job map --> processes all dataset

Using JCUDA with MapReduce

2013-01-20 Thread Amit Sela
Hi all, I was wondering if anyone here tried using the GPU of a Hadoop Node to enhance MapReduce processing ? I read about it but it always comes down to heavy computations such as Matrix multiplications and Mote Carlo algorithms. Did anyone try it with MapReduce jobs that analyze logs or any ot

Fair Scheduler of Hadoop

2013-01-20 Thread Lin Ma
Hi guys, I have a quick question regarding to fire scheduler of Hadoop, I am reading this article => http://blog.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/, my question is from the following statements, "There is currently no support for preemption of long tasks, but this is being added i

Re: Prolonged safemode

2013-01-20 Thread Harsh J
I am not aware of a direct regression in DN startup slowdown or block report slowdown; its hard to tell what exactly the regression is without more notes or logs on behavior. On Sun, Jan 20, 2013 at 5:43 PM, Mohammad Tariq wrote: > Thank you so much for the valuable reply Harsh. I'll > look int

Re: On a lighter note

2013-01-20 Thread Mohammad Tariq
Oh yeah Alex. Thank God that we have a German expert as well ;) Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com On Sun, Jan 20, 2013 at 1:28 PM, Alexander Alten-Lorenz wrote: > Actually "Der Untergang" ;) > > Alexander Alten-Lorenz > http://mapredit.blogspot.com > Twitter:

Re: Prolonged safemode

2013-01-20 Thread Mohammad Tariq
Hello Varun, Thank you so much for your reply. In most of the cases, it is not. But apart from that everything seems to be fine. I am not getting any notification about under replicated blocks or corrupted blocks. I will do a recheck though. Thank you. Warm Regards, Tariq https://mtariq.jux.c

Re: Prolonged safemode

2013-01-20 Thread Mohammad Tariq
Thank you so much for the valuable reply Harsh. I'll look into it. One quick question, why it it happening with 1.0.4? Is there any compulsion to set these two props, you have specified above. Earlier version were doing absolutely fine without these props? I am Sorry to be a pest of questions. But

Re: Prolonged safemode

2013-01-20 Thread varun kumar
Hi Tariq, When you start your namenode,Is it able to come out of Safemode Automatically. If no then there are under replicated blocks or corrupted blocks where namenode is trying to fetch it. Try to remove corrupted blocks. Regards, Varun Kumar.P On Sun, Jan 20, 2013 at 4:05 AM, Mohammad Tariq

Re: Prolonged safemode

2013-01-20 Thread Harsh J
If your DN is starting too slow, then you should investigate why. In any case, Apache Bigtop's (http://bigtop.apache.org) pseudo-distributed configs provide good values for 1-node setups. In your case, you seem to be missing dfs.safemode.min.datanodes set to 1, and dfs.safemode.extension set to 0.

Re: Prolonged safemode

2013-01-20 Thread shashwat shriparv
Check integrity of the file system, and check the replication factor, by mistake if default is left as 3 or so. if you have hbase configured check hbck if everything is fine with the cluster. ∞ Shashwat Shriparv On Sun, Jan 20, 2013 at 3:09 PM, xin jiang wrote: > > > On Sun, Jan 20, 2013 a

Re: Prolonged safemode

2013-01-20 Thread xin jiang
On Sun, Jan 20, 2013 at 7:50 AM, Mohammad Tariq wrote: > Hey Jean, > > Feels good to hear that ;) I don't have to feel > myself a solitary yonker anymore. > > Since I am working on a single node, the problem > becomes more sever. I don't have any other node > where MR files could get repl