Re: Does hadoop-default.xml + hadoop-site.xml matter for whole cluster or each node?

2009-03-09 Thread Rasit OZDAS
Some parameters are global (I can't give an example now), they are cluster-wide even if they're defined in hadoop-site.xml Rasit 2009/3/9 Nick Cen cenyo...@gmail.com for Q1: i think so , but i think it is a good practice to keep the hadoop-default.xml untouched. for Q2: i use this property

Re: master trying fetch data from slave using localhost hostname :)

2009-03-09 Thread Steve Loughran
pavelkolo...@gmail.com wrote: On Fri, 06 Mar 2009 14:41:57 -, jason hadoop jason.had...@gmail.com wrote: I see that when the host name of the node is also on the localhost line in /etc/hosts I erased all records with localhost from all /etc/hosts files and all fine now :) Thank you

Re: master trying fetch data from slave using localhost hostname :)

2009-03-09 Thread pavelkolodin
what does /etc/host look like now? I hit some problems with ubuntu and localhost last week; the hostname was set up in /etc/hosts not just to point to the loopback address, but to a different loopback address (127.0.1.1) from the normal value (127.0.0.1), so breaking everything.

MultipleOutputFormat with sorting functionality

2009-03-09 Thread Rasit OZDAS
Hi, all! I'm using multiple output format to write out 4 different files, each one has the same type. But it seems that outputs aren't being sorted. Should they be sorted? Or isn't it implemented for multiple output format? Here is some code: // in main function

Re: Does hadoop-default.xml + hadoop-site.xml matter for whole cluster or each node?

2009-03-09 Thread Owen O'Malley
On Mar 7, 2009, at 10:56 PM, pavelkolo...@gmail.com wrote: Does hadoop-default.xml + hadoop-site.xml of master host matter for whole Job or they matter for each node independently? Please never modify hadoop-default. That is for the system defaults. Please use hadoop-site for your

Re: question about released version id

2009-03-09 Thread Owen O'Malley
On Mar 2, 2009, at 11:46 PM, 鞠適存 wrote: I wonder how to make the hadoop version number. Each 0.18, 0.19 and 0.20 have their own branch. The first release on each branch is 0.X.0, and then 0.X.1 and so on. New features are only put into trunk and only important bug fixes are put into the

Re: Reducer goes past 100% complete?

2009-03-09 Thread Owen O'Malley
On Mar 9, 2009, at 1:00 PM, james warren wrote: Speculative execution has existed far before 0.19.x, but AFAIK the 100% issue has appeared (at least with greater frequency) since 0.19.0 came out. Are you saying there are changes in how task progress is being tracked? In the past, it

HDFS is corrupt, need to salvage the data.

2009-03-09 Thread Mayuran Yogarajah
Hello, it seems the HDFS in my cluster is corrupt. This is the output from hadoop fsck: Total size:9196815693 B Total dirs:17 Total files: 157 Total blocks: 157 (avg. block size 58578443 B) CORRUPT FILES:157 MISSING BLOCKS: 157