Re: rack awareness help

2010-03-19 Thread Mag Gam
Thanks everyone. I think everyone can agree that this part of the documentation is lacking for hadoop. Can someone please provide be a use case, for example: #server 1 Input > script.sh Output > rack01 #server 2 Input > script.sh Output > rack02 Is this how its supposed to work? I am bad with

Re: rack awareness help

2010-03-19 Thread Christopher Tubbs
More like the following (shown with the bash prompt). You could type this for testing. However, ultimately, hadoop itself will actually be executing this script and reading its output. $ ./script.sh server1.mydomain rack1 $ ./script.sh server2.mydomain rack1 $ ./script.sh server3.mydomain rack2 $

Re: rack awareness help

2010-03-19 Thread Allen Wittenauer
On 3/19/10 4:32 AM, "Mag Gam" wrote: > Thanks everyone. I think everyone can agree that this part of the > documentation is lacking for hadoop. > > Can someone please provide be a use case, for example: > > #server 1 > Input > script.sh > Output > rack01 > > #server 2 > Input > script.sh > O

0.21 Release

2010-03-19 Thread Jeremy Davis
Could someone clarify the position on the 0.21 release? This thread: http://mail-archives.apache.org/mod_mbox/hadoop-general/201002.mbox/> kind of died out (it seems) without any resolution. I saw two suggestions: Rebase 0.21 against trunk and then try to release, and I also saw release 0.21

Re: 0.21 Release

2010-03-19 Thread Stack
On Fri, Mar 19, 2010 at 9:43 AM, Jeremy Davis wrote: > ...and I also saw release 0.21 as is with stack as the release manager. Was > there a final decision on this off list? > On the above, I was toying with the idea of being release manager for releasing as hadoop 0.21.0 what is in current 0.21

Re: 0.21 Release

2010-03-19 Thread Jeremy Davis
Thank you for your reply, So what would be your (or anyones) advice on getting, HDFS sync/flush/ append functionality? You seem to indicate that 0.21 branch as is might not be the best idea, with your preference being a patch set against 0.20. We definitely have an application for this speci

Re: 0.21 Release

2010-03-19 Thread Todd Lipcon
On Fri, Mar 19, 2010 at 12:17 PM, Jeremy Davis wrote: > Thank you for your reply, > > So what would be your (or anyones) advice on getting, HDFS > sync/flush/append functionality? > You seem to indicate that 0.21 branch as is might not be the best idea, > with your preference being a patch set aga

Re: 0.21 Release

2010-03-19 Thread Ted Yu
I want to bring up the issue about shuffler from thread: Shuffle In Memory OutOfMemoryError I was expecting improved performance from MAPREDUCE-1182 That would be one of the reasons for people to try out 0.21 release Regards On Fri, Mar 19, 2010 at 12:31 PM, Todd Lipcon wrote: > On Fri, Mar 1

Re: 0.21 Release

2010-03-19 Thread Stack
On Fri, Mar 19, 2010 at 12:17 PM, Jeremy Davis wrote: > So what would be your (or anyones) advice on getting, HDFS sync/flush/append > functionality? > You seem to indicate that 0.21 branch as is might not be the best idea, with > your preference being a patch set against 0.20. > There'll be a ha