I want to bring up the issue about shuffler from thread: Shuffle In Memory OutOfMemoryError
I was expecting improved performance from MAPREDUCE-1182 That would be one of the reasons for people to try out 0.21 release Regards On Fri, Mar 19, 2010 at 12:31 PM, Todd Lipcon <t...@cloudera.com> wrote: > On Fri, Mar 19, 2010 at 12:17 PM, Jeremy Davis <jerda...@speakeasy.net > >wrote: > > > Thank you for your reply, > > > > So what would be your (or anyones) advice on getting, HDFS > > sync/flush/append functionality? > > You seem to indicate that 0.21 branch as is might not be the best idea, > > with your preference being a patch set against 0.20. > > > > We definitely have an application for this specific functionality, and I > > need to provide some direction/answers in this area for my colleagues. > > > > For example, I might say: We will install the current 0.21 now, and > develop > > against it.. But in X time we will install CDH2 (and get all the goodness > it > > brings), and then apply a given patch set against it. Is this in line > with > > what you are thinking? If so, could I get your perceived level of effort, > > maybe a time frame? April/May/June/ etc.. > > > > > FYI the 0.20 sync patches mentioned by Stack will be going into CDH3 at > some > point this spring. This is mainly for the benefit of HBase but of course > other applications will benefit as well. > > Thanks > -Todd > > > > I'm sure I'm not the only one that has plans for this feature, as it's in > > "Hadoop: the Definitive Guide" by O'Reilly published in September '09. > > > > Thanks, > > -JD > > > > > > > > > > > > On Mar 19, 2010, at 11:51 AM, Stack wrote: > > > > On Fri, Mar 19, 2010 at 9:43 AM, Jeremy Davis <jerda...@speakeasy.net> > >> wrote: > >> > >>> ...and I also saw release 0.21 as is with stack as the release manager. > >>> Was > >>> there a final decision on this off list? > >>> > >>> > >> On the above, I was toying with the idea of being release manager for > >> releasing as hadoop 0.21.0 what is in current 0.21 hadoop branch but I > >> subsequently decided against it after chatting with folks and figuring > >> that the only group that seemed interested in driving a release of the > >> hadoop 0.21 branch was the hbase crew. If I were to guess, an hadoop > >> vouched for by a couple of hbasers with their spotty hdfs and > >> mapreduce knowledge probably wouldn't have the penetration of a > >> release backed by, say, a Yahoo. No one would trust their data to > >> such a release. If no data in hadoop 0.21 clusters, hbase wouldn't > >> have anything to run against. So I let it go and figured time could > >> be spent better elsewhere; e.g. helping test the set of patches that > >> could get us a sync/flush/append on a patched hadoop 0.20 (hdfs-200, > >> etc.). > >> > >> Sorry, I should have added a note to cited thread that I'd wandered... > >> > >> St.Ack > >> > > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
