Hi Arun, Another feature that would be relevant and got deferred was the symlink work (HADOOP-10020) that Colin and Andrew were working on. Can we include this in hadoop-2.3.0 also?
thanks hari On Sun, Nov 10, 2013 at 2:07 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > Arun, thanks for jumping on this. > > On hadoop branch-2.2. I've quickly scanned the commit logs starting from > the 2.2.0 release and I've found around 20 JIRAs that I like seeing in > 2.2.1. Not all of them are bugs but the don't shake anything and improve > usability. > > I presume others will have their own laundry lists as well and I wonder the > union of all of them how much adds up to the current 81 commits. > > How about splitting the JIRAs among a few contributors to assert there is > nothing risky in there? And if so get discuss getting rid of those commits > for 2.2.1. IMO doing that would be cheaper than selectively applying > commits on a fresh branch. > > Said this, I think we should get 2.2.1 out of the door before switching > main efforts to 2.3.0. I volunteer myself to drive 2.2.1 a release if ASAP > if you don't have the bandwidth at the moment for it. > > Cheers. > > Alejandro > > > ************************************************************************************ > Commits in branch-2.2 that I'd like them to be in the 2.2.1 release: > > The ones prefixed with '*' technically are not bugs. > > YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed > containers. (Alejandro Abdelnur via Sandy Ryza) > YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza) > YARN-1044. used/min/max resources do not display info in the scheduler > page (Sangjin Lee via Sandy Ryza) > YARN-305. Fair scheduler logs too many "Node offered to app" messages. > (Lohit Vijayarenu via Sandy Ryza) > *MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy > Ryza) > YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps > switched. (Robert Kanter via Sandy Ryza) > YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text > file busy errors. (Sandy Ryza) > *MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write > out text files without separators (Sandy Ryza) > *YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza) > *YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza) > YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take > effect (Sandy Ryza) > HDFS-5403. WebHdfs client cannot communicate with older WebHdfs servers > post HDFS-5306. Contributed by Aaron T. Myers. > *YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp > into SchedulerApplication (Sandy Ryza) > *YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via > Sandy Ryza) > *MAPREDUCE-4680. Job history cleaner should only check timestamps of files > in old enough directories (Robert Kanter via Sandy Ryza) > YARN-1109. Demote NodeManager "Sending out status for container" logs to > debug (haosdent via Sandy Ryza) > *YARN-1321. Changed NMTokenCache to support both singleton and an instance > usage. Contributed by Alejandro Abdelnur > YARN-1343. NodeManagers additions/restarts are not reported as node > updates in AllocateResponse responses to AMs. (tucu) > YARN-1381. Same relaxLocality appears twice in exception message of > AMRMClientImpl#checkLocalityRelaxationConflict() (Ted Yu via Sandy Ryza) > HADOOP-9898. Set SO_KEEPALIVE on all our sockets. Contributed by Todd > Lipcon. > YARN-1388. Fair Scheduler page always displays blank fair share (Liyin > Liang via Sandy Ryza) > > > > On Fri, Nov 8, 2013 at 10:35 PM, Chris Nauroth <cnaur...@hortonworks.com > >wrote: > > > Arun, what are your thoughts on test-only patches? I know I've been > > merging a lot of Windows test stabilization patches down to branch-2.2. > > These can't rightly be called blockers, but they do improve dev > > experience, and there is no risk to product code. > > > > Chris Nauroth > > Hortonworks > > http://hortonworks.com/ > > > > > > > > On Fri, Nov 8, 2013 at 1:30 AM, Steve Loughran <ste...@hortonworks.com > > >wrote: > > > > > On 8 November 2013 02:42, Arun C Murthy <a...@hortonworks.com> wrote: > > > > > > > Gang, > > > > > > > > Thinking through the next couple of releases here, appreciate f/b. > > > > > > > > # hadoop-2.2.1 > > > > > > > > I was looking through commit logs and there is a *lot* of content > here > > > > (81 commits as on 11/7). Some are features/improvements and some are > > > fixes > > > > - it's really hard to distinguish what is important and what isn't. > > > > > > > > I propose we start with a blank slate (i.e. blow away branch-2.2 and > > > > start fresh from a copy of branch-2.2.0) and then be very careful > and > > > > meticulous about including only *blocker* fixes in branch-2.2. So, > most > > > of > > > > the content here comes via the next minor release (i.e. hadoop-2.3) > > > > > > > > In future, we continue to be *very* parsimonious about what gets > into > > a > > > > patch release (major.minor.patch) - in general, these should be only > > > > *blocker* fixes or key operational issues. > > > > > > > > > > +1 > > > > > > > > > > > > > > # hadoop-2.3 > > > > > > > > I'd like to propose the following features for YARN/MR to make it > into > > > > hadoop-2.3 and punt the rest to hadoop-2.4 and beyond: > > > > * Application History Server - This is happening in a branch and is > > > > close; with it we can provide a reasonable experience for new > > frameworks > > > > being built on top of YARN. > > > > * Bug-fixes in RM Restart > > > > * Minimal support for long-running applications (e.g. security) via > > > > YARN-896 > > > > > > > > > > +1 -the complete set isn't going to make it, but I'm sure we can > identify > > > the key ones > > > > > > > > > > > > > * RM Fail-over via ZKFC > > > > * Anything else? > > > > > > > > HDFS??? > > > > > > > > > > > > > > - If I had the time, I'd like to do some work on the HADOOP-9361 > > > filesystem spec & tests -this is mostly some specification, the > basis > > > of a > > > better test framework for newer FS tests, and some more tests, with > a > > > couple of minor changes to some of the FS code, mainly in terms of > > > tightening some of the exceptions thrown (IOE -> EOF) > > > > > > otherwise: > > > > > > - I'd like the hadoop-openstack JAR in; it's already in branch-2 so > > > it's a matter of ensuring testing during the release against as many > > > providers as possible. > > > - There are a fair few JIRAs about updating versions of dependencies > > > -the S3 JetS3t update went in this week, but there are more, as well > > as > > > cruft in the POMs which shows up downstream. I think we could update > > the > > > low-risk dependencies (test-time, log4j, &c), while avoiding those > we > > > know > > > will be trouble (jetty). This may seem minor but it does make a big > > > diff to > > > the downstream projects. > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > > > > -- > Alejandro >
