Arun, thanks for jumping on this. On hadoop branch-2.2. I've quickly scanned the commit logs starting from the 2.2.0 release and I've found around 20 JIRAs that I like seeing in 2.2.1. Not all of them are bugs but the don't shake anything and improve usability.
I presume others will have their own laundry lists as well and I wonder the union of all of them how much adds up to the current 81 commits. How about splitting the JIRAs among a few contributors to assert there is nothing risky in there? And if so get discuss getting rid of those commits for 2.2.1. IMO doing that would be cheaper than selectively applying commits on a fresh branch. Said this, I think we should get 2.2.1 out of the door before switching main efforts to 2.3.0. I volunteer myself to drive 2.2.1 a release if ASAP if you don't have the bandwidth at the moment for it. Cheers. Alejandro ************************************************************************************ Commits in branch-2.2 that I'd like them to be in the 2.2.1 release: The ones prefixed with '*' technically are not bugs. YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed containers. (Alejandro Abdelnur via Sandy Ryza) YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza) YARN-1044. used/min/max resources do not display info in the scheduler page (Sangjin Lee via Sandy Ryza) YARN-305. Fair scheduler logs too many "Node offered to app" messages. (Lohit Vijayarenu via Sandy Ryza) *MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy Ryza) YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps switched. (Robert Kanter via Sandy Ryza) YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text file busy errors. (Sandy Ryza) *MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators (Sandy Ryza) *YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza) *YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza) YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take effect (Sandy Ryza) HDFS-5403. WebHdfs client cannot communicate with older WebHdfs servers post HDFS-5306. Contributed by Aaron T. Myers. *YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication (Sandy Ryza) *YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via Sandy Ryza) *MAPREDUCE-4680. Job history cleaner should only check timestamps of files in old enough directories (Robert Kanter via Sandy Ryza) YARN-1109. Demote NodeManager "Sending out status for container" logs to debug (haosdent via Sandy Ryza) *YARN-1321. Changed NMTokenCache to support both singleton and an instance usage. Contributed by Alejandro Abdelnur YARN-1343. NodeManagers additions/restarts are not reported as node updates in AllocateResponse responses to AMs. (tucu) YARN-1381. Same relaxLocality appears twice in exception message of AMRMClientImpl#checkLocalityRelaxationConflict() (Ted Yu via Sandy Ryza) HADOOP-9898. Set SO_KEEPALIVE on all our sockets. Contributed by Todd Lipcon. YARN-1388. Fair Scheduler page always displays blank fair share (Liyin Liang via Sandy Ryza) On Fri, Nov 8, 2013 at 10:35 PM, Chris Nauroth <cnaur...@hortonworks.com>wrote: > Arun, what are your thoughts on test-only patches? I know I've been > merging a lot of Windows test stabilization patches down to branch-2.2. > These can't rightly be called blockers, but they do improve dev > experience, and there is no risk to product code. > > Chris Nauroth > Hortonworks > http://hortonworks.com/ > > > > On Fri, Nov 8, 2013 at 1:30 AM, Steve Loughran <ste...@hortonworks.com > >wrote: > > > On 8 November 2013 02:42, Arun C Murthy <a...@hortonworks.com> wrote: > > > > > Gang, > > > > > > Thinking through the next couple of releases here, appreciate f/b. > > > > > > # hadoop-2.2.1 > > > > > > I was looking through commit logs and there is a *lot* of content here > > > (81 commits as on 11/7). Some are features/improvements and some are > > fixes > > > - it's really hard to distinguish what is important and what isn't. > > > > > > I propose we start with a blank slate (i.e. blow away branch-2.2 and > > > start fresh from a copy of branch-2.2.0) and then be very careful and > > > meticulous about including only *blocker* fixes in branch-2.2. So, most > > of > > > the content here comes via the next minor release (i.e. hadoop-2.3) > > > > > > In future, we continue to be *very* parsimonious about what gets into > a > > > patch release (major.minor.patch) - in general, these should be only > > > *blocker* fixes or key operational issues. > > > > > > > +1 > > > > > > > > > > # hadoop-2.3 > > > > > > I'd like to propose the following features for YARN/MR to make it into > > > hadoop-2.3 and punt the rest to hadoop-2.4 and beyond: > > > * Application History Server - This is happening in a branch and is > > > close; with it we can provide a reasonable experience for new > frameworks > > > being built on top of YARN. > > > * Bug-fixes in RM Restart > > > * Minimal support for long-running applications (e.g. security) via > > > YARN-896 > > > > > > > +1 -the complete set isn't going to make it, but I'm sure we can identify > > the key ones > > > > > > > > > * RM Fail-over via ZKFC > > > * Anything else? > > > > > > HDFS??? > > > > > > > > > > - If I had the time, I'd like to do some work on the HADOOP-9361 > > filesystem spec & tests -this is mostly some specification, the basis > > of a > > better test framework for newer FS tests, and some more tests, with a > > couple of minor changes to some of the FS code, mainly in terms of > > tightening some of the exceptions thrown (IOE -> EOF) > > > > otherwise: > > > > - I'd like the hadoop-openstack JAR in; it's already in branch-2 so > > it's a matter of ensuring testing during the release against as many > > providers as possible. > > - There are a fair few JIRAs about updating versions of dependencies > > -the S3 JetS3t update went in this week, but there are more, as well > as > > cruft in the POMs which shows up downstream. I think we could update > the > > low-risk dependencies (test-time, log4j, &c), while avoiding those we > > know > > will be trouble (jetty). This may seem minor but it does make a big > > diff to > > the downstream projects. > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > -- Alejandro
