HADOOP-10020 is a JIRA that disables symlinks temporarily. They will be disabled in 2.2.1 as well, if the plan is to have only minor fixes in that branch.
To be honest, I'm not aware of anything in 2.2.1 that shouldn't be there. However, I have only been following the HDFS and common side of things so I may not have the full picture. Arun, can you give a specific example of something you'd like to "blow away"? Colin On Mon, Nov 11, 2013 at 10:06 AM, Hari Mankude <hmank...@talena-inc.com> wrote: > Hi Arun, > > Another feature that would be relevant and got deferred was the symlink > work (HADOOP-10020) that Colin and Andrew were working on. Can we include > this in hadoop-2.3.0 also? > > thanks > hari > > > On Sun, Nov 10, 2013 at 2:07 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > >> Arun, thanks for jumping on this. >> >> On hadoop branch-2.2. I've quickly scanned the commit logs starting from >> the 2.2.0 release and I've found around 20 JIRAs that I like seeing in >> 2.2.1. Not all of them are bugs but the don't shake anything and improve >> usability. >> >> I presume others will have their own laundry lists as well and I wonder the >> union of all of them how much adds up to the current 81 commits. >> >> How about splitting the JIRAs among a few contributors to assert there is >> nothing risky in there? And if so get discuss getting rid of those commits >> for 2.2.1. IMO doing that would be cheaper than selectively applying >> commits on a fresh branch. >> >> Said this, I think we should get 2.2.1 out of the door before switching >> main efforts to 2.3.0. I volunteer myself to drive 2.2.1 a release if ASAP >> if you don't have the bandwidth at the moment for it. >> >> Cheers. >> >> Alejandro >> >> >> ************************************************************************************ >> Commits in branch-2.2 that I'd like them to be in the 2.2.1 release: >> >> The ones prefixed with '*' technically are not bugs. >> >> YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed >> containers. (Alejandro Abdelnur via Sandy Ryza) >> YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza) >> YARN-1044. used/min/max resources do not display info in the scheduler >> page (Sangjin Lee via Sandy Ryza) >> YARN-305. Fair scheduler logs too many "Node offered to app" messages. >> (Lohit Vijayarenu via Sandy Ryza) >> *MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy >> Ryza) >> YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps >> switched. (Robert Kanter via Sandy Ryza) >> YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text >> file busy errors. (Sandy Ryza) >> *MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write >> out text files without separators (Sandy Ryza) >> *YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza) >> *YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza) >> YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take >> effect (Sandy Ryza) >> HDFS-5403. WebHdfs client cannot communicate with older WebHdfs servers >> post HDFS-5306. Contributed by Aaron T. Myers. >> *YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp >> into SchedulerApplication (Sandy Ryza) >> *YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via >> Sandy Ryza) >> *MAPREDUCE-4680. Job history cleaner should only check timestamps of files >> in old enough directories (Robert Kanter via Sandy Ryza) >> YARN-1109. Demote NodeManager "Sending out status for container" logs to >> debug (haosdent via Sandy Ryza) >> *YARN-1321. Changed NMTokenCache to support both singleton and an instance >> usage. Contributed by Alejandro Abdelnur >> YARN-1343. NodeManagers additions/restarts are not reported as node >> updates in AllocateResponse responses to AMs. (tucu) >> YARN-1381. Same relaxLocality appears twice in exception message of >> AMRMClientImpl#checkLocalityRelaxationConflict() (Ted Yu via Sandy Ryza) >> HADOOP-9898. Set SO_KEEPALIVE on all our sockets. Contributed by Todd >> Lipcon. >> YARN-1388. Fair Scheduler page always displays blank fair share (Liyin >> Liang via Sandy Ryza) >> >> >> >> On Fri, Nov 8, 2013 at 10:35 PM, Chris Nauroth <cnaur...@hortonworks.com >> >wrote: >> >> > Arun, what are your thoughts on test-only patches? I know I've been >> > merging a lot of Windows test stabilization patches down to branch-2.2. >> > These can't rightly be called blockers, but they do improve dev >> > experience, and there is no risk to product code. >> > >> > Chris Nauroth >> > Hortonworks >> > http://hortonworks.com/ >> > >> > >> > >> > On Fri, Nov 8, 2013 at 1:30 AM, Steve Loughran <ste...@hortonworks.com >> > >wrote: >> > >> > > On 8 November 2013 02:42, Arun C Murthy <a...@hortonworks.com> wrote: >> > > >> > > > Gang, >> > > > >> > > > Thinking through the next couple of releases here, appreciate f/b. >> > > > >> > > > # hadoop-2.2.1 >> > > > >> > > > I was looking through commit logs and there is a *lot* of content >> here >> > > > (81 commits as on 11/7). Some are features/improvements and some are >> > > fixes >> > > > - it's really hard to distinguish what is important and what isn't. >> > > > >> > > > I propose we start with a blank slate (i.e. blow away branch-2.2 and >> > > > start fresh from a copy of branch-2.2.0) and then be very careful >> and >> > > > meticulous about including only *blocker* fixes in branch-2.2. So, >> most >> > > of >> > > > the content here comes via the next minor release (i.e. hadoop-2.3) >> > > > >> > > > In future, we continue to be *very* parsimonious about what gets >> into >> > a >> > > > patch release (major.minor.patch) - in general, these should be only >> > > > *blocker* fixes or key operational issues. >> > > > >> > > >> > > +1 >> > > >> > > >> > > > >> > > > # hadoop-2.3 >> > > > >> > > > I'd like to propose the following features for YARN/MR to make it >> into >> > > > hadoop-2.3 and punt the rest to hadoop-2.4 and beyond: >> > > > * Application History Server - This is happening in a branch and is >> > > > close; with it we can provide a reasonable experience for new >> > frameworks >> > > > being built on top of YARN. >> > > > * Bug-fixes in RM Restart >> > > > * Minimal support for long-running applications (e.g. security) via >> > > > YARN-896 >> > > > >> > > >> > > +1 -the complete set isn't going to make it, but I'm sure we can >> identify >> > > the key ones >> > > >> > > >> > > >> > > > * RM Fail-over via ZKFC >> > > > * Anything else? >> > > > >> > > > HDFS??? >> > > > >> > > > >> > > >> > > - If I had the time, I'd like to do some work on the HADOOP-9361 >> > > filesystem spec & tests -this is mostly some specification, the >> basis >> > > of a >> > > better test framework for newer FS tests, and some more tests, with >> a >> > > couple of minor changes to some of the FS code, mainly in terms of >> > > tightening some of the exceptions thrown (IOE -> EOF) >> > > >> > > otherwise: >> > > >> > > - I'd like the hadoop-openstack JAR in; it's already in branch-2 so >> > > it's a matter of ensuring testing during the release against as many >> > > providers as possible. >> > > - There are a fair few JIRAs about updating versions of dependencies >> > > -the S3 JetS3t update went in this week, but there are more, as well >> > as >> > > cruft in the POMs which shows up downstream. I think we could update >> > the >> > > low-risk dependencies (test-time, log4j, &c), while avoiding those >> we >> > > know >> > > will be trouble (jetty). This may seem minor but it does make a big >> > > diff to >> > > the downstream projects. >> > > >> > > -- >> > > CONFIDENTIALITY NOTICE >> > > NOTICE: This message is intended for the use of the individual or >> entity >> > to >> > > which it is addressed and may contain information that is confidential, >> > > privileged and exempt from disclosure under applicable law. If the >> reader >> > > of this message is not the intended recipient, you are hereby notified >> > that >> > > any printing, copying, dissemination, distribution, disclosure or >> > > forwarding of this communication is strictly prohibited. If you have >> > > received this communication in error, please contact the sender >> > immediately >> > > and delete it from your system. Thank You. >> > > >> > >> > -- >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or entity >> to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the reader >> > of this message is not the intended recipient, you are hereby notified >> that >> > any printing, copying, dissemination, distribution, disclosure or >> > forwarding of this communication is strictly prohibited. If you have >> > received this communication in error, please contact the sender >> immediately >> > and delete it from your system. Thank You. >> > >> >> >> >> -- >> Alejandro >>
