Hi All, Thanks for the feedbacks. Inline with earlier communication dated 17th July 2018, I would like to provide some updates.
We are approaching previously proposed feature freeze date (Aug 21, about 7 days from today). If any features in branch which are targeted to 3.2.0, please reply to this email thread. Steve has mentioned about the s3 features which will come close to Code Freeze Date (Aug 31st). *Here's an updated 3.2.0 feature status:* 1. Merged & Completed features: - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning workloads Initial cut. - (Uma) HDFS-10285: HDFS Storage Policy Satisfier 2. Features close to finish: - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. Major patches are all in, only one last patch is in review state. - (Sunil) YARN-7494: Multi Node scheduling support in Capacity Scheduler. Close to commit. - (Chandni/Eric) YARN-7512: Support service upgrade via YARN Service API and CLI. 2 patches are pending which will be closed by Feature freeze date. - (Rohith) YARN-5742: Serve aggregated logs of historical apps from ATSv2. Patch in progress. - (Virajit) HDFS-12615: Router-based HDFS federation. Improvement works. - (Steve) S3Guard Phase III, S3a phase V, Support Windows Azure Storage. In progress. 3. Tentative features: - (Haibo Chen) YARN-1011: Resource overcommitment. Looks challenging to be done before Aug 2018. - (Eric) YARN-7129: Application Catalog for YARN applications. Challenging as more discussions are on-going. *Summary of 3.2.0 issues status:* 39 Blocker and Critical issues [1] are open, I am checking with owners to get status on each of them to get in by Code Freeze date. [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker, Critical) AND resolution = Unresolved AND "Target Version/s" = 3.2.0 ORDER BY priority DESC Thanks, Sunil On Fri, Jul 20, 2018 at 8:03 AM Sunil G <sun...@apache.org> wrote: > Thanks Subru for the thoughts. > One of the main reason for a major release is to push out critical > features with a faster cadence to the users. If we are pulling more and > more different types of features to a minor release, that branch will > become more destabilized and it may be tough to say that 3.1.2 is stable > that 3.1.1 for eg. We always tend to improve and stabilize features in > subsequent minor release. > For few companies, it makes sense to push out these new features faster to > make a reach to the users. Adding to the point to the backporting issues, I > agree that its a pain and we can workaround that with some git scripts. If > we can make such scripts available to committers, backport will be > seem-less across branches and we can achieve the faster release cadence > also. > > Thoughts? > > - Sunil > > > On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <su...@apache.org> wrote: > >> Thanks Sunil for volunteering to lead the release effort. I am generally >> supportive of a release but -1 on a 3.2 (prefer a 3.1.x) as feel we >> already >> have too many branches to be maintained. I already see many commits are in >> different branches with no apparent rationale, for e.g: 3.1 has commits >> which are absent in 3.0 etc. >> >> Additionally AFAIK 3.x has not been deployed in any major production >> setting so the cost of adding features should be minimal. >> >> Thoughts? >> >> -Subru >> >> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <sun...@apache.org> wrote: >> >> > Thanks Steve, Aaron, Wangda for sharing thoughts. >> > >> > Yes, important changes and features are much needed, hence we will be >> > keeping the door open for them as possible. Also considering few more >> > offline requests from other folks, I think extending the timeframe by >> > couple of weeks makes sense (including a second RC buffer) and this >> should >> > ideally help us to ship this by September itself. >> > >> > Revised dates (I will be updating same in Roadmap wiki as well) >> > >> > - Feature freeze date : all features to merge by August 21, 2018. >> > >> > - Code freeze date : blockers/critical only, no improvements and non >> > blocker/critical >> > >> > bug-fixes August 31, 2018. >> > >> > - Release date: September 15, 2018 >> > >> > Thank Eric and Zian, I think Wangda has already answered your questions. >> > >> > Thanks >> > Sunil >> > >> > >> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <wheele...@gmail.com> >> wrote: >> > >> > > Thanks Sunil for volunteering to be RM of 3.2 release, +1 for that. >> > > >> > > To concerns from Steve, >> > > >> > > It is a good idea to keep the door open to get important changes / >> > > features in before cutoff. I would prefer to keep the proposed release >> > date >> > > to make sure things can happen earlier instead of last minute and we >> all >> > > know that releases are always get delayed :). I'm also fine if we want >> > get >> > > another several weeks time. >> > > >> > > Regarding of 3.3 release, I would suggest doing that before >> thanksgiving. >> > > Do you think is it good or too early / late? >> > > >> > > Eric, >> > > >> > > The YARN-8220 will be replaced by YARN-8135, if YARN-8135 can get >> merged >> > > in time, we probably not need the YARN-8220. >> > > >> > > Sunil, >> > > >> > > Could u update https://cwiki.apache.org/confluence/display/HADOOP/ >> > Roadmap >> > > with proposed plan as well? We can fill feature list first before >> getting >> > > consensus of time. >> > > >> > > Thanks, >> > > Wangda >> > > >> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri >> <fab...@cloudera.com.invalid >> > > >> > > wrote: >> > > >> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran < >> ste...@hortonworks.com> >> > >> wrote: >> > >> >> > >> > >> > >> > >> > >> > On 16 Jul 2018, at 23:45, Sunil G <sun...@apache.org<mailto: >> > >> > sun...@apache.org>> wrote: >> > >> > >> > >> > I would also would like to take this opportunity to come up with a >> > >> detailed >> > >> > plan. >> > >> > >> > >> > - Feature freeze date : all features should be merged by August 10, >> > >> 2018. >> > >> > >> > >> > >> > >> > >> > >> > <snip> >> > >> >> > >> > >> > >> > Please let me know if I missed any features targeted to 3.2 per >> this >> > >> > >> > >> > >> > >> > Well there these big todo lists for S3 & S3Guard. >> > >> > >> > >> > https://issues.apache.org/jira/browse/HADOOP-15226 >> > >> > https://issues.apache.org/jira/browse/HADOOP-15220 >> > >> > >> > >> > >> > >> > There's a bigger bit of work coming on for Azure Datalake Gen 2 >> > >> > https://issues.apache.org/jira/browse/HADOOP-15407 >> > >> > >> > >> > I don't think this is quite ready yet, I've been doing work on it, >> but >> > >> if >> > >> > we have a 3 week deadline, I'm going to expect some timely reviews >> on >> > >> > https://issues.apache.org/jira/browse/HADOOP-15546 >> > >> > >> > >> > I've uprated that to a blocker feature; will review the S3 & >> S3Guard >> > >> JIRAs >> > >> > to see which of those are blocking. Then there are some pressing >> > "guave, >> > >> > java 9 prep" >> > >> > >> > >> > >> > >> I can help with this part if you like. >> > >> >> > >> >> > >> >> > >> > >> > >> > >> > >> > >> > >> > timeline. I would like to volunteer myself as release manager of >> 3.2.0 >> > >> > release. >> > >> > >> > >> > >> > >> > well volunteered! >> > >> > >> > >> > >> > >> > >> > >> Yes, thank you for stepping up. >> > >> >> > >> >> > >> > >> > >> > I think this raises a good q: what timetable should we have for the >> > >> 3.2. & >> > >> > 3.3 releases; if we do want a faster cadence, then having the >> outline >> > >> time >> > >> > from the 3.2 to the 3.3 release means that there's less concern >> about >> > >> > things not making the 3.2 dealine >> > >> > >> > >> > -Steve >> > >> > >> > >> > >> > >> Good idea to mitigate the short deadline. >> > >> >> > >> -AF >> > >> >> > > >> > >> >
