Thanks Vinod for working hard on this! It is really a huge effort to take care of a release with 154 commits.
I did a sanity check for commits on 2.7.2, all looks fine except HDFS-8767 is missing from CHANGES.txt due to original commit issue. I already update it, and I think we are in good shape now. Thanks, Junping ________________________________ From: Vinod Kumar Vavilapalli <vino...@apache.org> Sent: Wednesday, January 13, 2016 11:15 PM To: Junping Du Cc: mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org Subject: Re: [VOTE] Release Apache Hadoop 2.7.2 RC1 Okay, did a whole bunch of things on 2.7.2. Fixing CHANGES.txt entry that Junping reported on the voting thread - 56e2dcc HADOOP-5323. Trash documentation should describe its directory structure and configurations. (Weiwei Yang via ozawa) Moved entry from HDFS CHANGES.txt to that of common. Detected one more issue: YARN-2859 addendum’s got missed in 2.7.2 - 2e1c617 YARN-2859 addendum: move the entry from 2.6.2 to 2.6.3 in hadoop-yarn/CHANGES.txt. - d738e2e YARN-2859.addendum: fix the remaining issue from the previous patch Also removed multiple duplicate CHANGES.txt entries for MAPREDUCE-6549 Backports - 91b7f94 YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) - 3c1b25b HADOOP-12413. AccessControlList should avoid calling getGroupNames in isUserInList with empty groups. Contributed by Zhihai Xu. (cherry picked from commit b2017d9b032af20044fdf60ddbd1575a554ccb79) - 17ad3b1 HADOOP-12526. there are duplicate dependency definitions in pom's (sjlee) - c3ba72c YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. - 096ac9e MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee) - 3c0e4de HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula. - ad829ff MAPREDUCE-6377. JHS sorting on state column not working in webUi. Contributed by zhihai xu. (cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94) - 10f23dc HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. - 10a6298 YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev (cherry picked from commit d36b6e045f317c94e97cb41a163aa974d161a404) - 09d3e47 HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. Contributed by Chang Li. - 89f6716 MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts (cherry picked from commit cab3c7c8892ad33a7eb0955b01e99872ab95e192) - 4e84a8b YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla (cherry picked from commit f5acf94ecafb301a0cc8e8f91f19c8bcbc8da701) - 4765a65 MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) (cherry picked from commit 7fd00b3db4b7d73afd41276ba9a06ec06a0e1762) - 0fc19f6 YARN-4348. ZKRMStateStore.syncInternal shouldn't wait for sync completion for avoiding blocking ZK's event thread. (ozawa) - 3e192f8 YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. Ran these tests that got changed in the above patches, before pushing the backports: - TestAccessControlList,DFSTestUtil,TestCommitBlockWithInvalidGenStamp,TestHDFSConcat,TestLineRecordReader,TestMRTimelineEventHandling,TestDistributedShell,TestFileSystemNodeLabelsStore,TestCapacityScheduler @Junping, mind giving a look at the branch for sanity checks? Thanks +Vinod On Jan 13, 2016, at 11:02 AM, Vinod Kumar Vavilapalli <vino...@apache.org<mailto:vino...@apache.org>> wrote: Thanks for the comments Tsuyoshi / Andrew / Varun / Junping / Akira. I agree that where possible we should serialize releases and make them incremental w.r.t fixes. Will roll a new RC for 2.7.2 after the backports. If there are more thoughts on releases, which I’m sure we will all do have, let’s fork the thread off from this voting conversation. Thanks +Vinod On Dec 29, 2015, at 6:50 AM, Junping Du <j...@hortonworks.com<mailto:j...@hortonworks.com>> wrote: I am +1 with pulling all of these tickets into 2.7.2. - For “any fix in 2.6.3 to be there in all releases that get out after 2.6.3 release date” Shall we conclude this as a general rule - "any fix in 2.x.y to be there in all 2.b.c releases (while b>=x) that get out after 2.x.y release date"? I am generally fine with this, but just feel it sounds to set too strong restrictions among branches. Some fixes could be trivial (test case fix, etc.) enough to deserve more flexibility. I would prefer this rule only applies on critical/blocker fixes, but not applies on minor/trivial issues. Just 2 cents. Thanks, Junping ________________________________ From: Vinod Kumar Vavilapalli <vino...@apache.org<mailto:vino...@apache.org>> Sent: Thursday, December 24, 2015 12:47 AM To: Junping Du Cc: mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org>; yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org>; common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org> Subject: Re: [VOTE] Release Apache Hadoop 2.7.2 RC1 I retract my -1. I think we will need to discuss this a bit more. Beyond those two tickets, there are a bunch more (totaling to 16) that are in 2.6.3 but *not* in 2.7.2. See this: https://issues.apache.org/jira/issues/?jql=key%20in%20%28HADOOP-12526%2CHADOOP-12413%2CHADOOP-11267%2CHADOOP-10668%2CHADOOP-10134%2CYARN-4434%2CYARN-4365%2CYARN-4348%2CYARN-4344%2CYARN-4326%2CYARN-4241%2CYARN-2859%2CMAPREDUCE-6549%2CMAPREDUCE-6540%2CMAPREDUCE-6377%2CMAPREDUCE-5883%2CHDFS-9431%2CHDFS-9289%2CHDFS-8615%29%20and%20fixVersion%20!%3D%202.7.0<https://issues.apache.org/jira/issues/?jql=key%20in%20(HADOOP-12526,HADOOP-12413,HADOOP-11267,HADOOP-10668,HADOOP-10134,YARN-4434,YARN-4365,YARN-4348,YARN-4344,YARN-4326,YARN-4241,YARN-2859,MAPREDUCE-6549,MAPREDUCE-6540,MAPREDUCE-6377,MAPREDUCE-5883,HDFS-9431,HDFS-9289,HDFS-8615)%20and%20fixVersion%20!=%202.7.0> Two options here, depending on the importance of ‘causality' between 2.6.x and 2.7.x lines. - Ship 2.7.2 as we voted on here - Pull these 16 tickets into 2.7.2 and roll a new RC What do people think? Do folks expect “any fix in 2.6.3 to be there in all releases that get out after 2.6.3 release date (December 16th)”? Thanks +Vinod On Dec 23, 2015, at 12:37 PM, Vinod Kumar Vavilapalli <vino...@apache.org<mailto:vino...@apache.org>> wrote: Sigh. Missed this. To retain causality ("any fix in 2.6.3 will be there in all releases that got out after 2.6.3”), I’ll get these patches in. Reverting my +1, and casting -1 for the RC myself. Will spin a new RC, this voting thread is marked dead. Thanks +Vinod On Dec 22, 2015, at 8:24 AM, Junping Du <j...@hortonworks.com<mailto:j...@hortonworks.com>> wrote: However, when I look at our commit log and CHANGES.txt, I found something we are missing: 1. HDFS-9470 and YARN-4424 are missing from the 2.7.2 branch and RC1 tag. 2. HADOOP-5323, HDFS-8767 are missing in CHANGE.txt
