Re: [VOTE] Release Apache Hadoop 3.1.4 (RC3)
Thank you all for the suggestions and testing. As there's a data loss issue in the release, I've created a new RC with the patch included. I'll send the update soon. Regards, Gabor Bota On Thu, Jul 16, 2020 at 1:29 PM Stephen O'Donnell wrote: > > Hi Gabor, > > We recently discovered a HDFS data loss issue in any build which uses > snapshots containing HDFS-13101 but not including HDFS-15313. Unfortunately > 3.1.4 falls into this category: > > git log origin/branch-3.1.4 | egrep "HDFS-(15313|13101)" > HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. > Contributed by Shashikant Banerjee. > HDFS-13101. Yet another fsimage corruption related to snapshot. > Contributed by Shashikant Banerjee. > > See this comment for more information on the bug: > > https://issues.apache.org/jira/browse/HDFS-15313?focusedCommentId=17158140&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17158140 > > I think we should not make a release when we have a known data loss bug in > it. What do you think? > > I am going to commit HDFS-15313 onto branch-3.1 shortly, so maybe we should > cut a new RC after including that? > > Thanks, > > Stephen. > > On Mon, Jul 13, 2020 at 11:36 AM Gabor Bota > wrote: > > > Hi folks, > > > > I have put together a release candidate (RC3) for Hadoop 3.1.4. > > > > * > > The RC includes in addition to the previous ones: > > * fix of YARN-10347. Fix double locking in > > CapacityScheduler#reinitialize in branch-3.1 > > (https://issues.apache.org/jira/browse/YARN-10347) > > * the revert of HDFS-14941, as it caused > > HDFS-15421. IBR leak causes standby NN to be stuck in safe mode. > > (https://issues.apache.org/jira/browse/HDFS-15421) > > * HDFS-15323, as requested. > > (https://issues.apache.org/jira/browse/HDFS-15323) > > * > > > > The RC is available at: http://people.apache.org/~gabota/hadoop-3.1.4-RC3/ > > The RC tag in git is here: > > https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC3 > > The maven artifacts are staged at > > https://repository.apache.org/content/repositories/orgapachehadoop-1274/ > > > > You can find my public key at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > and http://keys.gnupg.net/pks/lookup?op=get&search=0xB86249D83539B38C > > > > Please try the release and vote. The vote will run for 7 weekdays, > > until July 22. 2020. 23:00 CET. > > > > > > Thanks, > > Gabor > > > > - > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.1.4 (RC3)
Hi Gabor, We recently discovered a HDFS data loss issue in any build which uses snapshots containing HDFS-13101 but not including HDFS-15313. Unfortunately 3.1.4 falls into this category: git log origin/branch-3.1.4 | egrep "HDFS-(15313|13101)" HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee. See this comment for more information on the bug: https://issues.apache.org/jira/browse/HDFS-15313?focusedCommentId=17158140&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17158140 I think we should not make a release when we have a known data loss bug in it. What do you think? I am going to commit HDFS-15313 onto branch-3.1 shortly, so maybe we should cut a new RC after including that? Thanks, Stephen. On Mon, Jul 13, 2020 at 11:36 AM Gabor Bota wrote: > Hi folks, > > I have put together a release candidate (RC3) for Hadoop 3.1.4. > > * > The RC includes in addition to the previous ones: > * fix of YARN-10347. Fix double locking in > CapacityScheduler#reinitialize in branch-3.1 > (https://issues.apache.org/jira/browse/YARN-10347) > * the revert of HDFS-14941, as it caused > HDFS-15421. IBR leak causes standby NN to be stuck in safe mode. > (https://issues.apache.org/jira/browse/HDFS-15421) > * HDFS-15323, as requested. > (https://issues.apache.org/jira/browse/HDFS-15323) > * > > The RC is available at: http://people.apache.org/~gabota/hadoop-3.1.4-RC3/ > The RC tag in git is here: > https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC3 > The maven artifacts are staged at > https://repository.apache.org/content/repositories/orgapachehadoop-1274/ > > You can find my public key at: > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > and http://keys.gnupg.net/pks/lookup?op=get&search=0xB86249D83539B38C > > Please try the release and vote. The vote will run for 7 weekdays, > until July 22. 2020. 23:00 CET. > > > Thanks, > Gabor > > - > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >
Re: [VOTE] Release Apache Hadoop 3.1.4 (RC3)
Hi Gabor Bota, > The RC is available at: http://people.apache.org/~gabota/hadoop-3.1.4-RC3/ I could not find .sha512 for src and bin tarballs. Could you upload the files too? I'm +1 (binding), pending on them. * verified the signature of the source tarball. * built from source tarball with native profile enabled on CentOS 7 and OpenJDK 8. * built documentation and skimmed the contents. * ran example jobs on 3 nodes docker cluster with NN-HA and RM-HA enblaed. * launched pseudo-distributed cluster with Kerberos and SSL enabled, ran basic EZ operation, ran example MR jobs. Thanks, Masatake Iwasaki On 2020/07/13 19:36, Gabor Bota wrote: Hi folks, I have put together a release candidate (RC3) for Hadoop 3.1.4. * The RC includes in addition to the previous ones: * fix of YARN-10347. Fix double locking in CapacityScheduler#reinitialize in branch-3.1 (https://issues.apache.org/jira/browse/YARN-10347) * the revert of HDFS-14941, as it caused HDFS-15421. IBR leak causes standby NN to be stuck in safe mode. (https://issues.apache.org/jira/browse/HDFS-15421) * HDFS-15323, as requested. (https://issues.apache.org/jira/browse/HDFS-15323) * The RC is available at: http://people.apache.org/~gabota/hadoop-3.1.4-RC3/ The RC tag in git is here: https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC3 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapachehadoop-1274/ You can find my public key at: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS and http://keys.gnupg.net/pks/lookup?op=get&search=0xB86249D83539B38C Please try the release and vote. The vote will run for 7 weekdays, until July 22. 2020. 23:00 CET. Thanks, Gabor - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[VOTE] Release Apache Hadoop 3.1.4 (RC3)
Hi folks, I have put together a release candidate (RC3) for Hadoop 3.1.4. * The RC includes in addition to the previous ones: * fix of YARN-10347. Fix double locking in CapacityScheduler#reinitialize in branch-3.1 (https://issues.apache.org/jira/browse/YARN-10347) * the revert of HDFS-14941, as it caused HDFS-15421. IBR leak causes standby NN to be stuck in safe mode. (https://issues.apache.org/jira/browse/HDFS-15421) * HDFS-15323, as requested. (https://issues.apache.org/jira/browse/HDFS-15323) * The RC is available at: http://people.apache.org/~gabota/hadoop-3.1.4-RC3/ The RC tag in git is here: https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC3 The maven artifacts are staged at https://repository.apache.org/content/repositories/orgapachehadoop-1274/ You can find my public key at: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS and http://keys.gnupg.net/pks/lookup?op=get&search=0xB86249D83539B38C Please try the release and vote. The vote will run for 7 weekdays, until July 22. 2020. 23:00 CET. Thanks, Gabor - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org