[DISCUSS] Hadoop 2019 Release Planning
Hi all, Hope this email finds you well I want to hear your thoughts about what should be the release plan for 2019. In 2018, we released: - 1 maintenance release of 2.6 - 3 maintenance releases of 2.7 - 3 maintenance releases of 2.8 - 3 releases of 2.9 - 4 releases of 3.0 - 2 releases of 3.1 Total 16 releases in 2018. In 2019, by far we only have two releases: - 1 maintenance release of 3.1 - 1 minor release of 3.2. However, the community put a lot of efforts to stabilize features of various release branches. There're: - 217 fixed patches in 3.1.3 [1] - 388 fixed patches in 3.2.1 [2] - 1172 fixed patches in 3.3.0 [3] (OMG!) I think it is the time to do maintenance releases of 3.1/3.2 and do a minor release for 3.3.0. In addition, I saw community discussion to do a 2.8.6 release for security fixes. Any other releases? I think there're release plans for Ozone as well. And please add your thoughts. Volunteers welcome! If you have interests to run a release as Release Manager (or co-Resource Manager), please respond to this email thread so we can coordinate. Thanks, Wangda Tan [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution = Fixed AND fixVersion = 3.1.3 [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution = Fixed AND fixVersion = 3.2.1 [3] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution = Fixed AND fixVersion = 3.3.0
Re: Aug Hadoop Community Meetup in China
Hi all, Kindly remind that we will have Beijing meetup on 8/10 soon, and the event will start since 10 am (CST time). The link has details on address: https://docs.google.com/document/d/1wDfyuQv6PDeKZGVdYJw6IfXUa_MIzIstbb8iva6Zt-U/edit?usp=sharing Thanks, Junping 俊平堵 于2019年7月23日周二 下午5:07写道： > Thanks for these positive feedbacks! The local community has voted the > date and location to be 8/10, Beijing. So please book your time ahead if > you have interest to join. > I have gathered a few topics, and some candidate places for hosting this > meetup. If you would like to propose more topics, please nominate it here > or ping me before this weekend (7/28, CST time). > Will update here when I have more to share. thx! > > > > > [image: Screen Shot 2019-07-23 at 10.15.54.png] > > [image: Screen Shot 2019-07-23 at 10.16.06.png] > > > > Thanks, > > Junping > > 俊平堵 于2019年7月18日周四 下午3:28写道： > >> Hi, all! >> >> I am glad to let you know that we are organizing >> Hadoop Contributors Meetup in China on Aug. >> >> >> This could be the first time hadoop community meetup in China and many >> attendees are expected to come from big data pioneers, such as: Cloudera, >> Tencent, Alibaba, Xiaomi, Didi, JD, Meituan, Toutiao, Sina, etc. >> >> >> We're still working out the details, such as dates, contents and >> locations. Here is a quick survey: https://www.surveymonkey.com/r/Y99RT3W >> where you can vote your prefer dates and locations if you would like to >> attend - the survey will end in July. 21. 12PM China Standard Time, and >> result will go public in next day. >> >> >> Also, please feel free to reach out to me if you have a topic to propose >> for the meetup. Will send out an update later with more details when I get >> more to share. Thanks! >> >> >> Cheers, >> >> >> Junping >> >
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/ [Aug 8, 2019 12:37:50 PM] (ericp) YARN-9685: NPE when rendering the info table of leaf queue in [Aug 8, 2019 1:41:04 PM] (weichiu) YARN-9711. Missing spaces in NMClientImpl (#1177) Contributed by Charles [Aug 8, 2019 1:52:04 PM] (elek) HDDS-1926. The new caching layer is used for old OM requests but not [Aug 8, 2019 4:55:46 PM] (github) HDDS-1619. Support volume acl operations for OM HA. Contributed by… [Aug 8, 2019 6:08:48 PM] (stevel) HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time [Aug 8, 2019 8:36:39 PM] (weichiu) HDFS-14459. ClosedChannelException silently ignored in [Aug 8, 2019 8:38:10 PM] (arp7) HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated. [Aug 8, 2019 8:45:29 PM] (weichiu) HDFS-14662. Document the usage of the new Balancer "asService" [Aug 8, 2019 8:46:31 PM] (weichiu) HDFS-14701. Change Log Level to warn in SlotReleaser. Contributed by [Aug 8, 2019 8:48:29 PM] (weichiu) HDFS-14705. Remove unused configuration dfs.min.replication. Contributed [Aug 8, 2019 8:50:30 PM] (weichiu) HDFS-14693. NameNode should log a warning when EditLog IPC logger's [Aug 8, 2019 10:40:19 PM] (github) HDDS-1863. Freon RandomKeyGenerator even if keySize is set to 0, it -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core Class org.apache.hadoop.applications.mawo.server.common.TaskStatus implements Cloneable but does not define or use clone method At TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines 39-346] Equals method for org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the argument is of type WorkerId At WorkerId.java:the argument is of type WorkerId At WorkerId.java:[line 114] org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object) does not check for null argument At WorkerId.java:null argument At WorkerId.java:[lines 114-115] Failed junit tests : hadoop.util.TestDiskCheckerWithDiskIo hadoop.util.TestBasicDiskValidator hadoop.util.TestReadWriteDiskValidator hadoop.hdfs.server.datanode.TestLargeBlockReport hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken hadoop.tools.dynamometer.TestDynamometerInfra hadoop.tools.dynamometer.TestDynamometerInfra cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/diff-compile-javac-root.txt [332K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/diff-patch-pylint.txt [220K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/diff-patch-shelldocs.txt [44K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifact/out/whitespace-eol.txt [9.6M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1223/artifa
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/ [Aug 8, 2019 5:18:00 PM] (ekrogen) HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao [Aug 8, 2019 9:55:41 PM] (weichiu) HDFS-14696. Backport HDFS-11273 to branch-2 (Move -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client Boxed value is unboxed and then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:[line 335] Failed junit tests : hadoop.hdfs.TestSafeMode hadoop.hdfs.server.balancer.TestBalancerRPCDelay hadoop.hdfs.server.datanode.TestFsDatasetCache hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.registry.secure.TestSecureLogins hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 hadoop.yarn.client.api.impl.TestNMClient hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema hadoop.mapreduce.v2.TestMRJobs cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-compile-cc-root-jdk1.7.0_95.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-compile-javac-root-jdk1.7.0_95.txt [328K] cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-compile-cc-root-jdk1.8.0_212.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-compile-javac-root-jdk1.8.0_212.txt [308K] checkstyle: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-patch-shellcheck.txt [72K] shelldocs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/whitespace-tabs.txt [1.2M] xml: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/xml.txt [12K] findbugs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-client-warnings.html [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-javadoc-javadoc-root-jdk1.7.0_95.txt [16K] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/diff-javadoc-javadoc-root-jdk1.8.0_212.txt [1.1M] unit: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/408/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [344K] https://builds.
