Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/379/ [Jan 6, 2021 3:12:46 AM] (noreply) YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2. (#2592) [Jan 6, 2021 2:32:13 PM] (noreply) HADOOP-17444. ADLS Gen1: Update adls SDK to 2.3.9 (#2551) [Jan 6, 2021 6:43:37 PM] (noreply) HADOOP-17404. ABFS: Small write - Merge append and flush - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5
+1 (non-binding) - verified checksums & signatures - started a single-node cluster in docker and did some simple tests, also checked NN web-ui - Did a full test run in Spark using the RC and it looks good On Thu, Jan 7, 2021 at 3:01 AM 孙立晟 wrote: > +1 (non-binding) > > - Build from source with CentOS 7.6 and openjdk 1.8.0_275 > - checksums and signatures > - Setup a pseudo cluster and run shell test > > Thanks, > Lisheng Sun > > > Masatake Iwasaki 于2021年1月7日周四 下午4:44写道： > > > +1(binding) > > > > * checked hadoop-3.2.2-src.tar.gz > >* verified signature and checksums. > >* built on CentOS 8 and OpenJDK 8 by `mvn install -DskipTests -Pnative > > -Pdist -Pyarn-ui`. > >* launched 3-node docker cluster with NN-HA and RM-HA enabled. > >* ran example MR jobs. > >* built site documentation by `mvn site` and skimmed the contents > > including top-level index.html. > > * checked hadoop-3.2.2-site.tar.gz > >* verified signature and checksums. > >* skimmed the contents including top-level index.html. > > > > Thanks, > > Masatake Iwasaki > > > > On 2021/01/03 23:39, Xiaoqiao He wrote: > > > Hi folks, > > > > > > The release candidate (RC5) for Hadoop-3.2.2 is available now. > > > There are 4 commits[1] differences between RC5 and RC4[2]. > > > > > > The RC5 is available at: > > > http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC5 > > > The RC5 tag in github is here: > > > https://github.com/apache/hadoop/tree/release-3.2.2-RC5 > > > The maven artifacts are staged at: > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1298 > > > > > > You can find my public key at: > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS or > > > https://people.apache.org/keys/committer/hexiaoqiao.asc directly. > > > > > > Please try the release and vote. > > > > > > I have done a simple test. > > > * Verify gpg sign and md5sum. > > > * Check staging repositories point to the correct artifact( > > > https://repository.apache.org/content/repositories/staging/). > > > * Setup pseudo cluster with HDFS and YARN. > > > * Run simple FsShell - mkdir/put/get/mv/rm. > > > * Submit example mr applications and check the result - Pi & wordcount. > > > * Check the web UI and the release year of > > > NameNode/DataNode/Resourcemanager/NodeManager including YARN UI2. > > > > > > My +1 to start. > > > > > > Thanks and Happy New Year! > > > - He Xiaoqiao > > > > > > [1] > > > > > > https://github.com/apache/hadoop/compare/release-3.2.2-RC4...release-3.2.2-RC5 > > > [2] > > > > > > https://lists.apache.org/thread.html/r8911d2934ebfe3869874842be94d1cfb00d99334bc93819d71466243%40%3Chdfs-dev.hadoop.apache.org%3E > > > [3] > > > > > > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12335948 > > > > > > > - > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > >
Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/ [Jan 5, 2021 2:43:01 PM] (noreply) HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0 (#2593) [Jan 5, 2021 9:09:41 PM] (noreply) HADOOP-17430. Restore ability to set Text to empty byte array (#2545) [Jan 6, 2021 3:12:46 AM] (noreply) YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2. (#2592) [Jan 6, 2021 2:32:13 PM] (noreply) HADOOP-17444. ADLS Gen1: Update adls SDK to 2.3.9 (#2551) [Jan 6, 2021 6:43:37 PM] (noreply) HADOOP-17404. ABFS: Small write - Merge append and flush -1 overall The following subsystems voted -1: mvnsite pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA hadoop.hdfs.TestFileChecksum hadoop.hdfs.TestWriteConfigurationToDFS hadoop.hdfs.TestViewDistributedFileSystem hadoop.hdfs.TestFileCreation hadoop.hdfs.TestErasureCodingExerciseAPIs hadoop.hdfs.TestDFSInputStream hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy hadoop.hdfs.TestReconstructStripedFile hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy hadoop.yarn.client.api.impl.TestTimelineClientV2Impl hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer hadoop.yarn.client.TestRMFailoverProxyProvider hadoop.yarn.client.TestNoHaRMFailoverProxyProvider hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.server.router.clientrm.TestFederationClientInterceptor hadoop.yarn.server.timelineservice.documentstore.TestDocumentStoreTimelineReaderImpl hadoop.yarn.server.timelineservice.documentstore.TestDocumentStoreCollectionCreator hadoop.yarn.server.timelineservice.documentstore.writer.cosmosdb.TestCosmosDBDocumentStoreWriter hadoop.yarn.server.timelineservice.documentstore.reader.cosmosdb.TestCosmosDBDocumentStoreReader hadoop.yarn.server.timelineservice.documentstore.TestDocumentStoreTimelineWriterImpl hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.tools.dynamometer.TestDynamometerInfra hadoop.tools.dynamometer.TestDynamometerInfra cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/diff-compile-cc-root.txt [48K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/diff-compile-javac-root.txt [596K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/diff-checkstyle-root.txt [16M] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/patch-mvnsite-root.txt [496K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/diff-patch-shelldocs.txt [96K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/whitespace-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/102/artifact/out/whitespace-tabs.txt [2.0M] xml:
[jira] [Created] (YARN-10563) Fix dependency exclusion problem in poms
Peter Bacsko created YARN-10563: --- Summary: Fix dependency exclusion problem in poms Key: YARN-10563 URL: https://issues.apache.org/jira/browse/YARN-10563 Project: Hadoop YARN Issue Type: Sub-task Reporter: Peter Bacsko Assignee: Peter Bacsko -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/ No changes -1 overall The following subsystems voted -1: asflicense hadolint jshint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml Failed junit tests : hadoop.util.TestReadWriteDiskValidator hadoop.util.TestDiskCheckerWithDiskIo hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.server.namenode.TestFsck hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer hadoop.yarn.server.resourcemanager.TestClientRMService hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.mapred.TestMiniMRChildTask hadoop.mapred.TestMRTimelineEventHandling hadoop.mapreduce.security.TestJHSSecurity hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.resourceestimator.solver.impl.TestLpSolver jshint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-patch-jshint.txt [208K] cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-compile-javac-root.txt [456K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-patch-shellcheck.txt [56K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/whitespace-tabs.txt [1.3M] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/xml.txt [4.0K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/diff-javadoc-javadoc-root.txt [20K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [220K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [276K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [12K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/170/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [36K]
Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5
+1 (non-binding) - Build from source with CentOS 7.6 and openjdk 1.8.0_275 - checksums and signatures - Setup a pseudo cluster and run shell test Thanks, Lisheng Sun Masatake Iwasaki 于2021年1月7日周四 下午4:44写道： > +1(binding) > > * checked hadoop-3.2.2-src.tar.gz >* verified signature and checksums. >* built on CentOS 8 and OpenJDK 8 by `mvn install -DskipTests -Pnative > -Pdist -Pyarn-ui`. >* launched 3-node docker cluster with NN-HA and RM-HA enabled. >* ran example MR jobs. >* built site documentation by `mvn site` and skimmed the contents > including top-level index.html. > * checked hadoop-3.2.2-site.tar.gz >* verified signature and checksums. >* skimmed the contents including top-level index.html. > > Thanks, > Masatake Iwasaki > > On 2021/01/03 23:39, Xiaoqiao He wrote: > > Hi folks, > > > > The release candidate (RC5) for Hadoop-3.2.2 is available now. > > There are 4 commits[1] differences between RC5 and RC4[2]. > > > > The RC5 is available at: > > http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC5 > > The RC5 tag in github is here: > > https://github.com/apache/hadoop/tree/release-3.2.2-RC5 > > The maven artifacts are staged at: > > https://repository.apache.org/content/repositories/orgapachehadoop-1298 > > > > You can find my public key at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS or > > https://people.apache.org/keys/committer/hexiaoqiao.asc directly. > > > > Please try the release and vote. > > > > I have done a simple test. > > * Verify gpg sign and md5sum. > > * Check staging repositories point to the correct artifact( > > https://repository.apache.org/content/repositories/staging/). > > * Setup pseudo cluster with HDFS and YARN. > > * Run simple FsShell - mkdir/put/get/mv/rm. > > * Submit example mr applications and check the result - Pi & wordcount. > > * Check the web UI and the release year of > > NameNode/DataNode/Resourcemanager/NodeManager including YARN UI2. > > > > My +1 to start. > > > > Thanks and Happy New Year! > > - He Xiaoqiao > > > > [1] > > > https://github.com/apache/hadoop/compare/release-3.2.2-RC4...release-3.2.2-RC5 > > [2] > > > https://lists.apache.org/thread.html/r8911d2934ebfe3869874842be94d1cfb00d99334bc93819d71466243%40%3Chdfs-dev.hadoop.apache.org%3E > > [3] > > > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12335948 > > > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5
+1(binding) * checked hadoop-3.2.2-src.tar.gz * verified signature and checksums. * built on CentOS 8 and OpenJDK 8 by `mvn install -DskipTests -Pnative -Pdist -Pyarn-ui`. * launched 3-node docker cluster with NN-HA and RM-HA enabled. * ran example MR jobs. * built site documentation by `mvn site` and skimmed the contents including top-level index.html. * checked hadoop-3.2.2-site.tar.gz * verified signature and checksums. * skimmed the contents including top-level index.html. Thanks, Masatake Iwasaki On 2021/01/03 23:39, Xiaoqiao He wrote: Hi folks, The release candidate (RC5) for Hadoop-3.2.2 is available now. There are 4 commits[1] differences between RC5 and RC4[2]. The RC5 is available at: http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC5 The RC5 tag in github is here: https://github.com/apache/hadoop/tree/release-3.2.2-RC5 The maven artifacts are staged at: https://repository.apache.org/content/repositories/orgapachehadoop-1298 You can find my public key at: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS or https://people.apache.org/keys/committer/hexiaoqiao.asc directly. Please try the release and vote. I have done a simple test. * Verify gpg sign and md5sum. * Check staging repositories point to the correct artifact( https://repository.apache.org/content/repositories/staging/). * Setup pseudo cluster with HDFS and YARN. * Run simple FsShell - mkdir/put/get/mv/rm. * Submit example mr applications and check the result - Pi & wordcount. * Check the web UI and the release year of NameNode/DataNode/Resourcemanager/NodeManager including YARN UI2. My +1 to start. Thanks and Happy New Year! - He Xiaoqiao [1] https://github.com/apache/hadoop/compare/release-3.2.2-RC4...release-3.2.2-RC5 [2] https://lists.apache.org/thread.html/r8911d2934ebfe3869874842be94d1cfb00d99334bc93819d71466243%40%3Chdfs-dev.hadoop.apache.org%3E [3] https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12335948 - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5
+1 (binding) On Mon, Jan 4, 2021 at 11:11 PM Akira Ajisaka wrote: > +1 (binding) > > - Verified the checksums and signatures > - Verified the source is the same as the RC5 tag > - Built from source with CentOS 7.9 and Java 1.8.0_275 > - Setup pseudo cluster and ran some FsShell commands > > Thanks, > Akira > > On Sun, Jan 3, 2021 at 11:40 PM Xiaoqiao He wrote: > > > > Hi folks, > > > > The release candidate (RC5) for Hadoop-3.2.2 is available now. > > There are 4 commits[1] differences between RC5 and RC4[2]. > > > > The RC5 is available at: > > http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC5 > > The RC5 tag in github is here: > > https://github.com/apache/hadoop/tree/release-3.2.2-RC5 > > The maven artifacts are staged at: > > https://repository.apache.org/content/repositories/orgapachehadoop-1298 > > > > You can find my public key at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS or > > https://people.apache.org/keys/committer/hexiaoqiao.asc directly. > > > > Please try the release and vote. > > > > I have done a simple test. > > * Verify gpg sign and md5sum. > > * Check staging repositories point to the correct artifact( > > https://repository.apache.org/content/repositories/staging/). > > * Setup pseudo cluster with HDFS and YARN. > > * Run simple FsShell - mkdir/put/get/mv/rm. > > * Submit example mr applications and check the result - Pi & wordcount. > > * Check the web UI and the release year of > > NameNode/DataNode/Resourcemanager/NodeManager including YARN UI2. > > > > My +1 to start. > > > > Thanks and Happy New Year! > > - He Xiaoqiao > > > > [1] > > > https://github.com/apache/hadoop/compare/release-3.2.2-RC4...release-3.2.2-RC5 > > [2] > > > https://lists.apache.org/thread.html/r8911d2934ebfe3869874842be94d1cfb00d99334bc93819d71466243%40%3Chdfs-dev.hadoop.apache.org%3E > > [3] > > > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12335948 > > - > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >
