[jira] [Created] (HADOOP-16042) Update the link to HadoopJavaVersion
Akira Ajisaka created HADOOP-16042: -- Summary: Update the link to HadoopJavaVersion Key: HADOOP-16042 URL: https://issues.apache.org/jira/browse/HADOOP-16042 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Akira Ajisaka Old: https://wiki.apache.org/hadoop/HadoopJavaVersions New: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-15994) Upgrade Jackson2 to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-15994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reopened HADOOP-15994: Hi [~Jack-Lee], please do not resolve this issue unless a committer commits a patch to the repository. Now I'm running full unit tests locally. If they passed, I'll commit your patch tomorrow. > Upgrade Jackson2 to the latest version > -- > > Key: HADOOP-15994 > URL: https://issues.apache.org/jira/browse/HADOOP-15994 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Akira Ajisaka >Assignee: lqjacklee >Priority: Major > Attachments: 460.patch, HADOOP-15994-001.patch, > HADOOP-15994-002.patch, HADOOP-15994-003.patch > > > Now Jackson 2.9.5 is used and it is vulnerable (CVE-2018-11307). Let's > upgrade to the latest version. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1
Thanks Sunil and everyone who has worked on this release. +1 from me. - Verified checksums for tar file. - Built from tar.gz. - Ran through S3A and S3Guard integration tests (in AWS us-west 2). This includes a yarn minicluster test but is mostly focused on s3a/s3guard. Cheers, Aaron On Thu, Jan 10, 2019 at 2:32 PM Kuhu Shukla wrote: > +1 (non-binding) > > - built from source on Mac > - deployed on a pseudo distributed one node cluster > - ran example jobs like sleep and wordcount. > > Thank you for all the work on this release. > Regards, > Kuhu > > On Thu, Jan 10, 2019 at 10:32 AM Craig.Condit > wrote: > > > +1 (non-binding) > > > > - built from source on CentOS 7.5 > > - deployed single node cluster > > - ran several yarn jobs > > - ran multiple docker jobs, including spark-on-docker > > > > On 1/8/19, 5:42 AM, "Sunil G" wrote: > > > > Hi folks, > > > > > > Thanks to all of you who helped in this release [1] and for helping > to > > vote > > for RC0. I have created second release candidate (RC1) for Apache > > Hadoop > > 3.2.0. > > > > > > Artifacts for this RC are available here: > > > > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/ > > > > > > RC tag in git is release-3.2.0-RC1. > > > > > > > > The maven artifacts are available via repository.apache.org at > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1178/ > > > > > > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 > pm > > PST. > > > > > > > > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature > > additions > > > > are the highlights of this release. > > > > 1. Node Attributes Support in YARN > > > > 2. Hadoop Submarine project for running Deep Learning workloads on > YARN > > > > 3. Support service upgrade via YARN Service API and CLI > > > > 4. HDFS Storage Policy Satisfier > > > > 5. Support Windows Azure Storage - Blob file system in Hadoop > > > > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a > > > > 7. Improvements in Router-based HDFS federation > > > > > > > > Thanks to Wangda, Vinod, Marton for helping me in preparing the > > release. > > > > I have done few testing with my pseudo cluster. My +1 to start. > > > > > > > > Regards, > > > > Sunil > > > > > > > > [1] > > > > > > > https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E > > > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in > > (3.2.0) > > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = > Resolved > > ORDER BY fixVersion ASC > > > > > > >
Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1
+1 (non-binding) - built from source on Mac - deployed on a pseudo distributed one node cluster - ran example jobs like sleep and wordcount. Thank you for all the work on this release. Regards, Kuhu On Thu, Jan 10, 2019 at 10:32 AM Craig.Condit wrote: > +1 (non-binding) > > - built from source on CentOS 7.5 > - deployed single node cluster > - ran several yarn jobs > - ran multiple docker jobs, including spark-on-docker > > On 1/8/19, 5:42 AM, "Sunil G" wrote: > > Hi folks, > > > Thanks to all of you who helped in this release [1] and for helping to > vote > for RC0. I have created second release candidate (RC1) for Apache > Hadoop > 3.2.0. > > > Artifacts for this RC are available here: > > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/ > > > RC tag in git is release-3.2.0-RC1. > > > > The maven artifacts are available via repository.apache.org at > > https://repository.apache.org/content/repositories/orgapachehadoop-1178/ > > > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm > PST. > > > > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature > additions > > are the highlights of this release. > > 1. Node Attributes Support in YARN > > 2. Hadoop Submarine project for running Deep Learning workloads on YARN > > 3. Support service upgrade via YARN Service API and CLI > > 4. HDFS Storage Policy Satisfier > > 5. Support Windows Azure Storage - Blob file system in Hadoop > > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a > > 7. Improvements in Router-based HDFS federation > > > > Thanks to Wangda, Vinod, Marton for helping me in preparing the > release. > > I have done few testing with my pseudo cluster. My +1 to start. > > > > Regards, > > Sunil > > > > [1] > > > https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in > (3.2.0) > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved > ORDER BY fixVersion ASC > > >
[jira] [Resolved] (HADOOP-10225) Publish Maven javadoc and sources artifacts with Hadoop releases.
[ https://issues.apache.org/jira/browse/HADOOP-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lewis John McGibbney resolved HADOOP-10225. --- Resolution: Won't Fix The original patch was allowed to stagnate. I've closed the Github PR. Someone else can take this on if they deem javadoc and sources being useful for consumers of the software. > Publish Maven javadoc and sources artifacts with Hadoop releases. > - > > Key: HADOOP-10225 > URL: https://issues.apache.org/jira/browse/HADOOP-10225 > Project: Hadoop Common > Issue Type: Improvement > Components: build, documentation >Affects Versions: 3.0.0-alpha1 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Labels: hadoop, javadoc, maven, sources > Attachments: HADOOP-10225.patch > > > Right now Maven javadoc and sources artifacts do not accompany Hadoop > releases within Maven central. This means that one needs to checkout source > code to DEBUG aspects of the codebase... this is not user friendly. > The build script(s) should be amended to accommodate publication of javadoc > and sources artifacts alongside pom and jar artifacts. > Some history on this conversation can be seen below > http://s.apache.org/7qR -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-15662) ABFS: Better exception handling of DNS errors
[ https://issues.apache.org/jira/browse/HADOOP-15662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou reopened HADOOP-15662: -- 001 patch doesn't handle transient DNS resolution error, it could require manual retry when such things happen. The more obvious issue in current implementation is that the HostNotFoundException is logged at DEBUG level, so error msg in the long time retry could not be captured by user until DEBUG level is set for the logger. Will provide a fix so that the error msg is logged at WARN level and make it easier for user to read. > ABFS: Better exception handling of DNS errors > - > > Key: HADOOP-15662 > URL: https://issues.apache.org/jira/browse/HADOOP-15662 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Thomas Marquardt >Assignee: Da Zhou >Priority: Major > Fix For: 3.2.1 > > Attachments: HADOOP-15662-001.patch > > > DNS errors are common during testing due to typos or misconfiguration. They > can also occur in production, as some transient DNS issues occur from time to > time. > 1) Let's investigate if we can distinguish between the two and fail fast for > the test issues, but continue to have retry logic for the transient DNS > issues in production. > 2) Let's improve the error handling of DNS failures, so the user has an > actionable error message. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1
+1 (non-binding) - built from source on CentOS 7.5 - deployed single node cluster - ran several yarn jobs - ran multiple docker jobs, including spark-on-docker On 1/8/19, 5:42 AM, "Sunil G" wrote: Hi folks, Thanks to all of you who helped in this release [1] and for helping to vote for RC0. I have created second release candidate (RC1) for Apache Hadoop 3.2.0. Artifacts for this RC are available here: http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/ RC tag in git is release-3.2.0-RC1. The maven artifacts are available via repository.apache.org at https://repository.apache.org/content/repositories/orgapachehadoop-1178/ This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST. 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature additions are the highlights of this release. 1. Node Attributes Support in YARN 2. Hadoop Submarine project for running Deep Learning workloads on YARN 3. Support service upgrade via YARN Service API and CLI 4. HDFS Storage Policy Satisfier 5. Support Windows Azure Storage - Blob file system in Hadoop 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a 7. Improvements in Router-based HDFS federation Thanks to Wangda, Vinod, Marton for helping me in preparing the release. I have done few testing with my pseudo cluster. My +1 to start. Regards, Sunil [1] https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0) AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved ORDER BY fixVersion ASC
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/ [Jan 9, 2019 4:55:25 AM] (tasanuma) HADOOP-15941. [JDK 11] Compilation failure: package com.sun.jndi.ldap is [Jan 9, 2019 5:57:58 PM] (sean) HADOOP-16027. [DOC] Effective use of FS instances during S3A integration [Jan 9, 2019 6:57:24 PM] (hanishakoneru) HDDS-947. Implement OzoneManager State Machine. [Jan 9, 2019 7:20:57 PM] (hanishakoneru) Revert "HDDS-947. Implement OzoneManager State Machine." [Jan 9, 2019 11:24:58 PM] (jlowe) Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.util.TestDiskCheckerWithDiskIo hadoop.util.TestReadWriteDiskValidator hadoop.security.ssl.TestSSLFactory hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap hadoop.yarn.client.api.impl.TestAMRMClient cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-compile-javac-root.txt [336K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/whitespace-eol.txt [9.3M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/whitespace-tabs.txt [1.1M] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-hdds_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-ozone_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-ozone_common.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/branch-findbugs-hadoop-ozone_tools.txt [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/diff-javadoc-javadoc-root.txt [752K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [184K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1012/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [332K]
