[jira] [Created] (HADOOP-16151) pip install pylint fails in branch-2.8 and branch-2.9
Akira Ajisaka created HADOOP-16151:
--
Summary: pip install pylint fails in branch-2.8 and branch-2.9
Key: HADOOP-16151
URL: https://issues.apache.org/jira/browse/HADOOP-16151
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Akira Ajisaka
https://builds.apache.org/job/PreCommit-HADOOP-Build/15982/consoleFull
{noformat}
Step 24/31 : RUN pip install pylint==1.9.2
---> Running in c0bed03b7115
Downloading/unpacking pylint==1.9.2
Downloading/unpacking configparser (from pylint==1.9.2)
Downloading configparser-3.7.3-py2.py3-none-any.whl
Downloading/unpacking backports.functools-lru-cache (from pylint==1.9.2)
Downloading backports.functools_lru_cache-1.5-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six in
/usr/lib/python2.7/dist-packages (from pylint==1.9.2)
Downloading/unpacking isort>=4.2.5 (from pylint==1.9.2)
Running setup.py (path:/tmp/pip_build_root/isort/setup.py) egg_info for
package isort
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution
option: 'python_requires'
warnings.warn(msg)
error in isort setup command: 'install_requires' must be a string or list
of strings containing valid project/version requirement specifiers
Complete output from command python setup.py egg_info:
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution
option: 'python_requires'
warnings.warn(msg)
error in isort setup command: 'install_requires' must be a string or list of
strings containing valid project/version requirement specifiers
Cleaning up...
Command python setup.py egg_info failed with error code 1 in
/tmp/pip_build_root/isort
Storing debug log for failure in /root/.pip/pip.log
The command '/bin/sh -c pip install pylint==1.9.2' returned a non-zero code: 1
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16110) ~/hadoop-env doesn't support HADOOP_OPTIONAL_TOOLS
[
https://issues.apache.org/jira/browse/HADOOP-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-16110.
-
Resolution: Won't Fix
> ~/hadoop-env doesn't support HADOOP_OPTIONAL_TOOLS
> --
>
> Key: HADOOP-16110
> URL: https://issues.apache.org/jira/browse/HADOOP-16110
> Project: Hadoop Common
> Issue Type: Bug
> Components: bin
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Priority: Minor
>
> if you set {{HADOOP_OPTIONAL_TOOLS}} in ~.hadoop-env, it doesn't get picked
> up because the HADOOP_OPTIONAL_TOOLS expansion takes place in the parse
> process way before {{hadoop_exec_user_hadoopenv}} is invoked.
> Unless I've really misunderstood what ~/.hadoop-env is meant to do "let me
> set hadoop env vars", I'd have expected that tools env var examining (and so:
> loading of optional tools) to take place after
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: builds failing on trunk, "convergence failure"?
I've been seeing this for months. I believe Shane saw it for a while as well, but his was fixed by removing his local maven repo. Billie On Tue, Feb 26, 2019 at 6:48 AM Steve Loughran wrote: > Is anyone else seeing mr-client-app failing to build with a convergence > error? It's started for me today on trunk > > > [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @ > hadoop-mapreduce-client-app --- > [WARNING] > Dependency convergence error for org.hamcrest:hamcrest-core:1.1 paths to > dependency are: > +-org.apache.hadoop:hadoop-mapreduce-client-app:3.3.0-SNAPSHOT > +-com.github.stefanbirkner:system-rules:1.18.0 > +-junit:junit-dep:4.11.20120805.1225 > +-org.hamcrest:hamcrest-core:1.1 > and > +-org.apache.hadoop:hadoop-mapreduce-client-app:3.3.0-SNAPSHOT > +-junit:junit:4.12 > +-org.hamcrest:hamcrest-core:1.3 > > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence > failed with message: > Failed while enforcing releasability. See above detailed error message. > > > It's only a trivial pom fixup, but I'm surprised its suddenly surfaced. Am > I the only person seeing it? >
[jira] [Created] (HADOOP-16149) hadoop-mapreduce-client-app build not converging due to transient dependencies
Steve Loughran created HADOOP-16149:
---
Summary: hadoop-mapreduce-client-app build not converging due to
transient dependencies
Key: HADOOP-16149
URL: https://issues.apache.org/jira/browse/HADOOP-16149
Project: Hadoop Common
Issue Type: Bug
Components: build
Affects Versions: 3.3.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Clean build of trunk failing today. No obvious change locally except I did
accidentally kick off a build which may have pulled in some -SNAPSHOT artifacts
from asf snapshot repo
{code}
Dependency convergence error for org.hamcrest:hamcrest-core:1.1 paths to
dependency are:
+-org.apache.hadoop:hadoop-mapreduce-client-app:3.3.0-SNAPSHOT
+-com.github.stefanbirkner:system-rules:1.18.0
+-junit:junit-dep:4.11.20120805.1225
+-org.hamcrest:hamcrest-core:1.1
and
+-org.apache.hadoop:hadoop-mapreduce-client-app:3.3.0-SNAPSHOT
+-junit:junit:4.12
+-org.hamcrest:hamcrest-core:1.3
{code}
This goes away if system-rules excludes junit and hamcrest
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
builds failing on trunk, "convergence failure"?
Is anyone else seeing mr-client-app failing to build with a convergence error? It's started for me today on trunk [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @ hadoop-mapreduce-client-app --- [WARNING] Dependency convergence error for org.hamcrest:hamcrest-core:1.1 paths to dependency are: +-org.apache.hadoop:hadoop-mapreduce-client-app:3.3.0-SNAPSHOT +-com.github.stefanbirkner:system-rules:1.18.0 +-junit:junit-dep:4.11.20120805.1225 +-org.hamcrest:hamcrest-core:1.1 and +-org.apache.hadoop:hadoop-mapreduce-client-app:3.3.0-SNAPSHOT +-junit:junit:4.12 +-org.hamcrest:hamcrest-core:1.3 [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message: Failed while enforcing releasability. See above detailed error message. It's only a trivial pom fixup, but I'm surprised its suddenly surfaced. Am I the only person seeing it?
review needed of HADOOP-16107; fixes failing build of MAPREDUCE-7184
Can someone take a look at HADOOP-16107 https://issues.apache.org/jira/browse/HADOOP-16107 It fixes the problem surfacing in jenkins builds and filed as MAPREDUCE-7184 https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1055/testReport/junit/org.apache.hadoop.mapred/TestJobCounters/testNewCounterA/ thanks
[jira] [Created] (HADOOP-16148) Cleanup LineReader Unit Test
BELUGA BEHR created HADOOP-16148:
Summary: Cleanup LineReader Unit Test
Key: HADOOP-16148
URL: https://issues.apache.org/jira/browse/HADOOP-16148
Project: Hadoop Common
Issue Type: Improvement
Components: common
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
I was trying to track down a bug and thought it might be coming from the
{{LineReader}} class. It wasn't. However, I did clean up the unit test for
this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/ [Feb 25, 2019 6:00:46 AM] (sunilg) YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for [Feb 25, 2019 9:41:43 AM] (elek) HDDS-1170. Fix the FindBugs issue in OzoneClientAdapterFactory. [Feb 25, 2019 9:58:45 AM] (sunilg) YARN-9168. DistributedShell client timeout should be -1 by default. [Feb 25, 2019 4:08:13 PM] (wwei) YARN-9329. updatePriority is blocked when using FairScheduler. [Feb 25, 2019 7:06:14 PM] (vinayakumarb) HDFS-7133. Support clearing namespace quota on '/'. Contributed by Ayush [Feb 25, 2019 7:45:37 PM] (gifuma) YARN-9287. Consecutive StringBuilder append should be reuse. Contributed [Feb 25, 2019 9:15:28 PM] (szetszwo) HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all [Feb 25, 2019 9:39:13 PM] (inigoiri) HADOOP-16125. Support multiple bind users in LdapGroupsMapping. [Feb 25, 2019 9:50:37 PM] (aengineer) HDDS-1151. Propagate the tracing id in ScmBlockLocationProtocol. [Feb 25, 2019 10:35:02 PM] (shv) HDFS-14130. [SBN read] Make ZKFC ObserverNode aware. Contributed by -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml Failed junit tests : hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.hdfs.qjournal.server.TestJournalNodeSync hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized hadoop.mapred.TestJobCounters hadoop.yarn.sls.appmaster.TestAMSimulator cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/diff-compile-javac-root.txt [336K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/diff-patch-hadolint.txt [8.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/diff-patch-pylint.txt [144K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/whitespace-eol.txt [9.6M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/xml.txt [16K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-hdds_client.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-ozone_client.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-ozone_common.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1059/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K]
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/ No changes -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml FindBugs : module:hadoop-common-project/hadoop-common Class org.apache.hadoop.fs.GlobalStorageStatistics defines non-transient non-serializable instance field map In GlobalStorageStatistics.java:instance field map In GlobalStorageStatistics.java FindBugs : module:hadoop-hdfs-project/hadoop-hdfs Dead store to state in org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Saver.save(OutputStream, INodeSymlink) At FSImageFormatPBINode.java:org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Saver.save(OutputStream, INodeSymlink) At FSImageFormatPBINode.java:[line 623] FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client Boxed value is unboxed and then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:[line 335] Failed junit tests : hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.namenode.TestNamenodeCapacityReport hadoop.registry.secure.TestSecureLogins hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 hadoop.yarn.client.api.impl.TestAMRMProxy hadoop.yarn.sls.nodemanager.TestNMSimulator cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-compile-cc-root-jdk1.7.0_95.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-compile-javac-root-jdk1.7.0_95.txt [328K] cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-compile-cc-root-jdk1.8.0_191.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-compile-javac-root-jdk1.8.0_191.txt [308K] checkstyle: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-patch-shellcheck.txt [72K] shelldocs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/whitespace-tabs.txt [1.2M] xml: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/xml.txt [20K] findbugs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/244/artifact/out/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html [8.0K]
