[jira] [Resolved] (HADOOP-18941) Modify HBase version in BUILDING.txt
[ https://issues.apache.org/jira/browse/HADOOP-18941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18941. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Modify HBase version in BUILDING.txt > > > Key: HADOOP-18941 > URL: https://issues.apache.org/jira/browse/HADOOP-18941 > Project: Hadoop Common > Issue Type: Bug >Reporter: Zepeng Zhang >Assignee: Zepeng Zhang >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > In current BUILDING.txt document, the version of HBase, which is used by YARN > Timeline Service V2 is a bit older than the actual one. Hence, I hereby > request to modify this uncertain description in the document. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18942. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18867.
---
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
Masatake Iwasaki created HADOOP-18942: - Summary: Upgrade ZooKeeper to 3.7.2 Key: HADOOP-18942 URL: https://issues.apache.org/jira/browse/HADOOP-18942 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
Masatake Iwasaki created HADOOP-18867:
-
Summary: Upgrade ZooKeeper to 3.6.4
Key: HADOOP-18867
URL: https://issues.apache.org/jira/browse/HADOOP-18867
Project: Hadoop Common
Issue Type: Improvement
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of the
ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
{noformat}
$ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
clean install
Dependency convergence error for
org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
are:
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
+-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
+-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
+-org.apache.yetus:audience-annotations:jar:0.13.0:compile
and
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
+-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
+-org.apache.yetus:audience-annotations:jar:0.13.0:compile
and
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
+-org.apache.hbase:hbase-common:jar:2.2.4:compile
+-org.apache.yetus:audience-annotations:jar:0.5.0:compile
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18598) maven site generation doesn't include javadocs
[ https://issues.apache.org/jira/browse/HADOOP-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18598. --- Fix Version/s: 3.4.0 3.3.5 Hadoop Flags: Reviewed Resolution: Fixed > maven site generation doesn't include javadocs > -- > > Key: HADOOP-18598 > URL: https://issues.apache.org/jira/browse/HADOOP-18598 > Project: Hadoop Common > Issue Type: Bug > Components: site >Affects Versions: 3.3.5 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Blocker > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > > the rc0 excluded all the site docs. running mvn site on trunk throws up site > plugin issues, which may be related, so start by updating that. > rc validation scripts to include checks for the api/index.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18609) Fix source code modification after building with docs profile
Masatake Iwasaki created HADOOP-18609:
-
Summary: Fix source code modification after building with docs
profile
Key: HADOOP-18609
URL: https://issues.apache.org/jira/browse/HADOOP-18609
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
MetricsSystem.java and MetricsSystemImpl.java are automatically modified by
building with {{-Pdocs}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18608) Add validation to check Javadocs are included in site documentation
Masatake Iwasaki created HADOOP-18608: - Summary: Add validation to check Javadocs are included in site documentation Key: HADOOP-18608 URL: https://issues.apache.org/jira/browse/HADOOP-18608 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki We should add validation to avoid an issue like HADOOP-18598 found in release process. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18601) Fix build failure with docs profile
Masatake Iwasaki created HADOOP-18601:
-
Summary: Fix build failure with docs profile
Key: HADOOP-18601
URL: https://issues.apache.org/jira/browse/HADOOP-18601
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Build with docs profile failed due to depcheck error.
{noformat}
$ mvn clean install -DskipTests -DskipShade -Pdocs
...
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-hdfs ---
[WARNING]
Dependency convergence error for xerces:xercesImpl:jar:2.12.2:provided paths to
dependency are:
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT:provided
+-xerces:xercesImpl:jar:2.12.2:provided
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
+-xerces:xercesImpl:jar:2.12.2:test
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-hdfs-client:jar:3.4.0-SNAPSHOT:provided
+-xerces:xercesImpl:jar:2.12.2:provided
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-com.google.code.findbugs:findbugs:jar:3.0.1:provided
+-jaxen:jaxen:jar:1.1.6:provided
+-xerces:xercesImpl:jar:2.6.2:provided
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-xerces:xercesImpl:jar:2.12.2:compile
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
failed with message:
Failed while enforcing releasability. See above detailed error message.
[INFO]
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18376) Fix failure of instanciation of s3gatewayrequestlog appender due to lack of HttpRequestLogAppender
[
https://issues.apache.org/jira/browse/HADOOP-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18376.
---
Resolution: Invalid
The cause was old log4j.properties in my testing environment.
> Fix failure of instanciation of s3gatewayrequestlog appender due to lack of
> HttpRequestLogAppender
> --
>
> Key: HADOOP-18376
> URL: https://issues.apache.org/jira/browse/HADOOP-18376
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {noformat}
> $ bin/hadoop version
> log4j:ERROR Could not instantiate class
> [org.apache.hadoop.http.HttpRequestLogAppender].
> java.lang.ClassNotFoundException:
> org.apache.hadoop.http.HttpRequestLogAppender
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
> at
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
> at org.apache.log4j.LogManager.(LogManager.java:127)
> at
> org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
> at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
> at
> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
> at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
> at org.apache.hadoop.util.VersionInfo.(VersionInfo.java:37)
> log4j:ERROR Could not instantiate appender named "s3gatewayrequestlog".
> Hadoop 3.4.0-SNAPSHOT
> Source code repository Unknown -r c92ff0b4f1135e84527e25087d9cc65250bd0222
> Compiled by rocky on 2022-07-27T23:17Z
> Compiled with protoc 3.7.1
> From source with checksum 486fac66d9b4349cc0fd3d5e8bc38e40
> This command was run using
> /home/rocky/dist/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.4.0-SNAPSHOT.jar
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18376) Fix failure of instanciation of s3gatewayrequestlog due to lack of HttpRequestLogAppender
Masatake Iwasaki created HADOOP-18376:
-
Summary: Fix failure of instanciation of s3gatewayrequestlog due
to lack of HttpRequestLogAppender
Key: HADOOP-18376
URL: https://issues.apache.org/jira/browse/HADOOP-18376
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
$ bin/hadoop version
log4j:ERROR Could not instantiate class
[org.apache.hadoop.http.HttpRequestLogAppender].
java.lang.ClassNotFoundException: org.apache.hadoop.http.HttpRequestLogAppender
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
at
org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)
at
org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)
at
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at
org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at
org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
at
org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at org.apache.hadoop.util.VersionInfo.(VersionInfo.java:37)
log4j:ERROR Could not instantiate appender named "s3gatewayrequestlog".
Hadoop 3.4.0-SNAPSHOT
Source code repository Unknown -r c92ff0b4f1135e84527e25087d9cc65250bd0222
Compiled by rocky on 2022-07-27T23:17Z
Compiled with protoc 3.7.1
>From source with checksum 486fac66d9b4349cc0fd3d5e8bc38e40
This command was run using
/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.4.0-SNAPSHOT.jar
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18375) Fix failure of shelltest for hadoop_add_ldlibpath
Masatake Iwasaki created HADOOP-18375: - Summary: Fix failure of shelltest for hadoop_add_ldlibpath Key: HADOOP-18375 URL: https://issues.apache.org/jira/browse/HADOOP-18375 Project: Hadoop Common Issue Type: Bug Components: test Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Test cases in hadoop_add_ldlibpath.bats failed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18100) Change scope of inner classes in InodeTree to make them accessible outside package
[ https://issues.apache.org/jira/browse/HADOOP-18100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18100. --- Fix Version/s: 3.4.0 3.2.4 3.3.9 Resolution: Fixed > Change scope of inner classes in InodeTree to make them accessible outside > package > -- > > Key: HADOOP-18100 > URL: https://issues.apache.org/jira/browse/HADOOP-18100 > Project: Hadoop Common > Issue Type: Bug >Reporter: Abhishek Das >Assignee: Abhishek Das >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.2.4, 3.3.9 > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18334) Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
Masatake Iwasaki created HADOOP-18334:
-
Summary: Fix create-release to address removal of GPG_AGENT_INFO
in branch-3.2
Key: HADOOP-18334
URL: https://issues.apache.org/jira/browse/HADOOP-18334
Project: Hadoop Common
Issue Type: Bug
Components: build
Affects Versions: 3.2.3
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
gpg v2.1 and above does not export GPG_AGENT_INFO. create-release script need
to export the info by itself to make {{--sign}} work. It was addressed as part
of HADOOP-16797 in branch-3.3 and trunk. Since we can not backport aarch64
support to branch-3.2, I filed this issue for branch-3.2 only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18309) Upgrade bundled Tomcat to 8.5.76 or higher
[
https://issues.apache.org/jira/browse/HADOOP-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18309.
---
Fix Version/s: 2.10.3
Hadoop Flags: Reviewed
Resolution: Fixed
> Upgrade bundled Tomcat to 8.5.76 or higher
> --
>
> Key: HADOOP-18309
> URL: https://issues.apache.org/jira/browse/HADOOP-18309
> Project: Hadoop Common
> Issue Type: Improvement
> Components: httpfs, kms
>Affects Versions: 2.10.1, 2.10.2
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.10.3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently we are using 8.5.75 which is affected by
> {color:#33}CVE-2022-25762{color}
> More Details -
> [https://lists.apache.org/thread/qzkqh2819x6zsmj7vwdf14ng2fdgckw7]
> Lets upgrade 8.5.76 or higher
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18299) Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
[
https://issues.apache.org/jira/browse/HADOOP-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18299.
---
Assignee: (was: Masatake Iwasaki)
Resolution: Cannot Reproduce
I can not reproduce this after wiping out ~/.m2/repository.
> Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
> -
>
> Key: HADOOP-18299
> URL: https://issues.apache.org/jira/browse/HADOOP-18299
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.3
>Reporter: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Build on aarch64 failed due to banned classes.
> {{com.github.stephenc.findbugs:findbugs-annotations}} seems to be pulled
> somewhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18299) Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
Masatake Iwasaki created HADOOP-18299:
-
Summary: Fix unexpected contents in
hadoop-client-check-test-invariants on aarch64
Key: HADOOP-18299
URL: https://issues.apache.org/jira/browse/HADOOP-18299
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.3.3
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Build on aarch64 failed due to banned classes.
{{com.github.stephenc.findbugs:findbugs-annotations}} seems to be pulled
somewhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18276) Rename CHANGES to CHANGELOG in branch-2.10
Masatake Iwasaki created HADOOP-18276: - Summary: Rename CHANGES to CHANGELOG in branch-2.10 Key: HADOOP-18276 URL: https://issues.apache.org/jira/browse/HADOOP-18276 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 2.10.2 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The file names of changelog generated by releasedocmaker of Yetus was changed from CHANGES.md to CHANGELOG.md. Hyperlinks in the generated index.html of "Changelog and Release Notes" page were also affected. The CHANGES*.md generated in previous releases must be renamed to avoid 404. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18267) Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc in the PATH
Masatake Iwasaki created HADOOP-18267: - Summary: Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc in the PATH Key: HADOOP-18267 URL: https://issues.apache.org/jira/browse/HADOOP-18267 Project: Hadoop Common Issue Type: Sub-task Components: build Affects Versions: 2.10.1 Reporter: Masatake Iwasaki We still need protoc in branch-2.10 even after HADOOP-16598. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17992) Disable JIRA plugin for YETUS on Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17992.
---
Resolution: Duplicate
Since the commit log of [#3623|https://github.com/apache/hadoop/pull/3623] is
mentioning HADOOP-17988 instead of HADOOP-17992, I changed the resolution to
"Duplicate" and removed "Fix Version/s" in order to avoid confusing
releasedocmaker.
> Disable JIRA plugin for YETUS on Hadoop
> ---
>
> Key: HADOOP-17992
> URL: https://issues.apache.org/jira/browse/HADOOP-17992
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.10.2
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I’ve been noticing an issue with Jenkins CI where a file jira-json goes
> missing all of a sudden – jenkins / hadoop-multibranch / PR-3588 / #2
> (apache.org)
> {code}
> [2021-10-27T17:52:58.787Z] Processing:
> https://github.com/apache/hadoop/pull/3588
> [2021-10-27T17:52:58.787Z] GITHUB PR #3588 is being downloaded from
> [2021-10-27T17:52:58.787Z]
> https://api.github.com/repos/apache/hadoop/pulls/3588
> [2021-10-27T17:52:58.787Z] JSON data at Wed Oct 27 17:52:55 UTC 2021
> [2021-10-27T17:52:58.787Z] Patch data at Wed Oct 27 17:52:56 UTC 2021
> [2021-10-27T17:52:58.787Z] Diff data at Wed Oct 27 17:52:56 UTC 2021
> [2021-10-27T17:52:59.814Z] awk: cannot open
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-3588/centos-7/out/jira-json
> (No such file or directory)
> [2021-10-27T17:52:59.814Z] ERROR: https://github.com/apache/hadoop/pull/3588
> issue status is not matched with "Patch Available".
> [2021-10-27T17:52:59.814Z]
> {code}
> This causes the pipeline run to fail. I’ve seen this in my multiple attempts
> to re-run the CI on my PR –
> # After 45 minutes – [jenkins / hadoop-multibranch / PR-3588 / #1
> (apache.org)|https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-3588/1/pipeline/]
> # After 1 minute – [jenkins / hadoop-multibranch / PR-3588 / #2
> (apache.org)|https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-3588/2/pipeline/]
> # After 17 minutes – [jenkins / hadoop-multibranch / PR-3588 / #3
> (apache.org)|https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-3588/3/pipeline/]
> The hadoop-multibranch pipeline doesn't use ASF JIRA, thus, we're disabling
> the *jira* plugin to fix this issue.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-17992) Disable JIRA plugin for YETUS on Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki reopened HADOOP-17992:
---
> Disable JIRA plugin for YETUS on Hadoop
> ---
>
> Key: HADOOP-17992
> URL: https://issues.apache.org/jira/browse/HADOOP-17992
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.10.2
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.10.2
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I’ve been noticing an issue with Jenkins CI where a file jira-json goes
> missing all of a sudden – jenkins / hadoop-multibranch / PR-3588 / #2
> (apache.org)
> {code}
> [2021-10-27T17:52:58.787Z] Processing:
> https://github.com/apache/hadoop/pull/3588
> [2021-10-27T17:52:58.787Z] GITHUB PR #3588 is being downloaded from
> [2021-10-27T17:52:58.787Z]
> https://api.github.com/repos/apache/hadoop/pulls/3588
> [2021-10-27T17:52:58.787Z] JSON data at Wed Oct 27 17:52:55 UTC 2021
> [2021-10-27T17:52:58.787Z] Patch data at Wed Oct 27 17:52:56 UTC 2021
> [2021-10-27T17:52:58.787Z] Diff data at Wed Oct 27 17:52:56 UTC 2021
> [2021-10-27T17:52:59.814Z] awk: cannot open
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-3588/centos-7/out/jira-json
> (No such file or directory)
> [2021-10-27T17:52:59.814Z] ERROR: https://github.com/apache/hadoop/pull/3588
> issue status is not matched with "Patch Available".
> [2021-10-27T17:52:59.814Z]
> {code}
> This causes the pipeline run to fail. I’ve seen this in my multiple attempts
> to re-run the CI on my PR –
> # After 45 minutes – [jenkins / hadoop-multibranch / PR-3588 / #1
> (apache.org)|https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-3588/1/pipeline/]
> # After 1 minute – [jenkins / hadoop-multibranch / PR-3588 / #2
> (apache.org)|https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-3588/2/pipeline/]
> # After 17 minutes – [jenkins / hadoop-multibranch / PR-3588 / #3
> (apache.org)|https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-3588/3/pipeline/]
> The hadoop-multibranch pipeline doesn't use ASF JIRA, thus, we're disabling
> the *jira* plugin to fix this issue.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18251) Fix failure of extracting JIRA id from commit message in git_jira_fix_version_check.py
Masatake Iwasaki created HADOOP-18251:
-
Summary: Fix failure of extracting JIRA id from commit message in
git_jira_fix_version_check.py
Key: HADOOP-18251
URL: https://issues.apache.org/jira/browse/HADOOP-18251
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
git_jira_fix_version_check.py is confused by commit message like {{"YARN-1151.
Ability to configure auxiliary services from HDFS-based JAR files."}} which
contains both {{YARN-}} and {{{}HDFS-{}}}. The latter {{HDFS-}} is unexpectedly
picked as JIRA issue id then 404 is thrown on accessing invalid URL like
"https://issues.apache.org/jira/rest/api/2/issue/HDFS-;.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18172) Change scope of getRootFallbackLink for InodeTree to make them accessible from outside package
[ https://issues.apache.org/jira/browse/HADOOP-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18172. --- Resolution: Fixed > Change scope of getRootFallbackLink for InodeTree to make them accessible > from outside package > -- > > Key: HADOOP-18172 > URL: https://issues.apache.org/jira/browse/HADOOP-18172 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Xing Lin >Assignee: Xing Lin >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.3.4 > > Time Spent: 5h 50m > Remaining Estimate: 0h > > Sometimes, we need to access rootFallBackLink in InodeTree from another > package. One such case is we extend from ViewFileSystem but want to put the > new filesystem in org.apache.hadoop.fs package, instead of > org.apache.hadoop.fs.viewfs package. As a result, we need make them public, > similar as what we did for getMountPoints() in HADOOP-18100. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18088) Replace log4j 1.x with reload4j
[ https://issues.apache.org/jira/browse/HADOOP-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18088. --- Hadoop Flags: Reviewed Release Note: log4j 1 was replaced with reload4j which is fork of log4j 1.2.17 with the goal of fixing pressing security issues. Resolution: Fixed > Replace log4j 1.x with reload4j > --- > > Key: HADOOP-18088 > URL: https://issues.apache.org/jira/browse/HADOOP-18088 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Fix For: 3.2.4, 3.3.3 > > Time Spent: 6h > Remaining Estimate: 0h > > As proposed in the dev mailing list > (https://lists.apache.org/thread/fdzkv80mzkf3w74z9120l0k0rc3v7kqk) let's > replace log4j 1 with reload4j in the maintenance releases (i.e. 3.3.x, 3.2.x > and 2.10.x) -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18192) Fix multiple_bindings warning about slf4j-relad4j
Masatake Iwasaki created HADOOP-18192:
-
Summary: Fix multiple_bindings warning about slf4j-relad4j
Key: HADOOP-18192
URL: https://issues.apache.org/jira/browse/HADOOP-18192
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Jar of slf4j-relad4j exists in libdirs of both common and hdfs.
{noformat}
$ ~/dist/hadoop-3.2.4-SNAPSHOT/bin/hdfs dfs -ls /
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/ext/dist/hadoop-3.2.4-SNAPSHOT/share/hadoop/common/lib/slf4j-reload4j-1.7.35.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/ext/dist/hadoop-3.2.4-SNAPSHOT/share/hadoop/hdfs/lib/slf4j-reload4j-1.7.35.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
$ find ~/dist/hadoop-3.2.4-SNAPSHOT/ -name 'slf4j-reload4j-*.jar'
/home/centos/dist/hadoop-3.2.4-SNAPSHOT/share/hadoop/common/lib/slf4j-reload4j-1.7.35.jar
/home/centos/dist/hadoop-3.2.4-SNAPSHOT/share/hadoop/hdfs/lib/slf4j-reload4j-1.7.35.jar
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18158) Fix failure of create-release script due to releasedocmaker changes in branch-2.10
[ https://issues.apache.org/jira/browse/HADOOP-18158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18158. --- Fix Version/s: 2.10.2 Hadoop Flags: Reviewed Resolution: Fixed > Fix failure of create-release script due to releasedocmaker changes in > branch-2.10 > -- > > Key: HADOOP-18158 > URL: https://issues.apache.org/jira/browse/HADOOP-18158 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 2.10.1 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 2.10.2 > > Time Spent: 1h > Remaining Estimate: 0h > > The file name generated by releasedocmaker of Yetus was changed from > CHANGES.md to CHANGELOG.md. dev-support scripts should be updated along with > the change. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18158) Fix failure of create-release script due to releasedocmaker changes in branch-2.10
Masatake Iwasaki created HADOOP-18158: - Summary: Fix failure of create-release script due to releasedocmaker changes in branch-2.10 Key: HADOOP-18158 URL: https://issues.apache.org/jira/browse/HADOOP-18158 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 2.10.1 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The file name generated by releasedocmaker of Yetus was changed from CHANGES.md to CHANGELOG.md. dev-support scripts should be updated along with the change. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17982) OpensslCipher initialization error should log a WARN message
[
https://issues.apache.org/jira/browse/HADOOP-17982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17982.
---
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> OpensslCipher initialization error should log a WARN message
>
>
> Key: HADOOP-17982
> URL: https://issues.apache.org/jira/browse/HADOOP-17982
> Project: Hadoop Common
> Issue Type: Improvement
> Components: kms, security
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> We spent months troubleshooting a RangerKMS performance problem, only to
> realize that the openssl library wasn't even loaded properly.
> The failure to load openssl lib is currently logged as a debug message during
> initialization. We really should upgrade it to at least INFO/WARN.
> {code}
> static {
> String loadingFailure = null;
> try {
> if (!NativeCodeLoader.buildSupportsOpenssl()) {
> PerformanceAdvisory.LOG.debug("Build does not support openssl");
> loadingFailure = "build does not support openssl.";
> } else {
> initIDs();
> }
> } catch (Throwable t) {
> loadingFailure = t.getMessage();
> LOG.debug("Failed to load OpenSSL Cipher.", t);
> } finally {
> loadingFailureReason = loadingFailure;
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18024) SocketChannel is not closed when IOException happens in Server$Listener.doAccept
[
https://issues.apache.org/jira/browse/HADOOP-18024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18024.
---
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> SocketChannel is not closed when IOException happens in
> Server$Listener.doAccept
>
>
> Key: HADOOP-18024
> URL: https://issues.apache.org/jira/browse/HADOOP-18024
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
>Affects Versions: 3.2.2
>Reporter: Haoze Wu
>Assignee: Haoze Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> This is a follow-up of HADOOP-17552.
> When the symptom described in HADOOP-17552 happens, the client may time out
> in 2min, according to the default RPC timeout configuration specified in
> HADOOP-17552. Before this timeout, the client just waits, and does not know
> this issue happens.
> However, we recently found that actually the client doesn’t need to waste
> this 2min, and the server’s availability can be also improved. If the
> IOException happens in line 1402 or 1403 or 1404, we can just close this
> problematic `SocketChannel` and continue to accept new socket connections.
> The client side can also be aware of the close socket immediately, instead of
> waiting 2min.
> The old implementation:
> {code:java}
> //hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
> public void run() {
> while (running) {
> // ...
> try {
> // ...
> while (iter.hasNext()) {
> // ...
> try {
> if (key.isValid()) {
> if (key.isAcceptable())
> doAccept(key); // line 1348
> }
> } catch (IOException e) { // line 1350
> }
> // ...
> }
> } catch (OutOfMemoryError e) {
> // ...
> } catch (Exception e) {
> // ...
> }
> }
> } {code}
> {code:java}
> //hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
> void doAccept(SelectionKey key) throws InterruptedException, IOException,
> OutOfMemoryError {
> ServerSocketChannel server = (ServerSocketChannel) key.channel();
> SocketChannel channel;
> while ((channel = server.accept()) != null) { // line 1400
> channel.configureBlocking(false); // line 1402
> channel.socket().setTcpNoDelay(tcpNoDelay); // line 1403
> channel.socket().setKeepAlive(true); // line 1404
> Reader reader = getReader();
> Connection c = connectionManager.register(channel,
> this.listenPort, this.isOnAuxiliaryPort);
> // If the connectionManager can't take it, close the connection.
> if (c == null) {
> if (channel.isOpen()) {
> IOUtils.cleanup(null, channel);
> }
> connectionManager.droppedConnections.getAndIncrement();
> continue;
> }
> key.attach(c); // so closeCurrentConnection can get the object
> reader.addConnection(c);
> }
> } {code}
>
> We propose that the following implementation is better:
> {code:java}
> void doAccept(SelectionKey key) throws InterruptedException, IOException,
> OutOfMemoryError {
> ServerSocketChannel server = (ServerSocketChannel) key.channel();
> SocketChannel channel;
> while ((channel = server.accept()) != null) { // line 1400
> try {
> channel.configureBlocking(false); // line 1402
> channel.socket().setTcpNoDelay(tcpNoDelay); // line 1403
> channel.socket().setKeepAlive(true); // line 1404
> } catch (IOException e) {
> LOG.warn(...);
> try {
> channel.socket().close();
> channel.close();
> } catch (IOException ignored) { }
> continue;
> }
> // ...
> }
> }{code}
> The advantages include:
> # {*}In the old implementation{*}, the `ServerSocketChannel` was abandoned
> due to the single exception in this single `SocketChannel`, because the
> exception handler is in line 1350. {*}In the new implementation{*}, we use a
> try-catch to handle the exception in line 1402 or 1403 or 1404, then the
> `ServerSocketChannel` can continue to accept new connections, and don’t need
> to go back to the line 1348 in the next while loop in the run method.
> # {*}In the old
[jira] [Resolved] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10
[
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17965.
---
Fix Version/s: 2.10.2
Hadoop Flags: Reviewed
Resolution: Fixed
> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.10.2
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not
> support Java 7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17964) Increase Java heap size for running Maven in Dockerfile of branch-2.10
[
https://issues.apache.org/jira/browse/HADOOP-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17964.
---
Fix Version/s: 2.10.2
Hadoop Flags: Reviewed
Resolution: Fixed
> Increase Java heap size for running Maven in Dockerfile of branch-2.10
> --
>
> Key: HADOOP-17964
> URL: https://issues.apache.org/jira/browse/HADOOP-17964
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.10.2
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> I got OOM on running create-release script in branch-2.10.
> {noformat}
> $ ./dev-support/bin/create-release --docker --dockercache --native
> $ tail -n 5 patchprocess/mvn_install.log
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 07:06 min
> Exception in thread "main"
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "main"
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17965) Fix documentation build failure using create-release on branch-2.10
Masatake Iwasaki created HADOOP-17965:
-
Summary: Fix documentation build failure using create-release on
branch-2.10
Key: HADOOP-17965
URL: https://issues.apache.org/jira/browse/HADOOP-17965
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-site-plugin:3.5:site (default-site) on project
hadoop-yarn-project: Execution default-site of goal
org.apache.maven.plugins:maven-site-plugin:3.5:site failed: An API
incompatibility was encountered while executing
org.apache.maven.plugins:maven-site-plugin:3.5:site:
java.lang.UnsupportedClassVersionError:
org/codehaus/mojo/spotbugs/SpotbugsViolationCheckMojo : Unsupported major.minor
version 52.0
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17964) Increase Java heap size for running Maven in Dockerfile of branch-2.10
Masatake Iwasaki created HADOOP-17964:
-
Summary: Increase Java heap size for running Maven in Dockerfile
of branch-2.10
Key: HADOOP-17964
URL: https://issues.apache.org/jira/browse/HADOOP-17964
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
I got OOM on running create-release script in branch-2.10.
{noformat}
$ ./dev-support/bin/create-release --docker --dockercache --native
$ tail -n 5 patchprocess/mvn_install.log
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 07:06 min
Exception in thread "main"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler
in thread "main"
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17916) Fix compilation error of ITUseHadoopCodecs with -DskipShade
Masatake Iwasaki created HADOOP-17916:
-
Summary: Fix compilation error of ITUseHadoopCodecs with
-DskipShade
Key: HADOOP-17916
URL: https://issues.apache.org/jira/browse/HADOOP-17916
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
(default-testCompile) on project hadoop-client-integration-tests: Compilation
failure: Compilation failure:
[ERROR]
/home/centos/srcs/hadoop/hadoop-client-modules/hadoop-client-integration-tests/src/test/java/org/apache/hadoop/example/ITUseHadoopCodecs.java:[34,28]
cannot find symbol
[ERROR] symbol: class RandomDatum
[ERROR] location: package org.apache.hadoop.io
[ERROR]
/home/centos/srcs/hadoop/hadoop-client-modules/hadoop-client-integration-tests/src/test/java/org/apache/hadoop/example/ITUseHadoopCodecs.java:[100,16]
package RandomDatum does not exist
[ERROR]
/home/centos/srcs/hadoop/hadoop-client-modules/hadoop-client-integration-tests/src/test/java/org/apache/hadoop/example/ITUseHadoopCodecs.java:[100,54]
package RandomDatum does not exist
[ERROR]
/home/centos/srcs/hadoop/hadoop-client-modules/hadoop-client-integration-tests/src/test/java/org/apache/hadoop/example/ITUseHadoopCodecs.java:[103,7]
cannot find symbol
[ERROR] symbol: class RandomDatum
[ERROR] location: class org.apache.hadoop.example.ITUseHadoopCodecs
[ERROR]
/home/centos/srcs/hadoop/hadoop-client-modules/hadoop-client-integration-tests/src/test/java/org/apache/hadoop/example/ITUseHadoopCodecs.java:[104,7]
cannot find symbol
[ERROR] symbol: class RandomDatum
[ERROR] location: class org.apache.hadoop.example.ITUseHadoopCodecs
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17908) Add missing RELEASENOTES and CHANGES to upstream
Masatake Iwasaki created HADOOP-17908: - Summary: Add missing RELEASENOTES and CHANGES to upstream Key: HADOOP-17908 URL: https://issues.apache.org/jira/browse/HADOOP-17908 Project: Hadoop Common Issue Type: Bug Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki RELEASENOTES and CHANGES of 2.10.1, 3.1.4 and 3.3.0 are missing in trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17870) HTTP Filesystem to qualify paths in open()/getFileStatus()
[
https://issues.apache.org/jira/browse/HADOOP-17870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17870.
---
Fix Version/s: 3.3.2
3.4.0
Resolution: Fixed
> HTTP Filesystem to qualify paths in open()/getFileStatus()
> --
>
> Key: HADOOP-17870
> URL: https://issues.apache.org/jira/browse/HADOOP-17870
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.3.1
>Reporter: VinothKumar Raman
>Assignee: VinothKumar Raman
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> HTTP filesystem is inconsistent with other filesystem (eg: S3AFilesystem) in
> that they don't accept Path which is relative to the URL that the filesystem
> is instantiated with.
>
> As in
> {code:java}
> FileSystem.get("http://www.example.com;).open(new Path("/test")){code}
> will fail with URI not absolute. But it should infact fetch
> {{http://www.example.com/test}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17899) Fix compilation error of TestHttpFileSystem
Masatake Iwasaki created HADOOP-17899:
-
Summary: Fix compilation error of TestHttpFileSystem
Key: HADOOP-17899
URL: https://issues.apache.org/jira/browse/HADOOP-17899
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.3.2
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
[ERROR]
/home/centos/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/http/TestHttpFileSystem.java:[29,29]
package org.junit.jupiter.api does not exist
[ERROR]
/home/centos/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/http/TestHttpFileSystem.java:[47,4]
cannot find symbol
symbol: class BeforeEach
location: class org.apache.hadoop.fs.http.TestHttpFileSystem
[INFO] 2 errors
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17897) Allow nested blocks in switch case in checkstyle settings
Masatake Iwasaki created HADOOP-17897: - Summary: Allow nested blocks in switch case in checkstyle settings Key: HADOOP-17897 URL: https://issues.apache.org/jira/browse/HADOOP-17897 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Nested blocks in switch case are used in existing code and checkstyle warning are just ignored. It should be allowed in project checkstyle settings. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17849) Exclude spotbugs-annotations from transitive dependencies on branch-3.2
[ https://issues.apache.org/jira/browse/HADOOP-17849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-17849. --- Fix Version/s: 3.2.3 Hadoop Flags: Reviewed Resolution: Fixed > Exclude spotbugs-annotations from transitive dependencies on branch-3.2 > --- > > Key: HADOOP-17849 > URL: https://issues.apache.org/jira/browse/HADOOP-17849 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.2.2 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 3.2.3 > > Time Spent: 40m > Remaining Estimate: 0h > > Building Hadoop in dist profile with ZooKeeper 3.4.14 fails on > hadoop-client-check-test-invariants. Excluding > com.github.spotbugs:spotbugs-annotation from transitive dependencies should > fix this for users needing zookeeer-3.4.14. Since the dependency is > provided/optional on ZooKeeper 3.5.x, branch-3.3 and above are not affected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17849) Exclude spotbugs-annotations from transitive dependencies on branch-3.2
Masatake Iwasaki created HADOOP-17849: - Summary: Exclude spotbugs-annotations from transitive dependencies on branch-3.2 Key: HADOOP-17849 URL: https://issues.apache.org/jira/browse/HADOOP-17849 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.2.2 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Building Hadoop in dist profile with ZooKeeper 3.4.14 fails on hadoop-client-check-test-invariants. Excluding com.github.spotbugs:spotbugs-annotation from transitive dependencies should fix this for users needing zookeeer-3.4.14. Since the dependency is provided/optional on ZooKeeper 3.5.x, branch-3.3 and above are not affected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17782) mvn install -DskipTests failed about hadoop@3.3.1
[
https://issues.apache.org/jira/browse/HADOOP-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17782.
---
Resolution: Cannot Reproduce
> mvn install -DskipTests failed about hadoop@3.3.1
> -
>
> Key: HADOOP-17782
> URL: https://issues.apache.org/jira/browse/HADOOP-17782
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.1
>Reporter: shixijun
>Priority: Major
>
> {panel:title=mvn install -DskipTests failed about hadoop@3.3.1}
> mvn install -DskipTests failed
> {panel}
> [root@centos8 spack-src]# mvn -version
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home:
> /home/spack/spack/opt/spack/linux-centos8-aarch64/gcc-8.4.1/maven-3.6.3-bnannw7m3zq6axy7nmovnlhjawdy6pzt
> Java version: 1.8.0_191, vendor: AdoptOpenJdk, runtime:
> /home/spack/spack/opt/spack/linux-centos8-aarch64/gcc-8.4.1/openjdk-1.8.0_191-b12-fidptihybskgklbjoo4lagkacm6n6lod/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "4.18.0-240.22.1.el8_3.aarch64", arch: "aarch64",
> family: "unix"
> [root@centos8 spack-src]# java -version
> openjdk version "1.8.0_191"
> OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_191-b12)
> OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.191-b12, mixed mode)
> [root@centos8 spack-src]#mvn install -DskipTests
> ……
> [INFO]
> [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @
> hadoop-yarn-applications-catalog-webapp ---
> [INFO] Executing tasks
> main:
> [mkdir] Created dir:
> /root/6/hadoop-trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/test-dir
> [INFO] Executed tasks
> [INFO]
> [INFO] --- maven-dependency-plugin:3.0.2:copy (default) @
> hadoop-yarn-applications-catalog-webapp ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (banned-illegal-imports) @
> hadoop-yarn-applications-catalog-webapp ---
> [INFO]
> [INFO] --- replacer:1.5.3:replace (replace-generated-sources) @
> hadoop-yarn-applications-catalog-webapp ---
> [INFO] Skipping
> [INFO]
> [INFO] --- replacer:1.5.3:replace (replace-sources) @
> hadoop-yarn-applications-catalog-webapp ---
> [INFO] Skipping
> [INFO]
> [INFO] --- replacer:1.5.3:replace (replace-guava) @
> hadoop-yarn-applications-catalog-webapp ---
> [INFO] Replacement run on 22 files.
> [INFO]
> [INFO] --- frontend-maven-plugin:1.11.2:install-node-and-yarn (install node
> and yarn) @ hadoop-yarn-applications-catalog-webapp ---
> [INFO] Installing node version v8.11.3
> [INFO] Downloading
> https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-arm64.tar.gz to
> /root/.m2/repository/com/github/eirslett/node/8.11.3/node-8.11.3-linux-arm64.tar.gz
> [INFO] No proxies configured
> [INFO] No proxy was configured, downloading directly
> Jun 30, 2021 4:56:30 PM org.apache.http.impl.execchain.RetryExec execute
> INFO: I/O exception (java.net.SocketException) caught when processing request
> to {s}->https://nodejs.org:443: Network is unreachable (connect failed)
> Jun 30, 2021 4:56:30 PM org.apache.http.impl.execchain.RetryExec execute
> INFO: Retrying request to {s}->https://nodejs.org:443
> Jun 30, 2021 5:00:56 PM org.apache.http.impl.execchain.RetryExec execute
> INFO: I/O exception (java.net.SocketException) caught when processing request
> to {s}->https://nodejs.org:443: Network is unreachable (connect failed)
> Jun 30, 2021 5:00:56 PM org.apache.http.impl.execchain.RetryExec execute
> INFO: Retrying request to {s}->https://nodejs.org:443
> Jun 30, 2021 5:05:22 PM org.apache.http.impl.execchain.RetryExec execute
> INFO: I/O exception (java.net.SocketException) caught when processing request
> to {s}->https://nodejs.org:443: Network is unreachable (connect failed)
> Jun 30, 2021 5:05:22 PM org.apache.http.impl.execchain.RetryExec execute
> INFO: Retrying request to {s}->https://nodejs.org:443
> [INFO]
>
> [INFO] Reactor Summary for Apache Hadoop Main 3.4.0-SNAPSHOT:
> [INFO]
> [INFO] Apache Hadoop Main . SUCCESS [ 1.394
> s]
> [INFO] Apache Hadoop Build Tools .. SUCCESS [ 1.139
> s]
> [INFO] Apache Hadoop Project POM .. SUCCESS [ 1.138
> s]
> [INFO] Apache Hadoop Annotations .. SUCCESS [ 1.608
> s]
> [INFO] Apache Hadoop Project Dist POM . SUCCESS [ 0.117
> s]
> [INFO] Apache Hadoop Assemblies ... SUCCESS [ 0.144
> s]
> [INFO] Apache Hadoop Maven Plugins SUCCESS [ 3.554
> s]
> [INFO] Apache Hadoop MiniKDC .. SUCCESS [ 0.875
> s]
> [INFO] Apache Hadoop Auth
[jira] [Reopened] (HADOOP-17719) DistCp with snapshot diff should support file systems other than DistributedFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-17719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki reopened HADOOP-17719:
---
> DistCp with snapshot diff should support file systems other than
> DistributedFileSystem
> --
>
> Key: HADOOP-17719
> URL: https://issues.apache.org/jira/browse/HADOOP-17719
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {{distcp -diff}} currently supports only DistributedFileSystem and
> WebHdfsFileSystem. Other file system could be used if getSnapshotDiffReport
> is supported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17719) DistCp with snapshot diff should support file systems other than DistributedFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-17719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17719.
---
Resolution: Duplicate
> DistCp with snapshot diff should support file systems other than
> DistributedFileSystem
> --
>
> Key: HADOOP-17719
> URL: https://issues.apache.org/jira/browse/HADOOP-17719
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{distcp -diff}} currently supports only DistributedFileSystem. Other file
> system could be used if getSnapshotDiffReport is supported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17775) Remove JavaScript package from Docker environment
Masatake Iwasaki created HADOOP-17775: - Summary: Remove JavaScript package from Docker environment Key: HADOOP-17775 URL: https://issues.apache.org/jira/browse/HADOOP-17775 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki As described in the [README of yarn-ui|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/README.md], required javascript modules are automatically pulled by frontend-maven-plugin. We can leverage them for local testing too. While hadoop-yarn-ui and hadoop-yarn-applications-catalog-webapp is using node.js, the version of node.js does not match. JavaScript related packages of the docker environment is not sure to work. * https://github.com/apache/hadoop/blob/538ce9c35403f0c8b595f42e835cc70c91c66621/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/pom.xml#L170-L212 * https://github.com/apache/hadoop/blob/538ce9c35403f0c8b595f42e835cc70c91c66621/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/pom.xml#L264-L290 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-11758) Add options to filter out too much granular tracing spans
[ https://issues.apache.org/jira/browse/HADOOP-11758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-11758. --- Resolution: Won't Fix > Add options to filter out too much granular tracing spans > - > > Key: HADOOP-11758 > URL: https://issues.apache.org/jira/browse/HADOOP-11758 > Project: Hadoop Common > Issue Type: Improvement > Components: tracing >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Attachments: testWriteTraceHooks-HDFS-8026.html, > testWriteTraceHooks.html > > > in order to avoid queue in span receiver spills -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17719) DistCp with snapshot diff should support file systems other than DistributedFileSystem
Masatake Iwasaki created HADOOP-17719:
-
Summary: DistCp with snapshot diff should support file systems
other than DistributedFileSystem
Key: HADOOP-17719
URL: https://issues.apache.org/jira/browse/HADOOP-17719
Project: Hadoop Common
Issue Type: Improvement
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{{distcp -diff}} currently supports only DistributedFileSystem. Other file
system could be used if getSnapshotDiffReport is supported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17609) Make SM4 support optional for OpenSSL native code
Masatake Iwasaki created HADOOP-17609: - Summary: Make SM4 support optional for OpenSSL native code Key: HADOOP-17609 URL: https://issues.apache.org/jira/browse/HADOOP-17609 Project: Hadoop Common Issue Type: Improvement Components: native Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki openssl-devel-1.1.1g provided by CentOS 8 does not work after HDFS-15098 because the SM4 is not enabled on the openssl package. We should not force users to install OpenSSL from source code even if they do not use SM4 feature. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17573) Fix compilation error of OBSFileSystem in trunk
Masatake Iwasaki created HADOOP-17573:
-
Summary: Fix compilation error of OBSFileSystem in trunk
Key: HADOOP-17573
URL: https://issues.apache.org/jira/browse/HADOOP-17573
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-huaweicloud: Compilation failure
[ERROR]
/home/centos/srcs/hadoop/hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSFileSystem.java:[396,58]
incompatible types: org.apache.hadoop.util.BlockingThreadPoolExecutorService
cannot be converted to
org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ListeningExecutorService
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17552) Change ipc.client.rpc-timeout.ms from 0 to 120000 by default to avoid potential hang
[
https://issues.apache.org/jira/browse/HADOOP-17552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17552.
---
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Change ipc.client.rpc-timeout.ms from 0 to 12 by default to avoid
> potential hang
>
>
> Key: HADOOP-17552
> URL: https://issues.apache.org/jira/browse/HADOOP-17552
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
>Affects Versions: 3.2.2
>Reporter: Haoze Wu
>Assignee: Haoze Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
> We are doing some systematic fault injection testing in Hadoop-3.2.2 and
> when we try to run a client (e.g., `bin/hdfs dfs -ls /`) to our HDFS cluster
> (1 NameNode, 2 DataNodes), the client gets stuck forever. After some
> investigation, we believe that it’s a bug in `hadoop.ipc.Client` because the
> read method of `hadoop.ipc.Client$Connection$PingInputStream` keeps
> swallowing `java.net.SocketTimeoutException` due to the mistaken usage of the
> `rpcTimeout` configuration in the `handleTimeout` method.
> *Reproduction*
> Start HDFS with the default configuration. Then execute a client (we used
> the command `bin/hdfs dfs -ls /` in the terminal). While HDFS is trying to
> accept the client’s socket, inject a socket error (java.net.SocketException
> or java.io.IOException), specifically at line 1402 (line 1403 or 1404 will
> also work).
> We prepare the scripts for reproduction in a gist
> ([https://gist.github.com/functioner/08bcd86491b8ff32860eafda8c140e24]).
> *Diagnosis*
> When the NameNode tries to accept a client’s socket, basically there are
> 4 steps:
> # accept the socket (line 1400)
> # configure the socket (line 1402-1404)
> # make the socket a Reader (after line 1404)
> # swallow the possible IOException in line 1350
> {code:java}
> //hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
> public void run() {
> while (running) {
> SelectionKey key = null;
> try {
> getSelector().select();
> Iterator iter =
> getSelector().selectedKeys().iterator();
> while (iter.hasNext()) {
> key = iter.next();
> iter.remove();
> try {
> if (key.isValid()) {
> if (key.isAcceptable())
> doAccept(key);
> }
> } catch (IOException e) { // line 1350
> }
> key = null;
> }
> } catch (OutOfMemoryError e) {
> // ...
> } catch (Exception e) {
> // ...
> }
> }
> }
> void doAccept(SelectionKey key) throws InterruptedException, IOException,
> OutOfMemoryError {
> ServerSocketChannel server = (ServerSocketChannel) key.channel();
> SocketChannel channel;
> while ((channel = server.accept()) != null) { // line 1400
> channel.configureBlocking(false); // line 1402
> channel.socket().setTcpNoDelay(tcpNoDelay); // line 1403
> channel.socket().setKeepAlive(true); // line 1404
>
> Reader reader = getReader();
> Connection c = connectionManager.register(channel,
> this.listenPort, this.isOnAuxiliaryPort);
> // If the connectionManager can't take it, close the connection.
> if (c == null) {
> if (channel.isOpen()) {
> IOUtils.cleanup(null, channel);
> }
> connectionManager.droppedConnections.getAndIncrement();
> continue;
> }
> key.attach(c); // so closeCurrentConnection can get the object
> reader.addConnection(c);
> }
> }
> {code}
> When a SocketException occurs in line 1402 (or 1403 or 1404), the
> server.accept() in line 1400 has finished, so we expect the following
> behavior:
> # The server (NameNode) accepts this connection but it will basically write
> nothing to this connection because it’s not added as a Reader data structure.
> # The client is aware that the connection has been established, and tries to
> read and write in this connection. After some time threshold, the client
> finds that it can’t read anything from this connection and exits with some
> exception or error.
> However, we do not observe behavior 2. The client just gets stuck forever
> (>10min). We re-examine the default configuration in
> [https://hadoop.apache.org/docs/r3.2.2/hadoop-project-dist/hadoop-common/core-default.xml]
> and we
[jira] [Created] (HADOOP-17538) Add kms-default.xml and httpfs-default.xml to site index
Masatake Iwasaki created HADOOP-17538: - Summary: Add kms-default.xml and httpfs-default.xml to site index Key: HADOOP-17538 URL: https://issues.apache.org/jira/browse/HADOOP-17538 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki While there are links to *-default.xml in the site index, kms-default.xml and httpfs-default.xml are not included. Adding them could be useful as a quick reference. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17515) Remove outdated contents from tracing documentation
Masatake Iwasaki created HADOOP-17515: - Summary: Remove outdated contents from tracing documentation Key: HADOOP-17515 URL: https://issues.apache.org/jira/browse/HADOOP-17515 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Tracer using HTrace and TraceAdmin were removed by HADOOP-17424. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17514) Remove trace subcommand from hadoop CLI
Masatake Iwasaki created HADOOP-17514: - Summary: Remove trace subcommand from hadoop CLI Key: HADOOP-17514 URL: https://issues.apache.org/jira/browse/HADOOP-17514 Project: Hadoop Common Issue Type: Improvement Components: scripts Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki TraceAdmin protocol and utility class were removed by HADOOP-17424. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17508) Simplify dependency installation instructions
[ https://issues.apache.org/jira/browse/HADOOP-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-17508. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Simplify dependency installation instructions > - > > Key: HADOOP-17508 > URL: https://issues.apache.org/jira/browse/HADOOP-17508 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.3.0 >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The instructions for installing Protocol buffers and Boost seem to just be > copied from Dockerfile and isn't friendly to read. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17507) Add build instructions for installing GCC 9 and CMake 3.19
[ https://issues.apache.org/jira/browse/HADOOP-17507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-17507. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Add build instructions for installing GCC 9 and CMake 3.19 > -- > > Key: HADOOP-17507 > URL: https://issues.apache.org/jira/browse/HADOOP-17507 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.0 >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17506) Fix typo in BUILDING.txt
[ https://issues.apache.org/jira/browse/HADOOP-17506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-17506. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix typo in BUILDING.txt > > > Key: HADOOP-17506 > URL: https://issues.apache.org/jira/browse/HADOOP-17506 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.0 >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17425) Bump up snappy-java to 1.1.8.2
[ https://issues.apache.org/jira/browse/HADOOP-17425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-17425. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Bump up snappy-java to 1.1.8.2 > -- > > Key: HADOOP-17425 > URL: https://issues.apache.org/jira/browse/HADOOP-17425 > Project: Hadoop Common > Issue Type: Improvement >Reporter: L. C. Hsieh >Assignee: L. C. Hsieh >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 50m > Remaining Estimate: 0h > > 1.1.8.2 includes: > * Support Apple Silicon (M1, Mac-aarch64) > * Fixed the pure-java Snappy fallback logic when no native library for your > platform is found. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17369) Bump up snappy-java to 1.1.8.1
Masatake Iwasaki created HADOOP-17369: - Summary: Bump up snappy-java to 1.1.8.1 Key: HADOOP-17369 URL: https://issues.apache.org/jira/browse/HADOOP-17369 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The native libraries provided by snappy-java does not work on some distros on aarch64 and ppc64le. Upgrading snappy-java should fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17270) Fix testCompressorDecompressorWithExeedBufferLimit to cover the intended scenario
Masatake Iwasaki created HADOOP-17270: - Summary: Fix testCompressorDecompressorWithExeedBufferLimit to cover the intended scenario Key: HADOOP-17270 URL: https://issues.apache.org/jira/browse/HADOOP-17270 Project: Hadoop Common Issue Type: Improvement Components: test Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The input data must be greater than internal buffer of Compressor/Decompressor as the test name implies. It must use strategy covering compression/decompression of multiple blocks. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16918) Dependency update for Hadoop 2.10
[ https://issues.apache.org/jira/browse/HADOOP-16918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-16918. --- Fix Version/s: 2.10.1 Resolution: Fixed Thanks. I'm closing this. > Dependency update for Hadoop 2.10 > - > > Key: HADOOP-16918 > URL: https://issues.apache.org/jira/browse/HADOOP-16918 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Priority: Major > Labels: release-blocker > Fix For: 2.10.1 > > Attachments: dependency-check-report.html, > dependency-check-report.html > > > A number of dependencies can be updated. > nimbus-jose-jwt > jetty > netty > zookeeper > hbase-common > jackson-databind > and many more. They should be updated in the 2.10.1 release. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17254) Upgrade hbase to 1.2.6.1 on branch-2
Masatake Iwasaki created HADOOP-17254: - Summary: Upgrade hbase to 1.2.6.1 on branch-2 Key: HADOOP-17254 URL: https://issues.apache.org/jira/browse/HADOOP-17254 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17253) Upgrade zookeeper to 3.4.14 on branch-2.10
Masatake Iwasaki created HADOOP-17253: - Summary: Upgrade zookeeper to 3.4.14 on branch-2.10 Key: HADOOP-17253 URL: https://issues.apache.org/jira/browse/HADOOP-17253 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Since versions of zookeeper and curator have different history between branch-2.10 and trunk, I filed this to upgrade both zookeeper and curator on branch-2.10. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17251) Upgrade netty-all to 4.1.50.Final on branch-2.10
Masatake Iwasaki created HADOOP-17251: - Summary: Upgrade netty-all to 4.1.50.Final on branch-2.10 Key: HADOOP-17251 URL: https://issues.apache.org/jira/browse/HADOOP-17251 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17249) Upgrade jackson-databind to 2.10 on branch-2.10
Masatake Iwasaki created HADOOP-17249: - Summary: Upgrade jackson-databind to 2.10 on branch-2.10 Key: HADOOP-17249 URL: https://issues.apache.org/jira/browse/HADOOP-17249 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.10.0 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki This is filed to test backporting HADOOP-16905 to branch-2.10. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17240) Fix wrong command line for setting up CentOS 8
Masatake Iwasaki created HADOOP-17240: - Summary: Fix wrong command line for setting up CentOS 8 Key: HADOOP-17240 URL: https://issues.apache.org/jira/browse/HADOOP-17240 Project: Hadoop Common Issue Type: Bug Reporter: Masatake Iwasaki Assignee: Takeru Kuramoto Instruction for setting up CentOS 8 in BUILDING.txt has invalid command line. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17095) findbugs warnings building branch-2.10
[
https://issues.apache.org/jira/browse/HADOOP-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-17095.
---
Resolution: Fixed
I'm closing this since all child issues are fixed.
> findbugs warnings building branch-2.10
> --
>
> Key: HADOOP-17095
> URL: https://issues.apache.org/jira/browse/HADOOP-17095
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.10.0, 2.10.1
>Reporter: Ahmed Hussein
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: build, findbugs, refactor
>
> precommit build for branch-2.10 generates finbugs warnings in several
> components.
> This is an umbrella to analyze those warnings and fix/ignore them as
> necessary.
>
> |{color:#FF}-1{color}|{color:#FF}findbugs{color}|{color:#FF}2m
> 1s{color}|{color:#FF}hadoop-common-project/hadoop-common in branch-2.10
> has 14 extant findbugs warnings.{color}|
> |{color:#FF}-1{color}|{color:#FF}findbugs{color}|{color:#FF}2m
> 54s{color}|{color:#FF}hadoop-hdfs-project/hadoop-hdfs in branch-2.10 has
> 10 extant findbugs warnings.{color}|
> |{color:#FF}-1{color}|{color:#FF}findbugs{color}|{color:#FF}2m
> 12s{color}|{color:#FF}hadoop-hdfs-project/hadoop-hdfs-client in
> branch-2.10 has 1 extant findbugs warnings.{color}|
> |{color:#FF}-1{color}|{color:#FF}findbugs{color}|{color:#FF}1m
> 35s{color}|{color:#FF}hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
> in branch-2.10 has 3 extant findbugs warnings.{color}|
> |{color:#FF}-1{color}|{color:#FF}findbugs{color}|{color:#FF}1m
> 50s{color}|{color:#FF}hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
> in branch-2.10 has 1 extant findbugs warnings.{color}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17206) Add python2 to required package on CentOS 8 for building documentation
Masatake Iwasaki created HADOOP-17206:
-
Summary: Add python2 to required package on CentOS 8 for building
documentation
Key: HADOOP-17206
URL: https://issues.apache.org/jira/browse/HADOOP-17206
Project: Hadoop Common
Issue Type: Improvement
Components: documentation
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{{mvn site}} fails without python2.
{noformat}
[INFO] --- exec-maven-plugin:1.3.1:exec (shelldocs) @ hadoop-common ---
/usr/bin/env: ‘python2’: No such file or directory
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17202) Fix findbugs warnings in hadoop-tools on branch-2.10
Masatake Iwasaki created HADOOP-17202: - Summary: Fix findbugs warnings in hadoop-tools on branch-2.10 Key: HADOOP-17202 URL: https://issues.apache.org/jira/browse/HADOOP-17202 Project: Hadoop Common Issue Type: Bug Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17171) Please fix CVEs by removing reference to htrace-core4
[ https://issues.apache.org/jira/browse/HADOOP-17171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-17171. --- Resolution: Duplicate Closing this as duplicate of HADOOP-15566. > Please fix CVEs by removing reference to htrace-core4 > - > > Key: HADOOP-17171 > URL: https://issues.apache.org/jira/browse/HADOOP-17171 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.3.0 >Reporter: Rodney Aaron Stainback >Priority: Major > > htrace-core4 is a retired project and even on the latest version they Shade > Jackson databind version 2.4.0 which has the following CVEs: > |cve|severity|cvss| > |CVE-2017-15095|critical|9.8| > |CVE-2018-1000873|medium|6.5| > |CVE-2018-14718|critical|9.8| > |CVE-2018-5968|high|8.1| > |CVE-2018-7489|critical|9.8| > |CVE-2019-14540|critical|9.8| > |CVE-2019-14893|critical|9.8| > |CVE-2019-16335|critical|9.8| > |CVE-2019-16942|critical|9.8| > |CVE-2019-16943|critical|9.8| > |CVE-2019-17267|critical|9.8| > |CVE-2019-17531|critical|9.8| > |CVE-2019-20330|critical|9.8| > |CVE-2020-10672|high|8.8| > |CVE-2020-10673|high|8.8| > |CVE-2020-10968|high|8.8| > |CVE-2020-10969|high|8.8| > |CVE-2020-1|high|8.8| > |CVE-2020-2|high|8.8| > |CVE-2020-3|high|8.8| > |CVE-2020-11619|critical|9.8| > |CVE-2020-11620|critical|9.8| > |CVE-2020-14060|high|8.1| > |CVE-2020-14061|high|8.1| > |CVE-2020-14062|high|8.1| > |CVE-2020-14195|high|8.1| > |CVE-2020-8840|critical|9.8| > |CVE-2020-9546|critical|9.8| > |CVE-2020-9547|critical|9.8| > |CVE-2020-9548|critical|9.8| > > Our security team is trying to block us from using hadoop because of this -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17153) Add boost installation steps to build instruction on CentOS 8
Masatake Iwasaki created HADOOP-17153: - Summary: Add boost installation steps to build instruction on CentOS 8 Key: HADOOP-17153 URL: https://issues.apache.org/jira/browse/HADOOP-17153 Project: Hadoop Common Issue Type: Bug Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki After HDFS-15385, -Pnative build fails without boost 1.72 used in libhdfs++. It must be installed from source since boost 1.66 packaged by the CentOS distribution does not match. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17138) Fix spotbugs warnings surfaced after upgrade to 4.0.6
Masatake Iwasaki created HADOOP-17138:
-
Summary: Fix spotbugs warnings surfaced after upgrade to 4.0.6
Key: HADOOP-17138
URL: https://issues.apache.org/jira/browse/HADOOP-17138
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Spotbugs 4.0.6 generated additional warnings.
{noformat}
$ mvn clean install findbugs:findbugs -DskipTests -DskipShade
$ find . -name findbugsXml.xml | xargs -n 1
/opt/spotbugs-4.0.6/bin/convertXmlToText
M D DLS: Dead store to $L5 in
org.apache.hadoop.ipc.Server$ConnectionManager.decrUserConnections(String) At
Server.java:[line 3729]
M D DLS: Dead store to $L5 in
org.apache.hadoop.ipc.Server$ConnectionManager.incrUserConnections(String) At
Server.java:[line 3717]
H D NP: Method
org.apache.hadoop.hdfs.server.datanode.checker.DatasetVolumeChecker$ResultHandler.onSuccess(Object)
overrides the nullness annotation of parameter $L1 in an incompatible way At
DatasetVolumeChecker.java:[line 322]
H D NP: Method
org.apache.hadoop.hdfs.server.datanode.checker.DatasetVolumeChecker$ResultHandler.onSuccess(VolumeCheckResult)
overrides the nullness annotation of parameter result in an incompatible way
At DatasetVolumeChecker.java:[lines 358-376]
M D NP: Method
org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker$2.onSuccess(Object)
overrides the nullness annotation of parameter result in an incompatible way
At ThrottledAsyncChecker.java:[lines 170-175]
M D DLS: Dead store to $L8 in
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.incrOpCount(FSEditLogOpCodes,
EnumMap, Step, StartupProgress$Counter) At FSEditLogLoader.java:[line 1241]
M D NP: result must be non-null but is marked as nullable At
LocatedFileStatusFetcher.java:[lines 380-397]
M D NP: result must be non-null but is marked as nullable At
LocatedFileStatusFetcher.java:[lines 291-309]
M D DLS: Dead store to $L6 in
org.apache.hadoop.yarn.sls.SLSRunner.increaseQueueAppNum(String) At
SLSRunner.java:[line 816]
H C UMAC: Uncallable method
org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage$1.getInstance()
defined in anonymous class At
TestTimelineReaderWebServicesHBaseStorage.java:[line 87]
M D DLS: Dead store to entities in
org.apache.hadoop.yarn.server.timelineservice.storage.TestTimelineReaderHBaseDown.checkQuery(HBaseTimelineReaderImpl)
At TestTimelineReaderHBaseDown.java:[line 190]
M V EI: org.apache.hadoop.fs.cosn.CosNInputStream$ReadBuffer.getBuffer() may
expose internal representation by returning CosNInputStream$ReadBuffer.buffer
At CosNInputStream.java:[line 87]
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17120) Fix failure of docker image creation due to pip2 install error on branch-3.1
Masatake Iwasaki created HADOOP-17120:
-
Summary: Fix failure of docker image creation due to pip2 install
error on branch-3.1
Key: HADOOP-17120
URL: https://issues.apache.org/jira/browse/HADOOP-17120
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.1.4
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
The command '/bin/sh -c pip2 install configparser==4.0.2 pylint==1.9.2'
returned a non-zero code: 1
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17073) Fix NoClassDefFoundError on 'mapred minicluster'
Masatake Iwasaki created HADOOP-17073:
-
Summary: Fix NoClassDefFoundError on 'mapred minicluster'
Key: HADOOP-17073
URL: https://issues.apache.org/jira/browse/HADOOP-17073
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
$ bin/mapred minicluster
2020-06-17 12:01:29,133 INFO mapreduce.MiniHadoopClusterManager: Updated 0
configuration settings from command line.
Exception in thread "main" java.lang.NoClassDefFoundError: org/junit/Assert
at
org.apache.hadoop.test.GenericTestUtils.assertExists(GenericTestUtils.java:298)
at
org.apache.hadoop.test.GenericTestUtils.getTestDir(GenericTestUtils.java:242)
at
org.apache.hadoop.test.GenericTestUtils.getTestDir(GenericTestUtils.java:251)
at
org.apache.hadoop.hdfs.MiniDFSCluster.getBaseDirectory(MiniDFSCluster.java:2982)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.(MiniDFSCluster.java:224)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:157)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
Caused by: java.lang.ClassNotFoundException: org.junit.Assert
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 8 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17040) Fix intermittent failure of ITestBlockingThreadPoolExecutorService
Masatake Iwasaki created HADOOP-17040: - Summary: Fix intermittent failure of ITestBlockingThreadPoolExecutorService Key: HADOOP-17040 URL: https://issues.apache.org/jira/browse/HADOOP-17040 Project: Hadoop Common Issue Type: Bug Components: fs/s3, test Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17034) Fix failure of TestSnappyCompressorDecompressor on CentOS 8
[ https://issues.apache.org/jira/browse/HADOOP-17034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-17034. --- Assignee: (was: Masatake Iwasaki) Resolution: Duplicate closing this as duplicate of HADOOP-16768. > Fix failure of TestSnappyCompressorDecompressor on CentOS 8 > --- > > Key: HADOOP-17034 > URL: https://issues.apache.org/jira/browse/HADOOP-17034 > Project: Hadoop Common > Issue Type: Bug > Environment: CentOS Linux release 8.0.1905 (Core), > snappy-devel-1.1.7-5.el8.x86_64 >Reporter: Masatake Iwasaki >Priority: Major > > testSnappyCompressDecompress testSnappyCompressDecompressInMultiThreads > reproducibly fails on CentOS 8. These tests has no issue on CentOS 7. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17034) Fix failure of TestSnappyCompressorDecompressor on CentOS 8
Masatake Iwasaki created HADOOP-17034: - Summary: Fix failure of TestSnappyCompressorDecompressor on CentOS 8 Key: HADOOP-17034 URL: https://issues.apache.org/jira/browse/HADOOP-17034 Project: Hadoop Common Issue Type: Bug Environment: CentOS Linux release 8.0.1905 (Core), snappy-devel-1.1.7-5.el8.x86_64 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki testSnappyCompressDecompress testSnappyCompressDecompressInMultiThreads reproducibly fails on CentOS 8. These tests has no issue on CentOS 7. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17025) Fix invalid metastore configuration in S3GuardTool tests
Masatake Iwasaki created HADOOP-17025:
-
Summary: Fix invalid metastore configuration in S3GuardTool tests
Key: HADOOP-17025
URL: https://issues.apache.org/jira/browse/HADOOP-17025
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3, test
Affects Versions: 3.3.0
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
The WARN message shown in S3GuardTool tests implies mismatch between property
name and the value.
{noformat}
2020-05-02 11:57:44,266 [setup] WARN conf.Configuration
(Configuration.java:getBoolean(1694)) - Invalid value for boolean:
org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore, choose default value: false
for fs.s3a.metadatastore.authoritative
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16972) Ignore AuthenticationFilterInitializer for KMSWebServer
Masatake Iwasaki created HADOOP-16972: - Summary: Ignore AuthenticationFilterInitializer for KMSWebServer Key: HADOOP-16972 URL: https://issues.apache.org/jira/browse/HADOOP-16972 Project: Hadoop Common Issue Type: Improvement Components: kms Affects Versions: 3.3.0 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki KMS does not work if hadoop.http.filter.initializers is set to AuthenticationFilterInitializer since KMS uses its own authentication filter. This is problematic when KMS is on the same node with other Hadoop services and shares core-site.xml with them. The filter initializers configuration should be tweaked as done for httpfs in HDFS-14845. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16905) Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches
[ https://issues.apache.org/jira/browse/HADOOP-16905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-16905. --- Fix Version/s: 3.3.0 Hadoop Flags: Reviewed Resolution: Fixed > Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches > > > Key: HADOOP-16905 > URL: https://issues.apache.org/jira/browse/HADOOP-16905 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: release-blocker > Fix For: 3.3.0 > > > Jackson-databind 2.10 should relieve us from the endless CVE patches > according to > https://medium.com/@cowtowncoder/jackson-2-10-features-cd880674d8a2 > Not sure if this is an easy update, but i think we should do this in the > Hadoop 3.3.0 and before removing jackson-databind entirely. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16744) Fix building instruction to enable zstd
Masatake Iwasaki created HADOOP-16744: - Summary: Fix building instruction to enable zstd Key: HADOOP-16744 URL: https://issues.apache.org/jira/browse/HADOOP-16744 Project: Hadoop Common Issue Type: Bug Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The instruction in the BUILDING.txt lacks libzstd-dev required on Ubuntu. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16739) Fix native build failure of hadoop-pipes on CentOS 8
Masatake Iwasaki created HADOOP-16739: - Summary: Fix native build failure of hadoop-pipes on CentOS 8 Key: HADOOP-16739 URL: https://issues.apache.org/jira/browse/HADOOP-16739 Project: Hadoop Common Issue Type: Improvement Components: tools/pipes Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Native build fails in hadoop-tools/hadoop-pips on CentOS 8 due to lack of rpc.h which was removed from glibc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16702) Move documentation of hadoop-toc to under src directory.
Masatake Iwasaki created HADOOP-16702: - Summary: Move documentation of hadoop-toc to under src directory. Key: HADOOP-16702 URL: https://issues.apache.org/jira/browse/HADOOP-16702 Project: Hadoop Common Issue Type: Bug Components: tools Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The documentation of hadoop-toc is not compiled because it is not under src directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16701) Fix broken links in site index
Masatake Iwasaki created HADOOP-16701: - Summary: Fix broken links in site index Key: HADOOP-16701 URL: https://issues.apache.org/jira/browse/HADOOP-16701 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 3.2.1 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Fix links to removed/unexisting pages in site index. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16689) Fix test failure of ITestAzureBlobFileSystemCLI.testMkdirRootNonExistentContainer
Masatake Iwasaki created HADOOP-16689: - Summary: Fix test failure of ITestAzureBlobFileSystemCLI.testMkdirRootNonExistentContainer Key: HADOOP-16689 URL: https://issues.apache.org/jira/browse/HADOOP-16689 Project: Hadoop Common Issue Type: Bug Components: fs/azure, test Affects Versions: 3.3.0 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The test on setup because AbstractAbfsIntegrationTest assumes the container to be exists. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16335) ERROR: namenode can only be executed by root while executing "hdfs namenode -format"
[
https://issues.apache.org/jira/browse/HADOOP-16335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-16335.
---
Resolution: Invalid
I think this is coufiguration issue. If you define env var
HDFS_NAMENODE_USER=root in hadoop-env.sh or somewhere and run the command by
another user, you will get the error.
{noformat}
$ HDFS_NAMENODE_USER=root bin/hadoop namenode -format
WARNING: Use of this script to execute namenode is deprecated.
WARNING: Attempting to execute replacement "hdfs namenode" instead.
ERROR: namenode can only be executed by root.
{noformat}
I'm closing this as invalid. Please reopen if this turns out to be bug in
scripts.
> ERROR: namenode can only be executed by root while executing "hdfs namenode
> -format"
>
>
> Key: HADOOP-16335
> URL: https://issues.apache.org/jira/browse/HADOOP-16335
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.2.0
>Reporter: Lakshmi Narayanan G
>Priority: Trivial
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16439) Upgrade bundled Tomcat in branch-2
Masatake Iwasaki created HADOOP-16439:
-
Summary: Upgrade bundled Tomcat in branch-2
Key: HADOOP-16439
URL: https://issues.apache.org/jira/browse/HADOOP-16439
Project: Hadoop Common
Issue Type: Improvement
Components: httpfs, kms
Reporter: Masatake Iwasaki
proposed by [~jojochuang] in mailing list:
{quote}We migrated from Tomcat to Jetty in Hadoop3, because Tomcat 6 went EOL in
2016. But we did not realize three years after Tomcat 6's EOL, a majority
of Hadoop users are still in Hadoop 2, and it looks like Hadoop 2 will stay
alive for another few years.
Backporting Jetty to Hadoop2 is probably too big of an imcompatibility.
How about migrating to Tomcat9?
{quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16418) Fix checkstyle and findbugs warnings in hadoop-dynamometer
Masatake Iwasaki created HADOOP-16418: - Summary: Fix checkstyle and findbugs warnings in hadoop-dynamometer Key: HADOOP-16418 URL: https://issues.apache.org/jira/browse/HADOOP-16418 Project: Hadoop Common Issue Type: Bug Components: tools Reporter: Masatake Iwasaki -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16411) Fix javadoc warnings in hadoop-dynamometer-workload
Masatake Iwasaki created HADOOP-16411: - Summary: Fix javadoc warnings in hadoop-dynamometer-workload Key: HADOOP-16411 URL: https://issues.apache.org/jira/browse/HADOOP-16411 Project: Hadoop Common Issue Type: Improvement Components: tools Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki "mvn package -Pdist" failed due to javadoc warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16099) Fix handling of path with multiple trailing slash in SwiftNativeFileSystem
Masatake Iwasaki created HADOOP-16099: - Summary: Fix handling of path with multiple trailing slash in SwiftNativeFileSystem Key: HADOOP-16099 URL: https://issues.apache.org/jira/browse/HADOOP-16099 Project: Hadoop Common Issue Type: Bug Components: fs/swift Affects Versions: 3.3.0 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Contract tests failed on Rackspace due to unremovable object from SwiftNativeFileSystem created by TestSwiftContractMkdir#testMkdirSlashHandling. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16098) Fix javadoc warnings in hadoop-aws
Masatake Iwasaki created HADOOP-16098: - Summary: Fix javadoc warnings in hadoop-aws Key: HADOOP-16098 URL: https://issues.apache.org/jira/browse/HADOOP-16098 Project: Hadoop Common Issue Type: Bug Components: fs/s3 Affects Versions: 3.3.0 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki mvn package -Pdist fails due to javadoc warnings in hadoop-aws. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14358) NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: Native Library /usr/java/jdk/jre/lib/amd64/libhadoop.so already loaded in a
[
https://issues.apache.org/jira/browse/HADOOP-14358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-14358.
---
Resolution: Invalid
Please use [user mailing list|https://hadoop.apache.org/mailing_lists.html] to
ask about operational issue. The log message implies misconfiguration.
> NativeCodeLoader: Failed to load native-hadoop with error:
> java.lang.UnsatisfiedLinkError: Native Library
> /usr/java/jdk/jre/lib/amd64/libhadoop.so already loaded in another classloader
> -
>
> Key: HADOOP-14358
> URL: https://issues.apache.org/jira/browse/HADOOP-14358
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Reporter: liuxian
>
> I am not sure whether this is a problem, in the "public final class
> NativeCodeLoader", when "loadLibrary("hadoop")", encounter this Exception.
> This time I want to confirm whether "nativeCodeLoaded" variables should be
> true
> Log print as follows:
> Line 10307: 2017-04-26 19:06:51,459 DEBUG NativeCodeLoader: Trying to load
> the custom-built native-hadoop library...
> Line 10308: 2017-04-26 19:06:51,460 DEBUG NativeCodeLoader: Failed to
> load native-hadoop with error: java.lang.UnsatisfiedLinkError: Native Library
> /usr/java/jdk/jre/lib/amd64/libhadoop.so already loaded in another classloader
> Line 10309: 2017-04-26 19:06:51,460 DEBUG NativeCodeLoader:
> java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> Line 10310: 2017-04-26 19:06:51,460 WARN NativeCodeLoader: Unable to
> load native-hadoop library for your platform... using builtin-java classes
> where applicable
>*/
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13839) Fix outdated tracing documentation
[ https://issues.apache.org/jira/browse/HADOOP-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-13839. --- Resolution: Fixed Fix Version/s: 2.7.4 Target Version/s: 2.7.4 I committed this to only branch-2.7. branch-2.8 and above uses htrace-4 and the documentation has been already updated. > Fix outdated tracing documentation > -- > > Key: HADOOP-13839 > URL: https://issues.apache.org/jira/browse/HADOOP-13839 > Project: Hadoop Common > Issue Type: Bug > Components: documentation, tracing >Affects Versions: 2.7.3 >Reporter: Masatake Iwasaki >Assignee: Elek, Marton >Priority: Minor > Fix For: 2.7.4 > > Attachments: HADOOP-13839-branch-2.7.001.patch > > > Sample code in tracing doc is based on older version of SpanReceiverHost. The > doc in branch-2 and trunk seems to be good. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13839) Fix outdated tracing documentation
Masatake Iwasaki created HADOOP-13839: - Summary: Fix outdated tracing documentation Key: HADOOP-13839 URL: https://issues.apache.org/jira/browse/HADOOP-13839 Project: Hadoop Common Issue Type: Bug Components: documentation, tracing Affects Versions: 2.7.3 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Priority: Minor Sample code in tracing doc is based on older version of SpanReceiverHost. The doc in branch-2 and trunk seems to be good. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13497) fix wrong command in CredentialProviderAPI.md
[
https://issues.apache.org/jira/browse/HADOOP-13497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-13497.
---
Resolution: Fixed
Fix Version/s: 3.0.0-alpha2
2.8.0
Committed. Thanks for the contribution, [~yuanbo].
> fix wrong command in CredentialProviderAPI.md
> -
>
> Key: HADOOP-13497
> URL: https://issues.apache.org/jira/browse/HADOOP-13497
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Reporter: Yuanbo Liu
>Assignee: Yuanbo Liu
>Priority: Trivial
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HADOOP-13497.001.patch
>
>
> In CredentialProviderAPI.md line 122
> {quote}
> Example: `hadoop credential create ssl.server.keystore.password
> jceks://file/tmp/test.jceks`
> {quote}
> should be
> {quote}
> Example: `hadoop credential create ssl.server.keystore.password -provider
> jceks://file/tmp/test.jceks`
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13439) Fix race between TestMetricsSystemImpl and TestGangliaMetrics
Masatake Iwasaki created HADOOP-13439:
-
Summary: Fix race between TestMetricsSystemImpl and
TestGangliaMetrics
Key: HADOOP-13439
URL: https://issues.apache.org/jira/browse/HADOOP-13439
Project: Hadoop Common
Issue Type: Bug
Components: t, test
Reporter: Masatake Iwasaki
Priority: Minor
TestGangliaMetrics#testGangliaMetrics2 set *.period to 120 but 8 was used.
{noformat}
2016-06-27 15:21:31,480 INFO impl.MetricsSystemImpl
(MetricsSystemImpl.java:startTimer(375)) - Scheduled snapshot period at 8
second(s).
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13152) Fix dynamic subcommands error on multiple arguments
Masatake Iwasaki created HADOOP-13152: - Summary: Fix dynamic subcommands error on multiple arguments Key: HADOOP-13152 URL: https://issues.apache.org/jira/browse/HADOOP-13152 Project: Hadoop Common Issue Type: Bug Affects Versions: HADOOP-12930 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-12588) Fix intermittent test failure of TestGangliaMetrics
[
https://issues.apache.org/jira/browse/HADOOP-12588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki reopened HADOOP-12588:
---
I saw this test failure again today. The fix seemed to be not enough.
{noformat}
java.lang.AssertionError: Missing metrics: test.s1rec.Xxx
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.metrics2.impl.TestGangliaMetrics.checkMetrics(TestGangliaMetrics.java:161)
at
org.apache.hadoop.metrics2.impl.TestGangliaMetrics.testGangliaMetrics2(TestGangliaMetrics.java:139)
{noformat}
> Fix intermittent test failure of TestGangliaMetrics
> ---
>
> Key: HADOOP-12588
> URL: https://issues.apache.org/jira/browse/HADOOP-12588
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Tsuyoshi Ozawa
>Assignee: Masatake Iwasaki
> Fix For: 2.8.0, 2.7.3
>
> Attachments: HADOOP-12588.001.patch
>
>
> Jenkins found this test failure on HADOOP-11149.
> {quote}
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.773 sec <<<
> FAILURE! - in org.apache.hadoop.metrics2.impl.TestGangliaMetrics
> testGangliaMetrics2(org.apache.hadoop.metrics2.impl.TestGangliaMetrics) Time
> elapsed: 0.39 sec <<< FAILURE!
> java.lang.AssertionError: Missing metrics: test.s1rec.Xxx
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.metrics2.impl.TestGangliaMetrics.checkMetrics(TestGangliaMetrics.java:159)
> at
> org.apache.hadoop.metrics2.impl.TestGangliaMetrics.testGangliaMetrics2(TestGangliaMetrics.java:137)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-12675) Fix description about retention period in usage of expunge command
Masatake Iwasaki created HADOOP-12675: - Summary: Fix description about retention period in usage of expunge command Key: HADOOP-12675 URL: https://issues.apache.org/jira/browse/HADOOP-12675 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 2.8.0 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12672) Split RPC timeout from IPC ping
Masatake Iwasaki created HADOOP-12672: - Summary: Split RPC timeout from IPC ping Key: HADOOP-12672 URL: https://issues.apache.org/jira/browse/HADOOP-12672 Project: Hadoop Common Issue Type: Improvement Components: ipc Reporter: Masatake Iwasaki -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12661) Add trash APIs to the FileSystem specification
Masatake Iwasaki created HADOOP-12661: - Summary: Add trash APIs to the FileSystem specification Key: HADOOP-12661 URL: https://issues.apache.org/jira/browse/HADOOP-12661 Project: Hadoop Common Issue Type: Sub-task Components: documentation Affects Versions: 2.8.0 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki HDFS-8831 added getTrashRoot and getTrashRoots. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12627) Add user documentation about metrics system
Masatake Iwasaki created HADOOP-12627: - Summary: Add user documentation about metrics system Key: HADOOP-12627 URL: https://issues.apache.org/jira/browse/HADOOP-12627 Project: Hadoop Common Issue Type: Improvement Components: documentation, metrics Affects Versions: 2.7.1 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Though there is [reference of metric items|https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Metrics.html] and [Javadoc|https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-project/api/org/apache/hadoop/metrics2/package-summary.html], we have no user documentation to configure metrics system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
