Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/148/ [Apr 1, 2021 9:39:40 AM] (Peter Bacsko) YARN-9618. NodesListManager event improvement. Contributed by Qi Zhu. [Apr 1, 2021 2:09:52 PM] (Peter Bacsko) YARN-10726. Log the size of DelegationTokenRenewer event queue in case of too many pending events. Contributed by Qi Zhu. [Apr 1, 2021 11:34:31 PM] (noreply) HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani [Apr 2, 2021 1:33:40 AM] (noreply) HDFS-15892. Add metric for editPendingQ in FSEditLogAsync (#2770) [Apr 2, 2021 2:28:17 AM] (noreply) HDFS-15938. Fix java doc in FSEditLog (#2837). Contributed by tomscut. [Apr 2, 2021 4:17:20 AM] (Brahma Reddy Battula) HDFS-15222. Correct the "hdfs fsck -list-corruptfileblocks" command output. Contributed by Ravuri Sushma sree. [Apr 2, 2021 4:26:00 AM] (Brahma Reddy Battula) HADOOP-17610. DelegationTokenAuthenticator prints token information. Contributed by Ravuri Sushma sree. [Apr 2, 2021 4:33:50 AM] (Brahma Reddy Battula) HADOOP-17587. Kinit with keytab should not display the keytab file's full path in any logs. Contributed by Ravuri Sushma sree. [Apr 2, 2021 4:37:33 PM] (noreply) HDFS-15944. Prevent truncation by snprintf (#2853) [Apr 2, 2021 5:37:04 PM] (noreply) HDFS-15946. Fix java doc in FSPermissionChecker (#2855). Contributed by tomscut. -1 overall The following subsystems voted -1: blanks mvnsite pathlen spotbugs unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml spotbugs : module:hadoop-hdfs-project/hadoop-hdfs Redundant nullcheck of oldLock, which is known to be non-null in org.apache.hadoop.hdfs.server.datanode.DataStorage.isPreUpgradableLayout(Storage$StorageDirectory) Redundant null check at DataStorage.java:is known to be non-null in org.apache.hadoop.hdfs.server.datanode.DataStorage.isPreUpgradableLayout(Storage$StorageDirectory) Redundant null check at DataStorage.java:[line 694] org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager$UserCounts doesn't override java.util.ArrayList.equals(Object) At RollingWindowManager.java:At RollingWindowManager.java:[line 1] spotbugs : module:hadoop-yarn-project/hadoop-yarn Redundant nullcheck of it, which is known to be non-null in org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker, NMStateStoreService$LocalResourceTrackerState) Redundant null check at ResourceLocalizationService.java:is known to be non-null in org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker, NMStateStoreService$LocalResourceTrackerState) Redundant null check at ResourceLocalizationService.java:[line 343] Redundant nullcheck of it, which is known to be non-null in org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker, NMStateStoreService$LocalResourceTrackerState) Redundant null check at ResourceLocalizationService.java:is known to be non-null in org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker, NMStateStoreService$LocalResourceTrackerState) Redundant null check at ResourceLocalizationService.java:[line 356] Boxed value is unboxed and then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:then immediately reboxed in
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/258/ No changes -1 overall The following subsystems voted -1: docker Powered by Apache Yetushttps://yetus.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-15933) distcp geo hdfs site is not working
[ https://issues.apache.org/jira/browse/HDFS-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HDFS-15933. - Resolution: Not A Bug > distcp geo hdfs site is not working > --- > > Key: HDFS-15933 > URL: https://issues.apache.org/jira/browse/HDFS-15933 > Project: Hadoop HDFS > Issue Type: Bug > Components: distcp >Affects Versions: 3.1.4 > Environment: Linux Redhat - RHEL 8,0 >Reporter: Suresh >Priority: Blocker > Attachments: GEO_Cluster.jpg > > > I am facing some issues in deploying HDFS at docker swarm network across geo > sites. > “distcp” command which is running at a HDFS datanodes container is not able > to pull the data from one site to another site(which is set to run at another > docker swarm network). It expects the docker swarm, HDFS datanodes to be in > host network configuration other than overlay(with ip forwarding), bridge > network. > We want to know how the shipment of docker based HDFS deployment would look > like, how distcp command would get invoked across docker instances sitting at > different Geo’s > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/ [Apr 2, 2021 2:28:17 AM] (noreply) HDFS-15938. Fix java doc in FSEditLog (#2837). Contributed by tomscut. [Apr 2, 2021 4:17:20 AM] (Brahma Reddy Battula) HDFS-15222. Correct the "hdfs fsck -list-corruptfileblocks" command output. Contributed by Ravuri Sushma sree. [Apr 2, 2021 4:26:00 AM] (Brahma Reddy Battula) HADOOP-17610. DelegationTokenAuthenticator prints token information. Contributed by Ravuri Sushma sree. [Apr 2, 2021 4:33:50 AM] (Brahma Reddy Battula) HADOOP-17587. Kinit with keytab should not display the keytab file's full path in any logs. Contributed by Ravuri Sushma sree. [Apr 2, 2021 4:37:33 PM] (noreply) HDFS-15944. Prevent truncation by snprintf (#2853) [Apr 2, 2021 5:37:04 PM] (noreply) HDFS-15946. Fix java doc in FSPermissionChecker (#2855). Contributed by tomscut. -1 overall The following subsystems voted -1: blanks pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.hdfs.server.datanode.TestBlockRecovery hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.mapred.TestJobEndNotifier hadoop.tools.fedbalance.procedure.TestBalanceProcedureScheduler hadoop.tools.fedbalance.TestDistCpProcedure hadoop.tools.dynamometer.TestDynamometerInfra hadoop.tools.dynamometer.TestDynamometerInfra hadoop.yarn.sls.TestReservationSystemInvariants cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/results-compile-cc-root.txt [100K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/results-compile-javac-root.txt [388K] blanks: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/blanks-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/blanks-tabs.txt [2.0M] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/results-checkstyle-root.txt [16M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/results-pathlen.txt [16K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/results-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/results-shellcheck.txt [28K] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/results-javadoc-javadoc-root.txt [1.1M] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/patch-unit-hadoop-common-project_hadoop-kms.txt [16K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [544K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt [76K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/466/artifact/out/patch-unit-hadoop-tools_hadoop-federation-balance.txt [12K]
[jira] [Created] (HDFS-15951) Remove unused parameters in NameNodeProxiesClient
tomscut created HDFS-15951: -- Summary: Remove unused parameters in NameNodeProxiesClient Key: HDFS-15951 URL: https://issues.apache.org/jira/browse/HDFS-15951 Project: Hadoop HDFS Issue Type: Wish Reporter: tomscut Assignee: tomscut Remove unused parameters in org.apache.hadoop.hdfs.NameNodeProxiesClient. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/ [Apr 2, 2021 12:48:27 AM] (Mingliang Liu) HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani -1 overall The following subsystems voted -1: asflicense hadolint mvnsite pathlen spotbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.util.TestDiskCheckerWithDiskIo hadoop.fs.TestFileUtil hadoop.crypto.key.kms.server.TestKMS hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.server.datanode.TestBlockRecovery hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.yarn.server.nodemanager.TestNodeStatusUpdater hadoop.yarn.server.resourcemanager.monitor.invariants.TestMetricsInvariantChecker hadoop.yarn.server.resourcemanager.TestClientRMService hadoop.mapreduce.task.reduce.TestFetcher hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.tools.TestDistCpSystem hadoop.yarn.sls.TestSLSRunner hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.resourceestimator.solver.impl.TestLpSolver cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-compile-javac-root.txt [496K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-patch-hadolint.txt [4.0K] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/patch-mvnsite-root.txt [584K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-patch-pylint.txt [48K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-patch-shellcheck.txt [56K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-patch-shelldocs.txt [48K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/diff-javadoc-javadoc-root.txt [20K] spotbugs: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-build-tools.txt [28K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-project.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-annotations.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-project-dist.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-assemblies.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-maven-plugins.txt [28K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-minikdc.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/257/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-auth.txt [28K]
[jira] [Created] (HDFS-15950) Remove unused hdfs.proto import
Gautham Banasandra created HDFS-15950:
-
Summary: Remove unused hdfs.proto import
Key: HDFS-15950
URL: https://issues.apache.org/jira/browse/HDFS-15950
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 3.4.0
Reporter: Gautham Banasandra
Assignee: Gautham Banasandra
hdfs.proto is imported in inotify.proto and is unused. This causes the
following warning to be generated -
{code}
inotify.proto:35:1: warning: Import hdfs.proto is unused.
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-15949) Fix integer overflow
Gautham Banasandra created HDFS-15949:
-
Summary: Fix integer overflow
Key: HDFS-15949
URL: https://issues.apache.org/jira/browse/HDFS-15949
Project: Hadoop HDFS
Issue Type: Bug
Components: libhdfs++
Affects Versions: 3.4.0
Reporter: Gautham Banasandra
Assignee: Gautham Banasandra
There are some instances where integer overflow warnings are reported. Need to
fix them.
{code}
[ 63%] Building CXX object
main/native/libhdfspp/tests/CMakeFiles/hdfs_ext_hdfspp_test_shim_static.dir/hdfs_ext_test.cc.o
In file included from
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googletest/include/gtest/gtest.h:375,
from
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googlemock/include/gmock/internal/gmock-internal-utils.h:47,
from
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googlemock/include/gmock/gmock-actions.h:51,
from
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googlemock/include/gmock/gmock.h:59,
from
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfspp_mini_dfs.h:24,
from
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:19:
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:
In member function ‘virtual void hdfs::HdfsExtTest_TestHosts_Test::TestBody()’:
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:456:95:
warning: integer overflow in expression of type ‘long int’ results in
‘-9223372036854775808’ [-Woverflow]
456 | EXPECT_EQ(nullptr, hdfsGetHosts(fs, filename.c_str(), 0,
std::numeric_limits::max()+1));
|
~~~^~
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:460:92:
warning: integer overflow in expression of type ‘long int’ results in
‘-9223372036854775808’ [-Woverflow]
460 | EXPECT_EQ(nullptr, hdfsGetHosts(fs, filename.c_str(),
std::numeric_limits::max()+1, std::numeric_limits::max()));
|
~~~^~
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
