[ https://issues.apache.org/jira/browse/HADOOP-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160029#comment-17160029 ]
Ahmed Hussein commented on HADOOP-17100: ---------------------------------------- [^HADOOP-17100.006.patch] submission results looks okay. I reviewed the failing unit tests. The patch did not break any test and the failures either exist on trunk, or caused by existing broken tests. |TestDistributedShell|Test is broken on trunk. see YARN-10334|Broken| |TestRMHATimelineCollectors|Test is broken on trunk. (YARN-10337)|Broken| |TestHDFSContractMultipartUploader|Broken on Trunk. HDFS-15471.|Broken| |TestFixKerberosTicketOrder|intermittently fails on trunk |Flaky| |TestGetFileChecksum|for HADOOP-17101, Filed HDFS-1546. an old jira exist HDFS-4723|Flaky| |TestGroupsCaching|intermittently fails due to other UT failures|Flaky| |TestNameNodeRetryCacheMetrics|filed a jira HDFS-1548|Flaky| |TestRaceWhenRelogin|intermittently fails due to other UT failures|Flaky| |TestRollingUpgrade|intermittently fails due to other UT failures |Flaky| |TestBalancerWithHANameNodes|intermittently fails due to other UT failures |Flaky| |TestDelegationTokenRenewer|intermittently fails due to other UT failures|Flaky| cc [~weichiu], can you please take a look at the patch and merge into 3.2.2, 3.3.1, 3.4.0, 3.1.5? The patch is very straightforward, changing import line in the files. > Replace Guava Supplier with Java8+ Supplier in Hadoop > ----------------------------------------------------- > > Key: HADOOP-17100 > URL: https://issues.apache.org/jira/browse/HADOOP-17100 > Project: Hadoop Common > Issue Type: Sub-task > Reporter: Ahmed Hussein > Assignee: Ahmed Hussein > Priority: Major > Attachments: HADOOP-17100.001.patch, HADOOP-17100.002.patch, > HADOOP-17100.003.patch, HADOOP-17100.006.patch > > > Replacing Usage of {{guava.Supplier<>}} are in Unit tests > {{GenereicTestUtils.waitFor()}} in Hadoop project. > * To make things more convenient for reviewers, I decided: > ** Not to replace Object instantiation by lambda expressions because this > will increase the patch size significantly and require code adjustments to > pass the checkstyle scripts. > ** Not to refactor the imports because this will make reading the patch more > difficult. > * Merge should be done to the following branches: 3.2.2, 3.3.1, 3.4.0, trunk > The task is straightforward because {{java.util.Supplier}} has the same API > as {{guava.Supplier<>}} and the vast majority of usage comes from Test-units. > Therefore, we need only to do the following a "one-line" change in all 147 > files. > {code:bash} > > -import com.google.common.base.Supplier; > +import java.util.function.Supplier; > {code} > The code change needs to be applied to the following list of files: > {code:java} > > Targets > Occurrences of 'com.google.common.base.Supplier' in project with mask > '*.java' > Found Occurrences (146 usages found) > org.apache.hadoop.conf (1 usage found) > TestReconfiguration.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.crypto.key.kms.server (1 usage found) > TestKMS.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.fs (2 usages found) > FCStatisticsBaseTest.java (1 usage found) > 40 import com.google.common.base.Supplier; > TestEnhancedByteBufferAccess.java (1 usage found) > 75 import com.google.common.base.Supplier; > org.apache.hadoop.fs.viewfs (1 usage found) > TestViewFileSystemWithTruncate.java (1 usage found) > 23 import com.google.common.base.Supplier; > org.apache.hadoop.ha (1 usage found) > TestZKFailoverController.java (1 usage found) > 25 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs (20 usages found) > DFSTestUtil.java (1 usage found) > 79 import com.google.common.base.Supplier; > MiniDFSCluster.java (1 usage found) > 78 import com.google.common.base.Supplier; > TestBalancerBandwidth.java (1 usage found) > 29 import com.google.common.base.Supplier; > TestClientProtocolForPipelineRecovery.java (1 usage found) > 30 import com.google.common.base.Supplier; > TestDatanodeRegistration.java (1 usage found) > 44 import com.google.common.base.Supplier; > TestDataTransferKeepalive.java (1 usage found) > 47 import com.google.common.base.Supplier; > TestDeadNodeDetection.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestDecommission.java (1 usage found) > 41 import com.google.common.base.Supplier; > TestDFSShell.java (1 usage found) > 37 import com.google.common.base.Supplier; > TestEncryptedTransfer.java (1 usage found) > 35 import com.google.common.base.Supplier; > TestEncryptionZonesWithKMS.java (1 usage found) > 22 import com.google.common.base.Supplier; > TestFileCorruption.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestLeaseRecovery2.java (1 usage found) > 32 import com.google.common.base.Supplier; > TestLeaseRecoveryStriped.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestMaintenanceState.java (1 usage found) > 63 import com.google.common.base.Supplier; > TestPread.java (1 usage found) > 61 import com.google.common.base.Supplier; > TestQuota.java (1 usage found) > 39 import com.google.common.base.Supplier; > TestReplaceDatanodeOnFailure.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestReplication.java (1 usage found) > 27 import com.google.common.base.Supplier; > TestSafeMode.java (1 usage found) > 62 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.client.impl (2 usages found) > TestBlockReaderLocalMetrics.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestLeaseRenewer.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.qjournal (1 usage found) > MiniJournalCluster.java (1 usage found) > 31 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.qjournal.client (1 usage found) > TestIPCLoggerChannel.java (1 usage found) > 43 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.qjournal.server (1 usage found) > TestJournalNodeSync.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.blockmanagement (7 usages found) > TestBlockManagerSafeMode.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestBlockReportRateLimiting.java (1 usage found) > 25 import com.google.common.base.Supplier; > TestNameNodePrunesMissingStorages.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestPendingInvalidateBlock.java (1 usage found) > 43 import com.google.common.base.Supplier; > TestPendingReconstruction.java (1 usage found) > 34 import com.google.common.base.Supplier; > TestRBWBlockInvalidation.java (1 usage found) > 49 import com.google.common.base.Supplier; > TestSlowDiskTracker.java (1 usage found) > 48 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.datanode (13 usages found) > DataNodeTestUtils.java (1 usage found) > 40 import com.google.common.base.Supplier; > TestBlockRecovery.java (1 usage found) > 120 import com.google.common.base.Supplier; > TestBlockScanner.java (1 usage found) > 43 import com.google.common.base.Supplier; > TestBPOfferService.java (1 usage found) > 92 import com.google.common.base.Supplier; > TestCorruptMetadataFile.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestDataNodeLifeline.java (1 usage found) > 74 import com.google.common.base.Supplier; > TestDataNodeMetrics.java (1 usage found) > 37 import com.google.common.base.Supplier; > TestDataNodeMetricsLogger.java (1 usage found) > 57 import com.google.common.base.Supplier; > TestDataNodeMultipleRegistrations.java (1 usage found) > 33 import com.google.common.base.Supplier; > TestDataNodeMXBean.java (1 usage found) > 31 import com.google.common.base.Supplier; > TestDatanodeProtocolRetryPolicy.java (1 usage found) > 32 import com.google.common.base.Supplier; > TestDataNodeVolumeFailure.java (1 usage found) > 94 import com.google.common.base.Supplier; > TestDiskError.java (1 usage found) > 31 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.datanode.checker (1 usage found) > TestThrottledAsyncChecker.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl (7 usages found) > TestCacheByPmemMappableBlockLoader.java (1 usage found) > 65 import com.google.common.base.Supplier; > TestFsDatasetCache.java (1 usage found) > 94 import com.google.common.base.Supplier; > TestFsDatasetImpl.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestFsVolumeList.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestLazyPersistLockedMemory.java (1 usage found) > 22 import com.google.common.base.Supplier; > TestPmemCacheRecovery.java (1 usage found) > 65 import com.google.common.base.Supplier; > TestSpaceReservation.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.datanode.metrics (1 usage found) > TestDataNodeOutlierDetectionViaMetrics.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.datanode.web.webhdfs (1 usage found) > TestDataNodeUGIProvider.java (1 usage found) > 54 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.diskbalancer (2 usages found) > TestDiskBalancer.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestDiskBalancerWithMockMover.java (1 usage found) > 23 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.federation (1 usage found) > FederationTestUtils.java (1 usage found) > 95 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.federation.router (4 usages found) > TestRouterAdminCLI.java (1 usage found) > 67 import com.google.common.base.Supplier; > TestRouterQuota.java (1 usage found) > 77 import com.google.common.base.Supplier; > TestRouterRpc.java (1 usage found) > 131 import com.google.common.base.Supplier; > TestRouterRPCClientRetries.java (1 usage found) > 57 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.mover (1 usage found) > TestMover.java (1 usage found) > 98 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.namenode (17 usages found) > TestAddStripedBlockInFBR.java (1 usage found) > 43 import com.google.common.base.Supplier; > TestBackupNode.java (1 usage found) > 59 import com.google.common.base.Supplier; > TestCacheDirectives.java (1 usage found) > 99 import com.google.common.base.Supplier; > TestCheckpoint.java (1 usage found) > 98 import com.google.common.base.Supplier; > TestDeadDatanode.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestEditLogAutoroll.java (1 usage found) > 49 import com.google.common.base.Supplier; > TestEditLogRace.java (1 usage found) > 41 import com.google.common.base.Supplier; > TestFsck.java (1 usage found) > 62 import com.google.common.base.Supplier; > TestFSNamesystemLock.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestMetaSave.java (1 usage found) > 33 import com.google.common.base.Supplier; > TestNameNodeMetadataConsistency.java (1 usage found) > 33 import com.google.common.base.Supplier; > TestNameNodeMetricsLogger.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestNameNodeMXBean.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestNameNodeStatusMXBean.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestPersistentStoragePolicySatisfier.java (1 usage found) > 40 import com.google.common.base.Supplier; > TestReencryption.java (1 usage found) > 34 import com.google.common.base.Supplier; > TestUpgradeDomainBlockPlacementPolicy.java (1 usage found) > 52 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.namenode.ha (11 usages found) > HATestUtil.java (1 usage found) > 64 import com.google.common.base.Supplier; > TestBootstrapStandby.java (1 usage found) > 30 import com.google.common.base.Supplier; > TestDNFencing.java (1 usage found) > 32 import com.google.common.base.Supplier; > TestDNFencingWithReplication.java (1 usage found) > 39 import com.google.common.base.Supplier; > TestEditLogTailer.java (1 usage found) > 65 import com.google.common.base.Supplier; > TestHASafeMode.java (1 usage found) > 75 import com.google.common.base.Supplier; > TestPendingCorruptDnMessages.java (1 usage found) > 44 import com.google.common.base.Supplier; > TestPipelinesFailover.java (1 usage found) > 65 import com.google.common.base.Supplier; > TestStandbyCheckpoints.java (1 usage found) > 20 import com.google.common.base.Supplier; > TestStandbyInProgressTail.java (1 usage found) > 52 import com.google.common.base.Supplier; > TestStandbyIsHot.java (1 usage found) > 46 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.namenode.snapshot (1 usage found) > TestRandomOpsWithSnapshots.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.namenode.sps (1 usage found) > TestStoragePolicySatisfierWithStripedFile.java (1 usage found) > 55 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.server.sps (1 usage found) > TestExternalStoragePolicySatisfier.java (1 usage found) > 101 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.shortcircuit (1 usage found) > TestShortCircuitCache.java (1 usage found) > 93 import com.google.common.base.Supplier; > org.apache.hadoop.hdfs.tools (2 usages found) > TestDFSAdmin.java (1 usage found) > 28 import com.google.common.base.Supplier; > TestDFSZKFailoverController.java (1 usage found) > 58 import com.google.common.base.Supplier; > org.apache.hadoop.http (1 usage found) > TestSSLHttpServerConfigs.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.ipc (1 usage found) > TestIPC.java (1 usage found) > 101 import com.google.common.base.Supplier; > org.apache.hadoop.mapred (2 usages found) > TestTaskAttemptListenerImpl.java (1 usage found) > 20 import com.google.common.base.Supplier; > UtilsForTests.java (1 usage found) > 64 import com.google.common.base.Supplier; > org.apache.hadoop.mapreduce.v2.app (4 usages found) > TestFetchFailure.java (1 usage found) > 29 import com.google.common.base.Supplier; > TestMRApp.java (1 usage found) > 31 import com.google.common.base.Supplier; > TestRecovery.java (1 usage found) > 31 import com.google.common.base.Supplier; > TestTaskHeartbeatHandler.java (1 usage found) > 28 import com.google.common.base.Supplier; > org.apache.hadoop.mapreduce.v2.app.rm (1 usage found) > TestRMContainerAllocator.java (1 usage found) > 156 import com.google.common.base.Supplier; > org.apache.hadoop.mapreduce.v2.hs (1 usage found) > TestJHSDelegationTokenSecretManager.java (1 usage found) > 30 import com.google.common.base.Supplier; > org.apache.hadoop.metrics2.impl (1 usage found) > TestMetricsSystemImpl.java (1 usage found) > 42 import com.google.common.base.Supplier; > org.apache.hadoop.metrics2.lib (1 usage found) > TestMutableRollingAverages.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.security (1 usage found) > TestGroupsCaching.java (1 usage found) > 37 import com.google.common.base.Supplier; > org.apache.hadoop.security.ssl (1 usage found) > TestReloadingX509TrustManager.java (1 usage found) > 24 import com.google.common.base.Supplier; > org.apache.hadoop.security.token.delegation (1 usage found) > TestZKDelegationTokenSecretManager.java (1 usage found) > 27 import com.google.common.base.Supplier; > org.apache.hadoop.test (2 usages found) > GenericTestUtils.java (1 usage found) > 64 import com.google.common.base.Supplier; > TestGenericTestUtils.java (1 usage found) > 26 import com.google.common.base.Supplier; > org.apache.hadoop.tracing (1 usage found) > SetSpanReceiver.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.util (1 usage found) > TestShell.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.applications.distributedshell (1 usage found) > TestDistributedShell.java (1 usage found) > 43 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.client (1 usage found) > TestRMFailover.java (1 usage found) > 64 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.client.api.impl (1 usage found) > TestYarnClientWithReservation.java (1 usage found) > 20 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.nodemanager.containermanager (1 usage > found) > TestContainerManager.java (1 usage found) > 51 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher (1 > usage found) > TestContainerLaunch.java (1 usage found) > 57 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer (1 > usage found) > TestContainerLocalizer.java (1 usage found) > 97 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation > (1 usage found) > TestLogAggregationService.java (1 usage found) > 150 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor (1 > usage found) > TestContainersMonitor.java (1 usage found) > 40 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.nodemanager.logaggregation.tracker (1 usage > found) > TestNMLogAggregationStatusTracker.java (1 usage found) > 24 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.resourcemanager (6 usages found) > TestApplicationMasterLauncher.java (1 usage found) > 95 import com.google.common.base.Supplier; > TestLeaderElectorService.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestRM.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestRMHA.java (1 usage found) > 21 import com.google.common.base.Supplier; > TestRMRestart.java (1 usage found) > 137 import com.google.common.base.Supplier; > TestWorkPreservingRMRestart.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.resourcemanager.recovery (1 usage found) > TestZKRMStateStore.java (1 usage found) > 75 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity (1 usage > found) > TestCapacityScheduler.java (1 usage found) > 192 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair (1 usage > found) > TestContinuousScheduling.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.resourcemanager.security (2 usages found) > TestDelegationTokenRenewer.java (1 usage found) > 117 import com.google.common.base.Supplier; > TestRMDelegationTokens.java (1 usage found) > 29 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.router.webapp (1 usage found) > TestRouterWebServicesREST.java (1 usage found) > 135 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.server.webproxy.amfilter (1 usage found) > TestAmFilter.java (1 usage found) > 53 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.service (1 usage found) > MockServiceAM.java (1 usage found) > 21 import com.google.common.base.Supplier; > org.apache.hadoop.yarn.sls.nodemanager (1 usage found) > TestNMSimulator.java (1 usage found) > 20 import com.google.common.base.Supplier; > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org