Thanks, Allen. On Thu, Mar 15, 2018 at 10:20 AM, Iñigo Goiri <elgo...@gmail.com> wrote: >> * It ALWAYS applies HADOOP-14667.05.patch prior to running. As a result, >> this is only set up for trunk with no parameterization to run other >> branches.
I tried to get this running in my environment a few weeks ago, but I suspect my problems are with my environment, not the patch. Let's at least get this committed to trunk. We can look at other branches later. -C >> * The URI handling for file paths in hadoop-common and elsewhere is pretty >> broken on Windows, so many many many unit tests are failing and I wouldn't >> be surprised if Windows hadoop installs are horked as a result. >> >> * Runtime is about 12-13 hours with many tests taking significantly longer >> than their UNIX counterparts. My guess is that this caused by winutils. >> Changing from winutils to Java 7 API calls would get this more in line and >> be a significant performance boost for Windows clients/servers as well. >> >> Have fun. >> >> ===== >> >> For more details, see https://builds.apache.org/job/hadoop-trunk-win/406/ >> <https://builds.apache.org/job/hadoop-trunk-win/406/> >> >> [Mar 14, 2018 6:26:58 PM] (xyao) HDFS-13251. Avoid using hard coded >> datanode data dirs in unit tests. >> [Mar 14, 2018 8:05:24 PM] (jlowe) MAPREDUCE-7064. Flaky test >> [Mar 14, 2018 8:14:36 PM] (inigoiri) HDFS-13198. RBF: >> RouterHeartbeatService throws out CachedStateStore >> [Mar 14, 2018 8:36:53 PM] (wangda) Revert "HADOOP-13707. If kerberos is >> enabled while HTTP SPNEGO is not >> [Mar 14, 2018 10:47:56 PM] (fabbri) HADOOP-15278 log s3a at info. >> Contributed by Steve Loughran. >> >> >> >> >> -1 overall >> >> >> The following subsystems voted -1: >> unit >> >> >> The following subsystems are considered long running: >> (runtime bigger than 1h 00m 00s) >> unit >> >> >> Specific tests: >> >> Failed CTEST tests : >> >> test_test_libhdfs_threaded_hdfs_static >> >> Failed junit tests : >> >> hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec >> hadoop.fs.contract.rawlocal.TestRawlocalContractAppend >> hadoop.fs.TestFsShellCopy >> hadoop.fs.TestFsShellList >> hadoop.fs.TestLocalFileSystem >> hadoop.http.TestHttpServer >> hadoop.http.TestHttpServerLogs >> hadoop.io.compress.TestCodec >> hadoop.io.nativeio.TestNativeIO >> hadoop.ipc.TestSocketFactory >> hadoop.metrics2.impl.TestStatsDMetrics >> hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal >> hadoop.security.TestSecurityUtil >> hadoop.security.TestShellBasedUnixGroupsMapping >> hadoop.security.token.TestDtUtilShell >> hadoop.util.TestNativeCodeLoader >> hadoop.fs.TestWebHdfsFileContextMainOperations >> hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy >> hadoop.hdfs.crypto.TestHdfsCryptoStreams >> hadoop.hdfs.qjournal.client.TestQuorumJournalManager >> hadoop.hdfs.qjournal.server.TestJournalNode >> hadoop.hdfs.qjournal.server.TestJournalNodeSync >> hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks >> hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped >> hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages >> hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks >> hadoop.hdfs.server.blockmanagement.TestReplicationPolicy >> hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl >> hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles >> hadoop.hdfs.server.datanode.fsdataset.impl. >> TestLazyPersistLockedMemory >> hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy >> hadoop.hdfs.server.datanode.fsdataset.impl. >> TestLazyPersistReplicaPlacement >> hadoop.hdfs.server.datanode.fsdataset.impl. >> TestLazyPersistReplicaRecovery >> hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter >> hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl >> hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation >> hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica >> hadoop.hdfs.server.datanode.TestBlockPoolSliceStorage >> hadoop.hdfs.server.datanode.TestBlockRecovery >> hadoop.hdfs.server.datanode.TestBlockScanner >> hadoop.hdfs.server.datanode.TestDataNodeFaultInjector >> hadoop.hdfs.server.datanode.TestDataNodeMetrics >> hadoop.hdfs.server.datanode.TestDataNodeUUID >> hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure >> hadoop.hdfs.server.datanode.TestDirectoryScanner >> hadoop.hdfs.server.datanode.TestHSync >> hadoop.hdfs.server.datanode.web.TestDatanodeHttpXFrame >> hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand >> hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC >> hadoop.hdfs.server.federation.router.TestRouterAdminCLI >> hadoop.hdfs.server.mover.TestStorageMover >> hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA >> hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA >> hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics >> hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstruction >> WithSnapshot >> hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot >> hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots >> hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff >> hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport >> hadoop.hdfs.server.namenode.TestAddBlock >> hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands >> hadoop.hdfs.server.namenode.TestCheckpoint >> hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate >> hadoop.hdfs.server.namenode.TestEditLogRace >> hadoop.hdfs.server.namenode.TestFileTruncate >> hadoop.hdfs.server.namenode.TestFsck >> hadoop.hdfs.server.namenode.TestFSImage >> hadoop.hdfs.server.namenode.TestFSImageWithSnapshot >> hadoop.hdfs.server.namenode.TestNamenodeCapacityReport >> hadoop.hdfs.server.namenode.TestNameNodeMXBean >> hadoop.hdfs.server.namenode.TestNestedEncryptionZones >> hadoop.hdfs.server.namenode.TestQuotaByStorageType >> hadoop.hdfs.server.namenode.TestStartup >> hadoop.hdfs.TestDatanodeRegistration >> hadoop.hdfs.TestDecommission >> hadoop.hdfs.TestDFSOutputStream >> hadoop.hdfs.TestDFSShell >> hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy >> hadoop.hdfs.TestDFSStripedOutputStreamWithFailure >> hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy >> hadoop.hdfs.TestFetchImage >> hadoop.hdfs.TestFileAppend >> hadoop.hdfs.TestFileConcurrentReader >> hadoop.hdfs.TestFileCreation >> hadoop.hdfs.TestFileLengthOnClusterRestart >> hadoop.hdfs.TestHDFSFileSystemContract >> hadoop.hdfs.TestHDFSServerPorts >> hadoop.hdfs.TestHDFSTrash >> hadoop.hdfs.TestHFlush >> hadoop.hdfs.TestLeaseRecovery >> hadoop.hdfs.TestLocalDFS >> hadoop.hdfs.TestMaintenanceState >> hadoop.hdfs.TestMiniDFSCluster >> hadoop.hdfs.TestPread >> hadoop.hdfs.TestQuota >> hadoop.hdfs.TestReadStripedFileWithDecodingCorruptData >> hadoop.hdfs.TestReadStripedFileWithDecodingDeletedData >> hadoop.hdfs.TestReadStripedFileWithMissingBlocks >> hadoop.hdfs.TestReconstructStripedFile >> hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy >> hadoop.hdfs.TestSecureEncryptionZoneWithKMS >> hadoop.hdfs.TestTrashWithSecureEncryptionZones >> hadoop.hdfs.tools.TestDebugAdmin >> hadoop.hdfs.tools.TestDFSAdmin >> hadoop.hdfs.tools.TestDFSAdminWithHA >> hadoop.hdfs.tools.TestGetConf >> hadoop.hdfs.web.TestWebHDFS >> hadoop.hdfs.web.TestWebHdfsUrl >> hadoop.fs.http.server.TestHttpFSServerWebServer >> hadoop.hdfs.nfs.nfs3.TestRpcProgramNfs3 >> hadoop.hdfs.nfs.nfs3.TestWrites >> hadoop.yarn.logaggregation.filecontroller.ifile. >> TestLogAggregationIndexFileController >> hadoop.yarn.logaggregation.TestAggregatedLogFormat >> hadoop.yarn.server.nodemanager.containermanager. >> launcher.TestContainerLaunch >> hadoop.yarn.server.nodemanager.containermanager.linux.privileged. >> TestPrivilegedOperationExecutor >> hadoop.yarn.server.nodemanager.containermanager.linux.resources. >> TestCGroupsHandlerImpl >> hadoop.yarn.server.nodemanager.containermanager.linux.runtime. >> TestDockerContainerRuntime >> hadoop.yarn.server.nodemanager.containermanager.linux.runtime. >> TestJavaSandboxLinuxContainerRuntime >> hadoop.yarn.server.nodemanager.containermanager.logaggregation. >> TestAppLogAggregatorImpl >> hadoop.yarn.server.nodemanager.containermanager.TestContainerManager >> hadoop.yarn.server.nodemanager.recovery. >> TestNMLeveldbStateStoreService >> hadoop.yarn.server.nodemanager.TestContainerExecutor >> hadoop.yarn.server.nodemanager.TestLocalDirsHandlerService >> hadoop.yarn.server.nodemanager.TestNodeManagerResync >> hadoop.yarn.server.nodemanager.webapp.TestContainerLogsPage >> hadoop.yarn.server.applicationhistoryservice. >> TestApplicationHistoryServer >> hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilt >> erForV1 >> hadoop.yarn.server.resourcemanager.metrics. >> TestSystemMetricsPublisher >> hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore >> hadoop.yarn.server.resourcemanager.scheduler.capacity.conf. >> TestLeveldbConfigurationStore >> hadoop.yarn.server.resourcemanager.scheduler. >> capacity.TestCapacityScheduler >> hadoop.yarn.server.resourcemanager.scheduler.constraint. >> TestPlacementProcessor >> hadoop.yarn.server.resourcemanager.scheduler.fair. >> TestAllocationFileLoaderService >> hadoop.yarn.server.resourcemanager.TestResourceTrackerService >> hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore >> hadoop.yarn.server.timelineservice.reader. >> TestTimelineReaderWebServicesHBaseStorage >> hadoop.yarn.server.timelineservice.storage.flow. >> TestHBaseStorageFlowActivity >> hadoop.yarn.server.timelineservice.storage.flow. >> TestHBaseStorageFlowRun >> hadoop.yarn.server.timelineservice.storage.flow. >> TestHBaseStorageFlowRunCompaction >> hadoop.yarn.server.timelineservice.storage. >> TestHBaseTimelineStorageApps >> hadoop.yarn.server.timelineservice.storage. >> TestHBaseTimelineStorageEntities >> hadoop.yarn.server.timelineservice.storage. >> TestHBaseTimelineStorageSchema >> hadoop.yarn.applications.distributedshell.TestDistributedShell >> hadoop.mapred.TestTaskProgressReporter >> hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter >> hadoop.mapreduce.lib.output.TestFileOutputCommitter >> hadoop.mapred.TestJavaSerialization >> hadoop.mapred.TestLocalJobSubmission >> hadoop.mapred.TestMiniMRChildTask >> hadoop.mapred.TestMRTimelineEventHandling >> hadoop.mapreduce.v2.TestMRJobs >> hadoop.mapreduce.v2.TestUberAM >> hadoop.yarn.service.client.TestBuildExternalComponents >> hadoop.yarn.service.client.TestServiceCLI >> hadoop.yarn.service.monitor.TestServiceMonitor >> hadoop.yarn.service.providers.TestAbstractClientProvider >> hadoop.yarn.service.TestServiceAM >> hadoop.yarn.service.TestYarnNativeServices >> hadoop.mapred.nativetask.handlers.TestNativeCollectorOnlyHandler >> hadoop.mapred.uploader.TestFrameworkUploader >> hadoop.tools.TestDistCpSystem >> hadoop.tools.TestHadoopArchiveLogs >> hadoop.mapred.gridmix.TestGridMixClasses >> hadoop.fs.azure.TestClientThrottlingAnalyzer >> hadoop.yarn.sls.TestReservationSystemInvariants >> >> >> CTEST: >> >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt> >> [28K] >> >> unit: >> >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-common-project_hadoop-common.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt> [276K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt> [1.5M] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt> >> [68K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt> [20K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-nfs.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-nfs.txt> [16K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-yarn-project_hadoop- >> yarn_hadoop-yarn-common.txt> [40K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop- >> yarn-server_hadoop-yarn-server-nodemanager.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit- >> hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server- >> nodemanager.txt> [216K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop- >> yarn-server_hadoop-yarn-server-applicationhistoryservice.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-yarn-project_hadoop- >> yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt> >> [32K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop- >> yarn-server_hadoop-yarn-server-resourcemanager.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-yarn-project_hadoop- >> yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt> [116K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop- >> yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-yarn-project_hadoop- >> yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt> >> [12K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop- >> yarn-server_hadoop-yarn-server-timelineservice-hbase-tests.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-yarn-project_hadoop- >> yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase-tests.txt> >> [12K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop- >> yarn-applications_hadoop-yarn-applications-distributedshell.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-yarn-project_hadoop- >> yarn_hadoop-yarn-applications_hadoop-yarn-applications- >> distributedshell.txt> [20K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop- >> mapreduce-client-core.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit-hadoop-mapreduce-project_ >> hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt> [136K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop- >> mapreduce-client-jobclient.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit-hadoop-mapreduce-project_ >> hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt> [108K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop- >> yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-yarn-project_hadoop- >> yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop- >> yarn-services-core.txt> [92K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop- >> mapreduce-client-nativetask.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit-hadoop-mapreduce-project_ >> hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt> [12K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop- >> mapreduce-client-uploader.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit-hadoop-mapreduce-project_ >> hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt> [20K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-tools_hadoop-distcp.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit- >> hadoop-tools_hadoop-distcp.txt> [16K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-tools_hadoop-archive-logs.txt < >> https://builds.apache.org/job/hadoop-trunk-win/406/ >> artifact/out/patch-unit-hadoop-tools_hadoop-archive-logs.txt> [8.0K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-tools_hadoop-gridmix.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit- >> hadoop-tools_hadoop-gridmix.txt> [12K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-tools_hadoop-azure.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit- >> hadoop-tools_hadoop-azure.txt> [16K] >> https://builds.apache.org/job/hadoop-trunk-win/406/artifact/ >> out/patch-unit-hadoop-tools_hadoop-sls.txt <https://builds.apache.org/ >> job/hadoop-trunk-win/406/artifact/out/patch-unit- >> hadoop-tools_hadoop-sls.txt> [12K] >> >> Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org < >> http://yetus.apache.org/> --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
