This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
discard fab0fa5 retrigger build discard c38b311 retrigger build discard 4e9c4dc retrigger build discard 9bba11f more debug discard b1a0c94 debug secure execution discard 1052564 try to fix tmate discard 8cb3c90 add timeout discard 6002b7d debug without tmate discard 3a8b003 sudo with power discard b2aa643 fix install script discard b849a80 fix build dependency discard e6b05fa debug-acceptace-timeout discard 4916923 switch to use ENV based variable with default discard 15ce5ea remove configs where we don't need them discard 9685ec0 HDDS-2646. Start acceptance tests only if at least one THREE pipeline is available add 968dec4 HDDS-2619. Fix Sonar issues in ReconTaskControllerImpl add afda1c4 HDDS-2655. Use pre-compiled Pattern in NetUtils#normalize add 7524ee4 HDDS-2658. Insight log level reset does not work add 3480c89 HDDS-2654. Conditionally enable profiling at the kernel level add a25b2b4 HDDS-2639. TestTableCacheImpl is flaky (#278) add f141097 HDDS-2657. Key get command creates the output file even in case of KEY_NOT_FOUND (#290) add 4d42714 HDDS-2455. Implement MiniOzoneHAClusterImpl#getOMLeader (#137) add 87e02a3 HDDS-2573. Handle InterruptedException in KeyOutputStream. (#257) add 9381dd0 HDDS-2637. Handle LeaderNot ready exception in OzoneManager StateMachine and upgrade ratis to latest version. (#276) add 2fb8747 HDDS-2386. Implement incremental ChunkBuffer. (#205) add dbd05b0 HDDS-2620. Fix listMultipartupload API. (#263) add ea6c2c9 HDDS-1722. Use the bindings in ReconSchemaGenerationModule to create Recon SQL tables on startup. (#294) add a3a2bfd HDDS-2571. Handle InterruptedException in SCMPipelineManager (#295) add c9f2b02 HDDS-2656. Prefer execute() over submit() if the returned Future is ignored (#302) add 7b71678 HDDS-2475. Unregister ContainerMetadataScrubberMetrics on thread exit (#286) add 5aa7383 HDDS-2653. Improve executor memory usage in new Freon tests (#284) add f686239 HDDS-2608. Provide command to wait until SCM is out from the safe-mode (#250) add 0ff53ef HDDS-2646. Start acceptance tests only if at least one THREE pipeline is available (#282) add a9990f6 HDDS-2622. Add config parameter for setting limit on total bytes of pending requests in Ratis. (#264) add 8e1e225 HDDS-2667. Prometheus reports invalid metric type add 6c1a9ff Addendum HDDS-2646. Start acceptance tests only if at least one THREE pipeline is available (#282) add b35ad98 HDDS-2506. Remove keyAllocationInfo and replication info from the auditLog (#288) add 4f1869c HDDS-2565. Handle InterruptedException in VolumeSet add 1998983 HDDS-2600. Move chaos test to org.apache.hadoop.ozone.chaos package. (#307) add 2f72319 HDDS-1873. Add API to get last completed times for every Recon task. (#233) add e359e66 HDDS-2666. Suppress loader constraint violation message in TestOzoneFileSystemWithMocks (#303) add 1af4f48 HDDS-2530. Sonar : refactor verifyResourceName in HddsClientUtils to fix Sonar errors (#293) add d18aa99 HDDS-2629. Ozone CLI: CreationTime/modifyTime of volume/bucket/key in… (#272) add 9b07c34 HDDS-2677. Acceptance test may fail despite success status (#312) add 1e18347 HDDS-2570. Handle InterruptedException in ProfileServlet. (#308) add 9593b25 HDDS-1987. Fix listStatus API (#108) add f69cb1e HDDS-2636. Refresh pipeline information in OzoneManager lookupFile call. (#301) add 76ad638 HDDS-2678. Add thread name to log pattern (#315) new 64caf25 debug acceptance test timeout This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (fab0fa5) \ N -- N -- N refs/heads/debug-acceptance-timeout (64caf25) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../hadoop/hdds/scm/client/HddsClientUtils.java | 75 +++--- .../hadoop/hdds/scm/storage/BlockOutputStream.java | 3 +- .../apache/hadoop/hdds/scm/storage/BufferPool.java | 4 +- hadoop-hdds/common/pom.xml | 5 + .../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 5 + .../org/apache/hadoop/hdds/scm/net/NetUtils.java | 19 +- .../apache/hadoop/hdds/utils/db/DBCheckpoint.java | 12 + .../hadoop/hdds/utils/db/RocksDBCheckpoint.java | 11 + .../hadoop/hdds/utils/db/cache/TableCacheImpl.java | 5 +- .../org/apache/hadoop/ozone/OzoneConfigKeys.java | 5 + .../java/org/apache/hadoop/ozone/OzoneConsts.java | 2 + .../apache/hadoop/ozone/common/ChunkBuffer.java | 49 +++- .../common/ChunkBufferImplWithByteBuffer.java | 10 +- .../ozone/common/IncrementalChunkBuffer.java | 267 +++++++++++++++++++++ .../apache/hadoop/ozone/web/utils/JsonUtils.java | 15 +- .../common/src/main/resources/ozone-default.xml | 8 + .../apache/hadoop/hdds/scm/net/TestNetUtils.java | 30 ++- .../hdds/utils/db/cache/TestTableCacheImpl.java | 62 ++--- .../hadoop/ozone/common/TestChunkBuffer.java | 18 ++ .../server/ratis/ContainerStateMachine.java | 7 + .../transport/server/ratis/XceiverServerRatis.java | 24 +- .../ozone/container/common/volume/VolumeSet.java | 1 + .../ozoneimpl/ContainerDataScrubberMetrics.java | 8 +- .../ozoneimpl/ContainerMetadataScanner.java | 18 +- .../ContainerMetadataScrubberMetrics.java | 8 +- .../replication/ReplicationSupervisor.java | 2 +- .../container/common/volume/TestVolumeSet.java | 24 ++ .../container/keyvalue/helpers/TestChunkUtils.java | 4 +- .../ozoneimpl/TestContainerScrubberMetrics.java | 35 ++- .../apache/hadoop/hdds/server/ProfileServlet.java | 7 +- .../hadoop/hdds/server/PrometheusMetricsSink.java | 5 +- .../hdds/server/TestPrometheusMetricsSink.java | 2 +- .../hdds/scm/pipeline/SCMPipelineManager.java | 1 + .../scm/safemode/HealthyPipelineSafeModeRule.java | 2 +- .../hdds/scm/server/StorageContainerManager.java | 7 +- .../hadoop/hdds/scm/cli/SafeModeCommands.java | 1 + .../hdds/scm/cli/SafeModeWaitSubcommand.java | 93 +++++++ .../apache/hadoop/ozone/client/OzoneBucket.java | 11 +- .../org/apache/hadoop/ozone/client/OzoneKey.java | 14 +- .../apache/hadoop/ozone/client/OzoneVolume.java | 9 +- .../hadoop/ozone/client/io/KeyOutputStream.java | 1 + .../hadoop/ozone/client/TestHddsClientUtils.java | 46 ++++ .../apache/hadoop/ozone/om/OMMetadataManager.java | 3 +- .../om/exceptions/OMLeaderNotReadyException.java | 22 +- ...derException.java => OMNotLeaderException.java} | 32 ++- .../ozone/om/ha/OMFailoverProxyProvider.java | 2 +- .../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 2 - .../ozone/om/protocol/OzoneManagerHAProtocol.java | 6 +- ...OzoneManagerProtocolClientSideTranslatorPB.java | 51 +++- .../java/org/apache/hadoop/ozone/TestOmUtils.java | 9 + hadoop-ozone/dist/src/main/compose/testlib.sh | 2 +- .../dist/src/main/dockerlibexec/entrypoint.sh | 8 + .../dist/src/main/smoketest/basic/basic.robot | 4 +- hadoop-ozone/dist/src/shell/conf/log4j.properties | 10 +- .../apache/hadoop/ozone/insight/LogSubcommand.java | 31 +-- .../integration-test/src/test/bin/start-chaos.sh | 2 +- .../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 10 +- .../hadoop/ozone/MiniOzoneHAClusterImpl.java | 22 +- .../hadoop/ozone/TestMiniOzoneHACluster.java | 112 +++++++++ .../ozone/{ => chaos}/MiniOzoneChaosCluster.java | 7 +- .../ozone/{ => chaos}/MiniOzoneLoadGenerator.java | 3 +- .../{ => chaos}/TestMiniChaosOzoneCluster.java | 2 +- .../hadoop/ozone/client/rpc/TestCommitWatcher.java | 4 +- .../client/rpc/TestOzoneAtRestEncryption.java | 14 +- .../client/rpc/TestOzoneRpcClientAbstract.java | 90 ++++--- .../ozone/client/rpc/TestSecureOzoneRpcClient.java | 8 +- .../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 8 +- .../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 235 ++++++++++++++++++ .../hadoop/ozone/om/TestOMRatisSnapshots.java | 20 +- .../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 30 +-- .../hadoop/ozone/ozShell/TestOzoneShellHA.java | 20 +- .../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 172 +++++++++---- .../hadoop/ozone/om/OMDBCheckpointServlet.java | 19 +- .../hadoop/ozone/om/OmMetadataManagerImpl.java | 32 ++- .../org/apache/hadoop/ozone/om/OzoneManager.java | 40 +-- .../hadoop/ozone/om/ratis/OMRatisSnapshotInfo.java | 15 +- .../ozone/om/ratis/OzoneManagerRatisServer.java | 59 +++-- .../ozone/om/ratis/OzoneManagerStateMachine.java | 69 ++++-- .../hadoop/ozone/om/request/RequestAuditor.java | 3 - .../om/snapshot/OzoneManagerSnapshotProvider.java | 10 + ...OzoneManagerProtocolServerSideTranslatorPB.java | 8 +- .../protocolPB/OzoneManagerRequestHandler.java | 1 + .../ozone/web/ozShell/keys/GetKeyHandler.java | 4 +- .../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 256 +++++++++++++++++++- .../ozone/om/ratis/TestOMRatisSnapshotInfo.java | 5 +- .../om/ratis/TestOzoneManagerRatisServer.java | 17 +- .../ozone/om/request/TestOMRequestUtils.java | 17 ++ .../ozone/web/ozShell/TestObjectPrinter.java | 6 +- .../fs/ozone/BasicOzoneClientAdapterImpl.java | 2 +- .../fs/ozone/TestOzoneFileSystemWithMocks.java | 2 + .../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 33 +-- .../hadoop/ozone/recon/ReconControllerModule.java | 4 - .../hadoop/ozone/recon/ReconSchemaManager.java | 55 +++++ .../org/apache/hadoop/ozone/recon/ReconServer.java | 26 +- ...lizationService.java => TaskStatusService.java} | 37 ++- .../recon/recovery/ReconOMMetadataManager.java | 6 + .../recon/recovery/ReconOmMetadataManagerImpl.java | 11 + .../spi/impl/OzoneManagerServiceProviderImpl.java | 32 ++- .../ozone/recon/tasks/ReconTaskController.java | 3 +- .../ozone/recon/tasks/ReconTaskControllerImpl.java | 70 +++--- .../ozone/recon/api/TestTaskStatusService.java | 86 +++++++ .../recon/tasks/TestReconTaskControllerImpl.java | 38 ++- .../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 4 +- .../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 8 +- .../hadoop/ozone/s3/endpoint/RootEndpoint.java | 4 +- .../hadoop/ozone/freon/BaseFreonGenerator.java | 116 ++++++--- .../hadoop/ozone/freon/RandomKeyGenerator.java | 2 +- pom.xml | 3 +- 108 files changed, 2353 insertions(+), 626 deletions(-) create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/IncrementalChunkBuffer.java copy hadoop-ozone/recon-codegen/src/main/java/org/hadoop/ozone/recon/schema/ReconSchemaDefinition.java => hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/net/TestNetUtils.java (56%) create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/SafeModeWaitSubcommand.java copy hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationException.java => hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/exceptions/OMLeaderNotReadyException.java (65%) rename hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/exceptions/{NotLeaderException.java => OMNotLeaderException.java} (71%) create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestMiniOzoneHACluster.java rename hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/{ => chaos}/MiniOzoneChaosCluster.java (97%) rename hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/{ => chaos}/MiniOzoneLoadGenerator.java (99%) rename hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/{ => chaos}/TestMiniChaosOzoneCluster.java (99%) create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconSchemaManager.java copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/{UtilizationService.java => TaskStatusService.java} (65%) create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestTaskStatusService.java --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org