[jira] [Created] (HDDS-1332) Skip flaky test - testStartStopDatanodeStateMachine
Arpit Agarwal created HDDS-1332:
---
Summary: Skip flaky test - testStartStopDatanodeStateMachine
Key: HDDS-1332
URL: https://issues.apache.org/jira/browse/HDDS-1332
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Arpit Agarwal
testStartStopDatanodeStateMachine fails frequently in Jenkins. It also seems to
have a timing issue which may be different from the Jenkins failure.
E.g. If I add a 10 second sleep as below I can get the test to fail 100%.
{code}
@@ -163,6 +163,7 @@ public void testStartStopDatanodeStateMachine() throws
IOException,
try (DatanodeStateMachine stateMachine =
new DatanodeStateMachine(getNewDatanodeDetails(), conf, null)) {
stateMachine.startDaemon();
+ Thread.sleep(10_000L);
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-1326) putkey operation failed with java.lang.ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HDDS-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoyu Yao resolved HDDS-1326.
--
Resolution: Duplicate
> putkey operation failed with java.lang.ArrayIndexOutOfBoundsException
> -
>
> Key: HDDS-1326
> URL: https://issues.apache.org/jira/browse/HDDS-1326
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Blocker
>
> steps taken :
> ---
> # trying to write key in 40 node cluster.
> # write failed.
> client output
> ---
>
> {noformat}
> e530-491c-ab03-3b1c34d1a751:c80390,
> 974a806d-bf7d-4f1b-adb4-d51d802d368a:c80390,
> 469bd8c4-5da2-43bb-bc4b-7edd884931e5:c80390]
> 2019-03-22 10:56:19,592 [main] WARN - Encountered exception {}
> java.io.IOException: Unexpected Storage Container Exception:
> java.util.concurrent.ExecutionException:
> java.util.concurrent.CompletionException:
> org.apache.ratis.protocol.StateMachineException:
> org.apache.hadoop.hdds.scm.container.common.helpers.ContainerNotOpenException
> from Server 5d3eb91f-e530-491c-ab03-3b1c34d1a751: Container 1269 in CLOSED
> state
> at
> org.apache.hadoop.hdds.scm.storage.BlockOutputStream.close(BlockOutputStream.java:511)
> at
> org.apache.hadoop.ozone.client.io.BlockOutputStreamEntry.close(BlockOutputStreamEntry.java:144)
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.handleFlushOrClose(KeyOutputStream.java:565)
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.handleWrite(KeyOutputStream.java:329)
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.write(KeyOutputStream.java:273)
> at
> org.apache.hadoop.ozone.client.io.OzoneOutputStream.write(OzoneOutputStream.java:49)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:96)
> at
> org.apache.hadoop.ozone.web.ozShell.keys.PutKeyHandler.call(PutKeyHandler.java:111)
> at
> org.apache.hadoop.ozone.web.ozShell.keys.PutKeyHandler.call(PutKeyHandler.java:53)
> at picocli.CommandLine.execute(CommandLine.java:919)
> at picocli.CommandLine.access$700(CommandLine.java:104)
> at picocli.CommandLine$RunLast.handle(CommandLine.java:1083)
> at picocli.CommandLine$RunLast.handle(CommandLine.java:1051)
> at
> picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:959)
> at picocli.CommandLine.parseWithHandlers(CommandLine.java:1242)
> at picocli.CommandLine.parseWithHandler(CommandLine.java:1181)
> at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:61)
> at org.apache.hadoop.ozone.web.ozShell.Shell.execute(Shell.java:82)
> at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:52)
> at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:93)
> Caused by: java.util.concurrent.ExecutionException:
> java.util.concurrent.CompletionException:
> org.apache.ratis.protocol.StateMachineException:
> org.apache.hadoop.hdds.scm.container.common.helpers.ContainerNotOpenException
> from Server 5d3eb91f-e530-491c-ab03-3b1c34d1a751: Container 1269 in CLOSED
> state
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hadoop.hdds.scm.storage.BlockOutputStream.waitOnFlushFutures(BlockOutputStream.java:529)
> at
> org.apache.hadoop.hdds.scm.storage.BlockOutputStream.handleFlush(BlockOutputStream.java:481)
> at
> org.apache.hadoop.hdds.scm.storage.BlockOutputStream.close(BlockOutputStream.java:496)
> ... 19 more
> Caused by: java.util.concurrent.CompletionException:
> org.apache.ratis.protocol.StateMachineException:
> org.apache.hadoop.hdds.scm.container.common.helpers.ContainerNotOpenException
> from Server 5d3eb91f-e530-491c-ab03-3b1c34d1a751: Container 1269 in CLOSED
> state
> at
> org.apache.ratis.client.impl.RaftClientImpl.handleStateMachineException(RaftClientImpl.java:402)
> at
> org.apache.ratis.client.impl.RaftClientImpl.lambda$sendAsync$3(RaftClientImpl.java:198)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.apache.ratis.client.impl.RaftClientImpl$PendingAsyncRequest.setReply(RaftClientImpl.java:95)
> at
> org.apache.ratis.client.impl.RaftClientImpl$PendingAsyncRequest.setReply(RaftClientImpl.java:75)
> at
> org.apache.ratis.util.SlidingWindow$RequestMap.setReply(SlidingWindow.java:127)
> at
> org.apache.ratis.util.SlidingWindow$Client.receiveReply(SlidingWindow.java:279)
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1084/ [Mar 22, 2019 10:26:24 AM] (stevel) HADOOP-16196. Path Parameterize Comparable. [Mar 22, 2019 10:29:27 AM] (stevel) HADOOP-16181. HadoopExecutors shutdown Cleanup. [Mar 22, 2019 10:35:30 AM] (stevel) HADOOP-16147. Allow CopyListing sequence file keys and values to be more [Mar 22, 2019 11:28:34 AM] (templedf) YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with [Mar 22, 2019 11:42:00 AM] (stevel) HADOOP-16201: S3AFileSystem#innerMkdirs builds needless lists (#636) [Mar 22, 2019 6:29:56 PM] (arp) HDDS-1323. Ignore unit test TestFailureHandlingByClient. Contributed by [Mar 22, 2019 6:45:39 PM] (gifuma) YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor [Mar 22, 2019 9:36:29 PM] (github) HDDS-1205. Refactor ReplicationManager to handle QUASI_CLOSED [Mar 22, 2019 10:08:28 PM] (7813154+ajayydv) HDDS-1291. Set OmKeyArgs#refreshPipeline flag properly to avoid reading [Mar 22, 2019 10:15:01 PM] (bharat) HDDS-1302. Fix SCM CLI does not list container with id 1. [Mar 22, 2019 10:53:12 PM] (ajay) HDDS-1310. In datanode once a container becomes unhealthy, datanode [Mar 22, 2019 11:20:44 PM] (ajay) Revert "HDDS-1310. In datanode once a container becomes unhealthy, [Mar 22, 2019 11:44:12 PM] (bharat) HDDS-918. Expose SCMMXBean as a MetricsSource. Contributed by Siddharth -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore org.apache.hadoop.yarn.server.timelineservice.documentstore.collection.document.entity.TimelineEntityDocument.setEvents(Map) makes inefficient use of keySet iterator instead of entrySet iterator At TimelineEntityDocument.java:keySet iterator instead of entrySet iterator At TimelineEntityDocument.java:[line 159] org.apache.hadoop.yarn.server.timelineservice.documentstore.collection.document.entity.TimelineEntityDocument.setMetrics(Map) makes inefficient use of keySet iterator instead of entrySet iterator At TimelineEntityDocument.java:keySet iterator instead of entrySet iterator At TimelineEntityDocument.java:[line 142] Unread field:TimelineEventSubDoc.java:[line 56] Unread field:TimelineMetricSubDoc.java:[line 44] Switch statement found in org.apache.hadoop.yarn.server.timelineservice.documentstore.collection.document.flowrun.FlowRunDocument.aggregate(TimelineMetric, TimelineMetric) where default case is missing At FlowRunDocument.java:TimelineMetric) where default case is missing At FlowRunDocument.java:[lines 121-136] org.apache.hadoop.yarn.server.timelineservice.documentstore.collection.document.flowrun.FlowRunDocument.aggregateMetrics(Map) makes inefficient use of keySet iterator instead of entrySet iterator At FlowRunDocument.java:keySet iterator instead of entrySet iterator At FlowRunDocument.java:[line 103] Possible doublecheck on org.apache.hadoop.yarn.server.timelineservice.documentstore.reader.cosmosdb.CosmosDBDocumentStoreReader.client in new org.apache.hadoop.yarn.server.timelineservice.documentstore.reader.cosmosdb.CosmosDBDocumentStoreReader(Configuration) At CosmosDBDocumentStoreReader.java:new org.apache.hadoop.yarn.server.timelineservice.documentstore.reader.cosmosdb.CosmosDBDocumentStoreReader(Configuration) At CosmosDBDocumentStoreReader.java:[lines 73-75] Possible doublecheck on org.apache.hadoop.yarn.server.timelineservice.documentstore.writer.cosmosdb.CosmosDBDocumentStoreWriter.client in new org.apache.hadoop.yarn.server.timelineservice.documentstore.writer.cosmosdb.CosmosDBDocumentStoreWriter(Configuration) At CosmosDBDocumentStoreWriter.java:new org.apache.hadoop.yarn.server.timelineservice.documentstore.writer.cosmosdb.CosmosDBDocumentStoreWriter(Configuration) At CosmosDBDocumentStoreWriter.java:[lines 66-68] Failed junit tests : hadoop.ha.TestZKFailoverController hadoop.hdfs.server.datanode.TestFsDatasetCache hadoop.hdfs.web.TestWebHdfsTimeouts cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1084/artifact/out/diff-compile-cc-root.txt [4.0K] jav
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/ No changes -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml FindBugs : module:hadoop-common-project/hadoop-common Class org.apache.hadoop.fs.GlobalStorageStatistics defines non-transient non-serializable instance field map In GlobalStorageStatistics.java:instance field map In GlobalStorageStatistics.java FindBugs : module:hadoop-hdfs-project/hadoop-hdfs Dead store to state in org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Saver.save(OutputStream, INodeSymlink) At FSImageFormatPBINode.java:org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Saver.save(OutputStream, INodeSymlink) At FSImageFormatPBINode.java:[line 623] FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client Boxed value is unboxed and then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:[line 335] Failed junit tests : hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.TestRollingUpgrade hadoop.registry.secure.TestSecureLogins hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 hadoop.yarn.client.api.impl.TestAMRMProxy cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-compile-cc-root-jdk1.7.0_95.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-compile-javac-root-jdk1.7.0_95.txt [328K] cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-compile-cc-root-jdk1.8.0_191.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-compile-javac-root-jdk1.8.0_191.txt [308K] checkstyle: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-patch-shellcheck.txt [72K] shelldocs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/whitespace-tabs.txt [1.2M] xml: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/xml.txt [20K] findbugs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html [8.0K] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html [8.0K] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/269/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice
