[ https://issues.apache.org/jira/browse/HDFS-16896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17683985#comment-17683985 ]
ASF GitHub Bot commented on HDFS-16896: --------------------------------------- simbadzina commented on PR #5322: URL: https://github.com/apache/hadoop/pull/5322#issuecomment-1416241192 I ran each of the failing unit test classes in Intellij individually. > [2023-01-26T21:52:13.365Z] Reason | Tests [2023-01-26T21:52:13.365Z] Failed junit tests | hadoop.hdfs.server.datanode.TestDiskError [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.namenode.TestNameNodeReconfigure [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.namenode.TestReencryption [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.datanode.TestDataNodeMetricsLogger [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.datanode.TestDataNodeReconfiguration [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.namenode.TestNameNodeMXBean [2023-01-26T21:52:13.365Z] | hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes They all passed with your PR. So the failures are just test flakiness. > HDFS Client hedged read has increased failure rate than without hedged read > --------------------------------------------------------------------------- > > Key: HDFS-16896 > URL: https://issues.apache.org/jira/browse/HDFS-16896 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client > Reporter: Tom McCormick > Assignee: Tom McCormick > Priority: Major > Labels: pull-request-available > > When hedged read is enabled by HDFS client, we see an increased failure rate > on reads. > *stacktrace* > > {code:java} > Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain > block: BP-1183972111-10.197.192.88-1590025572374:blk_17114848218_16043459722 > file=/data/tracking/streaming/AdImpressionEvent/daily/2022/07/18/compaction_1/part-r-1914862.1658217125623.1362294472.orc > at > org.apache.hadoop.hdfs.DFSInputStream.refetchLocations(DFSInputStream.java:1077) > at > org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1060) > at > org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1039) > at > org.apache.hadoop.hdfs.DFSInputStream.hedgedFetchBlockByteRange(DFSInputStream.java:1365) > at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1572) > at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1535) > at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:121) > at > org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:112) > at > org.apache.hadoop.fs.RetryingInputStream.lambda$readFully$3(RetryingInputStream.java:172) > at org.apache.hadoop.fs.RetryPolicy.lambda$run$0(RetryPolicy.java:137) > at org.apache.hadoop.fs.NoOpRetryPolicy.run(NoOpRetryPolicy.java:36) > at org.apache.hadoop.fs.RetryPolicy.run(RetryPolicy.java:136) > at > org.apache.hadoop.fs.RetryingInputStream.readFully(RetryingInputStream.java:168) > at > org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:112) > at > org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:112) > at > io.trino.plugin.hive.orc.HdfsOrcDataSource.readInternal(HdfsOrcDataSource.java:76) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org