[jira] [Created] (HDFS-12744) More logs when short-circuit read is failed and disabled
Weiwei Yang created HDFS-12744:
--
Summary: More logs when short-circuit read is failed and disabled
Key: HDFS-12744
URL: https://issues.apache.org/jira/browse/HDFS-12744
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Weiwei Yang
Assignee: Weiwei Yang
Short-circuit read (SCR) failed with following error
{noformat}
2017-10-21 16:42:28,024 WARN [B.defaultRpcServer.handler=7,queue=7,port=16020]
impl.BlockReaderFactory: BlockReaderFactory(xxx): unknown response code ERROR
while attempting to set up short-circuit access. Block xxx is not valid
{noformat}
then short-circuit read is disabled for *10 minutes* without any warning
message given in the log. This causes us spent some more time to figure out why
we had a long time window that SCR was not working. Propose to add a warning
log (other places already did) to indicate SCR is disabled and some more
logging in DN to display what happened.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/573/ No changes -1 overall The following subsystems voted -1: asflicense findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-tools/hadoop-resourceestimator Dead store to jobHistory in org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.getHistoryResourceSkyline(String, String) At ResourceEstimatorService.java:org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.getHistoryResourceSkyline(String, String) At ResourceEstimatorService.java:[line 196] Incorrect lazy initialization and update of static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.skylineStore in new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:of static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.skylineStore in new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[lines 78-82] Write to static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.config from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[line 80] Write to static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.gson from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[line 106] Write to static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.logParser from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[line 86] Write to static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.rleType from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[line 108] Write to static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.skylineStore from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[line 82] Write to static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.skylineStoreType from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[line 111] Write to static field org.apache.hadoop.resourceestimator.service.ResourceEstimatorService.solver from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:from instance method new org.apache.hadoop.resourceestimator.service.ResourceEstimatorService() At ResourceEstimatorService.java:[line 92] Found reliance on default encoding in org.apache.hadoop.resourceestimator.translator.impl.BaseLogParser.parseStream(InputStream):in org.apache.hadoop.resourceestimator.translator.impl.BaseLogParser.parseStream(InputStream): new java.io.InputStreamReader(InputStream) At BaseLogParser.java:[line 104] org.apache.hadoop.resourceestimator.translator.impl.LogParserUtil.parseLog(String) may fail to clean up java.io.InputStream Obligation to clean up resource created at LogParserUtil.java:up java.io.InputStream Obligation to clean up resource created at LogParserUtil.java:[line 94] is not discharged Failed junit tests : hadoop.security.TestRaceWhenRelogin hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting TEST-cetest hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer hadoop.yarn.server.TestDiskFailures Timed out junit tests :
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/15/ No changes - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/572/ [Oct 28, 2017 4:13:13 PM] (templedf) YARN-7397. Reduce lock contention in FairScheduler#getAppWeight() [Oct 29, 2017 5:38:44 AM] (sunilg) YARN-7224. Support GPU isolation for docker container. Contributed by ERROR: File 'out/email-report.txt' does not exist - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK9 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java9-linux-x86/7/ [Oct 25, 2017 5:17:56 AM] (haibochen) YARN-7389. Make TestResourceManager Scheduler agnostic. (Robert Kanter [Oct 25, 2017 8:17:21 AM] (aajisaka) HADOOP-14030. PreCommit TestKDiag failure. Contributed by Wei-Chiu [Oct 25, 2017 2:40:33 PM] (aajisaka) HADOOP-14979. Upgrade maven-dependency-plugin to 3.0.2. Contributed by [Oct 25, 2017 3:08:22 PM] (aw) HADOOP-14977. Xenial dockerfile needs ant in main build for findbugs [Oct 25, 2017 5:54:40 PM] (manojpec) HDFS-12544. SnapshotDiff - support diff generation on any snapshot root [Oct 25, 2017 9:11:30 PM] (xiao) HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in [Oct 25, 2017 9:24:22 PM] (arp) HDFS-12579. JournalNodeSyncer should use fromUrl field of [Oct 25, 2017 10:07:50 PM] (subru) YARN-4827. Document configuration of ReservationSystem for [Oct 25, 2017 10:51:27 PM] (subru) HADOOP-14840. Tool to estimate resource requirements of an application [Oct 26, 2017 5:25:10 PM] (rkanter) YARN-7358. TestZKConfigurationStore and TestLeveldbConfigurationStore [Oct 26, 2017 7:10:14 PM] (subu) YARN-5516. Add REST API for supporting recurring reservations. (Sean Po [Oct 26, 2017 10:50:14 PM] (rkanter) YARN-7320. Duplicate LiteralByteStrings in [Oct 27, 2017 12:47:32 AM] (rkanter) YARN-7262. Add a hierarchy into the ZKRMStateStore for delegation token [Oct 27, 2017 2:13:58 AM] (junping_du) Update CHANGES.md and RELEASENOTES for 2.8.2 release. [Oct 27, 2017 2:15:35 AM] (junping_du) Set jdiff stable version to 2.8.2. [Oct 27, 2017 2:30:48 AM] (junping_du) Add several jdiff xml files for 2.8.2 release. [Oct 27, 2017 3:15:19 AM] (wangda) YARN-7307. Allow client/AM update supported resource types via YARN [Oct 27, 2017 9:45:03 AM] (stevel) MAPREDUCE-6977 Move logging APIs over to slf4j in [Oct 27, 2017 2:43:54 PM] (arp) HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by [Oct 27, 2017 3:23:57 PM] (sunilg) YARN-7375. Possible NPE in RMWebapp when HA is enabled and the active [Oct 27, 2017 5:16:38 PM] (rohithsharmaks) YARN-7289. Application lifetime does not work with FairScheduler. [Oct 27, 2017 9:41:37 PM] (yufei) YARN-6505. Define the strings used in SLS JSON input file format. [Oct 27, 2017 9:56:05 PM] (jlowe) YARN-7244. ShuffleHandler is not aware of disks that are added. [Oct 27, 2017 10:36:03 PM] (cdouglas) HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. [Oct 27, 2017 11:46:05 PM] (inigoiri) YARN-7276. Federation Router Web Service fixes. Contributed by Inigo [Oct 28, 2017 1:47:45 AM] (cdouglas) HADOOP-14986. Enforce JDK limitations. [Oct 28, 2017 4:59:27 AM] (cdouglas) HADOOP-14991. Add missing figures to Resource Estimator tool. [Oct 28, 2017 5:56:41 AM] (Arun Suresh) YARN-7178. Add documentation for Container Update API. (asuresh) [Oct 28, 2017 6:08:18 AM] (Arun Suresh) YARN-7299. Fix TestDistributedScheduler. (asuresh) [Oct 28, 2017 4:13:13 PM] (templedf) YARN-7397. Reduce lock contention in FairScheduler#getAppWeight() [Oct 29, 2017 5:38:44 AM] (sunilg) YARN-7224. Support GPU isolation for docker container. Contributed by ERROR: File 'out/email-report.txt' does not exist - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
