[ https://issues.apache.org/jira/browse/HDFS-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521432#comment-16521432 ]
Hudson commented on HDFS-13692: ------------------------------- FAILURE: Integrated in Jenkins build Hadoop-precommit-ozone-acceptance #20 (See [https://builds.apache.org/job/Hadoop-precommit-ozone-acceptance/20/]) HDFS-13692. StorageInfoDefragmenter floods log when compacting (yqlin: [https://github.com/apache/hadoop/commit/30728aced4a6b05394b3fc8c613f39fade9cf3c2]) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java > StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet > ---------------------------------------------------------------------- > > Key: HDFS-13692 > URL: https://issues.apache.org/jira/browse/HDFS-13692 > Project: Hadoop HDFS > Issue Type: Improvement > Affects Versions: 3.1.0 > Reporter: Yiqun Lin > Assignee: Bharat Viswanadham > Priority: Minor > Fix For: 3.2.0, 3.1.1, 3.0.4 > > Attachments: HDFS-13692.00.patch > > > StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet. In > {{StorageInfoDefragmenter#scanAndCompactStorages}}, it prints for all the > StorageInfo under each DN. If there are 1k nodes in cluster, and each node > has 10 data dir configured, it will print 10k lines every compact interval > time (10 mins). The log looks large, we could switch log level from INFO to > DEBUG in {{StorageInfoDefragmenter#scanAndCompactStorages}}. > {noformat} > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-329bd988-a558-43a6-b31c-9142548b0179 : > 0.8222276264591439 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-b5505847-1389-4a80-b9d8-876172a83897 : 0.933351976137211 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-ca164b2f-0a2c-4b26-8e99-f0ece0909997 : > 0.9330040998881849 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-89b912ba-339b-45e3-b981-541b22690ccb : > 0.9314626719970249 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-89c0377b-a49c-4288-9304-e104d98de5bd : > 0.9309580852251582 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-5ffad4d2-168a-446d-a92e-ef46a82f26f8 : > 0.8938870614035088 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-eecbbd34-10f4-4647-8710-0f5963da3aaa : > 0.8963103205353998 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-7aafa122-433f-49c8-bf00-11bcdd8ce048 : > 0.8950508004926109 > 2018-06-19 10:18:48,849 INFO [StorageInfoMonitor] > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: StorageInfo > TreeSet fill ratio DS-eb9ba675-9c23-40a1-9241-c314dc0e2867 : > 0.8947356866877415 > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org