Re: Restrict Frequency of BlockReport To Namenode startup and failover

2020-02-06 Thread Surendra Singh Lilhore
Thanks Wei-Chiu, I feel now IBR is more stable in branch 3.x. If BR is just added to prevent bugs in IBR, I feel we should fix such bug in IBR. Adding one new functionality to prevent bug in other is not good. I also thing, DN should send BR in failure and process start scenario only. -Surendra

Re: Restrict Frequency of BlockReport To Namenode startup and failover

2020-02-06 Thread Ayush Saxena
Hi Wei-Chiu, Thanx for the response. Yes, We are talking about the FBR only. Increasing the frequency limits the problem, but doesn’t seems to be solving it. With increasing cluster size, the frequency needs to be increased, and we cannot increase it indefinitely, as in some case FBR is needed. O

Re: Restrict Frequency of BlockReport To Namenode startup and failover

2020-02-06 Thread Wei-Chiu Chuang
Hey Ayush, Thanks a lot for your proposal. Do you mean the Full Block Report that is sent out every 6 hours per DataNode? Someone told me they reduced the frequency of FBR to 24 hours and it seems okay. One of the purposes of FBR was to prevent bugs in incremental block report implementation. In

[jira] [Created] (HDFS-15156) In-Place Erasure Coding should factor in storage quota validations.

2020-02-06 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDFS-15156: Summary: In-Place Erasure Coding should factor in storage quota validations. Key: HDFS-15156 URL: https://issues.apache.org/jira/browse/HDFS-15156 Project: H

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2020-02-06 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1403/ [Feb 5, 2020 1:44:05 AM] (jhung) YARN-10116. Expose diagnostics in RMAppManager summary -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsyste

[jira] [Created] (HDFS-15155) writeIoRate of DataNodeVolumeMetrics is never used

2020-02-06 Thread Haibin Huang (Jira)
Haibin Huang created HDFS-15155: --- Summary: writeIoRate of DataNodeVolumeMetrics is never used Key: HDFS-15155 URL: https://issues.apache.org/jira/browse/HDFS-15155 Project: Hadoop HDFS Issue Ty

Apache Hadoop qbt Report: branch2.10+JDK7 on Linux/x86

2020-02-06 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/589/ [Feb 5, 2020 7:16:09 PM] (jhung) YARN-10116. Expose diagnostics in RMAppManager summary [Feb 5, 2020 7:33:30 PM] (kihwal) HADOOP-15787. TestIPC.testRTEDuringConnectionSetup fails with jdk8u242. -