答复: NN run progressively slower

2018-09-29 Thread Lin,Yiqun(vip.com)
is. Now we have reduced the young gen size and to see if NN is stable enough then, :). Thanks Yiqun -邮件原件- 发件人: Brahma Reddy Battula [mailto:brahmareddy.batt...@huawei.com] 发送时间: 2018年9月27日 22:49 收件人: 林意群[运营中心]; Wei-Chiu Chuang 抄送: Hdfs-dev 主题: RE: NN run progressively slower Looks you are

RE: NN run progressively slower

2018-09-27 Thread Brahma Reddy Battula
ra's which are fixed later to 2.6.0) like decommission nodes,... v) Any recursive calls are very frequently? -Original Message- From: Lin,Yiqun(vip.com) [mailto:yiqun01@vipshop.com] Sent: Wednesday, September 26, 2018 10:26 AM To: Wei-Chiu Chuang Cc: Hdfs-dev Subject: 答复: NN run progressiv

答复: NN run progressively slower

2018-09-25 Thread Lin,Yiqun(vip.com)
ime frame. We have customers at/beyond your scale in your version but >I don't think I've heard similar symptoms. Thanks for sharing this. Thanks Yiqun 发件人: Wei-Chiu Chuang [mailto:weic...@cloudera.com] 发送时间: 2018年9月26日 11:14 收件人: 林意群[运营中心] 抄送: Hdfs-dev 主题: Re: NN run progressively slower Y

Re: NN run progressively slower

2018-09-25 Thread Wei-Chiu Chuang
Yiqun, Is this related to HDFS-9260? Note that HDFS-9260 was backported since CDH5.7 and above. I'm interested to learn more. Did you observe clients failing to close file due to insufficient number of block replicas? Did NN fail over? Did you have gc logging enabled? Any chance to take a heap

答复: NN run progressively slower

2018-09-25 Thread Lin,Yiqun(vip.com)
9月26日 1:11 收件人: 林意群[运营中心] 抄送: Hdfs-dev 主题: Re: NN run progressively slower Are you using CMS? How big is young gen? How often does the NN do young gen collection when it is slow? On Tue, Sep 25, 2018 at 4:04 AM Lin,Yiqun(vip.com<http://vip.com>) mailto:yiqun01@vipshop.com>> wr

Re: NN run progressively slower

2018-09-25 Thread Kihwal Lee
Are you using CMS? How big is young gen? How often does the NN do young gen collection when it is slow? On Tue, Sep 25, 2018 at 4:04 AM Lin,Yiqun(vip.com) wrote: > Hi hdfs developers: > > We meet a bad problem after rolling upgrade our hadoop version from > 2.5.0-cdh5.3.2 to 2.6.0-cdh5.13.1.

NN run progressively slower

2018-09-25 Thread Lin,Yiqun(vip.com)
Hi hdfs developers: We meet a bad problem after rolling upgrade our hadoop version from 2.5.0-cdh5.3.2 to 2.6.0-cdh5.13.1. The problem is that we find NN running slow periodically (around a week). Concretely to say, For example, we startup NN on Monday, it will run fast. But time coming to