tell which commit increases heap usage worse during upgrade.
>
>
>
> On Tue, Oct 6, 2020 at 3:01 PM Kihwal Lee wrote:
>
>> Which layout change are you referring to? The only layout change I know
>> of was done in 2.7, IIRC. We backported that to 2.6 and did not see any
>
To be clear, we are running 2.8 and 2.10. Although we don't see any issues,
I am curious whether the change in heap usage is amplified on dense
datanodes.
Kihwal
On Tue, Oct 6, 2020 at 5:00 PM Kihwal Lee wrote:
> Which layout change are you referring to? The only layout change I know
Which layout change are you referring to? The only layout change I know of
was done in 2.7, IIRC. We backported that to 2.6 and did not see any
adverse effects at that time.
Is datanode using more heap all the time? Or is it running into trouble
when generating full block reports?
Kihwal
On Mon,
Hi Wei-Chiu,
We have experience with 5,000 - 6,000 node clusters. Although it ran/runs
fine, any heavy hitter activities such as decommissioning needed to be
carefully planned. In terms of files and blocks, we have multiple
clusters running stable with over 500M files and blocks. Some at over
It looks like the NN is having trouble after reading in the VERSION file, which
contains the new layout version. We can continue the discussion in the jira.
Kihwal
From: Ravi Prakash
To: Dinesh Kumar Prabakaran
Cc: user ; hdfs-dev
Sent: Thursday, October 13, 2016 3:29 PM
Subject: R
+1
From: Brahma Reddy Battula
To: "Gangumalla, Uma" ; Rakesh Radhakrishnan
; Sijie Guo
Cc: "d...@bookkeeper.apache.org" ; Uma gangumalla
; Vinayakumar B ;
"hdfs-...@hadoop.apache.org" ;
"user@hadoop.apache.org" ; "u...@bookkeeper.apache.org"
Sent: Thursday, July 28, 2016 4:21 AM
For the ext3 bug Colin mentioned, see
https://bugzilla.redhat.com/show_bug.cgi?id=592961. This was fixed in
2.6.32 and backported in RHEL 5.4 (or CENTOS). This has more to do with
file data and affects NN more. Since NN preallocates blocks for edits,
almost all data writes are done without modifyin
