I have anecdotally learned of multiple data points where during the upgrading from Hadoop 2 to Hadoop 3, DN heap usage increases to the point where it goes OOM.
Don't have much logs for this issue, but I suspect it's caused by the layout change added in Hadoop 2.8.0. Does anyone else observe the same issue and how do you mitigate this? For now we suggested increasing DN heap size prior to upgrade as part of pre-upgrade checklist. Thanks, Wei-Chiu
