----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69155/#review210029 -----------------------------------------------------------
Thanks for the patch Barna! Indeed this will be a sizeable memory saving! common/src/java/org/apache/hadoop/hive/common/HiveConfProperties.java Lines 56 (patched) <https://reviews.apache.org/r/69155/#comment294670> Why are we using interner here? common/src/java/org/apache/hadoop/hive/common/HiveConfProperties.java Lines 84 (patched) <https://reviews.apache.org/r/69155/#comment294669> Can we use interned.getProperty(key, default)? common/src/java/org/apache/hadoop/hive/common/HiveConfProperties.java Lines 198 (patched) <https://reviews.apache.org/r/69155/#comment294671> I think there are situations when this is not true, if we overwrite something the size will be smaller? - Peter Vary On okt. 25, 2018, 8:31 de, Barnabas Maidics wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69155/ > ----------------------------------------------------------- > > (Updated okt. 25, 2018, 8:31 de) > > > Review request for hive. > > > Repository: hive-git > > > Description > ------- > > The issue is that every Hive task has to load its own version of HiveConf. > When running with a large number of cores per executor (HoS), there is a > significant (~10%) amount of memory wasted due to this duplication. > See more: https://issues.apache.org/jira/browse/HIVE-20760 > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/common/HiveConfProperties.java > PRE-CREATION > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 07d5205bed > common/src/test/org/apache/hadoop/hive/conf/TestHiveConfProperties.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/69155/diff/1/ > > > Testing > ------- > > Created unit tests for the new Properties implementation. > Tested multiple queries output. > > > Thanks, > > Barnabas Maidics > >
