Xianjin YE created SPARK-6030: --------------------------------- Summary: SizeEstimator gives wrong result for Integer object on 64bit JVM with UseCompressedOops on Key: SPARK-6030 URL: https://issues.apache.org/jira/browse/SPARK-6030 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.2.1, 1.1.1, 1.0.2 Reporter: Xianjin YE Priority: Minor Fix For: 1.3.1
Integer on 64bit JVM with UseCompressedOops on is 16bytes (verified by a related article http://www.javaworld.com/article/2077496/testing-debugging/java-tip-130--do-you-know-your-data-size-.html, I created a gist for that code https://gist.github.com/advancedxy/2ae7c9cc7629f3aeb679), however SizeEstimator give 24bytes for Integer. SizeEstimator gives the wrong answer because it alignSize on internal shellSize. For Integer, there is a parent class called Number, which has zero fields. Thus the shellSize for Number is 12bytes but was aligned to 16bytes, which resulted the Integer's shellSize to be 20bytes, aligned to 24bytes. The right path should be 1. Object-> shellSize: 12bytes, realSize: 16bytes 2. Number -> shellSize: 12bytes + 0, realSize: 16bytes 3. Integer -> shellSize: 12bytes + 4bytes(the int value), realSize: 16bytes The fix is rather simple, I will submit a pr later. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org