Alex Behm has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8110 )
Change subject: IMPALA-5955: Use totalSize tblproperty instead of rawDataSize. ...................................................................... IMPALA-5955: Use totalSize tblproperty instead of rawDataSize. Today, Impala populates the 'rawDataSize' property during COMPUTE STATS for the purpose of extrapolating row counts based on file sizes. After this patch Impala will populate 'totalSize' instead of 'rawDataSize'. The 'rawDataSize' is not populated or used. Intended meaning/use of tblproperties: - rawDataSize' is the estimated in-memory size of a table (without encoding and compression) - 'totalSize' represents the on-disk size Using the fields correctly is important for compatibility with other users of the HMS such as Hive and SparkSQL. For example, SparkSQL relies on the 'totalSize' for join ordering. Testing: - core/hdfs run passed Change-Id: If7c2c4e1e99b297c849f9f0d18b2bef34ad811c6 Reviewed-on: http://gerrit.cloudera.org:8080/8110 Tested-by: Impala Public Jenkins Reviewed-by: Alex Behm <alex.b...@cloudera.com> --- M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test 4 files changed, 25 insertions(+), 25 deletions(-) Approvals: Impala Public Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/8110 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If7c2c4e1e99b297c849f9f0d18b2bef34ad811c6 Gerrit-Change-Number: 8110 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins
