kasakrisz commented on code in PR #4000:
URL: https://github.com/apache/hive/pull/4000#discussion_r1091850767
##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -277,23 +277,31 @@ private static Statistics collectStatistics(HiveConf
conf, PrunedPartitionList p
boolean metaTable = table.getMetaTable() != null;
if (!table.isPartitioned()) {
+ long ds, nr, fs;
+ if (table.isNonNative() &&
table.getStorageHandler().canProvideBasicStatistics()) {
+ Map<String, String> icebergBasicStatMap =
table.getStorageHandler().getBasicStatistics(Partish.buildFor(table));
+ ds = Long.parseLong(icebergBasicStatMap.get("totalSize"));
+ nr = Long.parseLong(icebergBasicStatMap.get("numRows"));
+ fs = Long.parseLong(icebergBasicStatMap.get("numFiles"));
Review Comment:
Should a null check be added here and/or handle NumberFormatException?
What would be the stats values if it can not be extracted via storage
handler?
In case of non-storage handler path there are so called `enhancers` to
estimate and fix the values:
```
basicStatsFactory.addEnhancer(new
BasicStats.RowNumEstimator(estimateRowSizeFromSchema(conf, schema)));
basicStatsFactory.addEnhancer(new BasicStats.SetMinRowNumber01());
```
##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -277,23 +277,31 @@ private static Statistics collectStatistics(HiveConf
conf, PrunedPartitionList p
boolean metaTable = table.getMetaTable() != null;
if (!table.isPartitioned()) {
+ long ds, nr, fs;
Review Comment:
Could you please declare each variable in separate lines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
