[ https://issues.apache.org/jira/browse/HIVE-26788?focusedWorklogId=833059&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-833059 ]
ASF GitHub Bot logged work on HIVE-26788: ----------------------------------------- Author: ASF GitHub Bot Created on: 13/Dec/22 14:29 Start Date: 13/Dec/22 14:29 Worklog Time Spent: 10m Work Description: SourabhBadhya commented on code in PR #3812: URL: https://github.com/apache/hive/pull/3812#discussion_r1047226607 ########## ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java: ########## @@ -428,6 +430,11 @@ protected Boolean findNextCompactionAndExecute(boolean collectGenericStats, bool AcidMetricService.updateMetricsFromWorker(ci.dbname, ci.tableName, ci.partName, ci.type, dir.getCurrentDirectories().size(), dir.getDeleteDeltas().size(), conf, msc); + + // Get columns with stats at the end. + // This will save computations if compaction has failed. + columnList = msc.findColumnsWithStats(CompactionInfo.compactionInfoToStruct(ci)); Review Comment: > Maybe would be even better to do it in StatsUpdater Implemented this way. Done Issue Time Tracking ------------------- Worklog Id: (was: 833059) Time Spent: 3h 10m (was: 3h) > Update stats of table/partition using noscan operation when stats autogather > is enabled > --------------------------------------------------------------------------------------- > > Key: HIVE-26788 > URL: https://issues.apache.org/jira/browse/HIVE-26788 > Project: Hive > Issue Type: Improvement > Reporter: Sourabh Badhya > Assignee: Sourabh Badhya > Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > Currently, statistics are always updated for major compaction and never > updated for minor compaction since minor compaction performs little updates > on the statistics (such as number of files in table/partition & total size of > the table/partition). It is better to utilize NOSCAN operation for both > these compaction since NOSCAN operations performs faster update of statistics > and updates the relevant fields such as number of files & total sizes of the > table/partitions if stats autogather is enabled. If autostats gather is > disabled then go for full update of statistics. -- This message was sent by Atlassian Jira (v8.20.10#820010)