[ https://issues.apache.org/jira/browse/HIVE-26479?focusedWorklogId=802855&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-802855 ]
ASF GitHub Bot logged work on HIVE-26479: ----------------------------------------- Author: ASF GitHub Bot Created on: 23/Aug/22 13:31 Start Date: 23/Aug/22 13:31 Worklog Time Spent: 10m Work Description: deniskuzZ commented on code in PR #3528: URL: https://github.com/apache/hive/pull/3528#discussion_r952632657 ########## ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java: ########## @@ -272,5 +277,23 @@ static void removeFilesForMmTable(HiveConf conf, AcidDirectory dir) throws IOExc fs.delete(dead, true); } } + + static void overrideConfProps(HiveConf conf, CompactionInfo ci, Map<String, String> properties) { + for (String key : properties.keySet()) { + if (key.startsWith(COMPACTOR_PREFIX)) { + String property = key.substring(10); // 10 is the length of "compactor." We only keep the rest. + conf.set(property, properties.get(key)); + } + } + + // Give preference to properties coming from compaction + // over table properties + for (String key : ci.getPropertiesMap().keySet()) { Review Comment: could we deduplicate this part of the code? something like: ```` static void overrideConfProps(HiveConf conf, CompactionInfo ci, Map<String, String> properties) { Stream.of(properties, new StringableMap(ci.properties)) .filter(Objects::nonNull) .flatMap(map -> map.entrySet().stream()) .filter(entry -> entry.getKey().startsWith(COMPACTOR_PREFIX)) .forEach(entry -> { String property = entry.getKey().substring(COMPACTOR_PREFIX.length()); conf.set(property, entry.getValue()); }); } ```` Issue Time Tracking ------------------- Worklog Id: (was: 802855) Time Spent: 1h 40m (was: 1.5h) > Add ability to set parameters for query-based compaction > -------------------------------------------------------- > > Key: HIVE-26479 > URL: https://issues.apache.org/jira/browse/HIVE-26479 > Project: Hive > Issue Type: Improvement > Reporter: Sourabh Badhya > Assignee: Sourabh Badhya > Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > [HIVE-13354|https://issues.apache.org/jira/browse/HIVE-13354] introduced the > ability to set some parameters for the compaction through table properties, > like the mapper memory size or compaction thresholds. This could be useful > for the query-based compaction as well, for example if the insert of the > query-based compaction is failing, we would have a possibility to tune the > compaction run directly. First it should be investigated which properties are > possible and would make sense to set for the query base compaction. Then > implement this feature for the query-based compaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)