[ https://issues.apache.org/jira/browse/HIVE-22977?focusedWorklogId=841210&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-841210 ]
ASF GitHub Bot logged work on HIVE-22977: ----------------------------------------- Author: ASF GitHub Bot Created on: 23/Jan/23 18:33 Start Date: 23/Jan/23 18:33 Worklog Time Spent: 10m Work Description: SourabhBadhya commented on code in PR #3801: URL: https://github.com/apache/hive/pull/3801#discussion_r1084408881 ########## ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorChain.java: ########## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hive.ql.txn.compactor; + +import org.apache.hadoop.hive.common.ValidWriteIdList; +import org.apache.hadoop.hive.conf.HiveConf; +import org.apache.hadoop.hive.metastore.api.Partition; +import org.apache.hadoop.hive.metastore.api.StorageDescriptor; +import org.apache.hadoop.hive.metastore.api.Table; +import org.apache.hadoop.hive.metastore.txn.CompactionInfo; +import org.apache.hadoop.hive.ql.io.AcidDirectory; +import org.apache.hadoop.hive.ql.metadata.HiveException; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; + +/** + * Runs different compactions based on the order provided in the list. + * Mainly used for fall back mechanism for Merge compaction. + */ +final class CompactorChain implements Compactor { Review Comment: Renamed it to `CompactorPipeline`. Also took some good ideas from pipeline design pattern and implemented here. Issue Time Tracking ------------------- Worklog Id: (was: 841210) Time Spent: 7h 20m (was: 7h 10m) > Merge delta files instead of running a query in major/minor compaction > ---------------------------------------------------------------------- > > Key: HIVE-22977 > URL: https://issues.apache.org/jira/browse/HIVE-22977 > Project: Hive > Issue Type: Improvement > Reporter: László Pintér > Assignee: Sourabh Badhya > Priority: Major > Labels: pull-request-available > Attachments: HIVE-22977.01.patch, HIVE-22977.02.patch > > Time Spent: 7h 20m > Remaining Estimate: 0h > > [Compaction Optimiziation] > We should analyse the possibility to move a delta file instead of running a > major/minor compaction query. > Please consider the following use cases: > - full acid table but only insert queries were run. This means that no > delete delta directories were created. Is it possible to merge the delta > directory contents without running a compaction query? > - full acid table, initiating queries through the streaming API. If there > are no abort transactions during the streaming, is it possible to merge the > delta directory contents without running a compaction query? -- This message was sent by Atlassian Jira (v8.20.10#820010)