----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19903/ -----------------------------------------------------------
Review request for hive. Bugs: HIVE-6809 https://issues.apache.org/jira/browse/HIVE-6809 Repository: hive-git Description ------- In busy hadoop system, dropping many of partitions takes much more time than expected. In hive-0.11.0, removing 1700 partitions by single partial spec took 90 minutes, which is reduced to 3 minutes when deleteData is set false. I couldn't test this in recent hive, which has HIVE-6256 but if the time-taking part is mostly from removing directories, it seemed not helpful to reduce whole processing time. Diffs ----- itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 47e94ea metastore/if/hive_metastore.thrift eef1b80 metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 2a1b4d7 metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 9567874 metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp b18009c metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 4f051af metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php c79624f metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote fdedb57 metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 23679be metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 56c23e6 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 27077b4 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 664dccd metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 0c2209b metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 6a0eabe metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java e0de0e0 metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 5c00aa1 metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 5025b83 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 5cb030c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 5d5fa78 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java a73a5e0 ql/src/java/org/apache/hadoop/hive/ql/plan/DropTableDesc.java ba30e1f ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionSpec.java PRE-CREATION ql/src/test/queries/clientpositive/drop_partitions_partialspec.q PRE-CREATION ql/src/test/results/clientpositive/drop_partitions_partialspec.q.out PRE-CREATION Diff: https://reviews.apache.org/r/19903/diff/ Testing ------- Thanks, Navis Ryu