[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=858776&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858776 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 24/Apr/23 20:02 Start Date: 24/Apr/23 20:02 Worklog Time Spent: 10m Work Description: deniskuzZ merged PR #4123: URL: https://github.com/apache/hive/pull/4123 Issue Time Tracking --- Worklog Id: (was: 858776) Time Spent: 5h 10m (was: 5h) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 5h 10m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=856150&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856150
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 11/Apr/23 15:05
Start Date: 11/Apr/23 15:05
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1162961355
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java:
##
@@ -498,6 +497,68 @@ public void testPartitionOpsWhenTableDoesNotExist() throws
InvalidObjectExceptio
}
}
+ @Test
+ public void testDropPartitionByName() throws Exception {
+Database db1 = new DatabaseBuilder()
+.setName(DB1)
+.setDescription("description")
+.setLocation("locationurl")
+.build(conf);
+try (AutoCloseable c = deadline()) {
+ objectStore.createDatabase(db1);
+}
+StorageDescriptor sd = createFakeSd("location");
+HashMap tableParams = new HashMap<>();
+tableParams.put("EXTERNAL", "false");
+FieldSchema partitionKey1 = new FieldSchema("Country",
ColumnType.STRING_TYPE_NAME, "");
+FieldSchema partitionKey2 = new FieldSchema("State",
ColumnType.STRING_TYPE_NAME, "");
+Table tbl1 =
+new Table(TABLE1, DB1, "owner", 1, 2, 3, sd,
Arrays.asList(partitionKey1, partitionKey2),
+tableParams, null, null, "MANAGED_TABLE");
+try (AutoCloseable c = deadline()) {
+ objectStore.createTable(tbl1);
+}
+HashMap partitionParams = new HashMap<>();
+partitionParams.put("PARTITION_LEVEL_PRIVILEGE", "true");
+List value1 = Arrays.asList("US", "CA");
+Partition part1 = new Partition(value1, DB1, TABLE1, 111, 111, sd,
partitionParams);
+part1.setCatName(DEFAULT_CATALOG_NAME);
+try (AutoCloseable c = deadline()) {
+ objectStore.addPartition(part1);
+}
+List value2 = Arrays.asList("US", "MA");
+Partition part2 = new Partition(value2, DB1, TABLE1, 222, 222, sd,
partitionParams);
+part2.setCatName(DEFAULT_CATALOG_NAME);
+try (AutoCloseable c = deadline()) {
+ objectStore.addPartition(part2);
+}
+
+List partitions;
+try (AutoCloseable c = deadline()) {
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/state=CA");
+ partitions = objectStore.getPartitions(DEFAULT_CATALOG_NAME, DB1,
TABLE1, 10);
+}
+Assert.assertEquals(1, partitions.size());
+Assert.assertEquals(222, partitions.get(0).getCreateTime());
+try (AutoCloseable c = deadline()) {
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/state=MA");
+ partitions = objectStore.getPartitions(DEFAULT_CATALOG_NAME, DB1,
TABLE1, 10);
+}
+Assert.assertEquals(0, partitions.size());
+
+try (AutoCloseable c = deadline()) {
+ // Illegal partName will do nothing, it doesn't matter
+ // because the real HMSHandler will guarantee the partName is legal and
exists.
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/state=NON_EXIST");
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/st=CA");
Review Comment:
It will not return false for non-existent partition, `HMSHandler` has
already checked the existence of the partition and it does not need a double
check in `ObjectStore`.
Issue Time Tracking
---
Worklog Id: (was: 856150)
Time Spent: 5h (was: 4h 50m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=856148&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856148
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 11/Apr/23 14:56
Start Date: 11/Apr/23 14:56
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1162948940
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##
@@ -5026,20 +5026,18 @@ private boolean drop_partition_common(RawStore ms,
String catName, String db_nam
verifyIsWritablePath(partPath);
}
- if (!ms.dropPartition(catName, db_name, tbl_name, part_vals)) {
-throw new MetaException("Unable to drop partition");
- } else {
-if (!transactionalListeners.isEmpty()) {
+ String partName = Warehouse.makePartName(tbl.getPartitionKeys(),
part_vals);
+ ms.dropPartition(catName, db_name, tbl_name, partName);
Review Comment:
Yes, it will throw a meta exception by `GetHelper`:
https://github.com/apache/hive/blob/e397acd81ce4a909c5a564008117067df73872a5/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L4435-L4439
Issue Time Tracking
---
Worklog Id: (was: 856148)
Time Spent: 4h 50m (was: 4h 40m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=855972&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-855972
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 10/Apr/23 22:55
Start Date: 10/Apr/23 22:55
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1162150844
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java:
##
@@ -498,6 +497,68 @@ public void testPartitionOpsWhenTableDoesNotExist() throws
InvalidObjectExceptio
}
}
+ @Test
+ public void testDropPartitionByName() throws Exception {
+Database db1 = new DatabaseBuilder()
+.setName(DB1)
+.setDescription("description")
+.setLocation("locationurl")
+.build(conf);
+try (AutoCloseable c = deadline()) {
+ objectStore.createDatabase(db1);
+}
+StorageDescriptor sd = createFakeSd("location");
+HashMap tableParams = new HashMap<>();
+tableParams.put("EXTERNAL", "false");
+FieldSchema partitionKey1 = new FieldSchema("Country",
ColumnType.STRING_TYPE_NAME, "");
+FieldSchema partitionKey2 = new FieldSchema("State",
ColumnType.STRING_TYPE_NAME, "");
+Table tbl1 =
+new Table(TABLE1, DB1, "owner", 1, 2, 3, sd,
Arrays.asList(partitionKey1, partitionKey2),
+tableParams, null, null, "MANAGED_TABLE");
+try (AutoCloseable c = deadline()) {
+ objectStore.createTable(tbl1);
+}
+HashMap partitionParams = new HashMap<>();
+partitionParams.put("PARTITION_LEVEL_PRIVILEGE", "true");
+List value1 = Arrays.asList("US", "CA");
+Partition part1 = new Partition(value1, DB1, TABLE1, 111, 111, sd,
partitionParams);
+part1.setCatName(DEFAULT_CATALOG_NAME);
+try (AutoCloseable c = deadline()) {
+ objectStore.addPartition(part1);
+}
+List value2 = Arrays.asList("US", "MA");
+Partition part2 = new Partition(value2, DB1, TABLE1, 222, 222, sd,
partitionParams);
+part2.setCatName(DEFAULT_CATALOG_NAME);
+try (AutoCloseable c = deadline()) {
+ objectStore.addPartition(part2);
+}
+
+List partitions;
+try (AutoCloseable c = deadline()) {
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/state=CA");
+ partitions = objectStore.getPartitions(DEFAULT_CATALOG_NAME, DB1,
TABLE1, 10);
+}
+Assert.assertEquals(1, partitions.size());
+Assert.assertEquals(222, partitions.get(0).getCreateTime());
+try (AutoCloseable c = deadline()) {
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/state=MA");
+ partitions = objectStore.getPartitions(DEFAULT_CATALOG_NAME, DB1,
TABLE1, 10);
+}
+Assert.assertEquals(0, partitions.size());
+
+try (AutoCloseable c = deadline()) {
+ // Illegal partName will do nothing, it doesn't matter
+ // because the real HMSHandler will guarantee the partName is legal and
exists.
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/state=NON_EXIST");
+ objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1,
"country=US/st=CA");
Review Comment:
If this API is returning false, we could assert false here.
Issue Time Tracking
---
Worklog Id: (was: 855972)
Time Spent: 4h 40m (was: 4.5h)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=855970&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-855970
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 10/Apr/23 22:39
Start Date: 10/Apr/23 22:39
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1162143358
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##
@@ -5026,20 +5026,18 @@ private boolean drop_partition_common(RawStore ms,
String catName, String db_nam
verifyIsWritablePath(partPath);
}
- if (!ms.dropPartition(catName, db_name, tbl_name, part_vals)) {
-throw new MetaException("Unable to drop partition");
- } else {
-if (!transactionalListeners.isEmpty()) {
+ String partName = Warehouse.makePartName(tbl.getPartitionKeys(),
part_vals);
+ ms.dropPartition(catName, db_name, tbl_name, partName);
Review Comment:
If dropPartition in the object store is not successful, then we should throw
a meta exception, right?
Issue Time Tracking
---
Worklog Id: (was: 855970)
Time Spent: 4.5h (was: 4h 20m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854269&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854269 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 31/Mar/23 21:02 Start Date: 31/Mar/23 21:02 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4123: URL: https://github.com/apache/hive/pull/4123#issuecomment-1492604592 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4123) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 854269) Time Spent: 4h 20m (was: 4h 10m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854230 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 31/Mar/23 17:44 Start Date: 31/Mar/23 17:44 Worklog Time Spent: 10m Work Description: wecharyu commented on PR #4123: URL: https://github.com/apache/hive/pull/4123#issuecomment-1492359832 Address comments. cc: @saihemanth-cloudera Issue Time Tracking --- Worklog Id: (was: 854230) Time Spent: 4h 10m (was: 4h) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854207&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854207
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 31/Mar/23 14:47
Start Date: 31/Mar/23 14:47
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1154570802
##
standalone-metastore/metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSBenchmarks.java:
##
@@ -246,26 +246,24 @@ static DescriptiveStatistics
benchmarkGetPartitions(@NotNull MicroBenchmark benc
}
static DescriptiveStatistics benchmarkDropPartition(@NotNull MicroBenchmark
bench,
- @NotNull BenchData data)
{
+ @NotNull BenchData data,
+ int count) {
final HMSClient client = data.getClient();
String dbName = data.dbName;
String tableName = data.tableName;
BenchmarkUtils.createPartitionedTable(client, dbName, tableName);
-final List values = Collections.singletonList("d1");
try {
- Table t = client.getTable(dbName, tableName);
- Partition partition = new Util.PartitionBuilder(t)
- .withValues(values)
- .build();
-
return bench.measure(
- () -> throwingSupplierWrapper(() -> client.addPartition(partition)),
- () -> throwingSupplierWrapper(() -> client.dropPartition(dbName,
tableName, values)),
+ () -> addManyPartitionsNoException(client, dbName, tableName, null,
+ Collections.singletonList("d"), count),
+ () -> throwingSupplierWrapper(() -> {
+List partNames = client.getPartitionNames(dbName,
tableName);
+partNames.forEach(partName ->
+throwingSupplierWrapper(() -> client.dropPartition(dbName,
tableName, partName)));
+return null;
+ }),
null);
-} catch (TException e) {
Review Comment:
I followed the `benchmarkDropPartitions()` and some others, catching
exception here should have nothing to do with benchmark statistics, because the
result statistic is in `measure()` method:
https://github.com/apache/hive/blob/745dbf7059de93a74bceda5e5acbb293008771dc/standalone-metastore/metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/MicroBenchmark.java#L84-L87
Also the `Err%` should be **Coefficient of variation** of successful
operations rather than percentage of error records:
https://github.com/apache/hive/blob/745dbf7059de93a74bceda5e5acbb293008771dc/standalone-metastore/metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/BenchmarkSuite.java#L207
Issue Time Tracking
---
Worklog Id: (was: 854207)
Time Spent: 4h (was: 3h 50m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854200&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854200
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 31/Mar/23 13:43
Start Date: 31/Mar/23 13:43
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1154494889
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##
@@ -5026,7 +5026,8 @@ private boolean drop_partition_common(RawStore ms, String
catName, String db_nam
verifyIsWritablePath(partPath);
}
- if (!ms.dropPartition(catName, db_name, tbl_name, part_vals)) {
+ String partName = Warehouse.makePartName(tbl.getPartitionKeys(),
part_vals);
+ if (!ms.dropPartition(catName, db_name, tbl_name, partName)) {
throw new MetaException("Unable to drop partition");
Review Comment:
`ms.dropPartition()` will throw the exception, we do not need to throw this
`MetaException` here, will remove this code.
Issue Time Tracking
---
Worklog Id: (was: 854200)
Time Spent: 3h 50m (was: 3h 40m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854154&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854154
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 31/Mar/23 11:02
Start Date: 31/Mar/23 11:02
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1154338570
##
standalone-metastore/metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSBenchmarks.java:
##
@@ -246,26 +246,24 @@ static DescriptiveStatistics
benchmarkGetPartitions(@NotNull MicroBenchmark benc
}
static DescriptiveStatistics benchmarkDropPartition(@NotNull MicroBenchmark
bench,
- @NotNull BenchData data)
{
+ @NotNull BenchData data,
+ int count) {
final HMSClient client = data.getClient();
String dbName = data.dbName;
String tableName = data.tableName;
BenchmarkUtils.createPartitionedTable(client, dbName, tableName);
-final List values = Collections.singletonList("d1");
try {
- Table t = client.getTable(dbName, tableName);
- Partition partition = new Util.PartitionBuilder(t)
- .withValues(values)
- .build();
-
return bench.measure(
- () -> throwingSupplierWrapper(() -> client.addPartition(partition)),
- () -> throwingSupplierWrapper(() -> client.dropPartition(dbName,
tableName, values)),
+ () -> addManyPartitionsNoException(client, dbName, tableName, null,
+ Collections.singletonList("d"), count),
+ () -> throwingSupplierWrapper(() -> {
+List partNames = client.getPartitionNames(dbName,
tableName);
+partNames.forEach(partName ->
+throwingSupplierWrapper(() -> client.dropPartition(dbName,
tableName, partName)));
+return null;
+ }),
null);
-} catch (TException e) {
Review Comment:
Why do we need to remove this? I think we are somehow the percentage of
error records is not correct.
Issue Time Tracking
---
Worklog Id: (was: 854154)
Time Spent: 3h 40m (was: 3.5h)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854153&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854153
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 31/Mar/23 10:49
Start Date: 31/Mar/23 10:49
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1154326620
##
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestObjectStore.java:
##
@@ -439,14 +439,14 @@ public void testPartitionOps() throws Exception {
Assert.assertEquals(2, numPartitions);
try (AutoCloseable c = deadline()) {
- objectStore.dropPartition(DEFAULT_CATALOG_NAME, DB1, TABLE1, value1);
Review Comment:
Ideally, I would like to keep the old tests and add new tests that takes in
"partName" as argument.
Issue Time Tracking
---
Worklog Id: (was: 854153)
Time Spent: 3.5h (was: 3h 20m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854142&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854142
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 31/Mar/23 10:19
Start Date: 31/Mar/23 10:19
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1154298770
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##
@@ -5026,7 +5026,8 @@ private boolean drop_partition_common(RawStore ms, String
catName, String db_nam
verifyIsWritablePath(partPath);
}
- if (!ms.dropPartition(catName, db_name, tbl_name, part_vals)) {
+ String partName = Warehouse.makePartName(tbl.getPartitionKeys(),
part_vals);
+ if (!ms.dropPartition(catName, db_name, tbl_name, partName)) {
throw new MetaException("Unable to drop partition");
Review Comment:
Should we throw the exception stack trace/reason for failure?
Issue Time Tracking
---
Worklog Id: (was: 854142)
Time Spent: 3h 20m (was: 3h 10m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853875&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853875 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 30/Mar/23 09:15 Start Date: 30/Mar/23 09:15 Worklog Time Spent: 10m Work Description: wecharyu commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1152971682 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: Got it! Thank you. Issue Time Tracking --- Worklog Id: (was: 853875) Time Spent: 3h 10m (was: 3h) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853871&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853871 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 30/Mar/23 08:42 Start Date: 30/Mar/23 08:42 Worklog Time Spent: 10m Work Description: saihemanth-cloudera commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1152931417 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: I'm Ok with your current changes(a new api in the raw store). Issue Time Tracking --- Worklog Id: (was: 853871) Time Spent: 3h (was: 2h 50m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853863&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853863 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 30/Mar/23 07:49 Start Date: 30/Mar/23 07:49 Worklog Time Spent: 10m Work Description: wecharyu commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1152866393 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: @saihemanth-cloudera Sry I did not get your point. Do you mean add this new API is OK? Thanks for your patience. Issue Time Tracking --- Worklog Id: (was: 853863) Time Spent: 2h 50m (was: 2h 40m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853827&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853827 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 30/Mar/23 04:17 Start Date: 30/Mar/23 04:17 Worklog Time Spent: 10m Work Description: saihemanth-cloudera commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1152712297 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: I would favor this change instead of passing the table into the drop partition API in the object store. Issue Time Tracking --- Worklog Id: (was: 853827) Time Spent: 2h 40m (was: 2.5h) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853513&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853513 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 28/Mar/23 20:08 Start Date: 28/Mar/23 20:08 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4123: URL: https://github.com/apache/hive/pull/4123#issuecomment-1487528392 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4123) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 853513) Time Spent: 2.5h (was: 2h 20m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853493&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853493 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 28/Mar/23 18:15 Start Date: 28/Mar/23 18:15 Worklog Time Spent: 10m Work Description: wecharyu commented on PR #4123: URL: https://github.com/apache/hive/pull/4123#issuecomment-1487395921 Put the benchmark in PR description, it shows some performance improvement. FYI: @deniskuzZ @saihemanth-cloudera @VenuReddy2103 Issue Time Tracking --- Worklog Id: (was: 853493) Time Spent: 2h 20m (was: 2h 10m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853492&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853492 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 28/Mar/23 18:13 Start Date: 28/Mar/23 18:13 Worklog Time Spent: 10m Work Description: wecharyu commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1150994392 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: If we reuse the existing `dropPartition()` method, we need to get the `table` and partition keys again in `ObjectStore` to make the partname, so I think it's reasonable to create this new API and deprecate the old one. Issue Time Tracking --- Worklog Id: (was: 853492) Time Spent: 2h 10m (was: 2h) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853389&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853389 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 28/Mar/23 11:48 Start Date: 28/Mar/23 11:48 Worklog Time Spent: 10m Work Description: VenuReddy2103 commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1150463962 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: IMHO, Instead of defining this new API, we can make the partname inside the existing `dropPartition()` method itself to invoke `dropPartitionsInternal()` or `dropPartitions()`. Because the new API signature is similar to exisiting `dropPartitions()` except the last argument(i.e., single partname vs list of partnames). May mark `dropPartition()` as deprecated and insist `dropPartitions()` directly in future. Issue Time Tracking --- Worklog Id: (was: 853389) Time Spent: 2h (was: 1h 50m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853387&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853387 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 28/Mar/23 11:43 Start Date: 28/Mar/23 11:43 Worklog Time Spent: 10m Work Description: VenuReddy2103 commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1150463962 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: IMHO, Instead of defining this new API, we can make the partname inside the existing `dropPartition()` method itself to invoke `dropPartitionsInternal()` or `dropPartitions()`. Because the new API signature is similar to exisiting `dropPartitions()` except the last argument(i.e., single partname vs list of partnames). May be, we can mark `dropPartition()` as deprecated and insist using `dropPartitions()` directly in future. Issue Time Tracking --- Worklog Id: (was: 853387) Time Spent: 1h 50m (was: 1h 40m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853306&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853306
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 28/Mar/23 06:36
Start Date: 28/Mar/23 06:36
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1150092003
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:
##
@@ -3101,6 +3100,22 @@ public boolean dropPartition(String catName, String
dbName, String tableName,
return success;
}
+ @Override
+ public boolean dropPartition(String catName, String dbName, String
tableName, String partName)
+ throws MetaException, NoSuchObjectException, InvalidObjectException,
InvalidInputException {
+boolean success = false;
+try {
+ openTransaction();
+ dropPartitionsInternal(catName, dbName, tableName,
Arrays.asList(partName), true, true);
Review Comment:
We have some test for direct sql and jdo sql before, it shows direct sql is
always faster even for simple operation like get connection. Let me add the
benchmark for dropping multiple single partitions.
Issue Time Tracking
---
Worklog Id: (was: 853306)
Time Spent: 1h 40m (was: 1.5h)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853290&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853290
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 28/Mar/23 05:37
Start Date: 28/Mar/23 05:37
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1150050461
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:
##
@@ -3101,6 +3100,22 @@ public boolean dropPartition(String catName, String
dbName, String tableName,
return success;
}
+ @Override
+ public boolean dropPartition(String catName, String dbName, String
tableName, String partName)
+ throws MetaException, NoSuchObjectException, InvalidObjectException,
InvalidInputException {
+boolean success = false;
+try {
+ openTransaction();
+ dropPartitionsInternal(catName, dbName, tableName,
Arrays.asList(partName), true, true);
Review Comment:
cc @VenuReddy2103
Issue Time Tracking
---
Worklog Id: (was: 853290)
Time Spent: 1.5h (was: 1h 20m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=853049&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853049
]
ASF GitHub Bot logged work on HIVE-27150:
-
Author: ASF GitHub Bot
Created on: 27/Mar/23 06:08
Start Date: 27/Mar/23 06:08
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1148824344
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:
##
@@ -3101,6 +3100,22 @@ public boolean dropPartition(String catName, String
dbName, String tableName,
return success;
}
+ @Override
+ public boolean dropPartition(String catName, String dbName, String
tableName, String partName)
+ throws MetaException, NoSuchObjectException, InvalidObjectException,
InvalidInputException {
+boolean success = false;
+try {
+ openTransaction();
+ dropPartitionsInternal(catName, dbName, tableName,
Arrays.asList(partName), true, true);
Review Comment:
I don't think this would improve the performance by any means. Consider
dropping 10k partitions, each partition would have to access same number of
tables in the underlying db to update the records, so it makes sense to batch
them and implement with direct SQL. But for a single partition since it'll
access same number of tables in the DB, I don't think it'll make sense to
implement this feature.
For example, [HIVE-26035](https://issues.apache.org/jira/browse/HIVE-26035)
(see the details in the jira) proved that implementing direct SQL actually
improved the performance by running against benchmark tests.
Similarly can you provide any evidence that this patch also has an edge by
running those tests? (Probably you might have to add some tests(e.g: dropping
10K+ single partitions).
Issue Time Tracking
---
Worklog Id: (was: 853049)
Time Spent: 1h 20m (was: 1h 10m)
> Drop single partition can also support direct sql
> -
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=852952&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852952 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 25/Mar/23 08:48 Start Date: 25/Mar/23 08:48 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4123: URL: https://github.com/apache/hive/pull/4123#issuecomment-1483766759 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4123) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 852952) Time Spent: 1h 10m (was: 1h) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=852945&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852945 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 25/Mar/23 04:28 Start Date: 25/Mar/23 04:28 Worklog Time Spent: 10m Work Description: wecharyu commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1148297352 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: Make sense, my initial thought was that this is an internal API which is not exposed to clients, so compatibility was not considered. I will follow your comment. Issue Time Tracking --- Worklog Id: (was: 852945) Time Spent: 1h (was: 50m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=852926&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852926 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 24/Mar/23 21:55 Start Date: 24/Mar/23 21:55 Worklog Time Spent: 10m Work Description: deniskuzZ commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1148088843 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: That is a change in API that breaks backward compatibility. We should keep the original signature and mark it as deprecated. In the following releases we could remove it. Issue Time Tracking --- Worklog Id: (was: 852926) Time Spent: 50m (was: 40m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=852925&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852925 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 24/Mar/23 21:51 Start Date: 24/Mar/23 21:51 Worklog Time Spent: 10m Work Description: deniskuzZ commented on code in PR #4123: URL: https://github.com/apache/hive/pull/4123#discussion_r1148088843 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java: ## @@ -459,16 +459,15 @@ boolean doesPartitionExist(String catName, String dbName, String tableName, * @param catName catalog name. * @param dbName database name. * @param tableName table name. - * @param part_vals list of partition values. + * @param partName partition name. * @return true if the partition was dropped. * @throws MetaException Error accessing the RDBMS. * @throws NoSuchObjectException no partition matching this description exists * @throws InvalidObjectException error dropping the statistics for the partition * @throws InvalidInputException error dropping the statistics for the partition */ - boolean dropPartition(String catName, String dbName, String tableName, - List part_vals) throws MetaException, NoSuchObjectException, InvalidObjectException, - InvalidInputException; + boolean dropPartition(String catName, String dbName, String tableName, String partName) Review Comment: That is a change in API that breaks backward compatibility. Issue Time Tracking --- Worklog Id: (was: 852925) Time Spent: 40m (was: 0.5h) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=852871&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852871 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 24/Mar/23 15:43 Start Date: 24/Mar/23 15:43 Worklog Time Spent: 10m Work Description: wecharyu commented on PR #4123: URL: https://github.com/apache/hive/pull/4123#issuecomment-1483018000 @deniskuzZ @kasakrisz @saihemanth-cloudera: Could you please review this PR? Issue Time Tracking --- Worklog Id: (was: 852871) Time Spent: 0.5h (was: 20m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=851646&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-851646 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 19/Mar/23 15:31 Start Date: 19/Mar/23 15:31 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4123: URL: https://github.com/apache/hive/pull/4123#issuecomment-1475294531 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4123) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4123&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [6 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4123&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4123&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 851646) Time Spent: 20m (was: 10m) > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27150) Drop single partition can also support direct sql
[ https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=851638&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-851638 ] ASF GitHub Bot logged work on HIVE-27150: - Author: ASF GitHub Bot Created on: 19/Mar/23 09:49 Start Date: 19/Mar/23 09:49 Worklog Time Spent: 10m Work Description: wecharyu opened a new pull request, #4123: URL: https://github.com/apache/hive/pull/4123 ### What changes were proposed in this pull request? We want to reuse the `dropPartitionsInternal()` method for `drop_partition_common` api to enjoy the high performance, to achieve this we will refactor the `RawStore#dropPartition` api. ### Why are the changes needed? 1. support direct sql for drop single partition 2. reduce one query to the DB, that is MPartition query in `ObjectStore#dropPartition` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass all existing tests. Issue Time Tracking --- Worklog Id: (was: 851638) Remaining Estimate: 0h Time Spent: 10m > Drop single partition can also support direct sql > - > > Key: HIVE-27150 > URL: https://issues.apache.org/jira/browse/HIVE-27150 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > *Background:* > [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct > sql for drop_partitions, we can reuse this huge improvement in drop_partition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
