[jira] [Commented] (HIVE-25935) Cleanup IMetaStoreClient#getPartitionsByNames APIs
[
https://issues.apache.org/jira/browse/HIVE-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500210#comment-17500210
]
Zoltán Borók-Nagy commented on HIVE-25935:
--
Hi [~pvary],
Thanks for reaching out. In Impala we are using the followings from
IMetastoreClient:
* {{getPartitionsByNames(String, String, List)}}
* {{getPartitionsByNames(GetPartitionsByNamesRequest)}}
And btw we also use the thrift interface directly:
* {{get_partitions_by_names}}
* {{get_partitions_by_names_req}}
> Cleanup IMetaStoreClient#getPartitionsByNames APIs
> --
>
> Key: HIVE-25935
> URL: https://issues.apache.org/jira/browse/HIVE-25935
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Reporter: Stamatis Zampetakis
>Assignee: Peter Vary
>Priority: Major
> Fix For: 4.0.0-alpha-1
>
>
> Currently the
> [IMetastoreClient|https://github.com/apache/hive/blob/4b7a948e45fd88372fef573be321cda40d189cc7/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java]
> interface has 8 variants of the {{getPartitionsByNames}} method. Going
> quickly over the concrete implementation it appears that not all of them are
> useful/necessary so a bit of cleanup is needed.
> Below a few potential problems I observed:
> * Some of the APIs are not used anywhere in the project (neither by
> production nor by test code).
> * Some of the APIs are deprecated in some concrete implementations but not
> globally at the interface level without an explanation why.
> * Some of the implementations simply throw without doing anything.
> * Many of the APIs are partially tested or not tested at all.
> HIVE-24743, HIVE-25281 are related since they introduce/deprecate some of the
> aforementioned APIs.
> It would be good to review the aforementioned APIs and decide what needs to
> stay and what needs to go as well as complete necessary when relevant.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-25935) Cleanup IMetaStoreClient#getPartitionsByNames APIs
[
https://issues.apache.org/jira/browse/HIVE-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500099#comment-17500099
]
Peter Vary commented on HIVE-25935:
---
So the HMS API contains 2 methods:
{code:java}
// get partitions give a list of partition names
list get_partitions_by_names(1:string db_name 2:string tbl_name
3:list names)
throws(1:MetaException o1, 2:NoSuchObjectException o2)
GetPartitionsByNamesResult
get_partitions_by_names_req(1:GetPartitionsByNamesRequest req)
throws(1:MetaException o1, 2:NoSuchObjectException o2)
{code}
The first one is an old version (2011)
The second one is a new version (2019)
We definitely need both.
So the question is mainly the HiveMetaStoreClient methods.
> Cleanup IMetaStoreClient#getPartitionsByNames APIs
> --
>
> Key: HIVE-25935
> URL: https://issues.apache.org/jira/browse/HIVE-25935
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Reporter: Stamatis Zampetakis
>Assignee: Peter Vary
>Priority: Major
> Fix For: 4.0.0-alpha-1
>
>
> Currently the
> [IMetastoreClient|https://github.com/apache/hive/blob/4b7a948e45fd88372fef573be321cda40d189cc7/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java]
> interface has 8 variants of the {{getPartitionsByNames}} method. Going
> quickly over the concrete implementation it appears that not all of them are
> useful/necessary so a bit of cleanup is needed.
> Below a few potential problems I observed:
> * Some of the APIs are not used anywhere in the project (neither by
> production nor by test code).
> * Some of the APIs are deprecated in some concrete implementations but not
> globally at the interface level without an explanation why.
> * Some of the implementations simply throw without doing anything.
> * Many of the APIs are partially tested or not tested at all.
> HIVE-24743, HIVE-25281 are related since they introduce/deprecate some of the
> aforementioned APIs.
> It would be good to review the aforementioned APIs and decide what needs to
> stay and what needs to go as well as complete necessary when relevant.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-25935) Cleanup IMetaStoreClient#getPartitionsByNames APIs
[
https://issues.apache.org/jira/browse/HIVE-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500078#comment-17500078
]
Peter Vary commented on HIVE-25935:
---
[~boroknagyz]: Can you please help us which getPartitionsByNames methods are
used by Impala?
Thanks,
Peter
> Cleanup IMetaStoreClient#getPartitionsByNames APIs
> --
>
> Key: HIVE-25935
> URL: https://issues.apache.org/jira/browse/HIVE-25935
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Reporter: Stamatis Zampetakis
>Priority: Major
> Fix For: 4.0.0-alpha-1
>
>
> Currently the
> [IMetastoreClient|https://github.com/apache/hive/blob/4b7a948e45fd88372fef573be321cda40d189cc7/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java]
> interface has 8 variants of the {{getPartitionsByNames}} method. Going
> quickly over the concrete implementation it appears that not all of them are
> useful/necessary so a bit of cleanup is needed.
> Below a few potential problems I observed:
> * Some of the APIs are not used anywhere in the project (neither by
> production nor by test code).
> * Some of the APIs are deprecated in some concrete implementations but not
> globally at the interface level without an explanation why.
> * Some of the implementations simply throw without doing anything.
> * Many of the APIs are partially tested or not tested at all.
> HIVE-24743, HIVE-25281 are related since they introduce/deprecate some of the
> aforementioned APIs.
> It would be good to review the aforementioned APIs and decide what needs to
> stay and what needs to go as well as complete necessary when relevant.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-25935) Cleanup IMetaStoreClient#getPartitionsByNames APIs
[
https://issues.apache.org/jira/browse/HIVE-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489276#comment-17489276
]
Peter Vary commented on HIVE-25935:
---
If the API methods are not released yet, then it is fine to remove them. If we
have fewer API methods to maintain, then it would be great.
Thanks for bringing this up!
> Cleanup IMetaStoreClient#getPartitionsByNames APIs
> --
>
> Key: HIVE-25935
> URL: https://issues.apache.org/jira/browse/HIVE-25935
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Reporter: Stamatis Zampetakis
>Priority: Major
>
> Currently the
> [IMetastoreClient|https://github.com/apache/hive/blob/4b7a948e45fd88372fef573be321cda40d189cc7/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java]
> interface has 8 variants of the {{getPartitionsByNames}} method. Going
> quickly over the concrete implementation it appears that not all of them are
> useful/necessary so a bit of cleanup is needed.
> Below a few potential problems I observed:
> * Some of the APIs are not used anywhere in the project (neither by
> production nor by test code).
> * Some of the APIs are deprecated in some concrete implementations but not
> globally at the interface level without an explanation why.
> * Some of the implementations simply throw without doing anything.
> * Many of the APIs are partially tested or not tested at all.
> HIVE-24743, HIVE-25281 are related since they introduce/deprecate some of the
> aforementioned APIs.
> It would be good to review the aforementioned APIs and decide what needs to
> stay and what needs to go as well as complete necessary when relevant.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-25935) Cleanup IMetaStoreClient#getPartitionsByNames APIs
[
https://issues.apache.org/jira/browse/HIVE-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489053#comment-17489053
]
Stamatis Zampetakis commented on HIVE-25935:
[~pvary] There are things that were introduced and partially deprecated before
ever being released. I am not talking about APIs already released.
> Cleanup IMetaStoreClient#getPartitionsByNames APIs
> --
>
> Key: HIVE-25935
> URL: https://issues.apache.org/jira/browse/HIVE-25935
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Reporter: Stamatis Zampetakis
>Priority: Major
>
> Currently the
> [IMetastoreClient|https://github.com/apache/hive/blob/4b7a948e45fd88372fef573be321cda40d189cc7/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java]
> interface has 8 variants of the {{getPartitionsByNames}} method. Going
> quickly over the concrete implementation it appears that not all of them are
> useful/necessary so a bit of cleanup is needed.
> Below a few potential problems I observed:
> * Some of the APIs are not used anywhere in the project (neither by
> production nor by test code).
> * Some of the APIs are deprecated in some concrete implementations but not
> globally at the interface level without an explanation why.
> * Some of the implementations simply throw without doing anything.
> * Many of the APIs are partially tested or not tested at all.
> HIVE-24743, HIVE-25281 are related since they introduce/deprecate some of the
> aforementioned APIs.
> It would be good to review the aforementioned APIs and decide what needs to
> stay and what needs to go as well as complete necessary when relevant.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-25935) Cleanup IMetaStoreClient#getPartitionsByNames APIs
[
https://issues.apache.org/jira/browse/HIVE-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489027#comment-17489027
]
Peter Vary commented on HIVE-25935:
---
We usually try to keep the API backwards compatible. There are plenty of other
projects using the API, so we keep every old method, so we can be sure that we
don't break anything.
> Cleanup IMetaStoreClient#getPartitionsByNames APIs
> --
>
> Key: HIVE-25935
> URL: https://issues.apache.org/jira/browse/HIVE-25935
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Reporter: Stamatis Zampetakis
>Priority: Major
>
> Currently the
> [IMetastoreClient|https://github.com/apache/hive/blob/4b7a948e45fd88372fef573be321cda40d189cc7/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java]
> interface has 8 variants of the {{getPartitionsByNames}} method. Going
> quickly over the concrete implementation it appears that not all of them are
> useful/necessary so a bit of cleanup is needed.
> Below a few potential problems I observed:
> * Some of the APIs are not used anywhere in the project (neither by
> production nor by test code).
> * Some of the APIs are deprecated in some concrete implementations but not
> globally at the interface level without an explanation why.
> * Some of the implementations simply throw without doing anything.
> * Many of the APIs are partially tested or not tested at all.
> HIVE-24743, HIVE-25281 are related since they introduce/deprecate some of the
> aforementioned APIs.
> It would be good to review the aforementioned APIs and decide what needs to
> stay and what needs to go as well as complete necessary when relevant.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
