[jira] [Commented] (HIVE-26227) Add support of catalog related statements for Hive ql
[
https://issues.apache.org/jira/browse/HIVE-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539577#comment-17539577
]
Wechar commented on HIVE-26227:
---
Not quite yet. In our scene, HMS is in charge of handling catalogs as the
metadata center, the other computing engines will only use catalogs while
querying. Simply put, we extend the original `db_name.tbl_name` to
`cat_name.db_name.tbl_name` in computing engines to support data from different
systems or sources.
So we do not plan to manage catalogs in other components now.
> Add support of catalog related statements for Hive ql
> -
>
> Key: HIVE-26227
> URL: https://issues.apache.org/jira/browse/HIVE-26227
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Catalog concept is proposed to Hive 3.0 to allow different systems to connect
> to different catalogs in the metastore. But so far we can not query catalog
> through Hive ql, this task aims to implement the ddl statements related to
> catalog.
> *Create Catalog*
> {code:sql}
> CREATE CATALOG [IF NOT EXISTS] catalog_name
> LOCATION hdfs_path
> [COMMENT catalog_comment];
> {code}
> LOCATION is required for creating a new catalog now.
> *Alter Catalog*
> {code:sql}
> ALTER CATALOG catalog_name SET LOCATION hdfs_path;
> {code}
> Only location metadata can be altered for catalog.
> *Drop Catalog*
> {code:sql}
> DROP CATALOG [IF EXISTS] catalog_name;
> {code}
> DROP CATALOG is always RESTRICT, which means DROP CATALOG will fail if there
> are non-default databases in the catalog.
> *Show Catalogs*
> {code:sql}
> SHOW CATALOGS [LIKE 'identifier_with_wildcards'];
> {code}
> SHOW CATALOGS lists all of the catalogs defined in the metastore.
> The optional LIKE clause allows the list of catalogs to be filtered using a
> regular expression.
> *Describe Catalog*
> {code:sql}
> DESC[RIBE] CATALOG [EXTENDED] cat_name;
> {code}
> DESCRIBE CATALOG shows the name of the catalog, its comment (if one has been
> set), and its root location on the filesystem.
> EXTENDED also shows the create time.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HIVE-26227) Add support of catalog related statements for Hive ql
[
https://issues.apache.org/jira/browse/HIVE-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17538836#comment-17538836
]
Stamatis Zampetakis commented on HIVE-26227:
Thanks for the explanation. If I recall well it is possible to manage catalogs
via direct metastore calls but I suppose that this is not enough for your
use-case. If I understand well you plan to introduce similar APIs for handling
catalogs (CREATE, ALTER, DROP) in Hbase, Kafka, etc. Are there any JIRA tickets
about existing or future changes in those projects you mentioned?
> Add support of catalog related statements for Hive ql
> -
>
> Key: HIVE-26227
> URL: https://issues.apache.org/jira/browse/HIVE-26227
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Catalog concept is proposed to Hive 3.0 to allow different systems to connect
> to different catalogs in the metastore. But so far we can not query catalog
> through Hive ql, this task aims to implement the ddl statements related to
> catalog.
> *Create Catalog*
> {code:sql}
> CREATE CATALOG [IF NOT EXISTS] catalog_name
> LOCATION hdfs_path
> [COMMENT catalog_comment];
> {code}
> LOCATION is required for creating a new catalog now.
> *Alter Catalog*
> {code:sql}
> ALTER CATALOG catalog_name SET LOCATION hdfs_path;
> {code}
> Only location metadata can be altered for catalog.
> *Drop Catalog*
> {code:sql}
> DROP CATALOG [IF EXISTS] catalog_name;
> {code}
> DROP CATALOG is always RESTRICT, which means DROP CATALOG will fail if there
> are non-default databases in the catalog.
> *Show Catalogs*
> {code:sql}
> SHOW CATALOGS [LIKE 'identifier_with_wildcards'];
> {code}
> SHOW CATALOGS lists all of the catalogs defined in the metastore.
> The optional LIKE clause allows the list of catalogs to be filtered using a
> regular expression.
> *Describe Catalog*
> {code:sql}
> DESC[RIBE] CATALOG [EXTENDED] cat_name;
> {code}
> DESCRIBE CATALOG shows the name of the catalog, its comment (if one has been
> set), and its root location on the filesystem.
> EXTENDED also shows the create time.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HIVE-26227) Add support of catalog related statements for Hive ql
[
https://issues.apache.org/jira/browse/HIVE-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17536689#comment-17536689
]
Wechar commented on HIVE-26227:
---
Sure [~zabetak]. We plan to provide a unified metadata management service
through the Hive metastore, which means the metadata of various systems are
stored in Hive metastore and divided by catalog.
Currently we want to manage the metadata from Hive, Hbase, Kafka, Jdbc, etc,
and computing engines like Hive, Spark, Presto, Flink can join data from
different systems based on the metadata in Hive metastore.
> Add support of catalog related statements for Hive ql
> -
>
> Key: HIVE-26227
> URL: https://issues.apache.org/jira/browse/HIVE-26227
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Catalog concept is proposed to Hive 3.0 to allow different systems to connect
> to different catalogs in the metastore. But so far we can not query catalog
> through Hive ql, this task aims to implement the ddl statements related to
> catalog.
> *Create Catalog*
> {code:sql}
> CREATE CATALOG [IF NOT EXISTS] catalog_name
> LOCATION hdfs_path
> [COMMENT catalog_comment];
> {code}
> LOCATION is required for creating a new catalog now.
> *Alter Catalog*
> {code:sql}
> ALTER CATALOG catalog_name SET LOCATION hdfs_path;
> {code}
> Only location metadata can be altered for catalog.
> *Drop Catalog*
> {code:sql}
> DROP CATALOG [IF EXISTS] catalog_name;
> {code}
> DROP CATALOG is always RESTRICT, which means DROP CATALOG will fail if there
> are non-default databases in the catalog.
> *Show Catalogs*
> {code:sql}
> SHOW CATALOGS [LIKE 'identifier_with_wildcards'];
> {code}
> SHOW CATALOGS lists all of the catalogs defined in the metastore.
> The optional LIKE clause allows the list of catalogs to be filtered using a
> regular expression.
> *Describe Catalog*
> {code:sql}
> DESC[RIBE] CATALOG [EXTENDED] cat_name;
> {code}
> DESCRIBE CATALOG shows the name of the catalog, its comment (if one has been
> set), and its root location on the filesystem.
> EXTENDED also shows the create time.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HIVE-26227) Add support of catalog related statements for Hive ql
[
https://issues.apache.org/jira/browse/HIVE-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17536533#comment-17536533
]
Stamatis Zampetakis commented on HIVE-26227:
Thanks for working on this [~wechar] ! Out of curiosity can you provide a few
more details on how do you plan to use this feature?
> Add support of catalog related statements for Hive ql
> -
>
> Key: HIVE-26227
> URL: https://issues.apache.org/jira/browse/HIVE-26227
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Catalog concept is proposed to Hive 3.0 to allow different systems to connect
> to different catalogs in the metastore. But so far we can not query catalog
> through Hive ql, this task aims to implement the ddl statements related to
> catalog.
> *Create Catalog*
> {code:sql}
> CREATE CATALOG [IF NOT EXISTS] catalog_name
> LOCATION hdfs_path
> [COMMENT catalog_comment];
> {code}
> LOCATION is required for creating a new catalog now.
> *Alter Catalog*
> {code:sql}
> ALTER CATALOG catalog_name SET LOCATION hdfs_path;
> {code}
> Only location metadata can be altered for catalog.
> *Drop Catalog*
> {code:sql}
> DROP CATALOG [IF EXISTS] catalog_name;
> {code}
> DROP CATALOG is always RESTRICT, which means DROP CATALOG will fail if there
> are non-default databases in the catalog.
> *Show Catalogs*
> {code:sql}
> SHOW CATALOGS [LIKE 'identifier_with_wildcards'];
> {code}
> SHOW CATALOGS lists all of the catalogs defined in the metastore.
> The optional LIKE clause allows the list of catalogs to be filtered using a
> regular expression.
> *Describe Catalog*
> {code:sql}
> DESC[RIBE] CATALOG [EXTENDED] cat_name;
> {code}
> DESCRIBE CATALOG shows the name of the catalog, its comment (if one has been
> set), and its root location on the filesystem.
> EXTENDED also shows the create time.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
