[
https://issues.apache.org/jira/browse/SPARK-34616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhang Jianguo updated SPARK-34616:
----------------------------------
Description:
Almost all SQL need to check if database exist before execution.
Every check needs to connect to Metastore and check if the DB exists in
Databse. This behavior could be too expensive.
A proposal is to add a switch to control if it's necessary to check DB's
existance.
[requireDbExists]([https://github.com/apache/spark/blob/53e4dba7c489ac5c0ad61f0121c4e247de5b485c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala#L197)]
!db_existance.PNG!
was:
Almost all SQL need to check if database exist before execution.
Every check needs to connect to Metastore and check if the DB exists in
Databse. This behavior could be too expensive.
A proposal is to add a switch to control if it's necessary to check DB's
existance.
[requireDbExists]([https://github.com/apache/spark/blob/53e4dba7c489ac5c0ad61f0121c4e247de5b485c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala#L197)]
> SessionCatalog#requireDbExists is called too many times affects SQL
> performance
> -------------------------------------------------------------------------------
>
> Key: SPARK-34616
> URL: https://issues.apache.org/jira/browse/SPARK-34616
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.4.5, 3.0.0
> Reporter: Zhang Jianguo
> Priority: Minor
> Attachments: db_existance.PNG
>
>
> Almost all SQL need to check if database exist before execution.
> Every check needs to connect to Metastore and check if the DB exists in
> Databse. This behavior could be too expensive.
>
> A proposal is to add a switch to control if it's necessary to check DB's
> existance.
>
> [requireDbExists]([https://github.com/apache/spark/blob/53e4dba7c489ac5c0ad61f0121c4e247de5b485c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala#L197)]
>
> !db_existance.PNG!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
