There were some extra discussions that happened at
https://github.com/apache/spark/pull/37105.
As of now we agreed to have a "soft deprecation" that
1. document the limitation of four API and suggest to use alternatives in
the API doc
2. do not use the @deprecation annotation.
Please let us know
Yes. The current goal is a pure educational deprecation.
So given the proposal:
1. existing users or users who do not care about catalog names in table
identifiers can still use all the API that maintain their past behavior.
2. new users who intend to use table identifiers with catalog names
get
It's better to keep all APIs working. But in this case, I really have no
idea how to make these 4 APIs reasonable. For example, tableExists(dbName:
String, tableName: String) currently checks if table "dbName.tableName"
exists in the Hive metastore, and does not work with v2 catalogs at all.
It's
Thank you for starting the official discussion, Rui.
'Unneeded API' doesn't sound like a good frame for this discussion
because it ignores the existing users and codes completely.
Technically, the above mentioned reasons look irrelevant to any
specific existing bugs or future maintenance cost
I want to highlight in case I missed this in the original email:
The 4 API will not be deleted. They will just be marked as deprecated
annotations and we encourage users to use their alternatives.
-Rui
On Thu, Jul 7, 2022 at 2:23 PM Rui Wang wrote:
> Hi Community,
>
> Proposal:
> I want to
Hi Community,
Proposal:
I want to discuss a proposal to deprecate the following Catalog API:
def listColumns(dbName: String, tableName: String): Dataset[Column]
def getTable(dbName: String, tableName: String): Table
def getFunction(dbName: String, functionName: String): Function
def
