Hi Spark devs, I'd like to bring more attention to this SPIP. As Dongjoon indicated in the email "Apache Spark 3.1 Feature Expectation (Dec. 2020)", this feature can be considered for 3.2 or even 3.1.
View catalog builds on top of the catalog plugin system introduced in DataSourceV2. It adds the “ViewCatalog” API to load, create, alter, and drop views. A catalog plugin can naturally implement both ViewCatalog and TableCatalog. Our internal implementation has been in production for over 8 months. Recently we extended it to support materialized views, for the read path initially. The PR has conflicts that I will resolve them shortly. Thanks, On Wed, Apr 22, 2020 at 12:24 AM John Zhuge <jzh...@apache.org> wrote: > Hi everyone, > > In order to disassociate view metadata from Hive Metastore and support > different storage backends, I am proposing a new view catalog API to load, > create, alter, and drop views. > > Document: > https://docs.google.com/document/d/1XOxFtloiMuW24iqJ-zJnDzHl2KMxipTjJoxleJFz66A/edit?usp=sharing > JIRA: https://issues.apache.org/jira/browse/SPARK-31357 > WIP PR: https://github.com/apache/spark/pull/28147 > > As part of a project to support common views across query engines like > Spark and Presto, my team used the view catalog API in Spark > implementation. The project has been in production over three months. > > Thanks, > John Zhuge > -- John Zhuge
