Re: [Spark Catalog API] Support for metadata Backup/Restore

2021-05-11 Thread Tianchen Zhang
ough and persistent enough, >>> why dont just use external metastore? It is fairly straightforward process. >>> Also regardless you are in cloud or not, database bkp is a routine and >>> established pattern in most organizations. >>> You can also enhance HA and DR by havin

Re: [Spark Catalog API] Support for metadata Backup/Restore

2021-05-07 Thread Tianchen Zhang
't see a strong reason to > expose the APIs to Spark. Do you plan to add new SQL commands in Spark to > backup/restore a catalog? > > On Tue, May 4, 2021 at 2:39 AM Tianchen Zhang > wrote: > >> Hi all, >> >> Currently the user-facing Catalog API doesn't support

[Spark Catalog API] Support for metadata Backup/Restore

2021-05-03 Thread Tianchen Zhang
Hi all, Currently the user-facing Catalog API doesn't support backup/restore metadata. Our customers are asking for such functionalities. Here is a usage example: 1. Read all metadata of one Spark cluster 2. Save them into a Parquet file on DFS 3. Read the Parquet file and restore all metadata in

[no subject]

2021-05-03 Thread Tianchen Zhang
Hi all, Currently the user-facing Catalog API doesn't support backup/restore metadata. Our customers are asking for such functionalities. Here is a usage example: 1. Read all metadata of one Spark cluster 2. Save them into a Parquet file on DFS 3. Read the Parquet file and restore all metadata in

[Spark Catalog API] Support for metadata Backup/Restore

2021-04-30 Thread Tianchen Zhang
Hi, Currently the user-facing Catalog API doesn't support backup/restore metadata. Our customers are asking for such functionalities. Here is a usage example: 1. Read all metadata of one Spark cluster 2. Save them into a Parquet file on DFS 3. Read the Parquet file and restore all metadata in