RussellSpitzer commented on code in PR #1889: URL: https://github.com/apache/polaris/pull/1889#discussion_r2145912516
########## site/content/in-dev/unreleased/generic-table.md: ########## @@ -0,0 +1,168 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +title: Generic Table (Beta) +type: docs +weight: 435 +--- + +The Generic Table in Apache Polaris provides basic management support for non-Iceberg tables. + +With the Generic Table API, you can: +- Create generic tables under a namespace +- Load a generic table +- Drop a generic table +- List all generic tables under a namespace + +**NOTE** The current generic table is in beta release, there can still be incomplete features and bugs. Please use it +with caution and report any issue if encountered. + +## What is a Generic Table? + +A generic table in Polaris is a structured entity that defines the necessary information. Each +generic table entity contains the following properties: + +- **name** (required): A unique identifier for the table within a namespace +- **format** (required): The format for the generic table, i.e. "delta", "csv" +- **base-location** (optional): Table base location in URI format. For example: s3://<my-bucket>/path/to/table + - The table base location is a location that includes all files for the table + - A table with multiple disjoint locations (i.e. containing files that are outside the configured base location) is not compliant with the current generic table support in Polaris. + - If no location is provided, clients or users are responsible for managing the location. +- **properties** (optional): Properties for the generic table passed on creation +- **doc** (optional): Comment or description for the table + +## Generic Table API Vs. Iceberg Table API + +Generic Table provides a different set of APIs to operate on the Generic Table entities while Iceberg APIs operates on +the Iceberg Table entities. + +| Operations | **Generic Table API** | **Iceberg Table API** | Review Comment: This is much closer to the Spark "HMS" catalog integration. The catalog itself is unaware of the anything about the underlying table except for some loosely defined metadata about it. It's up to the engine (and plugins in that engine) to determine exactly how loading data or committing data actually occur based on that metadata. You could imagine examples of use cases being something like a CSV based table, or a JDBC Table. When these are stored in the HMS by Spark, the HMS doesn't know how to actually interact with the metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
