rymurr commented on a change in pull request #3561: URL: https://github.com/apache/iceberg/pull/3561#discussion_r757589664
########## File path: rest_docs/rest-catalog-v0.1-SNAPSHOT.md ########## @@ -0,0 +1,1810 @@ +--- +title: Apache Iceberg REST Catalog API v1.0.0 +language_tabs: + - shell: Shell + - java: Java +language_clients: + - shell: "" + - java: "" +toc_footers: [] +includes: [] +search: true +highlight_theme: darkula +headingLevel: 2 + +--- + +<!-- Generator: Widdershins v4.0.1 --> + +<h1 id="apache-iceberg-rest-catalog-api">Apache Iceberg REST Catalog API v1.0.0</h1> + +> Scroll down for code samples, example requests and responses. Select a language for code samples from the tabs above or the mobile navigation menu. + +Defines the specification for the first version of the REST Catalog API. Implementations should support both Iceberg table specs v1 and v2, with priority given to v2. + +Base URLs: + +* <a href="http://127.0.0.1:1080";>http://127.0.0.1:1080</a> Review comment: specifying base url and port as part of the spec/docs seems strong. Shouldn't this be config? ########## File path: rest_docs/rest-catalog-open-api-v0.1.yaml ########## @@ -0,0 +1,727 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +openapi: 3.0.3 +info: + title: Apache Iceberg REST Catalog API + license: + name: Apache 2.0 + url: https://www.apache.org/licenses/LICENSE-2.0.html + version: 1.0.0 + description: + Defines the specification for the first version of the REST Catalog API. Implementations should support both Iceberg table specs v1 and v2, with priority given to v2. +servers: + - url: http://127.0.0.1:1080 + description: URL Used for Mock-Server Unit Tests +# All routes are currently configured using an Authorization header. +security: + - BearerAuth: [] +paths: + /v1/config: + get: + tags: + - Configuration API + summary: List all catalog configuration settings + operationId: getConfig + description: + All REST catalogs will be initialized by calling this route. This route + will return at least the minimum necessary metadata to initialize the + catalog. Optionally, it can also include server-side specific overrides. + For example, it might also include information used to initialize this catalog + such as the details of the Http connection pooling, etc. This route might + also advertise information about operations that are not implemented + so that the catalog can eagerly throw or go about another way of performing + the desired action. + responses: + default: + description: Server-Specific Configuration Overrides + content: + application/json: + schema: + $ref: '#/components/schemas/IcebergConfiguration' + "400": + description: Unknown Error + "401": + description: Unauthorized + + /v1/namespaces/{namespace}/tables/{table}: + parameters: + - name: namespace + in: path + description: Namespace the table is in + required: true + schema: + type: string + examples: + singlepart_namespace: + value: "accounting" + multipart_namespace: + value: "accounting.tax" + - name: table + in: path + description: Name of the table to load + required: true + schema: + type: string + example: "sales" + get: + tags: + - Catalog API + summary: Load a given table from a given namespace + operationId: loadTable + responses: + '200': + description: OK + content: + application/json: + schema: + $ref: '#/components/schemas/GetTableResponse' + '401': + description: Unauthorized + # Using 412, `Precondition Failed`, instead of 404, as 404 makes monitoring response codes from ELBs + # very difficult - Hard to tell if clients or servers are misconfigured and calling non-existent routes + # or missing routes versus expected error cases such as NoSuchTableException (expected meaning that + # a person who is on call shouldn't be paged for this but 404 they might need to be). + '412': + description: NoSuchTableException + content: + application/json: + schema: + $ref: '#/components/schemas/NoSuchTableError' + put: + tags: + - Catalog API + summary: Commit an in progress create (or replace) table transaction + operationId: commitTable + description: Commit a pending create (or replace) table transaction, e.g. for doCommit. + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/CommitTableRequest' + required: true + responses: + '200': + description: OK + content: + application/json: + schema: + $ref: '#/components/schemas/CommitTableResponse' + '401': + description: Unauthorized + content: + application/json: + schema: + $ref: '#/components/schemas/IcebergResponseObject' + delete: + tags: + - Catalog API + summary: Drop a table from the catalog + operationId: dropTable + description: Remove a table from the catalog + parameters: + - name: purgeRequested + in: query + required: false + description: Whether the user requested to purge the underlying table's data and metadata + schema: + type: boolean + default: false + responses: + '200': + description: OK + content: + 'application/json': + schema: + type: boolean + description: true if the table was dropped and false if the table did not exist + head: + tags: + - Catalog API + summary: Check if a table exists + operationId: tableExists + description: + Check if a table exists within a given namespace. Returns the standard response with `true` when found. Will return a TableNotFound error if not present. + parameters: + - name: namespace + in: path + required: true + schema: + type: string + - name: table + in: path + required: true + schema: + type: string + responses: + '200': + description: OK + '412': + description: Table Not Found + /v1/namespaces/{namespace}/tables: + parameters: + - name: namespace + description: Namespace under which to list tables. + in: path + required: true + schema: + type: string + examples: + singlepart_namespace: + value: "accounting" + multipart_namespace: + value: "accounting.tax" + get: + tags: + - Catalog API + summary: List all table identifiers underneath a given namespace + description: Return all table identifiers under this namespace + operationId: listTables + parameters: + - name: namespace + description: Namespace under which to list tables. + in: path + required: true + schema: + type: string + examples: + singlepart_namespace: + value: "accounting" + multipart_namespace: + value: "accounting.tax" + # TODO - There's a much more native way to handle pagination + - name: limit + description: number of values to return in one request + in: query + required: false + schema: + type: integer + example: 10 + - name: offset + description: Place in the response to continue from if paginating + in: query + required: false + schema: + type: integer + responses: Review comment: 404 if namespace doesn't exist? ########## File path: rest_docs/rest-catalog-v0.1-SNAPSHOT.md ########## @@ -0,0 +1,1810 @@ +--- +title: Apache Iceberg REST Catalog API v1.0.0 +language_tabs: + - shell: Shell + - java: Java +language_clients: + - shell: "" + - java: "" +toc_footers: [] +includes: [] +search: true +highlight_theme: darkula +headingLevel: 2 + +--- + +<!-- Generator: Widdershins v4.0.1 --> + +<h1 id="apache-iceberg-rest-catalog-api">Apache Iceberg REST Catalog API v1.0.0</h1> + +> Scroll down for code samples, example requests and responses. Select a language for code samples from the tabs above or the mobile navigation menu. + +Defines the specification for the first version of the REST Catalog API. Implementations should support both Iceberg table specs v1 and v2, with priority given to v2. + +Base URLs: + +* <a href="http://127.0.0.1:1080";>http://127.0.0.1:1080</a> + +License: <a href="https://www.apache.org/licenses/LICENSE-2.0.html";>Apache 2.0</a> + +# Authentication + +- HTTP Authentication, scheme: bearer + +<h1 id="apache-iceberg-rest-catalog-api-configuration-api">Configuration API</h1> + +## getConfig + +<a id="opIdgetConfig"></a> + +> Code samples + +```shell +# You can also use wget +curl -X GET http://127.0.0.1:1080/v1/config \ + -H 'Accept: application/json' \ + -H 'Authorization: Bearer {access-token}' + +``` + +```java +URL obj = new URL("http://127.0.0.1:1080/v1/config";); +HttpURLConnection con = (HttpURLConnection) obj.openConnection(); +con.setRequestMethod("GET"); +int responseCode = con.getResponseCode(); +BufferedReader in = new BufferedReader( + new InputStreamReader(con.getInputStream())); +String inputLine; +StringBuffer response = new StringBuffer(); +while ((inputLine = in.readLine()) != null) { + response.append(inputLine); +} +in.close(); +System.out.println(response.toString()); + +``` + +`GET /v1/config` + +*List all catalog configuration settings* + +All REST catalogs will be initialized by calling this route. This route will return at least the minimum necessary metadata to initialize the catalog. Optionally, it can also include server-side specific overrides. For example, it might also include information used to initialize this catalog such as the details of the Http connection pooling, etc. This route might also advertise information about operations that are not implemented so that the catalog can eagerly throw or go about another way of performing the desired action. Review comment: Could you clarify this a bit more for me? Maybe I am daft but it seems a bit circular to me. We need things like the url, auth, http client settings etc to call this endpoint but it returns the config we need to connect? Or is the idea that the http client would get re-configured once this call is made. Additionally, settings like the connection pooling sound like a concern of the client rather than the server (eg a lambda isn't going to use connection pooling whilst a long running service may use it a lot). Or is this meant more for the Spark (or equivalent) catalog settings? ########## File path: rest_docs/rest-catalog-v0.1-SNAPSHOT.md ########## @@ -0,0 +1,1810 @@ +--- +title: Apache Iceberg REST Catalog API v1.0.0 +language_tabs: + - shell: Shell + - java: Java +language_clients: + - shell: "" + - java: "" +toc_footers: [] +includes: [] +search: true +highlight_theme: darkula +headingLevel: 2 + +--- + +<!-- Generator: Widdershins v4.0.1 --> + +<h1 id="apache-iceberg-rest-catalog-api">Apache Iceberg REST Catalog API v1.0.0</h1> + +> Scroll down for code samples, example requests and responses. Select a language for code samples from the tabs above or the mobile navigation menu. + +Defines the specification for the first version of the REST Catalog API. Implementations should support both Iceberg table specs v1 and v2, with priority given to v2. + +Base URLs: + +* <a href="http://127.0.0.1:1080";>http://127.0.0.1:1080</a> + +License: <a href="https://www.apache.org/licenses/LICENSE-2.0.html";>Apache 2.0</a> + +# Authentication + +- HTTP Authentication, scheme: bearer Review comment: Same goes for authentication. I would expect auth would have to be pluggable in the spirit of `FileIO` rather than making the spec only support http bearer auth. ########## File path: rest_docs/rest-catalog-open-api-v0.1.yaml ########## @@ -0,0 +1,727 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +openapi: 3.0.3 +info: + title: Apache Iceberg REST Catalog API + license: + name: Apache 2.0 + url: https://www.apache.org/licenses/LICENSE-2.0.html + version: 1.0.0 + description: + Defines the specification for the first version of the REST Catalog API. Implementations should support both Iceberg table specs v1 and v2, with priority given to v2. +servers: + - url: http://127.0.0.1:1080 + description: URL Used for Mock-Server Unit Tests +# All routes are currently configured using an Authorization header. +security: + - BearerAuth: [] +paths: + /v1/config: + get: + tags: + - Configuration API + summary: List all catalog configuration settings + operationId: getConfig + description: + All REST catalogs will be initialized by calling this route. This route + will return at least the minimum necessary metadata to initialize the + catalog. Optionally, it can also include server-side specific overrides. + For example, it might also include information used to initialize this catalog + such as the details of the Http connection pooling, etc. This route might + also advertise information about operations that are not implemented + so that the catalog can eagerly throw or go about another way of performing + the desired action. + responses: + default: + description: Server-Specific Configuration Overrides + content: + application/json: + schema: + $ref: '#/components/schemas/IcebergConfiguration' + "400": + description: Unknown Error + "401": + description: Unauthorized + + /v1/namespaces/{namespace}/tables/{table}: + parameters: + - name: namespace + in: path + description: Namespace the table is in + required: true + schema: + type: string + examples: + singlepart_namespace: + value: "accounting" + multipart_namespace: + value: "accounting.tax" + - name: table + in: path + description: Name of the table to load + required: true + schema: + type: string + example: "sales" + get: + tags: + - Catalog API + summary: Load a given table from a given namespace + operationId: loadTable + responses: + '200': + description: OK + content: + application/json: + schema: + $ref: '#/components/schemas/GetTableResponse' + '401': + description: Unauthorized + # Using 412, `Precondition Failed`, instead of 404, as 404 makes monitoring response codes from ELBs + # very difficult - Hard to tell if clients or servers are misconfigured and calling non-existent routes + # or missing routes versus expected error cases such as NoSuchTableException (expected meaning that + # a person who is on call shouldn't be paged for this but 404 they might need to be). + '412': + description: NoSuchTableException + content: + application/json: + schema: + $ref: '#/components/schemas/NoSuchTableError' Review comment: just to be clear this is a new exception class that would be http server/clietn specific. Not the existing exception. Thats correct? ########## File path: rest_docs/rest-catalog-open-api-v0.1.yaml ########## @@ -0,0 +1,727 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +openapi: 3.0.3 +info: + title: Apache Iceberg REST Catalog API + license: + name: Apache 2.0 + url: https://www.apache.org/licenses/LICENSE-2.0.html + version: 1.0.0 + description: + Defines the specification for the first version of the REST Catalog API. Implementations should support both Iceberg table specs v1 and v2, with priority given to v2. +servers: + - url: http://127.0.0.1:1080 + description: URL Used for Mock-Server Unit Tests +# All routes are currently configured using an Authorization header. +security: + - BearerAuth: [] +paths: + /v1/config: + get: + tags: + - Configuration API + summary: List all catalog configuration settings + operationId: getConfig + description: + All REST catalogs will be initialized by calling this route. This route + will return at least the minimum necessary metadata to initialize the + catalog. Optionally, it can also include server-side specific overrides. + For example, it might also include information used to initialize this catalog + such as the details of the Http connection pooling, etc. This route might + also advertise information about operations that are not implemented + so that the catalog can eagerly throw or go about another way of performing + the desired action. + responses: + default: + description: Server-Specific Configuration Overrides + content: + application/json: + schema: + $ref: '#/components/schemas/IcebergConfiguration' + "400": + description: Unknown Error + "401": + description: Unauthorized + + /v1/namespaces/{namespace}/tables/{table}: Review comment: Why not just `v1/namespaces/{namespace}/{namespace}/{table}`? specifically why the extra `tables`? ########## File path: rest_docs/rest-catalog-open-api-v0.1.yaml ########## @@ -0,0 +1,727 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +openapi: 3.0.3 +info: + title: Apache Iceberg REST Catalog API + license: + name: Apache 2.0 + url: https://www.apache.org/licenses/LICENSE-2.0.html + version: 1.0.0 + description: + Defines the specification for the first version of the REST Catalog API. Implementations should support both Iceberg table specs v1 and v2, with priority given to v2. +servers: + - url: http://127.0.0.1:1080 + description: URL Used for Mock-Server Unit Tests +# All routes are currently configured using an Authorization header. +security: + - BearerAuth: [] +paths: + /v1/config: + get: + tags: + - Configuration API + summary: List all catalog configuration settings + operationId: getConfig + description: + All REST catalogs will be initialized by calling this route. This route + will return at least the minimum necessary metadata to initialize the + catalog. Optionally, it can also include server-side specific overrides. + For example, it might also include information used to initialize this catalog + such as the details of the Http connection pooling, etc. This route might + also advertise information about operations that are not implemented + so that the catalog can eagerly throw or go about another way of performing + the desired action. + responses: + default: + description: Server-Specific Configuration Overrides + content: + application/json: + schema: + $ref: '#/components/schemas/IcebergConfiguration' + "400": + description: Unknown Error + "401": + description: Unauthorized + + /v1/namespaces/{namespace}/tables/{table}: + parameters: + - name: namespace + in: path + description: Namespace the table is in + required: true + schema: + type: string + examples: + singlepart_namespace: Review comment: why not use path separators as namespace separators. Also, what happens if the namespace has a `.` in it or other url characters? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
