discivigour opened a new pull request, #160:
URL: https://github.com/apache/paimon-rust/pull/160
### Purpose
Linked issue: close #119
Implement a complete REST-based catalog (`RESTCatalog`) for Apache Paimon
Rust,
supporting database and table CRUD operations, token-based FileIO for OSS
data access,
and a mock server for integration testing.
### Brief change log
**New files:**
- `catalog/rest/mod.rs`: REST catalog module entry point.
- `catalog/rest/rest_catalog.rs`: `RESTCatalog` implementing the `Catalog`
trait with full
database and table CRUD (list, create, get, drop, alter, rename).
- `catalog/rest/rest_token.rs`: `RESTToken` struct for table-level data
access credentials.
- `catalog/rest/rest_token_file_io.rs`: `RESTTokenFileIO` — a `FileIO`
wrapper that lazily
fetches and caches table tokens from the REST server, enabling OSS data
access with
short-lived credentials.
- `catalog/database.rs`: `Database` struct representing a catalog database.
- `examples/rest_catalog_example.rs`: End-to-end example for REST catalog
operations.
- `examples/rest_catalog_read_append_example.rs`: Example for reading
append-only tables via REST catalog.
- `tests/mock_server.rs`: Mock HTTP server (axum-based) simulating the
Paimon REST API for testing.
- `tests/rest_catalog_test.rs`: Integration tests covering all CRUD
operations on the mock server.
**Modified files:**
- `api/api_response.rs`: Add `GetTableTokenResponse`; derive `PartialEq, Eq`.
- `api/auth/base.rs`: Add `Sync` bound to `AuthProvider` trait.
- `api/mod.rs`: Re-export `GetTableTokenResponse`.
- `api/rest_api.rs`: Add `options()` accessor and `load_table_token()`
method; fix variable
shadowing in config merge; remove `#[allow(dead_code)]` on `options` field.
- `api/rest_util.rs`: Add `RESTUtil::merge()` utility for merging property
maps.
- `catalog/mod.rs`: Register `database` and `rest` modules; add
`get_database` to `Catalog` trait.
- `catalog/filesystem.rs`: Implement `get_database` for `FileSystemCatalog`.
- `io/storage_oss.rs`: Use `HashMap::remove` instead of `get` + clone for
`security_token`.
- `integration_tests/tests/read_tables.rs`: Add REST catalog read tests
using mock server.
- `integration_tests/Cargo.toml`: Add `serde_json` dev dependency.
### Tests
- `tests/rest_catalog_test.rs`: Full CRUD integration tests
(list/create/get/drop/alter database,
list/create/get/drop/rename/alter table) against the mock server.
- `tests/rest_api_test.rs`: REST API layer tests including DLF ECS token
loader.
- `integration_tests/tests/read_tables.rs`: End-to-end read tests for
append-only and
primary-key (deletion-vector) tables via REST catalog backed by mock
server.
- `cargo test -p paimon` passes.
### API and Format
- New public types: `RESTCatalog`, `RESTToken`, `RESTTokenFileIO`,
`Database`, `GetTableTokenResponse`.
- New `Catalog::get_database` method added to the trait (breaking change for
existing implementors).
- No storage format changes.
### Documentation
- Two runnable examples added under `crates/paimon/examples/`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
