Davis-Zhang-Onehouse opened a new pull request, #20694: URL: https://github.com/apache/hudi/pull/20694
### Describe the issue this Pull Request addresses issue: #20693 Part of #20683, RFC-118 (#20684). Stacked on #20692: review only the last commit. Until the PRs below it merge, this PR also shows their commits. Adds the embedding table contract that the later RFC-118 preprocessing and compute stages use, and the dependencies those stages need. ### Summary and Changelog - Embedding table core: `EmbeddingTableConfig`, `EmbeddingTableSchema`, `EmbeddingModelSpec`, `EmbeddingTableChecks`, `EmbeddingTableVersionException` (key schema, the `prep_only` table shape, column names and schema checks). - Embedding client interface (`EmbeddingClient`, `EmbeddingInput`, `EmbeddingResult`, `EmbeddingClientNames`) and `DummyEmbeddingClient` for tests. - `BlobColumns` (shared BLOB column names) and `TransientRetry`. - New dependencies, pinned in the root `pom.xml`: Apache PDFBox 3.0.8 (with FontBox) and jtokkit 1.1.0 (MIT). Both are added to `hudi-utilities` and to the utilities and utilities-slim bundles, relocated under `org.apache.hudi`. - `LICENSE` / `NOTICE`: blocks for PDFBox/FontBox and their notices (including the Unicode license v3 text they carry), jtokkit (MIT) and the openai/tiktoken vocabulary files bundled in jtokkit (MIT). The bundles keep Hudi's existing LICENSE convention, as other shaded dependencies do. - 7 test files. No source code was copied from other projects; the new attributions are for bundled binaries and data. ### Impact Adds classes used by later stages; no behaviour of existing features changes. The utilities bundles grow by the two relocated dependencies. ### Risk Level medium: new dependencies and bundle relocations. Verified on JDK 11 / Spark 3.5 / Scala 2.12 with 37 tests, module compilation, checkstyle and RAT. Both utilities bundles were rebuilt; in each, the relocated tokenizer loads its bundled vocabularies and round-trips text. ### Documentation Update New configuration fields carry their documentation. RFC-118 describes the embedding table; a website page follows once the chain lands. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
