Davis-Zhang-Onehouse opened a new pull request, #20723:
URL: https://github.com/apache/hudi/pull/20723

   ### Describe the issue this Pull Request addresses
   
   issue: #20722
   Part of #20683, RFC-118 (#20684).
   Stacked on #20721: review only the last commit. Until the PRs below it 
merge, this PR also shows their commits.
   
   Speech interfaces of the embedding round (audio segments, transcripts, chunk 
ids and the transcript packer) and the Whisper speech model on whisper-jni 
(whisper.cpp and ggml). Whisper models are not shipped.
   
   ### Summary and Changelog
   
   - Speech interfaces: audio segments, transcripts, audio chunk ids and the 
transcript packer.
   - The Whisper speech model on whisper-jni (whisper.cpp and ggml): segment 
transcriber, context pool with handle release, native library loading and 
config.
   - The utilities bundles carry whisper-jni with its two Linux native folders 
only; the other platforms' prebuilt natives are filtered out. Whisper models 
are not shipped; a job points the config at a model file.
   - `LICENSE` gains the whisper.cpp/ggml MIT entry and a note on the 
public-domain Gutenberg text and LibriVox transcript used in test resources.
   - 7 test classes.
   
   Upstream files touched: root `pom.xml`, `hudi-utilities/pom.xml`, both 
utilities bundle poms, `LICENSE`. No source code was copied from other projects.
   
   ### Impact
   
   Adds speech transcription to the embedding stream. Bundles gain whisper-jni 
with Linux natives.
   
   ### Risk Level
   
   medium: new native dependency in the bundles. Verified on JDK 11 / Spark 3.5 
/ Scala 2.12 with 73 tests (none skipped), module compilation, checkstyle and 
RAT. Both bundles were checked to carry only the two Linux native folders.
   
   ### Documentation Update
   
   New speech settings carry their config documentation; a website page follows 
once the chain lands.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [PR] feat(ingest): RFC-118: ... Davis-Zhang-Onehouse (via GitHub) via commits

Reply via email to