morningman opened a new pull request, #66358: URL: https://github.com/apache/doris/pull/66358
### What problem does this PR solve? Related Issue: #65615 Related PR: #66331 Problem Summary: Split out of #66331, which adds an `adbc` catalog type that reads an external source through an [Arrow Database Connectivity](https://arrow.apache.org/adbc/) driver. This PR carries only that PR's `thirdparty/` half, so the dependency can be reviewed and the build-env image rebuilt before the code that links against it lands. Nothing in the tree consumes these artifacts yet -- this PR adds one package to the thirdparty build and declares its license, and changes nothing else. **What comes out of it** | Artifact | How | Used by | |---|---|---| | `libadbc_driver_manager.a` | built from source | to be statically linked into `doris_be` (#66331) | | `libadbc_driver_jni.so` | built from source | to be loaded by the FE ADBC connector (#66331) | | `libadbc_driver_sqlite.so` | built from source | tests only, **not shipped** | | `libadbc_driver_flightsql.so` | prebuilt, from the official release wheel | tests only, **not shipped** | Doris ships no ADBC driver to users; a deployment supplies its own. The two drivers above exist so the ADBC code paths can be tested at all. **Three things upstream does that do not carry over** - *The SQLite driver needs a system SQLite3 development package*, which Doris does not ship and most build hosts lack. The source tree vendors the amalgamation but never references it from CMake, so it is compiled here into a scratch static library, handed to `FindSQLite3`, and dropped afterwards. It ends up statically inside the driver, leaving no sqlite artifacts in thirdparty. - *The JNI bridge header is generated by shelling out to Maven* (`java/driver/jni/CMakeLists.txt` runs `mvn -Pjni,javah compile`). Doing that would make this the first thirdparty package to require Maven, a Maven Central connection and a JDK 11+, while the build-env image runs this script with `JAVA_HOME` on JDK 8. The `javah` output is checked in as a patch instead and `jni_wrapper.cc` is compiled against it directly, needing nothing but `jni.h`. The patch header records how to regenerate it on a version bump. The prebuilt JNI binary inside upstream's Maven jar is not used either: it requires `GLIBC_2.34` and `GLIBCXX_3.4.31`, which excludes CentOS 7/8, Rocky 8 and Ubuntu 20.04. - *The Flight SQL driver is written in Go and no bare shared library is published*, so it is taken from the official release wheel -- a zip the existing download step already knows how to unpack -- rather than adding a Go toolchain to the thirdparty build. It is skipped on platforms upstream publishes no prebuilt binary for, the same way hyperscan is. **On the version pin** The source tree is tag `apache-arrow-adbc-24`, which is release C/Go 1.12.0 (the tag carries neither number). The prebuilt Flight SQL driver is pinned to that same release, and that is not cosmetic: ADBC partition descriptors are driver-private bytes, so every process that hands one to another must have loaded the very same driver build. `dist/LICENSE-dist.txt` gets the corresponding Apache-2.0 entry. It is the only file outside `thirdparty/` here. ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [x] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [x] Other reason: thirdparty build scripts only. The gate for them is the `Build Third Party Libraries` workflow; this exact content passed its Linux, macOS and macOS-arm64 jobs in #66331 ([run](https://github.com/apache/doris/actions/runs/30679875385)). - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
