henrymai opened a new pull request, #35246:
URL: https://github.com/apache/arrow/pull/35246
<!--
Thanks for opening a pull request!
If this is your first pull request you can find detailed information on how
to contribute here:
* [New Contributor's
Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
* [Contributing
Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
If this is not a [minor
PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes).
Could you open an issue for this pull request on GitHub?
https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the
[Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.)
of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
In the case of PARQUET issues on JIRA the title also supports:
PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
-->
### Rationale for this change
Enables GCS when building the Arrow Dataset for Java and also fixes various
java build failures.
Currently we are using our own custom Arrow Dataset build with GCS turned
on, but we would rather this be enabled in the official releases from Arrow.
GCS support is already enabled for cpp, python, ruby, python, and r already,
so there should be no reason not to enable this on java as well.
### What changes are included in this PR?
- Changes to enable GCS for Java Arrow Dataset on both Linux and Mac Os
- Fixes to flight-sql-jdbc-driver/pom.xml. Without these fixes the
flight-sql-jdbc-driver build will fail with the following errors:
```
[WARNING] Used undeclared dependencies found:
[WARNING] org.bouncycastle:bcpkix-jdk15on:jar:1.61:runtime
[WARNING]
org.apache.arrow:arrow-memory-core:jar:12.0.0-SNAPSHOT:runtime
[WARNING] org.hamcrest:hamcrest:jar:2.2:runtime
[WARNING] org.apache.arrow:flight-sql:jar:12.0.0-SNAPSHOT:runtime
[WARNING] org.mockito:mockito-core:jar:2.25.1:test
[WARNING] org.apache.arrow:flight-core:jar:12.0.0-SNAPSHOT:runtime
[WARNING] org.slf4j:slf4j-api:jar:1.7.25:runtime
[WARNING] io.netty:netty-common:jar:4.1.82.Final:runtime
[WARNING] joda-time:joda-time:jar:2.10.14:runtime
[WARNING] org.apache.calcite.avatica:avatica:jar:1.18.0:runtime
[WARNING] com.google.protobuf:protobuf-java:jar:3.21.6:runtime
[WARNING] org.apache.arrow:arrow-vector:jar:12.0.0-SNAPSHOT:runtime
[WARNING] com.google.guava:guava:jar:31.1-jre:runtime
[...]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (analyze)
on project flight-sql-jdbc-driver: Dependency problems found -> [Help 1]
```
And also fail with:
Caused by: java.lang.NullPointerException: Could not find test data
path. Set the environment variable ARROW_TEST_DATA or the JVM property
arrow.test.dataRoot.
at java.util.Objects.requireNonNull(Objects.java:228)
at
org.apache.arrow.driver.jdbc.utils.FlightSqlTestCertificates.getTestDataRoot(FlightSqlTestCertificates.java:40)
at
org.apache.arrow.driver.jdbc.utils.FlightSqlTestCertificates.getFlightTestDataRoot(FlightSqlTestCertificates.java:51)
at
org.apache.arrow.driver.jdbc.utils.FlightSqlTestCertificates.exampleTlsCerts(FlightSqlTestCertificates.java:60)
at
org.apache.arrow.driver.jdbc.ConnectionTlsTest.<clinit>(ConnectionTlsTest.java:59)
### Are these changes tested?
I've tested the build with running:
```
$HOME/.local/bin/archery docker run java-jni-manylinux-2014
```
However, I have not tested the Mac Os build since I don't have a Mac to test
with.
I've also tested the resulting java-dist jar output from running the command
and have verified that GCS support is enabled.
### Are there any user-facing changes?
Yes, Java Arrow Dataset will now work with GCS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
