davisusanibar commented on issue #12570: URL: https://github.com/apache/arrow/issues/12570#issuecomment-1060621861
Hi Team, sorry to join late Thank you @GavinRay97 , library are downloaded but it's invalid pom/jar. Related to update [Arrow Java Nightly Doc](https://github.com/apache/arrow/blob/650f111b524fb1c5bfbfa6f533d15929c90ddc40/docs/source/java/install.rst#installing-nightly-packages) ... I just reviewing the issue and I see 02 options: 1. Analyze/review/configure how to integrate/use github nightly release in a transparent manner 2. Define workaround to build arrow java nightly dependencies locally: - Add your repo to documentation - Define more generic integration (without modifying/adding more configuration) to add to the documentation Just working on a generic nightly build implementation using this shell script: Code to add to the docs: ``` #!/bin/bash # Shell variables ARROW_JAVA_NIGHTLY_VERSION=${1:-'nightly-2022-03-03-0-github-java-jars'} DEPENDENCY_TO_INSTALL=${2:-'arrow'} # Local Variables TMP_FOLDER=arrow_java_$(date +"%d-%m-%Y") PATTERN_TO_GET_LIB_AND_VERSION='([a-z].+)-([0-9].[0-9].[0-9].dev[0-9]+).([a-z]+)' # Aplication logic echo $DEPENDENCY_TO_INSTALL mkdir -p $TMP_FOLDER pushd $TMP_FOLDER echo "**************** 1 - Download arrow-java $1 dependencies ****************" wget $( \ wget \ -qO- https://api.github.com/repos/ursacomputing/crossbow/releases/tags/$ARROW_JAVA_NIGHTLY_VERSION \ | jq -r '.assets[] | select((.name | endswith(".pom")) or (.name | endswith(".jar"))) | .browser_download_url' \ | grep $DEPENDENCY_TO_INSTALL ) echo "**************** 2 - Install arrow java libraries to local repository ****************" for LIBRARY in $(ls | grep -E '.jar' | grep dev); do [[ $LIBRARY =~ $PATTERN_TO_GET_LIB_AND_VERSION ]] FILE=$PWD/${BASH_REMATCH[0]} if [[ ( ${BASH_REMATCH[0]} == *"$DEPENDENCY_TO_INSTALL"* ) ]];then if [ -f "$FILE" ]; then FILE=$FILE else if [ -f "$FILE.jar" ]; then # Out of regex: -javadoc.jar / -sources.jar FILE=$FILE.jar else if [ -f "$FILE-with-dependencies.jar" ]; then # Out of regex: -with-dependencies.jar FILE=$FILE-with-dependencies.jar else echo "Please! Review $FILE, it was not intalled on m2 locally." fi fi fi echo "$FILE" mvn install:install-file \ -Dfile="$FILE" \ -DgroupId=org.apache.arrow \ -DartifactId=${BASH_REMATCH[1]} \ -Dversion=${BASH_REMATCH[2]} \ -Dpackaging=${BASH_REMATCH[3]} \ -DcreateChecksum=true \ -Dgenerate.pom=true fi done popd # rm -rf $TMP_FOLDER echo "Go to your project and execute: mvn clean install" ``` Execute: Download all dependencies / Or only jar needed ``` # Download all dependencies sh arrow_java_nightly.sh nightly-2022-03-03-0-github-java-jars # Download needed library, for example: memory sh arrow_java_nightly.sh nightly-2022-03-03-0-github-java-jars memory ``` Use: Go to your pom.xml add dependencies and version needed ``` ... <properties> <arrow.version>8.0.0.dev165</arrow.version> </properties> <dependencies> <dependency> <groupId>org.apache.arrow</groupId> <artifactId>arrow-memory-core</artifactId> <version>${arrow.version}</version> </dependency> <dependency> <groupId>org.apache.arrow</groupId> <artifactId>arrow-memory-netty</artifactId> <version>${arrow.version}</version> </dependency> <dependency> <groupId>ch.qos.logback</groupId> <artifactId>logback-classic</artifactId> <version>${logback.version}</version> </dependency> <dependency> <groupId>org.apache.arrow</groupId> <artifactId>flight-core</artifactId> <version>${arrow.version}</version> </dependency> </dependencies> ... ``` Run: ``` mvn clean install ``` Please if you could help me if this work on your side. Thank you in advance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
