This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.1 by this push: new 4bda955 [SPARK-34762][BUILD] Fix the build failure with Scala 2.13 which is related to commons-cli 4bda955 is described below commit 4bda9558b80ae2cf28c62377c0633012f1189e72 Author: Kousuke Saruta <saru...@oss.nttdata.com> AuthorDate: Thu Mar 18 12:31:50 2021 +0900 [SPARK-34762][BUILD] Fix the build failure with Scala 2.13 which is related to commons-cli ### What changes were proposed in this pull request? This PR fixes the build failure with Scala 2.13 which is related to `commons-cli`. The last few days, build with Scala 2.13 on GA continues to fail and the error message says like as follows. ``` [error] /home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/server/HiveServer2.java:26:1: error: package org.apache.commons.cli does not exist 1278[error] import org.apache.commons.cli.GnuParser; ``` The reason is that `mvn help` in `change-scala-version.sh` downloads the POM file of `commons-cli` but doesn't download the JAR file, leading the build failure. This PR also adds `commons-cli` to the dependencies explicitly because HiveThriftServer depends on it. ### Why are the changes needed? Expect to fix the build failure with Scala 2.13. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? I confirmed that build successfully finishes with Scala 2.13 on my laptop. ``` find ~/.m2 -name commons-cli -exec rm -rf {} \; find ~/.ivy2 -name commons-cli -exec rm -rf {} \; find ~/.cache/ -name commons-cli -exec rm -rf {} \; // For Linux find ~/Library/Caches -name commons-cli -exec rm -rf {} \; // For macOS dev/change-scala-version 2.13 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Pdocker-integration-tests -Pkubernetes-integration-tests -Pspark-ganglia-lgpl -Pscala-2.13 clean compile test:compile ``` Closes #31862 from sarutak/commons-cli. Authored-by: Kousuke Saruta <saru...@oss.nttdata.com> Signed-off-by: HyukjinKwon <gurwls...@apache.org> (cherry picked from commit c5cadfefdf9b4c6135355b49366fd9e9d1e3fcd0) Signed-off-by: HyukjinKwon <gurwls...@apache.org> --- dev/change-scala-version.sh | 5 +++++ pom.xml | 6 ++++++ sql/hive-thriftserver/pom.xml | 4 ++++ 3 files changed, 15 insertions(+) diff --git a/dev/change-scala-version.sh b/dev/change-scala-version.sh index 9cdc7d9..6adfb23 100755 --- a/dev/change-scala-version.sh +++ b/dev/change-scala-version.sh @@ -60,6 +60,11 @@ BASEDIR=$(dirname $0)/.. find "$BASEDIR" -name 'pom.xml' -not -path '*target*' -print \ -exec bash -c "sed_i 's/\(artifactId.*\)_'$FROM_VERSION'/\1_'$TO_VERSION'/g' {}" \; +# dependency:get is workaround for SPARK-34762 to download the JAR file of commons-cli. +# Without this, build with Scala 2.13 using SBT will fail because the help plugin used below downloads only the POM file. +COMMONS_CLI_VERSION=`build/mvn help:evaluate -Dexpression=commons-cli.version -q -DforceStdout` +build/mvn dependency:get -Dartifact=commons-cli:commons-cli:${COMMONS_CLI_VERSION} -q + # Update <scala.version> in parent POM # First find the right full version from the profile's build SCALA_VERSION=`build/mvn help:evaluate -Pscala-${TO_VERSION} -Dexpression=scala.version -q -DforceStdout` diff --git a/pom.xml b/pom.xml index 6f2dfd1..77c4ea8 100644 --- a/pom.xml +++ b/pom.xml @@ -199,6 +199,7 @@ <paranamer.version>2.8</paranamer.version> <maven-antrun.version>1.8</maven-antrun.version> <commons-crypto.version>1.1.0</commons-crypto.version> + <commons-cli.version>1.2</commons-cli.version> <!-- If you are changing Arrow version specification, please check ./python/pyspark/sql/pandas/utils.py, and ./python/setup.py too. @@ -2379,6 +2380,11 @@ </exclusion> </exclusions> </dependency> + <dependency> + <groupId>commons-cli</groupId> + <artifactId>commons-cli</artifactId> + <version>${commons-cli.version}</version> + </dependency> </dependencies> </dependencyManagement> diff --git a/sql/hive-thriftserver/pom.xml b/sql/hive-thriftserver/pom.xml index 7cc9c3b..92893af 100644 --- a/sql/hive-thriftserver/pom.xml +++ b/sql/hive-thriftserver/pom.xml @@ -133,6 +133,10 @@ <groupId>net.sf.jpam</groupId> <artifactId>jpam</artifactId> </dependency> + <dependency> + <groupId>commons-cli</groupId> + <artifactId>commons-cli</artifactId> + </dependency> </dependencies> <build> <outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org