[
https://issues.apache.org/jira/browse/SPARK-49875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arsen Gumin updated SPARK-49875:
--------------------------------
Issue Type: Bug (was: Improvement)
> error build spark 3.5.3
> -----------------------
>
> Key: SPARK-49875
> URL: https://issues.apache.org/jira/browse/SPARK-49875
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.5.3
> Environment: docker image in description
> Reporter: Arsen Gumin
> Priority: Major
>
> i try create make-distribution.sh and get error. maybe I did something wrong?
> what did you try to do:
> # Update scala-maven-plugin verison (all upper 4.8)
> # Downgrade scala-maven-plugin (4.7.*)
> # Build without my patch (patch -p1 < $SPARK_HOME/spark-42411.patch)
> # change scala and java version(8,11,17)
>
> {code:java}
> // docker image with build
> FROM eclipse-temurin:17
> ENV MAVEN_OPTS="-Xss64m -Xmx2g -XX:ReservedCodeCacheSize=1g"
> ENV SCALA_VERSION=2.13
> ENV SPARK_VERSION=3.5.3
> ENV SPARK_HOME=/opt/spark
> ENV
> SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}.tgz
> WORKDIR /opt
> ARG spark_uid=185
> RUN groupadd --system --gid=${spark_uid} spark && \
> useradd --system --uid=${spark_uid} --gid=spark spark
> RUN apt-get update; \
> apt-get install -y wget patch gettext-base gnupg2 bash tini libc6
> libpam-modules krb5-user libnss3 procps net-tools gosu libnss-wrapper; \
> rm -rf /var/lib/apt/lists/*
> RUN set -ex; \
> mkdir -p $SPARK_HOME; \
> wget -nv -O /opt/spark.tgz "$SPARK_TGZ_URL"; \
> tar -zxf /opt/spark.tgz --strip-components=1 --directory=$SPARK_HOME; \
> chown -R spark:spark .;
> WORKDIR $SPARK_HOME
> COPY mtls/spark-42411.patch .
> RUN patch -p1 < $SPARK_HOME/spark-42411.patch
> RUN ./dev/change-scala-version.sh ${SCALA_VERSION}
> RUN ./dev/make-distribution.sh \
> --name spark-mtls \
> --pip \
> -P"scala-${SCALA_VERSION}" \
> -Pconnect \
> -Pkubernetes
> {code}
>
>
> error
>
>
> {code:java}
> //
> [INFO] Compiler bridge file:
> /root/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.13-1.9.5-bin_2.13.8__61.0-1.9.5_20230915T135648.jar
> [INFO] Compiler bridge file is not installed yet
> Downloading from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/scala-sbt/compiler-bridge_2.13/1.9.5/compiler-bridge_2.13-1.9.5-sources.jar
> Downloaded from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/scala-sbt/compiler-bridge_2.13/1.9.5/compiler-bridge_2.13-1.9.5-sources.jar
> (53 kB at 228 kB/s)
> Downloading from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/jline/jline/3.22.0/jline-3.22.0.pom
> Downloaded from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/jline/jline/3.22.0/jline-3.22.0.pom
> (18 kB at 99 kB/s)
> Downloading from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/jline/jline-parent/3.22.0/jline-parent-3.22.0.pom
> Downloaded from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/jline/jline-parent/3.22.0/jline-parent-3.22.0.pom
> (28 kB at 137 kB/s)
> Downloading from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/jline/jline/3.22.0/jline-3.22.0.jar
> Downloaded from gcs-maven-central-mirror:
> https://maven-central.storage-download.googleapis.com/maven2/org/jline/jline/3.22.0/jline-3.22.0.jar
> (1.1 MB at 1.2 MB/s)
> warning: 4 deprecations (since 2.12.0)
> warning: 1 deprecation (since 2.12.5)
> warning: 2 deprecations (since 2.12.9)
> warning: 2 deprecations (since 2.13)
> warning: 8 deprecations (since 2.13.0)
> warning: 1 deprecation (since 2.13.4)
> warning: 18 deprecations in total; re-run with -deprecation for details
> warning: 2 feature warnings; re-run with -feature for details
> 8 warnings
> [INFO] Compiler bridge installed
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary for Spark Project Parent POM 3.5.3:
> [INFO]
> [INFO] Spark Project Parent POM ........................... SUCCESS [03:47
> min]
> [INFO] Spark Project Sketch ............................... FAILURE [ 20.618
> s]
> [INFO] Spark Project Local DB ............................. SKIPPED
> [INFO] Spark Project Networking ........................... SKIPPED
> [INFO] Spark Project Shuffle Streaming Service ............ SKIPPED
> [INFO] Spark Project Unsafe ............................... SKIPPED
> [INFO] Spark Project Common Utils ......................... SKIPPED
> [INFO] Spark Project Tags ................................. SKIPPED
> [INFO] Spark Project Core ................................. SKIPPED
> [INFO] Spark Project GraphX ............................... SKIPPED
> [INFO] Spark Project ML Library ........................... SKIPPED
> [INFO] Spark Project ML Local Library ..................... SKIPPED
> [INFO] Spark Project Tools ................................ SKIPPED
> [INFO] Spark Project Streaming ............................ SKIPPED
> [INFO] Spark Project SQL API .............................. SKIPPED
> [INFO] Spark Project Catalyst ............................. SKIPPED
> [INFO] Spark Project SQL .................................. SKIPPED
> [INFO] Spark Project Hive ................................. SKIPPED
> [INFO] Spark Project Assembly ............................. SKIPPED
> [INFO] Spark Project Examples ............................. SKIPPED
> [INFO] Spark Project REPL ................................. SKIPPED
> [INFO] Spark Project Launcher ............................. SKIPPED
> [INFO] Kafka 0.10+ Token Provider for Streaming ........... SKIPPED
> [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
> [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
> [INFO] Kafka 0.10+ Source for Structured Streaming ........ SKIPPED
> [INFO] Spark Avro ......................................... SKIPPED
> [INFO] Spark Project Connect Server ....................... SKIPPED
> [INFO] Spark Project Connect Common ....................... SKIPPED
> [INFO] Spark Project Connect Client ....................... SKIPPED
> [INFO] Spark Protobuf ..................................... SKIPPED
> [INFO] Spark Project Kubernetes ........................... SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 04:09 min
> [INFO] Finished at: 2024-10-04T10:16:17Z
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:4.9.0:compile (scala-compile-first) on
> project spark-sketch_2.12: Execution scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:4.9.0:compile failed:
> org.apache.commons.exec.ExecuteException: Process exited with an error: 1
> (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <args> -rf :spark-sketch_2.12
> Error: building at STEP "RUN ./dev/make-distribution.sh --name
> spark-mtls --pip -P"scala-${SCALA_VERSION}" -Pconnect
> -Pkubernetes -Dscala-maven-plugin.version=4.9.0": while running
> runtime: exit status 1 {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
