sarutak opened a new pull request, #55198: URL: https://github.com/apache/spark/pull/55198
### What changes were proposed in this pull request? This PR fixes a release build failure which recently happens at document generation phase due to error on fetching artifacts. https://github.com/apache/spark/actions/runs/23083193766/job/67055792458 ``` Error: lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts: Error: file:/home/spark-rm/.m2/repository/io/netty/netty-codec-protobuf/4.2.10.Final/netty-codec-protobuf-4.2.10.Final.jar: not found: /home/spark-rm/.m2/repository/io/netty/netty-codec-protobuf/4.2.10.Final/netty-codec-protobuf-4.2.10.Final.jar Error: file:/home/spark-rm/.m2/repository/io/netty/netty-codec-marshalling/4.2.10.Final/netty-codec-marshalling-4.2.10.Final.jar: not found: /home/spark-rm/.m2/repository/io/netty/netty-codec-marshalling/4.2.10.Final/netty-codec-marshalling-4.2.10.Final.jar Error: Error: at lmcoursier.internal.shaded.coursier.Artifacts$.$anonfun$fetchArtifacts$9(Artifacts.scala:365) Error: at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$extension$1(Task.scala:14) Error: at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$extension$1$adapted(Task.scala:14) Error: at lmcoursier.internal.shaded.coursier.util.Task$.wrap(Task.scala:82) Error: at lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14) Error: at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307) Error: at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:51) Error: at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74) Error: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) Error: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) Error: at java.base/java.lang.Thread.run(Thread.java:840) Error: Caused by: lmcoursier.internal.shaded.coursier.cache.ArtifactError$NotFound: not found: /home/spark-rm/.m2/repository/io/netty/netty-codec-protobuf/4.2.10.Final/netty-codec-protobuf-4.2.10.Final.jar Error: at lmcoursier.internal.shaded.coursier.cache.internal.Downloader.$anonfun$checkFileExists$1(Downloader.scala:603) Error: at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) Error: at scala.util.Success.$anonfun$map$1(Try.scala:255) Error: at scala.util.Success.map(Try.scala:213) Error: at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) Error: at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42) Error: at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74) Error: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) Error: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) Error: at java.base/java.lang.Thread.run(Thread.java:840) Error: (streaming-kinesis-asl / update) lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts: Error: file:/home/spark-rm/.m2/repository/io/netty/netty-codec-protobuf/4.2.10.Final/netty-codec-protobuf-4.2.10.Final.jar: not found: /home/spark-rm/.m2/repository/io/netty/netty-codec-protobuf/4.2.10.Final/netty-codec-protobuf-4.2.10.Final.jar Error: file:/home/spark-rm/.m2/repository/io/netty/netty-codec-marshalling/4.2.10.Final/netty-codec-marshalling-4.2.10.Final.jar: not found: /home/spark-rm/.m2/repository/io/netty/netty-codec-marshalling/4.2.10.Final/netty-codec-marshalling-4.2.10.Final.jar Error: Total time: 368 s (0:06:08.0), completed Mar 14, 2026, 8:40:17 AM ------------------------------------------------ Jekyll 4.4.1 Please append `--trace` to the `build` command for any additional information or backtrace. ------------------------------------------------ ``` This issue is similar to SPARK-34762 and SPARK-37302 in that there are pom files but are not corresponding jar files under `.m2` for some dependencies. In this case, the following command is executed through `make-distribution.sh` downloads pom files for `xz:1.10`, `netty-codec-protobuf` and `netty-codec-marshalling` ``` build/mvn clean package -DskipTests -Dmaven.javadoc.skip=true -Dmaven.scaladoc.skip=true -Dmaven.source.skip -Dcyclonedx.skip=true -B -Pyarn -Pkubernetes -Phadoop-3 -Phive -Phive-thriftserver ``` And when building documents, the following command is executed, requires the dependencies. ``` NO_PROVIDED_SPARK_JARS=0 build/sbt -Phive -Pkinesis-asl clean package ``` Regarding xz, `1.12` is declared in `pom.xml` so this PR fixes `SparkBuild.scala` to pin the version. Regarding `netty-codec-protobuf` and `netty-codec-marshalling`, they are declared in pom.xml to be excluded. So, this PR fixes `SparkBuild.scala` to exclude them. ### Why are the changes needed? To recover the release build. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? The following command successfully finishes on my laptop. ``` $ build/mvn clean package -DskipTests -Dmaven.javadoc.skip=true -Dmaven.scaladoc.skip=true -Dmaven.source.skip -Dcyclonedx.skip=true -B -Pyarn -Pkubernetes -Phadoop-3 -Phive -Phive-thriftserver $ SKIP_SCALADOC=1 SKIP_RDOC=1 SKIP_SQLDOC=1 bundle exec jekyll build ``` Note that to build documents, please follow the instructions in `docs/README.md` ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
