eejbyfeldt opened a new pull request, #41943:
URL: https://github.com/apache/spark/pull/41943
### What changes were proposed in this pull request?
Drop hardcoded `--target:jvm-1.8` value from scalac argument in pom.xml.
<!--
Please clarify what changes you are proposing. The purpose of this section
is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster
reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class
hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other
DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
Build using maven is broken using 2.13 and Java 11 or later.
It fails with
```
$ ./build/mvn compile -Pscala-2.13 -Djava.version=11 -X
...
[WARNING] [Warn] : [deprecation @ | origin= | version=] -target is
deprecated: Use -release instead to compile against the correct platform API.
[ERROR] [Error] : target platform version 8 is older than the release
version 11
[WARNING] one warning found
[ERROR] one error found
...
```
if setting the `java.version` property or
```
$ ./build/mvn compile -Pscala-2.13
...
[WARNING] [Warn] : [deprecation @ | origin= | version=] -target is
deprecated: Use -release instead to compile against the correct platform API.
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala:71:
not found: value sun
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:26:
not found: object sun
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:27:
not found: object sun
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:206:
not found: type DirectBuffer
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:210:
not found: type Unsafe
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:212:
not found: type Unsafe
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:213:
not found: type DirectBuffer
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:216:
not found: type DirectBuffer
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:236:
not found: type DirectBuffer
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:26:
Unused import
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:27:
Unused import
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala:452:
not found: value sun
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:26:
not found: object sun
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99:
not found: type SignalHandler
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99:
not found: type Signal
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:83:
not found: type Signal
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108:
not found: type SignalHandler
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108:
not found: value Signal
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:114:
not found: type Signal
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:116:
not found: value Signal
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:128:
not found: value Signal
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:26:
Unused import
[ERROR] [Error]
/home/eejbyfeldt/dev/apache/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:26:
Unused import
[WARNING] one warning found
[ERROR] 23 errors found
...
```
This is caused by that we in pom.xml hardcode that scalac should run with
`-target:jvm-1.8` (regardless of the value of `java.version`) this was fine for
scala 2.12.18 and scala 2.13.8 as the scala-maven-plugin would add the arg
`-target` based on the `java.version` property.
(https://github.com/davidB/scala-maven-plugin/blob/4.8.0/src/main/java/scala_maven/ScalaMojoSupport.java#L629-L648)
since this argument is later it took precedence over the value we hardcoded in
maven and everything works as expected.
The problem comes in scala 2.13.11 where `-target` is deprecated and
therefore the scala-maven-plugin uses the `-release` argument instead. The
first second failure about not being able to accessing `sun._` packages which
is expected behvaior when using `-release 8` see:
https://github.com/scala/bug/issues/12643 but if one sets `--release 11` when
using Java 11 access to `sun._` compile just fine.
Note: That builds using scala 2.13 and java 11 or later without setting
`java.version` to the appropriate value will still fail.
Note2: The java 8 builds still succeeds as the `rt.jar` is pased on the
`-bootclasspath` when using java8.
### Does this PR introduce _any_ user-facing change?
Fixes the maven build when using scala 2.13 and Java 11 or later.
### How was this patch tested?
Exising CI builds and manual builds locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
