Re: [PR] [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' [spark]
yaooqinn closed pull request #46457: [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' URL: https://github.com/apache/spark/pull/46457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' [spark]
LuciferYang commented on code in PR #46457:
URL: https://github.com/apache/spark/pull/46457#discussion_r1593672542
##
sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:
##
@@ -197,8 +197,8 @@ trait SparkDateTimeUtils {
rebaseJulianToGregorianDays(julianDays)
}
- private val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
- private val getOffsetsByWallHandle = {
+ private lazy val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
Review Comment:
Not sure, but:
1. Using `TimeZone.getDefault.getOffset(localMillis)`, all tests can pass,
perhaps the unit test coverage is not comprehensive, but what would be the
counterexample?
2. Looking at the inverse function `fromJavaDate` of `toJavaDate`, it does
not take into account the scenario of `ZoneInfo`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' [spark]
yaooqinn commented on code in PR #46457:
URL: https://github.com/apache/spark/pull/46457#discussion_r1593525386
##
sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:
##
@@ -197,8 +197,8 @@ trait SparkDateTimeUtils {
rebaseJulianToGregorianDays(julianDays)
}
- private val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
- private val getOffsetsByWallHandle = {
+ private lazy val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
Review Comment:
```suggestion
private val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' [spark]
yaooqinn commented on code in PR #46457:
URL: https://github.com/apache/spark/pull/46457#discussion_r1593465736
##
sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:
##
@@ -197,8 +197,8 @@ trait SparkDateTimeUtils {
rebaseJulianToGregorianDays(julianDays)
}
- private val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
- private val getOffsetsByWallHandle = {
+ private lazy val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
Review Comment:
> On the other hand, if TimeZone.getDefault.getOffset(localMillis) can be
uniformly used to get timeZoneOffset in the toJavaDate function, then the use
of the MethodHandle mechanism can be avoided here.
It seems that `ZoneInfo.getOffset` doesn't support daylight saving
transitions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' [spark]
LuciferYang commented on code in PR #46457:
URL: https://github.com/apache/spark/pull/46457#discussion_r1593383893
##
sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:
##
@@ -197,8 +197,8 @@ trait SparkDateTimeUtils {
rebaseJulianToGregorianDays(julianDays)
}
- private val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
- private val getOffsetsByWallHandle = {
+ private lazy val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
Review Comment:
On the other hand, if `TimeZone.getDefault.getOffset(localMillis)` can be
uniformly used to get `timeZoneOffset` in the `toJavaDate` function, then the
use of the `MethodHandle` mechanism can be avoided here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' [spark]
LuciferYang commented on code in PR #46457:
URL: https://github.com/apache/spark/pull/46457#discussion_r1593382052
##
sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:
##
@@ -197,8 +197,8 @@ trait SparkDateTimeUtils {
rebaseJulianToGregorianDays(julianDays)
}
- private val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
- private val getOffsetsByWallHandle = {
+ private lazy val zoneInfoClassName = "sun.util.calendar.ZoneInfo"
Review Comment:
Is it necessary to use `lazy` for this since `zoneInfoClassName` is just a
`String`? 'Lazy' comes with the cost of locking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[PR] [SPARK-48185][SQL] Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' [spark]
yaooqinn opened a new pull request, #46457: URL: https://github.com/apache/spark/pull/46457 ### What changes were proposed in this pull request? I met the error below while debugging UTs because of loading `sun.util.calendar.ZoneInfo` eagerly. This PR makes the relevant variables lazy. ```log Caused by: java.lang.IllegalAccessException: symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo, from interface org.apache.spark.sql.catalyst.util.SparkDateTimeUtils (unnamed module @65d6b83b) at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:955) at java.base/java.lang.invoke.MethodHandles$Lookup.checkSymbolicClass(MethodHandles.java:3686) at java.base/java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:3646) at java.base/java.lang.invoke.MethodHandles$Lookup.findVirtual(MethodHandles.java:2680) at org.apache.spark.sql.catalyst.util.SparkDateTimeUtils.$init$(SparkDateTimeUtils.scala:206) at org.apache.spark.sql.catalyst.util.DateTimeUtils$.(DateTimeUtils.scala:41) ... 82 more ``` ### Why are the changes needed? sun.util.calendar.ZoneInfo is inaccessible in some scenarios. ### Does this PR introduce _any_ user-facing change? Yes, such errors might be delayed from backend-scheduling to job-scheduling ### How was this patch tested? I tested with idea and UT debugging locally ### Was this patch authored or co-authored using generative AI tooling? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
