Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun closed pull request #45075: [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 URL: https://github.com/apache/spark/pull/45075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1961643981 Thank you, @LuciferYang and @srowen . Let me merge this first to see Daily Maven runs, too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on code in PR #45075: URL: https://github.com/apache/spark/pull/45075#discussion_r1500902404 ## docs/core-migration-guide.md: ## @@ -36,6 +36,8 @@ license: | - Since Spark 4.0, Spark uses `ReadWriteOncePod` instead of `ReadWriteOnce` access mode in persistence volume claims. To restore the legacy behavior, you can set `spark.kubernetes.legacy.useReadWriteOnceAccessMode` to `true`. +- Since Spark 4.0, Spark uses `~/.ivy2.5.2` as Ivy user directory by default to isolate the existing systems from Apache Ivy's incompatibility. To restore the legacy behavior, you can set `spark.jars.ivy` to `~/.ivy2`. Review Comment: I also thought like that. Something like `.ivy2.5.2+`. After receiving your comment, I'm rethinking about that. The bottom line is that the compatibility and release cycle depends on the Apache Ivy community, not Apache Spark community. - `.ivy2.5.2` literally means Apache Ivy format written by Apache Ivy 2.5.2 . - If Apache Ivy 2.5.3 is not going to introduce any new change, it's still Apache Ivy 2.5.2-format. - If Apache Ivy 2.5.3 breaks the format again, we need to use `.ivy2.5.3` at that time. - In addition, if we use `.ivy2.5.2_or_higher`, it could be an over-claim because Apache Spark community is unable to guarantee any compatibility for Apache Ivy 2.5.3 or higher which implies the naming. - Let's say we used `.ivy2.5.2_or_higher` and Apache Ivy 2.5.3 breaks the compatibility again. Then, we should change it again to `.ivy2.5.3_or_higher`. So, it introduces the same cost. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
LuciferYang commented on code in PR #45075: URL: https://github.com/apache/spark/pull/45075#discussion_r1500911196 ## docs/core-migration-guide.md: ## @@ -36,6 +36,8 @@ license: | - Since Spark 4.0, Spark uses `ReadWriteOncePod` instead of `ReadWriteOnce` access mode in persistence volume claims. To restore the legacy behavior, you can set `spark.kubernetes.legacy.useReadWriteOnceAccessMode` to `true`. +- Since Spark 4.0, Spark uses `~/.ivy2.5.2` as Ivy user directory by default to isolate the existing systems from Apache Ivy's incompatibility. To restore the legacy behavior, you can set `spark.jars.ivy` to `~/.ivy2`. Review Comment: SGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on code in PR #45075: URL: https://github.com/apache/spark/pull/45075#discussion_r1500902404 ## docs/core-migration-guide.md: ## @@ -36,6 +36,8 @@ license: | - Since Spark 4.0, Spark uses `ReadWriteOncePod` instead of `ReadWriteOnce` access mode in persistence volume claims. To restore the legacy behavior, you can set `spark.kubernetes.legacy.useReadWriteOnceAccessMode` to `true`. +- Since Spark 4.0, Spark uses `~/.ivy2.5.2` as Ivy user directory by default to isolate the existing systems from Apache Ivy's incompatibility. To restore the legacy behavior, you can set `spark.jars.ivy` to `~/.ivy2`. Review Comment: I also thought like that. Something like `.ivy2.5.2+`. After receiving your comment, I'm rethinking about that. The bottom line is that the compatibility and release cycle depends on the Apache Ivy community, not Apache Spark community. - `.ivy2.5.2` literally means Apache Ivy format written by Apache Ivy 2.5.2 . - If Apache Ivy 2.5.3 is not going to introduce any new change, it's still Apache Ivy 2.5.2-format. - If Apache Ivy 2.5.3 breaks the format again, we need to use `.ivy2.5.3` at that time. - In addition, if we use `ivy2.5.2_or_higher`, it could be an over-claim because Apache Spark community is unable to guarantee any compatibility for Apache Ivy 2.5.3 or higher which implies the naming. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on code in PR #45075: URL: https://github.com/apache/spark/pull/45075#discussion_r1500903505 ## docs/core-migration-guide.md: ## @@ -36,6 +36,8 @@ license: | - Since Spark 4.0, Spark uses `ReadWriteOncePod` instead of `ReadWriteOnce` access mode in persistence volume claims. To restore the legacy behavior, you can set `spark.kubernetes.legacy.useReadWriteOnceAccessMode` to `true`. +- Since Spark 4.0, Spark uses `~/.ivy2.5.2` as Ivy user directory by default to isolate the existing systems from Apache Ivy's incompatibility. To restore the legacy behavior, you can set `spark.jars.ivy` to `~/.ivy2`. Review Comment: Like `.ivy2`, we don't need to change this until the next Apache Ivy breaking change, @LuciferYang . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on code in PR #45075: URL: https://github.com/apache/spark/pull/45075#discussion_r1500902404 ## docs/core-migration-guide.md: ## @@ -36,6 +36,8 @@ license: | - Since Spark 4.0, Spark uses `ReadWriteOncePod` instead of `ReadWriteOnce` access mode in persistence volume claims. To restore the legacy behavior, you can set `spark.kubernetes.legacy.useReadWriteOnceAccessMode` to `true`. +- Since Spark 4.0, Spark uses `~/.ivy2.5.2` as Ivy user directory by default to isolate the existing systems from Apache Ivy's incompatibility. To restore the legacy behavior, you can set `spark.jars.ivy` to `~/.ivy2`. Review Comment: I also thought like that. Something like `.ivy2.5.2+`. After receiving your comment, I'm rethinking about that. The bottom line is that the compatibility and release cycle depends on the Apache Ivy community. - `.ivy2.5.2` literally means Apache Ivy format written by Apache Ivy 2.5.2 . - If Apache Ivy 2.5.3 is not going to introduce any new change, it's still Apache Ivy 2.5.2-format. - If Apache Ivy 2.5.3 breaks the format again, we need to use `.ivy2.5.3` at that time. - In addition, if we use `ivy2.5.2_or_higher`, it could be an over-claim because Apache Spark community is unable to guarantee any compatibility for Apache Ivy 2.5.3 or higher which implies the naming. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
LuciferYang commented on code in PR #45075: URL: https://github.com/apache/spark/pull/45075#discussion_r1500400758 ## docs/core-migration-guide.md: ## @@ -36,6 +36,8 @@ license: | - Since Spark 4.0, Spark uses `ReadWriteOncePod` instead of `ReadWriteOnce` access mode in persistence volume claims. To restore the legacy behavior, you can set `spark.kubernetes.legacy.useReadWriteOnceAccessMode` to `true`. +- Since Spark 4.0, Spark uses `~/.ivy2.5.2` as Ivy user directory by default to isolate the existing systems from Apache Ivy's incompatibility. To restore the legacy behavior, you can set `spark.jars.ivy` to `~/.ivy2`. Review Comment: Will it need to be changed again if we upgrade to use ivy 2.5.3 or 2.6.x in the future? Or can the name of this directory be: ``` .ivy2.5.2_and_above .ivy2.5.2_plus .ivy2.5.2_upwards .ivy2.5.2_or_higher ``` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1960966196 Could you review this again, @LuciferYang ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
srowen commented on code in PR #45075:
URL: https://github.com/apache/spark/pull/45075#discussion_r1499724171
##
common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala:
##
@@ -335,10 +335,12 @@ private[spark] object MavenUtils extends Logging {
/* Set ivy settings for location of cache, if option is supplied */
private def processIvyPathArg(ivySettings: IvySettings, ivyPath:
Option[String]): Unit = {
-ivyPath.filterNot(_.trim.isEmpty).foreach { alternateIvyDir =>
- ivySettings.setDefaultIvyUserDir(new File(alternateIvyDir))
- ivySettings.setDefaultCache(new File(alternateIvyDir, "cache"))
+val alternateIvyDir = ivyPath.filterNot(_.trim.isEmpty).getOrElse {
+ // To protect old Ivy-based systems like old Spark from Apache Ivy
2.5.2's incompatibility.
+ System.getProperty("user.home") + File.separator + ".ivy2.5.2"
Review Comment:
Sounds good to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on code in PR #45075:
URL: https://github.com/apache/spark/pull/45075#discussion_r1499722837
##
common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala:
##
@@ -335,10 +335,12 @@ private[spark] object MavenUtils extends Logging {
/* Set ivy settings for location of cache, if option is supplied */
private def processIvyPathArg(ivySettings: IvySettings, ivyPath:
Option[String]): Unit = {
-ivyPath.filterNot(_.trim.isEmpty).foreach { alternateIvyDir =>
- ivySettings.setDefaultIvyUserDir(new File(alternateIvyDir))
- ivySettings.setDefaultCache(new File(alternateIvyDir, "cache"))
+val alternateIvyDir = ivyPath.filterNot(_.trim.isEmpty).getOrElse {
+ // To protect old Ivy-based systems like old Spark from Apache Ivy
2.5.2's incompatibility.
+ System.getProperty("user.home") + File.separator + ".ivy2.5.2"
Review Comment:
Ya, we can do that. I also thought `.ivy2.5` first, but it was misleading
because Ivy 2.5.0 and 2.5.1 are incompatible with 2.5.2. I originally wanted to
say `2.5.2+` here, but I dropped `+`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on code in PR #45075:
URL: https://github.com/apache/spark/pull/45075#discussion_r1499722837
##
common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala:
##
@@ -335,10 +335,12 @@ private[spark] object MavenUtils extends Logging {
/* Set ivy settings for location of cache, if option is supplied */
private def processIvyPathArg(ivySettings: IvySettings, ivyPath:
Option[String]): Unit = {
-ivyPath.filterNot(_.trim.isEmpty).foreach { alternateIvyDir =>
- ivySettings.setDefaultIvyUserDir(new File(alternateIvyDir))
- ivySettings.setDefaultCache(new File(alternateIvyDir, "cache"))
+val alternateIvyDir = ivyPath.filterNot(_.trim.isEmpty).getOrElse {
+ // To protect old Ivy-based systems like old Spark from Apache Ivy
2.5.2's incompatibility.
+ System.getProperty("user.home") + File.separator + ".ivy2.5.2"
Review Comment:
Ya, we can do that. I also thought `.ivy2.5` first, but it was misleading
because Ivy 2.5.0 and 2.5.1 are incompatible with 2.5.2. I want to say `2.5.2+`
here, but I dropped `+`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
srowen commented on code in PR #45075:
URL: https://github.com/apache/spark/pull/45075#discussion_r1499710183
##
common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala:
##
@@ -335,10 +335,12 @@ private[spark] object MavenUtils extends Logging {
/* Set ivy settings for location of cache, if option is supplied */
private def processIvyPathArg(ivySettings: IvySettings, ivyPath:
Option[String]): Unit = {
-ivyPath.filterNot(_.trim.isEmpty).foreach { alternateIvyDir =>
- ivySettings.setDefaultIvyUserDir(new File(alternateIvyDir))
- ivySettings.setDefaultCache(new File(alternateIvyDir, "cache"))
+val alternateIvyDir = ivyPath.filterNot(_.trim.isEmpty).getOrElse {
+ // To protect old Ivy-based systems like old Spark from Apache Ivy
2.5.2's incompatibility.
+ System.getProperty("user.home") + File.separator + ".ivy2.5.2"
Review Comment:
Is ".ivy2.5.2" too specific? like would the new ivy dir be used, probably,
by subsequent ivy versions? maybe ".ivy2.5" or something. It doesn't really
matter though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
LuciferYang commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958784809 hmm... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958772546 Too bad. It still fails. ``` [info] *** 1 SUITE ABORTED *** [error] Error: Total 1553, Failed 0, Errors 1, Passed 1552, Ignored 597 [error] Error during tests: [error] org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958760347 Anyway, let's wait and see as you told. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958759588 IIUC, both Apache Spark 4.0.0 and 3.5.1 has the same patch and they should work together without Ivy 2.5.2 issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958757622 Yes, I already verify SBT failure on this PR, @LuciferYang . That's the reason why this PR can be a way to verify Ivy issue before going with Daily Maven CI. > Previously, upgrading ivy to 2.5.2 would cause SBT testing to fail, not just Maven -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
LuciferYang commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958755539 https://github.com/apache/spark/pull/42668 I recorded the issue in the PR of revert SPARK-44914 and provided a manual reproduce method when attempting to upgrade again https://github.com/apache/spark/pull/44477#issuecomment-1869925312  ``` [info] : java.lang.RuntimeException: problem during retrieve of org.apache.spark#spark-submit-parent-8bd00540-3ae3-45c0-b8cb-adf54c547a85: java.lang.RuntimeException: Multiple artifacts of the module log4j#log4j;1.2.17 are retrieved to the same file! Update the retrieve pattern to fix this error. [info]at org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(RetrieveEngine.java:238)` ``` Previously, upgrading ivy to 2.5.2 would cause SBT testing to fail, not just Maven -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958750324 To @LuciferYang , in any way, if we have more issues, our Maven CI will be broken from Today because we didn't protect them from 3.5.1. Let me make a PR for them while waiting this. https://github.com/apache/spark/blob/a87015efb5cf36103bc4eb82ae8613874e2eb408/.github/workflows/build_maven.yml#L36 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958747343 Oh, I didn't realize them. Do we have any JIRA issues? Then, I can track them together. > IIRC, these are two different issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
LuciferYang commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958746192 hmm... I'm not sure if SPARK-46400 can fix this issue at the same time. IIRC, these are two different issues. Let's wait for the test results from ci. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1958733681 Could you review this, @LuciferYang ? Apache Spark 3.5.1 is accessible now. I believe we are ready for Ivy upgrade. For further clean-ups, I'll proceed separate in SPARK-47126 (which is added as TODO). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1936836929 Ya, I'm digging differently with fresh eyes, @LuciferYang . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
LuciferYang commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1936809922 Previously, my workaround solution was here: https://github.com/apache/spark/pull/44477/files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-44914][BUILD] Upgrade Apache Ivy to 2.5.2 [spark]
dongjoon-hyun commented on PR #45075: URL: https://github.com/apache/spark/pull/45075#issuecomment-1935641577 Hmm. Although it passed locally, this seems to fail still in CI. ``` [info] *** 1 SUITE ABORTED *** [error] Error: Total 1555, Failed 0, Errors 1, Passed 1554, Ignored 597 [error] Error during tests: [error] org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
