LuciferYang commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1433767075
##
common/utils/src/main/scala/org/apache/spark/util/IvyTestUtils.scala:
##
@@ -15,30 +15,26 @@
* limitations under the License.
*/
-package
nchammas commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1429570569
##
core/src/main/scala/org/apache/spark/internal/config/package.scala:
##
@@ -2452,7 +2452,7 @@ package object config {
.createOptional
private[spark] val
cloud-fan commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1427451350
##
common/utils/src/main/scala/org/apache/spark/util/IvyTestUtils.scala:
##
@@ -15,30 +15,26 @@
* limitations under the License.
*/
-package
LuciferYang commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1395428501
##
common/utils/src/main/scala/org/apache/spark/util/IvyTestUtils.scala:
##
@@ -230,12 +226,11 @@ private[deploy] object IvyTestUtils {
val inside =
LuciferYang commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1395397883
##
common/utils/src/main/scala/org/apache/spark/util/IvyTestUtils.scala:
##
@@ -230,12 +226,11 @@ private[deploy] object IvyTestUtils {
val inside =
LuciferYang commented on PR #43354:
URL: https://github.com/apache/spark/pull/43354#issuecomment-1813722799
@vsevolodstep-db
I found that after moving MavenUtilsSuite.scala to the common-utils module,
it cannot pass the test. Do you know why? The current GA does not test this
case
HyukjinKwon closed pull request #43354: [SPARK-45506][CONNECT] Add ivy URI
support to SparkConnect addArtifact
URL: https://github.com/apache/spark/pull/43354
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
HyukjinKwon commented on PR #43354:
URL: https://github.com/apache/spark/pull/43354#issuecomment-1786218198
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on PR #43354:
URL: https://github.com/apache/spark/pull/43354#issuecomment-1786218108
Yeah, the test failure at
https://github.com/vsevolodstep-db/spark/actions/runs/6691537121/job/18179083580
seems unrelated.
--
This is an automated message from the Apache Git
hvanhovell closed pull request #43354: [SPARK-45506][CONNECT] Add ivy URI
support to SparkConnect addArtifact
URL: https://github.com/apache/spark/pull/43354
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
hvanhovell commented on PR #43354:
URL: https://github.com/apache/spark/pull/43354#issuecomment-1785922075
@HyukjinKwon are the current python failures a known problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
hvanhovell commented on PR #43354:
URL: https://github.com/apache/spark/pull/43354#issuecomment-1780279706
@vsevolodstep-db can you retrigger tests? I am not able to do so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
HyukjinKwon commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361984891
##
common/utils/src/test/scala/org/apache/spark/util/MavenUtilsSuite.scala:
##
@@ -94,19 +97,20 @@ class SparkSubmitUtilsSuite extends SparkFunSuite {
}
HyukjinKwon commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361984474
##
common/utils/src/test/scala/org/apache/spark/util/MavenUtilsSuite.scala:
##
@@ -248,15 +252,18 @@ class SparkSubmitUtilsSuite extends SparkFunSuite {
val
vsevolodstep-db commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361957662
##
common/utils/src/main/scala/org/apache/spark/util/IvyTestUtils.scala:
##
@@ -330,58 +341,78 @@ private[deploy] object IvyTestUtils {
/**
* Creates
vsevolodstep-db commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361948894
##
common/utils/pom.xml:
##
@@ -51,6 +51,20 @@
org.apache.commons
commons-text
+
Review Comment:
~400 Kb from commons-io and ~1Mb
vsevolodstep-db commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361945559
##
connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/ArtifactSuite.scala:
##
@@ -268,4 +269,14 @@ class ArtifactSuite extends
vsevolodstep-db commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361937361
##
common/utils/src/main/scala/org/apache/spark/util/SparkTestUtils.scala:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
vsevolodstep-db commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361934438
##
common/utils/pom.xml:
##
@@ -51,6 +51,20 @@
org.apache.commons
commons-text
+
+ commons-io
+ commons-io
+
+
+
hvanhovell commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361415253
##
connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/ArtifactSuite.scala:
##
@@ -268,4 +269,14 @@ class ArtifactSuite extends
hvanhovell commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361414066
##
common/utils/src/main/scala/org/apache/spark/util/SparkTestUtils.scala:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
hvanhovell commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361057882
##
common/utils/pom.xml:
##
@@ -51,6 +51,20 @@
org.apache.commons
commons-text
+
+ commons-io
+ commons-io
+
+
+
hvanhovell commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1361051725
##
common/utils/pom.xml:
##
@@ -51,6 +51,20 @@
org.apache.commons
commons-text
+
Review Comment:
How much deps do we add to the client
HyukjinKwon commented on code in PR #43354:
URL: https://github.com/apache/spark/pull/43354#discussion_r1360080331
##
common/utils/src/main/scala/org/apache/spark/util/IvyTestUtils.scala:
##
@@ -330,58 +341,78 @@ private[deploy] object IvyTestUtils {
/**
* Creates a
vsevolodstep-db opened a new pull request, #43354:
URL: https://github.com/apache/spark/pull/43354
### What changes were proposed in this pull request?
This PR extends existing SparkConnect `SparkSession.addArtifact` API to
support Apache Ivy URIs in extend to the existing local `.jar` &
25 matches
Mail list logo
