HyukjinKwon closed pull request #46324: [SPARK-48075] [SS] Add type checking
for PySpark avro functions
URL: https://github.com/apache/spark/pull/46324
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
HyukjinKwon commented on PR #46324:
URL: https://github.com/apache/spark/pull/46324#issuecomment-2089620300
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on PR #46298:
URL: https://github.com/apache/spark/pull/46298#issuecomment-2089586686
Doctest:
```
File
"/home/runner/work/spark/spark-35/python/pyspark/sql/connect/dataframe.py",
line 1057, in pyspark.sql.connect.dataframe.DataFrame.union
Failed example:
dongjoon-hyun commented on PR #46329:
URL: https://github.com/apache/spark/pull/46329#issuecomment-2089570488
Thank you, @viirya !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
cxzl25 commented on code in PR #46273:
URL: https://github.com/apache/spark/pull/46273#discussion_r1587027095
##
core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala:
##
@@ -710,7 +711,7 @@ private[spark] class ExternalSorter[K, V, C](
dongjoon-hyun closed pull request #46332: [SPARK-48080][K8S] Promote
`*MainAppResource` and `NonJVMResource` to `DeveloperApi`
URL: https://github.com/apache/spark/pull/46332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
dongjoon-hyun commented on PR #46332:
URL: https://github.com/apache/spark/pull/46332#issuecomment-2089465359
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
dongjoon-hyun commented on PR #46332:
URL: https://github.com/apache/spark/pull/46332#issuecomment-2089464647
Could you review this too, @HyukjinKwon ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
HyukjinKwon closed pull request #46310: [SPARK-48064][SQL] Update error
messages for routine related error classes
URL: https://github.com/apache/spark/pull/46310
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
HyukjinKwon commented on PR #46310:
URL: https://github.com/apache/spark/pull/46310#issuecomment-2089462468
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon closed pull request #46331: [SPARK-47683][PYTHON][BUILD][FOLLOW-UP]
Exclude `lib/py4j*zip` in `pyspark-connect` package
URL: https://github.com/apache/spark/pull/46331
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
HyukjinKwon closed pull request #46300:
[SPARK-48058][SPARK-43727][PYTHON][CONNECT] `UserDefinedFunction.returnType`
parse the DDL string
URL: https://github.com/apache/spark/pull/46300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
HyukjinKwon commented on code in PR #44920:
URL: https://github.com/apache/spark/pull/44920#discussion_r1587017977
##
python/MANIFEST.in:
##
@@ -14,13 +14,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
HyukjinKwon commented on PR #46300:
URL: https://github.com/apache/spark/pull/46300#issuecomment-2089461191
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1587017101
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the
dongjoon-hyun opened a new pull request, #46332:
URL: https://github.com/apache/spark/pull/46332
…
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
panbingkun commented on PR #46301:
URL: https://github.com/apache/spark/pull/46301#issuecomment-2089426127
> @panbingkun I see. There are about 88 loggings with variables
>
> ```
> find . -name "*.java"|xargs grep -i
"logger.info\|logger.warn\|logger.error"|grep "{}"| grep -v
dongjoon-hyun closed pull request #46329: [SPARK-48078][K8S] Promote
`o.a.s.d.k8s.Constants` to `DeveloperApi`
URL: https://github.com/apache/spark/pull/46329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
dongjoon-hyun commented on PR #46329:
URL: https://github.com/apache/spark/pull/46329#issuecomment-2089425612
Thank you, @HyukjinKwon !
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
nchammas commented on code in PR #44920:
URL: https://github.com/apache/spark/pull/44920#discussion_r1586990733
##
python/MANIFEST.in:
##
@@ -14,13 +14,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
HyukjinKwon commented on code in PR #44920:
URL: https://github.com/apache/spark/pull/44920#discussion_r1586988988
##
python/MANIFEST.in:
##
@@ -14,13 +14,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
HyukjinKwon commented on code in PR #46300:
URL: https://github.com/apache/spark/pull/46300#discussion_r1586986177
##
python/pyspark/sql/connect/udf.py:
##
@@ -148,15 +150,35 @@ def __init__(
)
self.func = func
-self.returnType: DataType = (
-
nchammas commented on code in PR #44920:
URL: https://github.com/apache/spark/pull/44920#discussion_r1586985679
##
python/MANIFEST.in:
##
@@ -14,13 +14,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
panbingkun opened a new pull request, #46330:
URL: https://github.com/apache/spark/pull/46330
### What changes were proposed in this pull request?
The pr aims to upgrade:
- `maven-install-plugin` from `3.1.1` to `3.1.2`
- `maven-deploy-plugin` from `3.1.1` to `3.1.2`
### Why
HyukjinKwon closed pull request #46328: [SPARK-46894][PYTHON][FOLLOW-UP]
Includes `error-conditions.json` into PyPI package
URL: https://github.com/apache/spark/pull/46328
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
HyukjinKwon commented on PR #46328:
URL: https://github.com/apache/spark/pull/46328#issuecomment-2089397703
Merged to master.
It will be tested in the scheduled job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
HyukjinKwon commented on code in PR #44920:
URL: https://github.com/apache/spark/pull/44920#discussion_r1586979834
##
python/MANIFEST.in:
##
@@ -14,13 +14,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
dongjoon-hyun commented on PR #46328:
URL: https://github.com/apache/spark/pull/46328#issuecomment-2089396116
Oh, much simpler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
yaooqinn commented on PR #46133:
URL: https://github.com/apache/spark/pull/46133#issuecomment-2089391772
Thank you @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on code in PR #46328:
URL: https://github.com/apache/spark/pull/46328#discussion_r1586976770
##
python/packaging/classic/setup.py:
##
@@ -307,6 +307,7 @@ def run(self):
"pyspark.errors",
"pyspark.errors.exceptions",
HyukjinKwon commented on code in PR #46328:
URL: https://github.com/apache/spark/pull/46328#discussion_r1586971280
##
python/packaging/classic/setup.py:
##
@@ -307,6 +307,7 @@ def run(self):
"pyspark.errors",
"pyspark.errors.exceptions",
HyukjinKwon commented on PR #46328:
URL: https://github.com/apache/spark/pull/46328#issuecomment-2089381467
There is a test at
https://github.com/apache/spark/blob/master/dev/pip-sanity-check.py but that
only runs a basic test that does not import pyspark.errors whereas
dongjoon-hyun commented on PR #46329:
URL: https://github.com/apache/spark/pull/46329#issuecomment-2089379790
Could you review this PR too, @viirya ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586968858
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppResourceSpec.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the
dongjoon-hyun opened a new pull request, #46329:
URL: https://github.com/apache/spark/pull/46329
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
###
zhengruifeng commented on PR #46299:
URL: https://github.com/apache/spark/pull/46299#issuecomment-2089374790
thank you @dongjoon-hyun for reviews
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
dongjoon-hyun commented on PR #46327:
URL: https://github.com/apache/spark/pull/46327#issuecomment-2089367196
Merged to master for Apache Spark 4.0.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
dongjoon-hyun closed pull request #46327: [SPARK-48077][K8S] Promote
`KubernetesClientUtils` to `DeveloperApi`
URL: https://github.com/apache/spark/pull/46327
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
dongjoon-hyun commented on PR #46326:
URL: https://github.com/apache/spark/pull/46326#issuecomment-2089359042
Thank you, @viirya !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun commented on PR #46327:
URL: https://github.com/apache/spark/pull/46327#issuecomment-2089358862
Thank you, @viirya !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586956299
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppDriverConf.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache
nchammas commented on code in PR #46328:
URL: https://github.com/apache/spark/pull/46328#discussion_r1586954148
##
python/packaging/classic/setup.py:
##
@@ -307,6 +307,7 @@ def run(self):
"pyspark.errors",
"pyspark.errors.exceptions",
dongjoon-hyun commented on PR #46311:
URL: https://github.com/apache/spark/pull/46311#issuecomment-2089353288
Yep. Merged to master for Apache Spark 4.0.0. Thank you, @harshmotw-db .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun commented on PR #46327:
URL: https://github.com/apache/spark/pull/46327#issuecomment-2089352549
Thank you, @HyukjinKwon !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
dongjoon-hyun closed pull request #46311: [SPARK-45891][SQL][FOLLOW-UP] Added
length check to the is_variant_null expression
URL: https://github.com/apache/spark/pull/46311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
dongjoon-hyun commented on PR #46326:
URL: https://github.com/apache/spark/pull/46326#issuecomment-2089352104
Merged to master for Apache Spark 4.0.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
dongjoon-hyun closed pull request #46326: [SPARK-48076][K8S] Promote
`KubernetesVolumeUtils` to `DeveloperApi`
URL: https://github.com/apache/spark/pull/46326
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
dongjoon-hyun commented on PR #46326:
URL: https://github.com/apache/spark/pull/46326#issuecomment-2089351522
Thank you, @HyukjinKwon !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
dongjoon-hyun commented on PR #46327:
URL: https://github.com/apache/spark/pull/46327#issuecomment-2089349241
Could you review this PR, @HyukjinKwon ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
dongjoon-hyun commented on PR #46326:
URL: https://github.com/apache/spark/pull/46326#issuecomment-2089349123
Could you review this PR, @HyukjinKwon ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
harshmotw-db commented on PR #46311:
URL: https://github.com/apache/spark/pull/46311#issuecomment-2089347930
Thanks for the reviews @cloud-fan @HyukjinKwon @dongjoon-hyun. The tests are
completed and it is ready to be merged now.
--
This is an automated message from the Apache Git
HyukjinKwon commented on PR #46328:
URL: https://github.com/apache/spark/pull/46328#issuecomment-2089346151
and @itholic too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
HyukjinKwon commented on PR #46328:
URL: https://github.com/apache/spark/pull/46328#issuecomment-2089346079
cc @nchammas FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
HyukjinKwon opened a new pull request, #46328:
URL: https://github.com/apache/spark/pull/46328
### What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/44920 that
includes `error-conditions.json` into PyPI package.
###
dongjoon-hyun commented on PR #46327:
URL: https://github.com/apache/spark/pull/46327#issuecomment-2089344533
Could you review this PR, @viirya ? This is for `Spark K8s Operator`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun commented on PR #46326:
URL: https://github.com/apache/spark/pull/46326#issuecomment-2089344351
Could you review this PR, @viirya ? This is for `Spark K8s Operator`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun opened a new pull request, #46327:
URL: https://github.com/apache/spark/pull/46327
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
###
allisonwang-db commented on PR #46310:
URL: https://github.com/apache/spark/pull/46310#issuecomment-2089335214
@HyukjinKwon the failed test doesn't seem to be related
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
dtenedor commented on PR #46309:
URL: https://github.com/apache/spark/pull/46309#issuecomment-2089333094
```
[info] *** 1 TEST FAILED ***
[error] Failed: Total 4098, Failed 1, Errors 0, Passed 4097, Ignored 10,
Canceled 2
[error] Failed tests:
[error]
PaysonXu commented on PR #46320:
URL: https://github.com/apache/spark/pull/46320#issuecomment-2089329577
@MaxGekk @cloud-fan would you please review this PR ? Actually, I'm not sure
if rename 44 to internal err is appropriate. thx.
--
This is an automated message from the Apache Git
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586929083
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppDriverConf.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache
dongjoon-hyun opened a new pull request, #46326:
URL: https://github.com/apache/spark/pull/46326
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
###
HyukjinKwon commented on code in PR #46307:
URL: https://github.com/apache/spark/pull/46307#discussion_r1586928314
##
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CanonicalizeSuite.scala:
##
@@ -454,4 +454,29 @@ class CanonicalizeSuite extends
HyukjinKwon closed pull request #46321: [MINOR] Fix the grammar of some
comments on renaming error classes
URL: https://github.com/apache/spark/pull/46321
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
HyukjinKwon commented on code in PR #46319:
URL: https://github.com/apache/spark/pull/46319#discussion_r1586927903
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala:
##
@@ -98,7 +98,7 @@ case class ExpressionWithUnresolvedIdentifier(
/**
HyukjinKwon commented on PR #46321:
URL: https://github.com/apache/spark/pull/46321#issuecomment-2089300250
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
panbingkun commented on PR #46322:
URL: https://github.com/apache/spark/pull/46322#issuecomment-2089299522
+1, LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
HyukjinKwon closed pull request #46306: [SPARK-48062][PYTHON][SS][TESTS] Add
pyspark test for SimpleDataSourceStreamingReader
URL: https://github.com/apache/spark/pull/46306
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
HyukjinKwon commented on PR #46306:
URL: https://github.com/apache/spark/pull/46306#issuecomment-2089292078
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on code in PR #46324:
URL: https://github.com/apache/spark/pull/46324#discussion_r1586922452
##
python/pyspark/sql/avro/functions.py:
##
@@ -80,6 +81,25 @@ def from_avro(
from py4j.java_gateway import JVMView
from pyspark.sql.classic.column
dongjoon-hyun commented on PR #46184:
URL: https://github.com/apache/spark/pull/46184#issuecomment-2089283472
For the record, Apache Spark 4.0.0-preview is scheduled on next Monday.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586909100
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppResourceSpec.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586908945
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppResourceSpec.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586908000
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppResourceSpec.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586908280
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppResourceSpec.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586907715
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppResourceSpec.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586907367
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppResourceSpec.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586906766
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586906065
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppDriverConf.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586905774
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppDriverConf.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586903355
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppDriverConf.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586903355
##
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppDriverConf.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586902370
##
spark-submission-worker/build.gradle:
##
@@ -0,0 +1,18 @@
+dependencies {
+ implementation project(":spark-operator-api")
+
+
dongjoon-hyun commented on code in PR #10:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/10#discussion_r1586901378
##
gradle.properties:
##
@@ -18,17 +18,23 @@
group=org.apache.spark.k8s.operator
version=0.1.0
-fabric8Version=6.12.1
+# Caution: fabric8
dongjoon-hyun closed pull request #46133: [SPARK-47911][SQL] Introduces a
universal BinaryFormatter to make binary output consistent
URL: https://github.com/apache/spark/pull/46133
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
dongjoon-hyun commented on PR #46133:
URL: https://github.com/apache/spark/pull/46133#issuecomment-2089240086
All tests passed except irrelevant linter failures. Let me merge this for
Apache Spark 4.0.0-preview.
--
This is an automated message from the Apache Git Service.
To respond to
viirya commented on code in PR #46273:
URL: https://github.com/apache/spark/pull/46273#discussion_r1586888674
##
core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala:
##
@@ -710,7 +711,7 @@ private[spark] class ExternalSorter[K, V, C](
viirya commented on code in PR #46273:
URL: https://github.com/apache/spark/pull/46273#discussion_r1586884528
##
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala:
##
@@ -2410,6 +2413,26 @@ class AdaptiveQueryExecSuite
}
}
+
bogao007 commented on code in PR #46287:
URL: https://github.com/apache/spark/pull/46287#discussion_r1586884029
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListenerBus.scala:
##
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
viirya commented on code in PR #46273:
URL: https://github.com/apache/spark/pull/46273#discussion_r1586883895
##
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala:
##
@@ -2410,6 +2413,26 @@ class AdaptiveQueryExecSuite
}
}
+
dongjoon-hyun commented on PR #46157:
URL: https://github.com/apache/spark/pull/46157#issuecomment-2089225792
Welcome to the Apache Spark community, @huangzhir .
I added you to the Apache Spark contributor group and assigned SPARK-47934
to you.
Congratulations for your first commit!
dongjoon-hyun closed pull request #46157: [SPARK-47934] [CORE] Ensure trailing
slashes in `HistoryServer` URL redirections
URL: https://github.com/apache/spark/pull/46157
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
WweiL commented on code in PR #46287:
URL: https://github.com/apache/spark/pull/46287#discussion_r1586877487
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListenerBus.scala:
##
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache
dongjoon-hyun commented on PR #46288:
URL: https://github.com/apache/spark/pull/46288#issuecomment-2089216404
Although we are waiting for `Ammonite` still, could you base this PR once
more, @panbingkun ?
- https://github.com/com-lihaoyi/Ammonite/pull/1482
--
This is an automated
dongjoon-hyun commented on PR #46273:
URL: https://github.com/apache/spark/pull/46273#issuecomment-2089211702
Also, cc @viirya , too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun commented on PR #46273:
URL: https://github.com/apache/spark/pull/46273#issuecomment-2089211257
I merged #46316 . Could you rebase this PR to the master branch, @cxzl25 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun closed pull request #46322: [SPARK-48074][Core] Improve the
readability of JSON loggings
URL: https://github.com/apache/spark/pull/46322
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dongjoon-hyun commented on PR #46322:
URL: https://github.com/apache/spark/pull/46322#issuecomment-2089202215
Merged to master for Apache spark 4.0.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
allisonwang-db commented on code in PR #46139:
URL: https://github.com/apache/spark/pull/46139#discussion_r1586863445
##
python/docs/source/user_guide/sql/python_data_source.rst:
##
@@ -84,6 +93,131 @@ Define the reader logic to generate synthetic data. Use the
`faker` library
dongjoon-hyun commented on PR #46323:
URL: https://github.com/apache/spark/pull/46323#issuecomment-2089200222
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
1 - 100 of 185 matches
Mail list logo
