dongjoon-hyun closed pull request #46436: [SPARK-48171][CORE] Clean up the use
of deprecated constructors of `o.rocksdb.Logger`
URL: https://github.com/apache/spark/pull/46436
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
dongjoon-hyun commented on PR #46402:
URL: https://github.com/apache/spark/pull/46402#issuecomment-2098687774
I merged the following. Could you rebase this PR?
- https://github.com/apache/spark/pull/46427
--
This is an automated message from the Apache Git Service.
To respond to the
dongjoon-hyun closed pull request #46438: [SPARK-48169][SQL] Use lazy
BadRecordException cause in all parsers and remove the old constructor, which
was meant for the migration
URL: https://github.com/apache/spark/pull/46438
--
This is an automated message from the Apache Git Service.
To
stefankandic commented on PR #46180:
URL: https://github.com/apache/spark/pull/46180#issuecomment-2098821075
will we have to do the same for pyspark - as `StringType` there only
supports 4 initial collations?
--
This is an automated message from the Apache Git Service.
To respond to the
dongjoon-hyun commented on PR #46433:
URL: https://github.com/apache/spark/pull/46433#issuecomment-2098820860
Merged to master for Apache Spark 4.0.0-preview.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
dongjoon-hyun closed pull request #46433: [SPARK-48170][PYTHON][CONNECT][TESTS]
Enable `ArrowPythonUDFParityTests.test_err_return_type`
URL: https://github.com/apache/spark/pull/46433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun commented on PR #46414:
URL: https://github.com/apache/spark/pull/46414#issuecomment-2098834940
Merged to master for Apache Spark 4.0.0-preview.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
dongjoon-hyun closed pull request #46414: [SPARK-48035][SQL][FOLLOWUP] Fix
try_add/try_multiply being semantic equal to add/multiply
URL: https://github.com/apache/spark/pull/46414
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
viirya commented on PR #46441:
URL: https://github.com/apache/spark/pull/46441#issuecomment-2098866163
Looks good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
dongjoon-hyun closed pull request #46441: [SPARK-48174][INFRA] Merge `connect`
back to the original test pipeline
URL: https://github.com/apache/spark/pull/46441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
dongjoon-hyun commented on PR #46441:
URL: https://github.com/apache/spark/pull/46441#issuecomment-2098866227
Thank you, @viirya !
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
WweiL opened a new pull request, #46444:
URL: https://github.com/apache/spark/pull/46444
This reverts commit df633091b7147dea84a5a51a30dcf690ca7d1124.
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592826648
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592837929
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Mode.scala:
##
@@ -17,28 +17,36 @@
package
dongjoon-hyun commented on PR #45408:
URL: https://github.com/apache/spark/pull/45408#issuecomment-2098956928
Gentle ping, @ted-jenks .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592830626
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Mode.scala:
##
@@ -162,10 +196,10 @@ object ModeBuilder extends ExpressionBuilder {
dongjoon-hyun commented on PR #46386:
URL: https://github.com/apache/spark/pull/46386#issuecomment-2098985001
WDYT, @gengliangwang ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
kelvinjian-db commented on PR #46443:
URL: https://github.com/apache/spark/pull/46443#issuecomment-2099000299
@cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
dongjoon-hyun opened a new pull request, #46441:
URL: https://github.com/apache/spark/pull/46441
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
###
dongjoon-hyun closed pull request #46439: [SPARK-48173][SQL] CheckAnalysis
should see the entire query plan
URL: https://github.com/apache/spark/pull/46439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
dongjoon-hyun commented on PR #46442:
URL: https://github.com/apache/spark/pull/46442#issuecomment-2098777015
BTW, could you reiview this INFRA PR when you have some time, @cloud-fan ?
There are not many people at this point of time. Sorry for asking you.
- #46441
The above is a
dongjoon-hyun closed pull request #46432: [SPARK-41547][CONNECT][TESTS]
Re-eneable Spark Connect function tests with ANSI mode
URL: https://github.com/apache/spark/pull/46432
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
sahnib commented on code in PR #45376:
URL: https://github.com/apache/spark/pull/45376#discussion_r1592763066
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUpdateEventTimeWatermarkColumn.scala:
##
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache
cloud-fan commented on PR #46441:
URL: https://github.com/apache/spark/pull/46441#issuecomment-2098943607
late LGTM!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
cloud-fan commented on PR #46423:
URL: https://github.com/apache/spark/pull/46423#issuecomment-2098622408
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
uros-db commented on code in PR #46166:
URL: https://github.com/apache/spark/pull/46166#discussion_r1592640514
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteCollationJoin.scala:
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software
uros-db commented on code in PR #46166:
URL: https://github.com/apache/spark/pull/46166#discussion_r1592640514
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteCollationJoin.scala:
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software
nchammas commented on PR #44755:
URL: https://github.com/apache/spark/pull/44755#issuecomment-2098662588
Following up regarding your feedback on the API change, @holdenk. Do you
recall what it was?
--
This is an automated message from the Apache Git Service.
To respond to the message,
dongjoon-hyun commented on PR #46438:
URL: https://github.com/apache/spark/pull/46438#issuecomment-2098821219
Merged to master for Apache Spark 4.0.0-preview.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
dongjoon-hyun commented on PR #46302:
URL: https://github.com/apache/spark/pull/46302#issuecomment-2098823631
I converted this to `Draft` PR because the CI is broken and the PR title has
`[WIP]`.
--
This is an automated message from the Apache Git Service.
To respond to the message,
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592820188
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
dongjoon-hyun closed pull request #46273: [SPARK-48037][CORE] Fix
SortShuffleWriter lacks shuffle write related metrics resulting in potentially
inaccurate data
URL: https://github.com/apache/spark/pull/46273
--
This is an automated message from the Apache Git Service.
To respond to the
dongjoon-hyun commented on PR #46273:
URL: https://github.com/apache/spark/pull/46273#issuecomment-2098934956
Merged to master for Apache Spark 4.0.0-preview.
Could you make backporting PRs to the release branches, @cxzl25 ?
--
This is an automated message from the Apache Git
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592824395
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592822463
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592834974
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Mode.scala:
##
@@ -17,28 +17,36 @@
package
dongjoon-hyun commented on PR #46441:
URL: https://github.com/apache/spark/pull/46441#issuecomment-2098974153
Thank you, @cloud-fan !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun commented on PR #46434:
URL: https://github.com/apache/spark/pull/46434#issuecomment-2098973037
Could you fix Python linter failure, @grundprinzip ?
```
./python/pyspark/sql/tests/connect/test_connect_column.py:1029:5: F401 'os'
imported but unused
import os
dongjoon-hyun commented on PR #46439:
URL: https://github.com/apache/spark/pull/46439#issuecomment-2098635035
Merged to master for Apache Spark 4.0.0-preview.
Please make backporting PRs to the release branches, @cloud-fan . I believe
we need to pass the CIs for this backporting.
dongjoon-hyun commented on code in PR #46402:
URL: https://github.com/apache/spark/pull/46402#discussion_r1592672474
##
connector/profiler/pom.xml:
##
@@ -45,6 +48,7 @@
me.bechberger
ap-loader-all
3.0-8
+ provided
Review Comment:
cc @parthchandra ,
dongjoon-hyun commented on code in PR #46402:
URL: https://github.com/apache/spark/pull/46402#discussion_r1592673906
##
connector/profiler/README.md:
##
@@ -23,7 +23,7 @@ Code profiling is currently only supported for
To get maximum profiling information set the following jvm
sahnib commented on code in PR #45376:
URL: https://github.com/apache/spark/pull/45376#discussion_r1592763363
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/IncrementalExecution.scala:
##
@@ -347,6 +347,28 @@ class IncrementalExecution(
sahnib commented on code in PR #45376:
URL: https://github.com/apache/spark/pull/45376#discussion_r1592764603
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/EventTimeWatermarkExec.scala:
##
@@ -107,25 +109,70 @@ case class EventTimeWatermarkExec(
}
sahnib commented on code in PR #45376:
URL: https://github.com/apache/spark/pull/45376#discussion_r1592763254
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/IncrementalExecution.scala:
##
@@ -347,6 +347,28 @@ class IncrementalExecution(
parthchandra commented on PR #46427:
URL: https://github.com/apache/spark/pull/46427#issuecomment-2098862359
Thanks @dongjoon-hyun. Noted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
parthchandra commented on code in PR #46402:
URL: https://github.com/apache/spark/pull/46402#discussion_r1592803402
##
connector/profiler/pom.xml:
##
@@ -45,6 +48,7 @@
me.bechberger
ap-loader-all
3.0-8
+ provided
Review Comment:
It's great to
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592822463
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592822930
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592822930
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
uros-db commented on code in PR #46166:
URL: https://github.com/apache/spark/pull/46166#discussion_r1592640514
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteCollationJoin.scala:
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software
dongjoon-hyun commented on PR #46288:
URL: https://github.com/apache/spark/pull/46288#issuecomment-2098787633
Thank you for confirming. Ya, let's wait.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
kelvinjian-db opened a new pull request, #46443:
URL: https://github.com/apache/spark/pull/46443
### What changes were proposed in this pull request?
This PR fixes an edge case where an aggregate function is in the child of a
`With` expression, which previously
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592827444
##
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/CollationBenchmark.scala:
##
Review Comment:
did you run this benchmark? I wonder how this
cloud-fan commented on code in PR #46273:
URL: https://github.com/apache/spark/pull/46273#discussion_r1592828452
##
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala:
##
@@ -2502,6 +2502,26 @@ class AdaptiveQueryExecSuite
}
}
cloud-fan closed pull request #46423: [SPARK-47297][SQL] Add collation support
for format expressions
URL: https://github.com/apache/spark/pull/46423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
cloud-fan opened a new pull request, #46442:
URL: https://github.com/apache/spark/pull/46442
backport https://github.com/apache/spark/pull/46439 to 3.5
### What changes were proposed in this pull request?
This is a follow-up of https://github.com/apache/spark/pull/38029
dongjoon-hyun commented on PR #46441:
URL: https://github.com/apache/spark/pull/46441#issuecomment-2098838055
Could you review this PR, @viirya ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
GideonPotok commented on PR #46404:
URL: https://github.com/apache/spark/pull/46404#issuecomment-2098916903
@cloud-fan @MaxGekk @dbatomic just letting y'all know this is ready for
first round of review. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to
dongjoon-hyun commented on PR #46427:
URL: https://github.com/apache/spark/pull/46427#issuecomment-2098639686
Merged to master for Apache Spark 4.0.0-preview.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
dongjoon-hyun closed pull request #46427: [SPARK-48165][BUILD] Update
`ap-loader` to 3.0-9
URL: https://github.com/apache/spark/pull/46427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
dongjoon-hyun commented on PR #46441:
URL: https://github.com/apache/spark/pull/46441#issuecomment-2098682742
Could you review this INFRA PR, @cloud-fan ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
dongjoon-hyun commented on code in PR #46437:
URL: https://github.com/apache/spark/pull/46437#discussion_r1592733398
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:
##
@@ -351,7 +351,7 @@ abstract class JdbcDialect extends Serializable with
Logging {
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592825050
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -800,6 +804,61 @@ class CollationStringExpressionsSuite
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592833471
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Mode.scala:
##
@@ -70,20 +78,46 @@ case class Mode(
buffer
}
- override
uros-db commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592832622
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Mode.scala:
##
@@ -70,20 +78,46 @@ case class Mode(
buffer
}
- override
dongjoon-hyun commented on PR #46386:
URL: https://github.com/apache/spark/pull/46386#issuecomment-2098984770
I understand your point, @mridulm . Do you use the following syntax?
```
spark.sparkContext.setLocalProperty("mdc." + name, "value")
```
Initially, I thought it's
dongjoon-hyun commented on PR #46456:
URL: https://github.com/apache/spark/pull/46456#issuecomment-2099681039
Ah, I got your point. It's a very interesting `connector` bug.
> df2 should be immutable.
I was thinking the following. My bad.
```
scala>
dongjoon-hyun commented on PR #46440:
URL: https://github.com/apache/spark/pull/46440#issuecomment-2099685242
No problem, @yaooqinn . We have enough time for 4.0.0. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
viirya commented on code in PR #39615:
URL: https://github.com/apache/spark/pull/39615#discussion_r1593353161
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala:
##
@@ -166,317 +148,58 @@ object JavaTypeInference {
uros-db opened a new pull request, #46461:
URL: https://github.com/apache/spark/pull/46461
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How
dongjoon-hyun commented on code in PR #46456:
URL: https://github.com/apache/spark/pull/46456#discussion_r1593374153
##
python/pyspark/sql/tests/test_dataframe.py:
##
@@ -430,6 +430,11 @@ def test_sample(self):
IllegalArgumentException, lambda:
dongjoon-hyun commented on code in PR #46456:
URL: https://github.com/apache/spark/pull/46456#discussion_r1593374153
##
python/pyspark/sql/tests/test_dataframe.py:
##
@@ -430,6 +430,11 @@ def test_sample(self):
IllegalArgumentException, lambda:
LuciferYang commented on code in PR #46457:
URL: https://github.com/apache/spark/pull/46457#discussion_r1593383893
##
sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:
##
@@ -197,8 +197,8 @@ trait SparkDateTimeUtils {
yaooqinn commented on PR #46440:
URL: https://github.com/apache/spark/pull/46440#issuecomment-2099683026
Thank you @dongjoon-hyun
Let me look into this issue, it might take a while
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
cxzl25 commented on code in PR #46273:
URL: https://github.com/apache/spark/pull/46273#discussion_r1593350763
##
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala:
##
@@ -2502,6 +2502,26 @@ class AdaptiveQueryExecSuite
}
}
+
yaooqinn opened a new pull request, #46457:
URL: https://github.com/apache/spark/pull/46457
### What changes were proposed in this pull request?
I met the error below while debugging UTs because of loading
`sun.util.calendar.ZoneInfo` eagerly. This PR makes the
yaooqinn closed pull request #46136: [SPARK-47914][SQL] Do not display the
splits parameter in Range
URL: https://github.com/apache/spark/pull/46136
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
dongjoon-hyun commented on PR #46455:
URL: https://github.com/apache/spark/pull/46455#issuecomment-2099711660
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun closed pull request #46455: [SPARK-48183][PYTHON][DOCS] Update
error contribution guide to respect new error class file
URL: https://github.com/apache/spark/pull/46455
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
LuciferYang commented on code in PR #46457:
URL: https://github.com/apache/spark/pull/46457#discussion_r1593382052
##
sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:
##
@@ -197,8 +197,8 @@ trait SparkDateTimeUtils {
dongjoon-hyun commented on PR #46459:
URL: https://github.com/apache/spark/pull/46459#issuecomment-2099760350
Could you make a backporting PR to branch-3.4 too, @cxzl25 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
chaoqin-li1123 commented on PR #45950:
URL: https://github.com/apache/spark/pull/45950#issuecomment-2099774647
This seems to be broken in the main function of pyspark init(), what is the
expected action item we should take? @HyukjinKwon
--
This is an automated message from the Apache
dongjoon-hyun commented on PR #46459:
URL: https://github.com/apache/spark/pull/46459#issuecomment-2099780272
Merged to branch-3.5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun closed pull request #46459: [SPARK-48037][CORE][3.5] Fix
SortShuffleWriter lacks shuffle write related metrics resulting in potentially
inaccurate data
URL: https://github.com/apache/spark/pull/46459
--
This is an automated message from the Apache Git Service.
To respond to
uros-db opened a new pull request, #46458:
URL: https://github.com/apache/spark/pull/46458
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How
yaooqinn commented on PR #46136:
URL: https://github.com/apache/spark/pull/46136#issuecomment-2099698487
Merged to master
Thank you @guixiaowen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
cxzl25 opened a new pull request, #46459:
URL: https://github.com/apache/spark/pull/46459
### What changes were proposed in this pull request?
This PR aims to fix SortShuffleWriter lacks shuffle write related metrics
resulting in potentially inaccurate data.
### Why are the
uros-db opened a new pull request, #46460:
URL: https://github.com/apache/spark/pull/46460
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How
uros-db opened a new pull request, #46462:
URL: https://github.com/apache/spark/pull/46462
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How
GideonPotok commented on code in PR #46404:
URL: https://github.com/apache/spark/pull/46404#discussion_r1592912017
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Mode.scala:
##
@@ -70,20 +78,46 @@ case class Mode(
buffer
}
-
ianmcook commented on code in PR #45481:
URL: https://github.com/apache/spark/pull/45481#discussion_r1592915751
##
python/pyspark/sql/pandas/conversion.py:
##
@@ -225,15 +225,68 @@ def toPandas(self) -> "PandasDataFrameLike":
else:
return pdf
-def
dongjoon-hyun commented on PR #46445:
URL: https://github.com/apache/spark/pull/46445#issuecomment-2099174845
cc @HyukjinKwon and @zhengruifeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
gengliangwang commented on PR #46390:
URL: https://github.com/apache/spark/pull/46390#issuecomment-2099214104
Thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
Fokko opened a new pull request, #46447:
URL: https://github.com/apache/spark/pull/46447
### What changes were proposed in this pull request?
### Why are the changes needed?
Fixes quite a few bugs on the Parquet side:
dongjoon-hyun opened a new pull request, #46448:
URL: https://github.com/apache/spark/pull/46448
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
###
dongjoon-hyun commented on PR #46447:
URL: https://github.com/apache/spark/pull/46447#issuecomment-2099295786
cc @cloud-fan , @HyukjinKwon , @mridulm , @sunchao , @yaooqinn ,
@LuciferYang , @steveloughran , @viirya , @huaxin, @parthchandra , too.
--
This is an automated message from the
dongjoon-hyun opened a new pull request, #46449:
URL: https://github.com/apache/spark/pull/46449
### What changes were proposed in this pull request?
This PR aims to run `build`, `scala-213`, and `java-11-17` job of
`branch-3.5` only if needed to reduce the maximum concurrency of
gengliangwang commented on PR #46386:
URL: https://github.com/apache/spark/pull/46386#issuecomment-2099078293
@dongjoon-hyun Thanks, I will add a configuration about the unification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
ianmcook commented on code in PR #45481:
URL: https://github.com/apache/spark/pull/45481#discussion_r1592915751
##
python/pyspark/sql/pandas/conversion.py:
##
@@ -225,15 +225,68 @@ def toPandas(self) -> "PandasDataFrameLike":
else:
return pdf
-def
SemyonSinchenko commented on PR #46368:
URL: https://github.com/apache/spark/pull/46368#issuecomment-2099107642
New changes:
- fixes from comments
- **changing the type from Long to BigInteger** (`bytes` in proto)
--
This is an automated message from the Apache Git Service.
To
1 - 100 of 292 matches
Mail list logo
