LuciferYang commented on PR #44070:
URL: https://github.com/apache/spark/pull/44070#issuecomment-1831379447
Merged into master. Thanks @zhengruifeng ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
igreenfield commented on PR #44053:
URL: https://github.com/apache/spark/pull/44053#issuecomment-1831379592
Hi @dongjoon-hyun,
1. About the case you can look at the jira case.
2. I will look at the other PR.
--
This is an automated message from the Apache Git Service.
To respond to t
LuciferYang closed pull request #44070: [SPARK-46154][INFRA] Add a new Daily
testing GitHub Action job for Maven with Java 21
URL: https://github.com/apache/spark/pull/44070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
LuciferYang commented on PR #44070:
URL: https://github.com/apache/spark/pull/44070#issuecomment-1831377349
>
https://github.com/LuciferYang/spark/actions/runs/7028585225/job/19124783706
ulysses-you commented on PR #44074:
URL: https://github.com/apache/spark/pull/44074#issuecomment-1831367808
cc @dongjoon-hyun @cloud-fan this pr is only for adding Adaptive Query Post
Planner Strategy Rules part.
--
This is an automated message from the Apache Git Service.
To respond to
ulysses-you commented on PR #44074:
URL: https://github.com/apache/spark/pull/44074#issuecomment-1831368538
CI link https://github.com/ulysses-you/spark/actions/runs/7029365126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
panbingkun opened a new pull request, #44075:
URL: https://github.com/apache/spark/pull/44075
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How
ulysses-you opened a new pull request, #44074:
URL: https://github.com/apache/spark/pull/44074
### What changes were proposed in this pull request?
This pr adds a new extension entrance `queryPostPlannerStrategyRules` in
`SparkSessionExtensions`. It will be applied bet
itholic opened a new pull request, #44073:
URL: https://github.com/apache/spark/pull/44073
### What changes were proposed in this pull request?
This PR proposes to update TODO comments within the Pandas API on Spark
codebase, particularly those associated with missing parameter from D
itholic commented on PR #44068:
URL: https://github.com/apache/spark/pull/44068#issuecomment-1831320232
I guess maybe you wanted to make it looks like:
Hide the "More" drop-down button in the PySpark docs navigation bar ?
or
Hide the `"More"` drop-down button in the PySp
LuciferYang commented on PR #44070:
URL: https://github.com/apache/spark/pull/44070#issuecomment-1831312425
Thanks @zhengruifeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comm
zhengruifeng commented on PR #44070:
URL: https://github.com/apache/spark/pull/44070#issuecomment-1831310289
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubsc
zhengruifeng commented on PR #44067:
URL: https://github.com/apache/spark/pull/44067#issuecomment-1831309126
merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
zhengruifeng closed pull request #44067: [SPARK-46150][INFRA] Combine python
codegen check and protobuf braking change
URL: https://github.com/apache/spark/pull/44067
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408809625
##
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala:
##
@@ -1871,21 +1871,6 @@ private[spark] class DAGScheduler(
markStageAsFinis
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408807471
##
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala:
##
@@ -1871,21 +1871,6 @@ private[spark] class DAGScheduler(
markStageAsFinis
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408807471
##
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala:
##
@@ -1871,21 +1871,6 @@ private[spark] class DAGScheduler(
markStageAsFinis
panbingkun commented on PR #44071:
URL: https://github.com/apache/spark/pull/44071#issuecomment-1831291281
https://github.com/apache/spark/assets/15246973/a3556fdb-0636-4921-a26f-6654dedd7e58";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please
panbingkun commented on PR #44071:
URL: https://github.com/apache/spark/pull/44071#issuecomment-1831291414
> Let's try to use text as much as possible in the PR description, as images
are not easily searchable in the git log.
Done.
--
This is an automated message from the Apache Gi
LuciferYang commented on PR #44070:
URL: https://github.com/apache/spark/pull/44070#issuecomment-1831290943
cc @zhengruifeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
LuciferYang commented on PR #44070:
URL: https://github.com/apache/spark/pull/44070#issuecomment-1831289972
The installed Java version is correct.
--
This is an automated message f
panbingkun commented on PR #44068:
URL: https://github.com/apache/spark/pull/44068#issuecomment-1831288840
> Btw, seems like the code-block style is not applied properly in the PR
title?
>
> https://private-user-images.githubusercontent.com/44108233/286495201-c80c6c7e-544a-4940-8e87-7
LuciferYang commented on PR #44071:
URL: https://github.com/apache/spark/pull/44071#issuecomment-1831288805
Let's try to use text as much as possible in the PR description, as images
are not easily searchable in the git log.
--
This is an automated message from the Apache Git Service.
To
itholic commented on PR #44068:
URL: https://github.com/apache/spark/pull/44068#issuecomment-1831286173
Btw, seems like the code-block style is not applied properly in the PR title?
https://github.com/apache/spark/assets/44108233/c80c6c7e-544a-4940-8e87-7c0fed25a5ce";>
Could you
panbingkun commented on PR #44071:
URL: https://github.com/apache/spark/pull/44071#issuecomment-1831286430
cc @dongjoon-hyun @LuciferYang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spec
itholic commented on code in PR #44068:
URL: https://github.com/apache/spark/pull/44068#discussion_r1408794458
##
python/docs/source/conf.py:
##
@@ -196,6 +196,7 @@
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
+
panbingkun opened a new pull request, #44071:
URL: https://github.com/apache/spark/pull/44071
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How
HyukjinKwon closed pull request #44035: [SPARK-46117][DOCS][PYTHON] Enhancing
readability of PySpark API reference by hiding verbose typehints.
URL: https://github.com/apache/spark/pull/44035
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HyukjinKwon commented on PR #44035:
URL: https://github.com/apache/spark/pull/44035#issuecomment-1831274086
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
beliefer commented on PR #44066:
URL: https://github.com/apache/spark/pull/44066#issuecomment-1831271248
The GA failure is unrelated. cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t
LuciferYang commented on PR #44070:
URL: https://github.com/apache/spark/pull/44070#issuecomment-1831267774
https://github.com/LuciferYang/spark/actions/runs/7028585225/job/19124783706
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
LuciferYang commented on code in PR #44070:
URL: https://github.com/apache/spark/pull/44070#discussion_r1408781274
##
.github/workflows/build_and_test.yml:
##
@@ -283,6 +283,14 @@ jobs:
name: unit-tests-log-${{ matrix.modules }}-${{ matrix.comment }}-${{
matrix.java }}
LuciferYang opened a new pull request, #44070:
URL: https://github.com/apache/spark/pull/44070
### What changes were proposed in this pull request?
This pr aims add a new Daily testing GitHub Action job for Maven with Java 21
### Why are the changes needed?
Need Daily testing for
wangyum commented on PR #43982:
URL: https://github.com/apache/spark/pull/43982#issuecomment-1831263113
cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To un
sandip-db opened a new pull request, #44069:
URL: https://github.com/apache/spark/pull/44069
### What changes were proposed in this pull request?
Add DecimalType support in XML schema inference
### Why are the changes needed?
Add DecimalType support in XML schema inference
itholic commented on PR #44049:
URL: https://github.com/apache/spark/pull/44049#issuecomment-1831260186
Nice fix! +1 for
https://github.com/apache/spark/pull/44049#discussion_r1408578398, otherwise it
looks good to me.
--
This is an automated message from the Apache Git Service.
To respo
panbingkun commented on code in PR #44068:
URL: https://github.com/apache/spark/pull/44068#discussion_r1408773499
##
python/docs/source/conf.py:
##
@@ -196,6 +196,7 @@
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
panbingkun commented on PR #44068:
URL: https://github.com/apache/spark/pull/44068#issuecomment-1831256603
cc @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
panbingkun opened a new pull request, #44068:
URL: https://github.com/apache/spark/pull/44068
### What changes were proposed in this pull request?
The pr aims to hide the `More drop-down button` in the PySpark docs
navigation bar.
### Why are the changes needed?
Because there is
zwangsheng commented on PR #44054:
URL: https://github.com/apache/spark/pull/44054#issuecomment-1831253378
Hi, @cloud-fan @yaooqinn refine the comment related to
`ShuffleWriteProcessor#write`, PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message,
zhengruifeng opened a new pull request, #44067:
URL: https://github.com/apache/spark/pull/44067
### What changes were proposed in this pull request?
Combine python codegen check and protobuf braking change
### Why are the changes needed?
1, `uses: bufbuild/buf-setup-action@v1
LuciferYang commented on code in PR #43526:
URL: https://github.com/apache/spark/pull/43526#discussion_r1408707826
##
core/src/main/scala/org/apache/spark/deploy/StandaloneResourceUtils.scala:
##
@@ -23,7 +23,7 @@ import java.nio.file.Files
import scala.collection.mutable
impo
LuciferYang commented on code in PR #44055:
URL: https://github.com/apache/spark/pull/44055#discussion_r1408703884
##
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLEnv.scala:
##
@@ -33,11 +33,11 @@ import org.apache.spark.util.Utils
private
LuciferYang commented on PR #44057:
URL: https://github.com/apache/spark/pull/44057#issuecomment-1831166449
Thanks @HyukjinKwon @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
HyukjinKwon closed pull request #44065: [SPARK-46149][ML][PYTHON][TESTS] Skip
'TorchDistributorLocalUnitTests.test_end_to_end_run_locally' with Python 3.12
URL: https://github.com/apache/spark/pull/44065
--
This is an automated message from the Apache Git Service.
To respond to the message, p
HyukjinKwon commented on PR #44065:
URL: https://github.com/apache/spark/pull/44065#issuecomment-1831144154
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
HyukjinKwon closed pull request #44064: [SPARK-46148][PS] Fix
pyspark.pandas.mlflow.load_model test (Python 3.12)
URL: https://github.com/apache/spark/pull/44064
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abo
HyukjinKwon commented on PR #44064:
URL: https://github.com/apache/spark/pull/44064#issuecomment-1831143512
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
zhengruifeng commented on PR #44062:
URL: https://github.com/apache/spark/pull/44062#issuecomment-1831141319
thanks, merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
zhengruifeng closed pull request #44062: [SPARK-46146][INFRA] Unpin `markupsafe`
URL: https://github.com/apache/spark/pull/44062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408670937
##
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala:
##
@@ -1671,37 +1671,6 @@ class TaskSchedulerImplSuite extends SparkFunSuite with
Local
beliefer commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408664994
##
core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala:
##
@@ -54,7 +54,7 @@ private[spark] trait TaskScheduler {
// Submit a sequence of tasks to ru
panbingkun commented on code in PR #44049:
URL: https://github.com/apache/spark/pull/44049#discussion_r1408661534
##
python/docs/source/getting_started/quickstart_df.ipynb:
##
@@ -424,76 +424,22 @@
"metadata": {},
"outputs": [
{
- "data": {
- "text/html": [
panbingkun commented on code in PR #44049:
URL: https://github.com/apache/spark/pull/44049#discussion_r1408661534
##
python/docs/source/getting_started/quickstart_df.ipynb:
##
@@ -424,76 +424,22 @@
"metadata": {},
"outputs": [
{
- "data": {
- "text/html": [
panbingkun commented on code in PR #44049:
URL: https://github.com/apache/spark/pull/44049#discussion_r1408660391
##
python/docs/source/getting_started/quickstart_df.ipynb:
##
@@ -424,76 +424,22 @@
"metadata": {},
"outputs": [
{
- "data": {
- "text/html": [
panbingkun commented on code in PR #44049:
URL: https://github.com/apache/spark/pull/44049#discussion_r1408656787
##
python/docs/source/getting_started/quickstart_df.ipynb:
##
@@ -424,76 +424,22 @@
"metadata": {},
"outputs": [
{
- "data": {
- "text/html": [
laglangyue commented on code in PR #43526:
URL: https://github.com/apache/spark/pull/43526#discussion_r1408656173
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala:
##
@@ -49,9 +50,10 @@ abstract class CompactibleFileStreamLog[T <
laglangyue commented on code in PR #43526:
URL: https://github.com/apache/spark/pull/43526#discussion_r1408655902
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSourceLog.scala:
##
@@ -36,8 +33,8 @@ class FileStreamSourceLog(
path: String)
laglangyue commented on code in PR #43526:
URL: https://github.com/apache/spark/pull/43526#discussion_r1408654883
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/OperatorStateMetadata.scala:
##
@@ -64,8 +65,9 @@ case class OperatorStateMetadataV1(
ob
HyukjinKwon commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408649869
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
return
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408649384
##
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala:
##
@@ -296,18 +296,32 @@ private[spark] class TaskSchedulerImpl(
new TaskSetManager(this
beliefer opened a new pull request, #44066:
URL: https://github.com/apache/spark/pull/44066
### What changes were proposed in this pull request?
This PR used to back port https://github.com/apache/spark/pull/43801 to
branch-3.5
### Why are the changes needed?
Escape the sing
dongjoon-hyun commented on PR #43924:
URL: https://github.com/apache/spark/pull/43924#issuecomment-1831101933
Thank you so much, @panbingkun !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
beliefer commented on PR #44016:
URL: https://github.com/apache/spark/pull/44016#issuecomment-1831101286
@srowen @LuciferYang Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
yaooqinn commented on PR #43903:
URL: https://github.com/apache/spark/pull/43903#issuecomment-1831091175
Thank you again, @dongjoon-hyun.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
dongjoon-hyun closed pull request #43903: [SPARK-46003][UI][TESTS] Create an
ui-test module with Jest to test ui javascript code
URL: https://github.com/apache/spark/pull/43903
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
dongjoon-hyun commented on PR #43903:
URL: https://github.com/apache/spark/pull/43903#issuecomment-1831086471
Sure, thank you for proceeding the process.
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
dongjoon-hyun closed pull request #44063: [SPARK-46147][PS][TESTS] Fix the
doctest in pyspark.pandas.series.Seires.to_dict (Python 3.12)
URL: https://github.com/apache/spark/pull/44063
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to Git
yaooqinn commented on PR #43903:
URL: https://github.com/apache/spark/pull/43903#issuecomment-1831082839
Hi @dongjoon-hyun, thanks for your suggestions and vote for the SPIP. As the
vote result is positive,
https://lists.apache.org/thread/1m94wwkd0wfx9j7rb5p4kf1kgd8rgg89, Would you
like to
dongjoon-hyun commented on PR #44063:
URL: https://github.com/apache/spark/pull/44063#issuecomment-1831077704
`pandas-slow` passed. Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dongjoon-hyun commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408626232
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
retu
beliefer commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408625247
##
core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala:
##
@@ -66,10 +66,6 @@ private[spark] trait TaskScheduler {
*/
def killTaskAttempt(taskId:
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408624999
##
core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala:
##
@@ -54,7 +54,7 @@ private[spark] trait TaskScheduler {
// Submit a sequence of tasks to run
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408623095
##
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala:
##
@@ -2205,12 +2190,12 @@ private[spark] class DAGScheduler(
val message = s"Stage fai
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408622291
##
core/src/main/scala/org/apache/spark/internal/config/package.scala:
##
@@ -2603,4 +2603,13 @@ package object config {
.stringConf
.toSequence
.c
Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408621909
##
core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala:
##
@@ -66,10 +66,6 @@ private[spark] trait TaskScheduler {
*/
def killTaskAttempt(taskId:
HyukjinKwon commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408617846
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
return
HyukjinKwon commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408617846
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
return
dongjoon-hyun commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408617098
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
retu
dongjoon-hyun commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408617098
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
retu
dongjoon-hyun commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408617098
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
retu
panbingkun commented on PR #43924:
URL: https://github.com/apache/spark/pull/43924#issuecomment-1831061274
> Thank you for keeping tracking this, @panbingkun . How about Today's
status?
@dongjoon-hyun
- From the past `11` test processes, no similar `fatal errors`
(https://github.com
dongjoon-hyun commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408615284
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
retu
dongjoon-hyun commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408614954
##
python/pyspark/ml/torch/tests/test_distributor.py:
##
@@ -398,6 +398,9 @@ def test_local_file_with_pytorch(self) -> None:
test_file_path, learning
HyukjinKwon commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408614514
##
python/pyspark/tests/test_worker.py:
##
@@ -230,7 +230,7 @@ def conf(cls):
_conf.set("spark.python.worker.faulthandler.enabled", "true")
return
HyukjinKwon commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408613472
##
python/pyspark/ml/torch/tests/test_distributor.py:
##
@@ -398,6 +398,9 @@ def test_local_file_with_pytorch(self) -> None:
test_file_path, learning_r
dongjoon-hyun closed pull request #44057: [SPARK-46140][SQL][TESTS] Remove
comments about JVM module options from `HiveExternalCatalogVersionsSuite`
URL: https://github.com/apache/spark/pull/44057
--
This is an automated message from the Apache Git Service.
To respond to the message, please l
HyukjinKwon commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408613472
##
python/pyspark/ml/torch/tests/test_distributor.py:
##
@@ -398,6 +398,9 @@ def test_local_file_with_pytorch(self) -> None:
test_file_path, learning_r
HyukjinKwon commented on code in PR #44065:
URL: https://github.com/apache/spark/pull/44065#discussion_r1408612421
##
python/pyspark/ml/torch/tests/test_distributor.py:
##
@@ -398,6 +398,9 @@ def test_local_file_with_pytorch(self) -> None:
test_file_path, learning_r
dongjoon-hyun commented on PR #44065:
URL: https://github.com/apache/spark/pull/44065#issuecomment-1831055718
Thank you for taking care of them, @HyukjinKwon !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
dongjoon-hyun commented on PR #44059:
URL: https://github.com/apache/spark/pull/44059#issuecomment-1831053820
Thank you, @LuciferYang !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specifi
HyukjinKwon commented on PR #44065:
URL: https://github.com/apache/spark/pull/44065#issuecomment-1831051264
Build:
https://github.com/HyukjinKwon/spark/actions/runs/7026809016/job/19120174720
--
This is an automated message from the Apache Git Service.
To respond to the message, please lo
panbingkun commented on PR #43924:
URL: https://github.com/apache/spark/pull/43924#issuecomment-1831051103
- 20231125
https://github.com/panbingkun/spark/actions/runs/6987302202/job/1901476
RocksDBStateStoreStreamingAggregationSuite
https://github.com/apache/spark/assets/15246973
HyukjinKwon opened a new pull request, #44065:
URL: https://github.com/apache/spark/pull/44065
### What changes were proposed in this pull request?
This PR proposes to skip doctest, `pyspark.pandas.series.Seires.to_dict`,
compatible with Python 3.12.
### Why are the changes nee
HyukjinKwon commented on PR #44064:
URL: https://github.com/apache/spark/pull/44064#issuecomment-1831045562
Build: https://github.com/HyukjinKwon/spark/actions/runs/7026607283
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub an
HyukjinKwon opened a new pull request, #44064:
URL: https://github.com/apache/spark/pull/44064
### What changes were proposed in this pull request?
This PR proposes to explicitly name the columns for DataFrame used when
predicting in mlflow (according to the mlflow spec) in order to f
HyukjinKwon commented on PR #44063:
URL: https://github.com/apache/spark/pull/44063#issuecomment-1831028623
Build: https://github.com/HyukjinKwon/spark/actions/runs/7026487431
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub an
HyukjinKwon opened a new pull request, #44063:
URL: https://github.com/apache/spark/pull/44063
### What changes were proposed in this pull request?
This PR proposes to fix doctest, `pyspark.pandas.series.Seires.to_dict`,
compatible with Python 3.12.
```
File "/__w/spark/spar
cloud-fan commented on code in PR #44061:
URL: https://github.com/apache/spark/pull/44061#discussion_r1408585088
##
sql/core/src/test/scala/org/apache/spark/sql/internal/CatalogSuite.scala:
##
@@ -1050,6 +1052,15 @@ class CatalogSuite extends SharedSparkSession with
AnalysisTes
cloud-fan commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1408583579
##
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala:
##
@@ -296,18 +296,32 @@ private[spark] class TaskSchedulerImpl(
new TaskSetManager(th
1 - 100 of 230 matches
Mail list logo
