[spark] branch master updated (edeecad -> e21bb71)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from edeecad [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission add e21bb71 [SPARK-32991][SQL] Use conf in shared state as the original configuraion for RESET No new revisions were added by this update. Summary of changes: .../spark/sql/execution/command/SetCommand.scala | 2 +- .../apache/spark/sql/internal/SharedState.scala| 15 -- .../spark/sql/SparkSessionBuilderSuite.scala | 54 ++ .../spark/sql/hive/HiveSharedStateSuite.scala | 46 +- 4 files changed, 99 insertions(+), 18 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (edeecad -> e21bb71)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from edeecad [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission add e21bb71 [SPARK-32991][SQL] Use conf in shared state as the original configuraion for RESET No new revisions were added by this update. Summary of changes: .../spark/sql/execution/command/SetCommand.scala | 2 +- .../apache/spark/sql/internal/SharedState.scala| 15 -- .../spark/sql/SparkSessionBuilderSuite.scala | 54 ++ .../spark/sql/hive/HiveSharedStateSuite.scala | 46 +- 4 files changed, 99 insertions(+), 18 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (87b32f6 -> edeecad)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document add edeecad [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission No new revisions were added by this update. Summary of changes: .../apache/spark/ExecutorAllocationClient.scala| 19 +++-- .../apache/spark/ExecutorAllocationManager.scala | 5 +- .../org/apache/spark/deploy/DeployMessage.scala| 32 ++-- .../org/apache/spark/deploy/master/Master.scala| 21 +++-- .../org/apache/spark/deploy/worker/Worker.scala| 29 --- .../executor/CoarseGrainedExecutorBackend.scala| 70 +--- .../cluster/CoarseGrainedClusterMessage.scala | 16 ++-- .../cluster/CoarseGrainedSchedulerBackend.scala| 93 -- .../cluster/StandaloneSchedulerBackend.scala | 7 +- .../org/apache/spark/storage/BlockManager.scala| 9 ++- .../spark/storage/BlockManagerMasterEndpoint.scala | 25 ++ .../storage/BlockManagerStorageEndpoint.scala | 2 +- .../spark/deploy/DecommissionWorkerSuite.scala | 4 +- .../spark/deploy/client/AppClientSuite.scala | 8 +- .../apache/spark/deploy/master/MasterSuite.scala | 18 +++-- .../spark/scheduler/WorkerDecommissionSuite.scala | 7 +- .../BlockManagerDecommissionIntegrationSuite.scala | 40 ++ .../k8s/integrationtest/DecommissionSuite.scala| 12 +-- .../scheduler/ExecutorAllocationManagerSuite.scala | 6 +- 19 files changed, 257 insertions(+), 166 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (87b32f6 -> edeecad)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document add edeecad [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission No new revisions were added by this update. Summary of changes: .../apache/spark/ExecutorAllocationClient.scala| 19 +++-- .../apache/spark/ExecutorAllocationManager.scala | 5 +- .../org/apache/spark/deploy/DeployMessage.scala| 32 ++-- .../org/apache/spark/deploy/master/Master.scala| 21 +++-- .../org/apache/spark/deploy/worker/Worker.scala| 29 --- .../executor/CoarseGrainedExecutorBackend.scala| 70 +--- .../cluster/CoarseGrainedClusterMessage.scala | 16 ++-- .../cluster/CoarseGrainedSchedulerBackend.scala| 93 -- .../cluster/StandaloneSchedulerBackend.scala | 7 +- .../org/apache/spark/storage/BlockManager.scala| 9 ++- .../spark/storage/BlockManagerMasterEndpoint.scala | 25 ++ .../storage/BlockManagerStorageEndpoint.scala | 2 +- .../spark/deploy/DecommissionWorkerSuite.scala | 4 +- .../spark/deploy/client/AppClientSuite.scala | 8 +- .../apache/spark/deploy/master/MasterSuite.scala | 18 +++-- .../spark/scheduler/WorkerDecommissionSuite.scala | 7 +- .../BlockManagerDecommissionIntegrationSuite.scala | 40 ++ .../k8s/integrationtest/DecommissionSuite.scala| 12 +-- .../scheduler/ExecutorAllocationManagerSuite.scala | 6 +- 19 files changed, 257 insertions(+), 166 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (87b32f6 -> edeecad)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document add edeecad [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission No new revisions were added by this update. Summary of changes: .../apache/spark/ExecutorAllocationClient.scala| 19 +++-- .../apache/spark/ExecutorAllocationManager.scala | 5 +- .../org/apache/spark/deploy/DeployMessage.scala| 32 ++-- .../org/apache/spark/deploy/master/Master.scala| 21 +++-- .../org/apache/spark/deploy/worker/Worker.scala| 29 --- .../executor/CoarseGrainedExecutorBackend.scala| 70 +--- .../cluster/CoarseGrainedClusterMessage.scala | 16 ++-- .../cluster/CoarseGrainedSchedulerBackend.scala| 93 -- .../cluster/StandaloneSchedulerBackend.scala | 7 +- .../org/apache/spark/storage/BlockManager.scala| 9 ++- .../spark/storage/BlockManagerMasterEndpoint.scala | 25 ++ .../storage/BlockManagerStorageEndpoint.scala | 2 +- .../spark/deploy/DecommissionWorkerSuite.scala | 4 +- .../spark/deploy/client/AppClientSuite.scala | 8 +- .../apache/spark/deploy/master/MasterSuite.scala | 18 +++-- .../spark/scheduler/WorkerDecommissionSuite.scala | 7 +- .../BlockManagerDecommissionIntegrationSuite.scala | 40 ++ .../k8s/integrationtest/DecommissionSuite.scala| 12 +-- .../scheduler/ExecutorAllocationManagerSuite.scala | 6 +- 19 files changed, 257 insertions(+), 166 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (87b32f6 -> edeecad)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document add edeecad [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission No new revisions were added by this update. Summary of changes: .../apache/spark/ExecutorAllocationClient.scala| 19 +++-- .../apache/spark/ExecutorAllocationManager.scala | 5 +- .../org/apache/spark/deploy/DeployMessage.scala| 32 ++-- .../org/apache/spark/deploy/master/Master.scala| 21 +++-- .../org/apache/spark/deploy/worker/Worker.scala| 29 --- .../executor/CoarseGrainedExecutorBackend.scala| 70 +--- .../cluster/CoarseGrainedClusterMessage.scala | 16 ++-- .../cluster/CoarseGrainedSchedulerBackend.scala| 93 -- .../cluster/StandaloneSchedulerBackend.scala | 7 +- .../org/apache/spark/storage/BlockManager.scala| 9 ++- .../spark/storage/BlockManagerMasterEndpoint.scala | 25 ++ .../storage/BlockManagerStorageEndpoint.scala | 2 +- .../spark/deploy/DecommissionWorkerSuite.scala | 4 +- .../spark/deploy/client/AppClientSuite.scala | 8 +- .../apache/spark/deploy/master/MasterSuite.scala | 18 +++-- .../spark/scheduler/WorkerDecommissionSuite.scala | 7 +- .../BlockManagerDecommissionIntegrationSuite.scala | 40 ++ .../k8s/integrationtest/DecommissionSuite.scala| 12 +-- .../scheduler/ExecutorAllocationManagerSuite.scala | 6 +- 19 files changed, 257 insertions(+), 166 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (87b32f6 -> edeecad)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document add edeecad [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission No new revisions were added by this update. Summary of changes: .../apache/spark/ExecutorAllocationClient.scala| 19 +++-- .../apache/spark/ExecutorAllocationManager.scala | 5 +- .../org/apache/spark/deploy/DeployMessage.scala| 32 ++-- .../org/apache/spark/deploy/master/Master.scala| 21 +++-- .../org/apache/spark/deploy/worker/Worker.scala| 29 --- .../executor/CoarseGrainedExecutorBackend.scala| 70 +--- .../cluster/CoarseGrainedClusterMessage.scala | 16 ++-- .../cluster/CoarseGrainedSchedulerBackend.scala| 93 -- .../cluster/StandaloneSchedulerBackend.scala | 7 +- .../org/apache/spark/storage/BlockManager.scala| 9 ++- .../spark/storage/BlockManagerMasterEndpoint.scala | 25 ++ .../storage/BlockManagerStorageEndpoint.scala | 2 +- .../spark/deploy/DecommissionWorkerSuite.scala | 4 +- .../spark/deploy/client/AppClientSuite.scala | 8 +- .../apache/spark/deploy/master/MasterSuite.scala | 18 +++-- .../spark/scheduler/WorkerDecommissionSuite.scala | 7 +- .../BlockManagerDecommissionIntegrationSuite.scala | 40 ++ .../k8s/integrationtest/DecommissionSuite.scala| 12 +-- .../scheduler/ExecutorAllocationManagerSuite.scala | 6 +- 19 files changed, 257 insertions(+), 166 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3819d39 -> 87b32f6)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one add 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document No new revisions were added by this update. Summary of changes: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 2 +- .../org/apache/spark/sql/catalyst/optimizer/OptimizerLoggingSuite.scala | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3819d39 -> 87b32f6)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one add 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document No new revisions were added by this update. Summary of changes: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 2 +- .../org/apache/spark/sql/catalyst/optimizer/OptimizerLoggingSuite.scala | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3819d39 -> 87b32f6)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one add 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document No new revisions were added by this update. Summary of changes: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 2 +- .../org/apache/spark/sql/catalyst/optimizer/OptimizerLoggingSuite.scala | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3819d39 -> 87b32f6)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one add 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document No new revisions were added by this update. Summary of changes: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 2 +- .../org/apache/spark/sql/catalyst/optimizer/OptimizerLoggingSuite.scala | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (3819d39 -> 87b32f6)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one add 87b32f6 [MINOR][DOCS][TESTS] Fix PLAN_CHANGE_LOG_LEVEL document No new revisions were added by this update. Summary of changes: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 2 +- .../org/apache/spark/sql/catalyst/optimizer/OptimizerLoggingSuite.scala | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a03d77d -> 3819d39)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` add 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one No new revisions were added by this update. Summary of changes: build/sbt-launch-lib.bash | 6 +- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala | 5 - .../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 3 ++- 3 files changed, 11 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a03d77d -> 3819d39)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` add 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one No new revisions were added by this update. Summary of changes: build/sbt-launch-lib.bash | 6 +- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala | 5 - .../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 3 ++- 3 files changed, 11 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a03d77d -> 3819d39)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` add 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one No new revisions were added by this update. Summary of changes: build/sbt-launch-lib.bash | 6 +- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala | 5 - .../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 3 ++- 3 files changed, 11 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a03d77d -> 3819d39)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` add 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one No new revisions were added by this update. Summary of changes: build/sbt-launch-lib.bash | 6 +- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala | 5 - .../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 3 ++- 3 files changed, 11 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a03d77d -> 3819d39)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` add 3819d39 [SPARK-32998][BUILD] Add ability to override default remote repos with internal one No new revisions were added by this update. Summary of changes: build/sbt-launch-lib.bash | 6 +- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala | 5 - .../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 3 ++- 3 files changed, 11 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (b38f3a5 -> a03d77d)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct add a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` No new revisions were added by this update. Summary of changes: .../execution/datasources/DataSourceUtils.scala| 22 ++ .../datasources/parquet/ParquetWriteSupport.scala | 6 +++--- .../main/scala/org/apache/spark/sql/package.scala | 4 ++-- .../datasources/parquet/ParquetIOSuite.scala | 16 +++- .../apache/spark/sql/hive/StatisticsSuite.scala| 2 +- 5 files changed, 31 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (b38f3a5 -> a03d77d)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct add a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` No new revisions were added by this update. Summary of changes: .../execution/datasources/DataSourceUtils.scala| 22 ++ .../datasources/parquet/ParquetWriteSupport.scala | 6 +++--- .../main/scala/org/apache/spark/sql/package.scala | 4 ++-- .../datasources/parquet/ParquetIOSuite.scala | 16 +++- .../apache/spark/sql/hive/StatisticsSuite.scala| 2 +- 5 files changed, 31 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (b38f3a5 -> a03d77d)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct add a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` No new revisions were added by this update. Summary of changes: .../execution/datasources/DataSourceUtils.scala| 22 ++ .../datasources/parquet/ParquetWriteSupport.scala | 6 +++--- .../main/scala/org/apache/spark/sql/package.scala | 4 ++-- .../datasources/parquet/ParquetIOSuite.scala | 16 +++- .../apache/spark/sql/hive/StatisticsSuite.scala| 2 +- 5 files changed, 31 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (b38f3a5 -> a03d77d)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct add a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` No new revisions were added by this update. Summary of changes: .../execution/datasources/DataSourceUtils.scala| 22 ++ .../datasources/parquet/ParquetWriteSupport.scala | 6 +++--- .../main/scala/org/apache/spark/sql/package.scala | 4 ++-- .../datasources/parquet/ParquetIOSuite.scala | 16 +++- .../apache/spark/sql/hive/StatisticsSuite.scala| 2 +- 5 files changed, 31 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (b38f3a5 -> a03d77d)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct add a03d77d [SPARK-33160][SQL][FOLLOWUP] Replace the parquet metadata key `org.apache.spark.int96NoRebase` by `org.apache.spark.legacyINT96` No new revisions were added by this update. Summary of changes: .../execution/datasources/DataSourceUtils.scala| 22 ++ .../datasources/parquet/ParquetWriteSupport.scala | 6 +++--- .../main/scala/org/apache/spark/sql/package.scala | 4 ++-- .../datasources/parquet/ParquetIOSuite.scala | 16 +++- .../apache/spark/sql/hive/StatisticsSuite.scala| 2 +- 5 files changed, 31 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a1629b4 -> b38f3a5)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location add b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct No new revisions were added by this update. Summary of changes: ...ithDynamicPartitionsBenchmark-jdk11-results.txt | 8 ++ ...TableWithDynamicPartitionsBenchmark-results.txt | 8 ++ .../datasources/BasicWriteStatsTracker.scala | 16 ++-- ...InsertTableWithDynamicPartitionsBenchmark.scala | 103 + .../BasicWriteJobStatsTrackerMetricSuite.scala | 59 5 files changed, 187 insertions(+), 7 deletions(-) create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-jdk11-results.txt create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-results.txt create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/InsertTableWithDynamicPartitionsBenchmark.scala create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/BasicWriteJobStatsTrackerMetricSuite.scala - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a1629b4 -> b38f3a5)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location add b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct No new revisions were added by this update. Summary of changes: ...ithDynamicPartitionsBenchmark-jdk11-results.txt | 8 ++ ...TableWithDynamicPartitionsBenchmark-results.txt | 8 ++ .../datasources/BasicWriteStatsTracker.scala | 16 ++-- ...InsertTableWithDynamicPartitionsBenchmark.scala | 103 + .../BasicWriteJobStatsTrackerMetricSuite.scala | 59 5 files changed, 187 insertions(+), 7 deletions(-) create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-jdk11-results.txt create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-results.txt create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/InsertTableWithDynamicPartitionsBenchmark.scala create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/BasicWriteJobStatsTrackerMetricSuite.scala - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (8cae7f8 -> a1629b4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect) add a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/hive/HiveUtils.scala | 92 ++ 1 file changed, 77 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a1629b4 -> b38f3a5)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location add b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct No new revisions were added by this update. Summary of changes: ...ithDynamicPartitionsBenchmark-jdk11-results.txt | 8 ++ ...TableWithDynamicPartitionsBenchmark-results.txt | 8 ++ .../datasources/BasicWriteStatsTracker.scala | 16 ++-- ...InsertTableWithDynamicPartitionsBenchmark.scala | 103 + .../BasicWriteJobStatsTrackerMetricSuite.scala | 59 5 files changed, 187 insertions(+), 7 deletions(-) create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-jdk11-results.txt create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-results.txt create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/InsertTableWithDynamicPartitionsBenchmark.scala create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/BasicWriteJobStatsTrackerMetricSuite.scala - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (8cae7f8 -> a1629b4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect) add a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/hive/HiveUtils.scala | 92 ++ 1 file changed, 77 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (d9ee33c -> 8cae7f8)
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on
thriftserver
add 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table
Catalog: add, update type and nullability of columns (MySQL dialect)
No new revisions were added by this update.
Summary of changes:
...tionSuite.scala => MySQLIntegrationSuite.scala} | 55 +++---
.../org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala | 34 +++--
.../org/apache/spark/sql/jdbc/MySQLDialect.scala | 24 +-
3 files changed, 69 insertions(+), 44 deletions(-)
copy
external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/{OracleIntegrationSuite.scala
=> MySQLIntegrationSuite.scala} (56%)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a1629b4 -> b38f3a5)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location add b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct No new revisions were added by this update. Summary of changes: ...ithDynamicPartitionsBenchmark-jdk11-results.txt | 8 ++ ...TableWithDynamicPartitionsBenchmark-results.txt | 8 ++ .../datasources/BasicWriteStatsTracker.scala | 16 ++-- ...InsertTableWithDynamicPartitionsBenchmark.scala | 103 + .../BasicWriteJobStatsTrackerMetricSuite.scala | 59 5 files changed, 187 insertions(+), 7 deletions(-) create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-jdk11-results.txt create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-results.txt create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/InsertTableWithDynamicPartitionsBenchmark.scala create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/BasicWriteJobStatsTrackerMetricSuite.scala - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (8cae7f8 -> a1629b4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect) add a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/hive/HiveUtils.scala | 92 ++ 1 file changed, 77 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (d9ee33c -> 8cae7f8)
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on
thriftserver
add 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table
Catalog: add, update type and nullability of columns (MySQL dialect)
No new revisions were added by this update.
Summary of changes:
...tionSuite.scala => MySQLIntegrationSuite.scala} | 55 +++---
.../org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala | 34 +++--
.../org/apache/spark/sql/jdbc/MySQLDialect.scala | 24 +-
3 files changed, 69 insertions(+), 44 deletions(-)
copy
external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/{OracleIntegrationSuite.scala
=> MySQLIntegrationSuite.scala} (56%)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a1629b4 -> b38f3a5)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location add b38f3a5 [SPARK-32978][SQL] Make sure the number of dynamic part metric is correct No new revisions were added by this update. Summary of changes: ...ithDynamicPartitionsBenchmark-jdk11-results.txt | 8 ++ ...TableWithDynamicPartitionsBenchmark-results.txt | 8 ++ .../datasources/BasicWriteStatsTracker.scala | 16 ++-- ...InsertTableWithDynamicPartitionsBenchmark.scala | 103 + .../BasicWriteJobStatsTrackerMetricSuite.scala | 59 5 files changed, 187 insertions(+), 7 deletions(-) create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-jdk11-results.txt create mode 100644 sql/core/benchmarks/InsertTableWithDynamicPartitionsBenchmark-results.txt create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/InsertTableWithDynamicPartitionsBenchmark.scala create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/BasicWriteJobStatsTrackerMetricSuite.scala - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (8cae7f8 -> a1629b4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect) add a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/hive/HiveUtils.scala | 92 ++ 1 file changed, 77 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (d9ee33c -> 8cae7f8)
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on
thriftserver
add 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table
Catalog: add, update type and nullability of columns (MySQL dialect)
No new revisions were added by this update.
Summary of changes:
...tionSuite.scala => MySQLIntegrationSuite.scala} | 55 +++---
.../org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala | 34 +++--
.../org/apache/spark/sql/jdbc/MySQLDialect.scala | 24 +-
3 files changed, 69 insertions(+), 44 deletions(-)
copy
external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/{OracleIntegrationSuite.scala
=> MySQLIntegrationSuite.scala} (56%)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (8cae7f8 -> a1629b4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect) add a1629b4 [SPARK-32852][SQL] spark.sql.hive.metastore.jars support HDFS location No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/hive/HiveUtils.scala | 92 ++ 1 file changed, 77 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (d9ee33c -> 8cae7f8)
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on
thriftserver
add 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table
Catalog: add, update type and nullability of columns (MySQL dialect)
No new revisions were added by this update.
Summary of changes:
...tionSuite.scala => MySQLIntegrationSuite.scala} | 55 +++---
.../org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala | 34 +++--
.../org/apache/spark/sql/jdbc/MySQLDialect.scala | 24 +-
3 files changed, 69 insertions(+), 44 deletions(-)
copy
external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/{OracleIntegrationSuite.scala
=> MySQLIntegrationSuite.scala} (56%)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (d9ee33c -> 8cae7f8)
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on
thriftserver
add 8cae7f8 [SPARK-33095][SQL] Support ALTER TABLE in JDBC v2 Table
Catalog: add, update type and nullability of columns (MySQL dialect)
No new revisions were added by this update.
Summary of changes:
...tionSuite.scala => MySQLIntegrationSuite.scala} | 55 +++---
.../org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala | 34 +++--
.../org/apache/spark/sql/jdbc/MySQLDialect.scala | 24 +-
3 files changed, 69 insertions(+), 44 deletions(-)
copy
external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/{OracleIntegrationSuite.scala
=> MySQLIntegrationSuite.scala} (56%)
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated: [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 8b9036f [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 8b9036f is described below commit 8b9036fb684d1621452c22115345ddfcda6e07c5 Author: HyukjinKwon AuthorDate: Thu Oct 22 18:17:36 2020 +0900 [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 ### What changes were proposed in this pull request? This PR proposes to set the upper bound of PyArrow and Pandas versions to 0.12.0 and 0.24.0 respectively. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 and https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 were not ported back so it fails the tests. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 contains Arrow dependency upgrade so it cannot be cleanly ported back. Note that I _think_ these tests were broken from the very first place at https://github.com/apache/spark/commit/7c65f7680ffbe2c03e444ec60358cbf912c27d13#diff-bdcc6a2a85f645f62724fe8dafbf0581cb0c1d65f6a76cb2985a9172e31a473c. There was one flaky test in ML that stops other tests so SQL and Arrow related tests were not shown. ### Why are the changes needed? 1. Spark 2.4.x already declared that higher versions might not work at https://github.com/apache/spark/blob/branch-2.4/docs/sql-pyspark-pandas-with-arrow.md#recommended-pandas-and-pyarrow-versions. 2. We're currently unable to test all combinations (due to the lack of resources in GitHub Actions, see SPARK-32264). It should be best to pick one combination to test. 3. Just to clarify, Spark 2.4 works with the latest PyArrow and pandas 99% correctly. Most of are just test only issues. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions in this build should test. Closes #30128 from HyukjinKwon/SPARK-33217. Lead-authored-by: HyukjinKwon Co-authored-by: Dongjoon Hyun Signed-off-by: HyukjinKwon --- .github/workflows/build_and_test.yml | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml index 8f46250..9390248 100644 --- a/.github/workflows/build_and_test.yml +++ b/.github/workflows/build_and_test.yml @@ -130,16 +130,16 @@ jobs: if: contains(matrix.modules, 'pyspark') # PyArrow is not supported in PyPy yet, see ARROW-2651. run: | -python3.6 -m pip install numpy pyarrow pandas scipy xmlrunner +python3.6 -m pip install numpy 'pyarrow<0.12.0' 'pandas<0.24.0' scipy xmlrunner python3.6 -m pip list -# PyPy does not have xmlrunner -pypy3 -m pip install numpy pandas scipy +# PyPy does not have xmlrunner, and pandas<0.24.0 installation fails in PyPy3, just skipping. +pypy3 -m pip install numpy scipy pypy3 -m pip list - name: Install Python packages (Python 2.7) if: contains(matrix.modules, 'pyspark') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-')) run: | # Some tests do not pass in PySpark with PyArrow, for example, pyspark.sql.tests.ArrowTests. -python2.7 -m pip install numpy pandas scipy xmlrunner +python2.7 -m pip install numpy 'pandas<0.24.0' scipy xmlrunner python2.7 -m pip list # SparkR - name: Install R 4.0 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated: [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 8b9036f [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 8b9036f is described below commit 8b9036fb684d1621452c22115345ddfcda6e07c5 Author: HyukjinKwon AuthorDate: Thu Oct 22 18:17:36 2020 +0900 [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 ### What changes were proposed in this pull request? This PR proposes to set the upper bound of PyArrow and Pandas versions to 0.12.0 and 0.24.0 respectively. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 and https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 were not ported back so it fails the tests. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 contains Arrow dependency upgrade so it cannot be cleanly ported back. Note that I _think_ these tests were broken from the very first place at https://github.com/apache/spark/commit/7c65f7680ffbe2c03e444ec60358cbf912c27d13#diff-bdcc6a2a85f645f62724fe8dafbf0581cb0c1d65f6a76cb2985a9172e31a473c. There was one flaky test in ML that stops other tests so SQL and Arrow related tests were not shown. ### Why are the changes needed? 1. Spark 2.4.x already declared that higher versions might not work at https://github.com/apache/spark/blob/branch-2.4/docs/sql-pyspark-pandas-with-arrow.md#recommended-pandas-and-pyarrow-versions. 2. We're currently unable to test all combinations (due to the lack of resources in GitHub Actions, see SPARK-32264). It should be best to pick one combination to test. 3. Just to clarify, Spark 2.4 works with the latest PyArrow and pandas 99% correctly. Most of are just test only issues. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions in this build should test. Closes #30128 from HyukjinKwon/SPARK-33217. Lead-authored-by: HyukjinKwon Co-authored-by: Dongjoon Hyun Signed-off-by: HyukjinKwon --- .github/workflows/build_and_test.yml | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml index 8f46250..9390248 100644 --- a/.github/workflows/build_and_test.yml +++ b/.github/workflows/build_and_test.yml @@ -130,16 +130,16 @@ jobs: if: contains(matrix.modules, 'pyspark') # PyArrow is not supported in PyPy yet, see ARROW-2651. run: | -python3.6 -m pip install numpy pyarrow pandas scipy xmlrunner +python3.6 -m pip install numpy 'pyarrow<0.12.0' 'pandas<0.24.0' scipy xmlrunner python3.6 -m pip list -# PyPy does not have xmlrunner -pypy3 -m pip install numpy pandas scipy +# PyPy does not have xmlrunner, and pandas<0.24.0 installation fails in PyPy3, just skipping. +pypy3 -m pip install numpy scipy pypy3 -m pip list - name: Install Python packages (Python 2.7) if: contains(matrix.modules, 'pyspark') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-')) run: | # Some tests do not pass in PySpark with PyArrow, for example, pyspark.sql.tests.ArrowTests. -python2.7 -m pip install numpy pandas scipy xmlrunner +python2.7 -m pip install numpy 'pandas<0.24.0' scipy xmlrunner python2.7 -m pip list # SparkR - name: Install R 4.0 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated: [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 8b9036f [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 8b9036f is described below commit 8b9036fb684d1621452c22115345ddfcda6e07c5 Author: HyukjinKwon AuthorDate: Thu Oct 22 18:17:36 2020 +0900 [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 ### What changes were proposed in this pull request? This PR proposes to set the upper bound of PyArrow and Pandas versions to 0.12.0 and 0.24.0 respectively. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 and https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 were not ported back so it fails the tests. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 contains Arrow dependency upgrade so it cannot be cleanly ported back. Note that I _think_ these tests were broken from the very first place at https://github.com/apache/spark/commit/7c65f7680ffbe2c03e444ec60358cbf912c27d13#diff-bdcc6a2a85f645f62724fe8dafbf0581cb0c1d65f6a76cb2985a9172e31a473c. There was one flaky test in ML that stops other tests so SQL and Arrow related tests were not shown. ### Why are the changes needed? 1. Spark 2.4.x already declared that higher versions might not work at https://github.com/apache/spark/blob/branch-2.4/docs/sql-pyspark-pandas-with-arrow.md#recommended-pandas-and-pyarrow-versions. 2. We're currently unable to test all combinations (due to the lack of resources in GitHub Actions, see SPARK-32264). It should be best to pick one combination to test. 3. Just to clarify, Spark 2.4 works with the latest PyArrow and pandas 99% correctly. Most of are just test only issues. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions in this build should test. Closes #30128 from HyukjinKwon/SPARK-33217. Lead-authored-by: HyukjinKwon Co-authored-by: Dongjoon Hyun Signed-off-by: HyukjinKwon --- .github/workflows/build_and_test.yml | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml index 8f46250..9390248 100644 --- a/.github/workflows/build_and_test.yml +++ b/.github/workflows/build_and_test.yml @@ -130,16 +130,16 @@ jobs: if: contains(matrix.modules, 'pyspark') # PyArrow is not supported in PyPy yet, see ARROW-2651. run: | -python3.6 -m pip install numpy pyarrow pandas scipy xmlrunner +python3.6 -m pip install numpy 'pyarrow<0.12.0' 'pandas<0.24.0' scipy xmlrunner python3.6 -m pip list -# PyPy does not have xmlrunner -pypy3 -m pip install numpy pandas scipy +# PyPy does not have xmlrunner, and pandas<0.24.0 installation fails in PyPy3, just skipping. +pypy3 -m pip install numpy scipy pypy3 -m pip list - name: Install Python packages (Python 2.7) if: contains(matrix.modules, 'pyspark') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-')) run: | # Some tests do not pass in PySpark with PyArrow, for example, pyspark.sql.tests.ArrowTests. -python2.7 -m pip install numpy pandas scipy xmlrunner +python2.7 -m pip install numpy 'pandas<0.24.0' scipy xmlrunner python2.7 -m pip list # SparkR - name: Install R 4.0 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated: [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 8b9036f [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 8b9036f is described below commit 8b9036fb684d1621452c22115345ddfcda6e07c5 Author: HyukjinKwon AuthorDate: Thu Oct 22 18:17:36 2020 +0900 [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 ### What changes were proposed in this pull request? This PR proposes to set the upper bound of PyArrow and Pandas versions to 0.12.0 and 0.24.0 respectively. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 and https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 were not ported back so it fails the tests. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 contains Arrow dependency upgrade so it cannot be cleanly ported back. Note that I _think_ these tests were broken from the very first place at https://github.com/apache/spark/commit/7c65f7680ffbe2c03e444ec60358cbf912c27d13#diff-bdcc6a2a85f645f62724fe8dafbf0581cb0c1d65f6a76cb2985a9172e31a473c. There was one flaky test in ML that stops other tests so SQL and Arrow related tests were not shown. ### Why are the changes needed? 1. Spark 2.4.x already declared that higher versions might not work at https://github.com/apache/spark/blob/branch-2.4/docs/sql-pyspark-pandas-with-arrow.md#recommended-pandas-and-pyarrow-versions. 2. We're currently unable to test all combinations (due to the lack of resources in GitHub Actions, see SPARK-32264). It should be best to pick one combination to test. 3. Just to clarify, Spark 2.4 works with the latest PyArrow and pandas 99% correctly. Most of are just test only issues. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions in this build should test. Closes #30128 from HyukjinKwon/SPARK-33217. Lead-authored-by: HyukjinKwon Co-authored-by: Dongjoon Hyun Signed-off-by: HyukjinKwon --- .github/workflows/build_and_test.yml | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml index 8f46250..9390248 100644 --- a/.github/workflows/build_and_test.yml +++ b/.github/workflows/build_and_test.yml @@ -130,16 +130,16 @@ jobs: if: contains(matrix.modules, 'pyspark') # PyArrow is not supported in PyPy yet, see ARROW-2651. run: | -python3.6 -m pip install numpy pyarrow pandas scipy xmlrunner +python3.6 -m pip install numpy 'pyarrow<0.12.0' 'pandas<0.24.0' scipy xmlrunner python3.6 -m pip list -# PyPy does not have xmlrunner -pypy3 -m pip install numpy pandas scipy +# PyPy does not have xmlrunner, and pandas<0.24.0 installation fails in PyPy3, just skipping. +pypy3 -m pip install numpy scipy pypy3 -m pip list - name: Install Python packages (Python 2.7) if: contains(matrix.modules, 'pyspark') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-')) run: | # Some tests do not pass in PySpark with PyArrow, for example, pyspark.sql.tests.ArrowTests. -python2.7 -m pip install numpy pandas scipy xmlrunner +python2.7 -m pip install numpy 'pandas<0.24.0' scipy xmlrunner python2.7 -m pip list # SparkR - name: Install R 4.0 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated: [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 8b9036f [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 8b9036f is described below commit 8b9036fb684d1621452c22115345ddfcda6e07c5 Author: HyukjinKwon AuthorDate: Thu Oct 22 18:17:36 2020 +0900 [SPARK-33217][INFRA][PYTHON][2.4] Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4 ### What changes were proposed in this pull request? This PR proposes to set the upper bound of PyArrow and Pandas versions to 0.12.0 and 0.24.0 respectively. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 and https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 were not ported back so it fails the tests. https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 contains Arrow dependency upgrade so it cannot be cleanly ported back. Note that I _think_ these tests were broken from the very first place at https://github.com/apache/spark/commit/7c65f7680ffbe2c03e444ec60358cbf912c27d13#diff-bdcc6a2a85f645f62724fe8dafbf0581cb0c1d65f6a76cb2985a9172e31a473c. There was one flaky test in ML that stops other tests so SQL and Arrow related tests were not shown. ### Why are the changes needed? 1. Spark 2.4.x already declared that higher versions might not work at https://github.com/apache/spark/blob/branch-2.4/docs/sql-pyspark-pandas-with-arrow.md#recommended-pandas-and-pyarrow-versions. 2. We're currently unable to test all combinations (due to the lack of resources in GitHub Actions, see SPARK-32264). It should be best to pick one combination to test. 3. Just to clarify, Spark 2.4 works with the latest PyArrow and pandas 99% correctly. Most of are just test only issues. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions in this build should test. Closes #30128 from HyukjinKwon/SPARK-33217. Lead-authored-by: HyukjinKwon Co-authored-by: Dongjoon Hyun Signed-off-by: HyukjinKwon --- .github/workflows/build_and_test.yml | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml index 8f46250..9390248 100644 --- a/.github/workflows/build_and_test.yml +++ b/.github/workflows/build_and_test.yml @@ -130,16 +130,16 @@ jobs: if: contains(matrix.modules, 'pyspark') # PyArrow is not supported in PyPy yet, see ARROW-2651. run: | -python3.6 -m pip install numpy pyarrow pandas scipy xmlrunner +python3.6 -m pip install numpy 'pyarrow<0.12.0' 'pandas<0.24.0' scipy xmlrunner python3.6 -m pip list -# PyPy does not have xmlrunner -pypy3 -m pip install numpy pandas scipy +# PyPy does not have xmlrunner, and pandas<0.24.0 installation fails in PyPy3, just skipping. +pypy3 -m pip install numpy scipy pypy3 -m pip list - name: Install Python packages (Python 2.7) if: contains(matrix.modules, 'pyspark') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-')) run: | # Some tests do not pass in PySpark with PyArrow, for example, pyspark.sql.tests.ArrowTests. -python2.7 -m pip install numpy pandas scipy xmlrunner +python2.7 -m pip install numpy 'pandas<0.24.0' scipy xmlrunner python2.7 -m pip list # SparkR - name: Install R 4.0 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a908b67 -> d9ee33c)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks add d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on thriftserver No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 11 + .../service/cli/operation/OperationManager.java| 4 +- .../hive/service/cli/operation/SQLOperation.java | 7 +-- .../sql/hive/thriftserver/HiveThriftServer2.scala | 2 +- .../SparkExecuteStatementOperation.scala | 45 +- .../server/SparkSQLOperationManager.scala | 5 +- .../ui/HiveThriftServer2AppStatusStore.scala | 1 + .../ui/HiveThriftServer2EventManager.scala | 7 +++ .../ui/HiveThriftServer2Listener.scala | 10 .../thriftserver/HiveThriftServer2Suites.scala | 55 +- .../SparkExecuteStatementOperationSuite.scala | 3 +- .../ui/HiveThriftServer2ListenerSuite.scala| 1 + 12 files changed, 140 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a908b67 -> d9ee33c)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks add d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on thriftserver No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 11 + .../service/cli/operation/OperationManager.java| 4 +- .../hive/service/cli/operation/SQLOperation.java | 7 +-- .../sql/hive/thriftserver/HiveThriftServer2.scala | 2 +- .../SparkExecuteStatementOperation.scala | 45 +- .../server/SparkSQLOperationManager.scala | 5 +- .../ui/HiveThriftServer2AppStatusStore.scala | 1 + .../ui/HiveThriftServer2EventManager.scala | 7 +++ .../ui/HiveThriftServer2Listener.scala | 10 .../thriftserver/HiveThriftServer2Suites.scala | 55 +- .../SparkExecuteStatementOperationSuite.scala | 3 +- .../ui/HiveThriftServer2ListenerSuite.scala| 1 + 12 files changed, 140 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a908b67 -> d9ee33c)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks add d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on thriftserver No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 11 + .../service/cli/operation/OperationManager.java| 4 +- .../hive/service/cli/operation/SQLOperation.java | 7 +-- .../sql/hive/thriftserver/HiveThriftServer2.scala | 2 +- .../SparkExecuteStatementOperation.scala | 45 +- .../server/SparkSQLOperationManager.scala | 5 +- .../ui/HiveThriftServer2AppStatusStore.scala | 1 + .../ui/HiveThriftServer2EventManager.scala | 7 +++ .../ui/HiveThriftServer2Listener.scala | 10 .../thriftserver/HiveThriftServer2Suites.scala | 55 +- .../SparkExecuteStatementOperationSuite.scala | 3 +- .../ui/HiveThriftServer2ListenerSuite.scala| 1 + 12 files changed, 140 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a908b67 -> d9ee33c)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks add d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on thriftserver No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 11 + .../service/cli/operation/OperationManager.java| 4 +- .../hive/service/cli/operation/SQLOperation.java | 7 +-- .../sql/hive/thriftserver/HiveThriftServer2.scala | 2 +- .../SparkExecuteStatementOperation.scala | 45 +- .../server/SparkSQLOperationManager.scala | 5 +- .../ui/HiveThriftServer2AppStatusStore.scala | 1 + .../ui/HiveThriftServer2EventManager.scala | 7 +++ .../ui/HiveThriftServer2Listener.scala | 10 .../thriftserver/HiveThriftServer2Suites.scala | 55 +- .../SparkExecuteStatementOperationSuite.scala | 3 +- .../ui/HiveThriftServer2ListenerSuite.scala| 1 + 12 files changed, 140 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (a908b67 -> d9ee33c)
This is an automated email from the ASF dual-hosted git repository. yamamuro pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks add d9ee33c [SPARK-26533][SQL] Support query auto timeout cancel on thriftserver No new revisions were added by this update. Summary of changes: .../org/apache/spark/sql/internal/SQLConf.scala| 11 + .../service/cli/operation/OperationManager.java| 4 +- .../hive/service/cli/operation/SQLOperation.java | 7 +-- .../sql/hive/thriftserver/HiveThriftServer2.scala | 2 +- .../SparkExecuteStatementOperation.scala | 45 +- .../server/SparkSQLOperationManager.scala | 5 +- .../ui/HiveThriftServer2AppStatusStore.scala | 1 + .../ui/HiveThriftServer2EventManager.scala | 7 +++ .../ui/HiveThriftServer2Listener.scala | 10 .../thriftserver/HiveThriftServer2Suites.scala | 55 +- .../SparkExecuteStatementOperationSuite.scala | 3 +- .../ui/HiveThriftServer2ListenerSuite.scala| 1 + 12 files changed, 140 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (eb33bcb4 -> a908b67)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from eb33bcb4 [SPARK-30796][SQL] Add parameter position for REGEXP_REPLACE add a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks No new revisions were added by this update. Summary of changes: .../spark/shuffle/IndexShuffleBlockResolver.scala | 2 +- .../spark/storage/BlockManagerDecommissioner.scala | 64 -- 2 files changed, 35 insertions(+), 31 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (eb33bcb4 -> a908b67)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from eb33bcb4 [SPARK-30796][SQL] Add parameter position for REGEXP_REPLACE add a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks No new revisions were added by this update. Summary of changes: .../spark/shuffle/IndexShuffleBlockResolver.scala | 2 +- .../spark/storage/BlockManagerDecommissioner.scala | 64 -- 2 files changed, 35 insertions(+), 31 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (eb33bcb4 -> a908b67)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from eb33bcb4 [SPARK-30796][SQL] Add parameter position for REGEXP_REPLACE add a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks No new revisions were added by this update. Summary of changes: .../spark/shuffle/IndexShuffleBlockResolver.scala | 2 +- .../spark/storage/BlockManagerDecommissioner.scala | 64 -- 2 files changed, 35 insertions(+), 31 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (eb33bcb4 -> a908b67)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from eb33bcb4 [SPARK-30796][SQL] Add parameter position for REGEXP_REPLACE add a908b67 [SPARK-33218][CORE] Update misleading log messages for removed shuffle blocks No new revisions were added by this update. Summary of changes: .../spark/shuffle/IndexShuffleBlockResolver.scala | 2 +- .../spark/storage/BlockManagerDecommissioner.scala | 64 -- 2 files changed, 35 insertions(+), 31 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (cb3fa6c -> eb33bcb4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from cb3fa6c [SPARK-33212][BUILD] Move to shaded clients for Hadoop 3.x profile add eb33bcb4 [SPARK-30796][SQL] Add parameter position for REGEXP_REPLACE No new revisions were added by this update. Summary of changes: .../catalyst/expressions/regexpExpressions.scala | 101 - .../expressions/RegexpExpressionsSuite.scala | 14 +++ .../sql-functions/sql-expression-schema.md | 2 +- .../sql-tests/inputs/regexp-functions.sql | 12 +++ .../sql-tests/results/regexp-functions.sql.out | 84 - 5 files changed, 188 insertions(+), 25 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (cb3fa6c -> eb33bcb4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from cb3fa6c [SPARK-33212][BUILD] Move to shaded clients for Hadoop 3.x profile add eb33bcb4 [SPARK-30796][SQL] Add parameter position for REGEXP_REPLACE No new revisions were added by this update. Summary of changes: .../catalyst/expressions/regexpExpressions.scala | 101 - .../expressions/RegexpExpressionsSuite.scala | 14 +++ .../sql-functions/sql-expression-schema.md | 2 +- .../sql-tests/inputs/regexp-functions.sql | 12 +++ .../sql-tests/results/regexp-functions.sql.out | 84 - 5 files changed, 188 insertions(+), 25 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (cb3fa6c -> eb33bcb4)
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from cb3fa6c [SPARK-33212][BUILD] Move to shaded clients for Hadoop 3.x profile add eb33bcb4 [SPARK-30796][SQL] Add parameter position for REGEXP_REPLACE No new revisions were added by this update. Summary of changes: .../catalyst/expressions/regexpExpressions.scala | 101 - .../expressions/RegexpExpressionsSuite.scala | 14 +++ .../sql-functions/sql-expression-schema.md | 2 +- .../sql-tests/inputs/regexp-functions.sql | 12 +++ .../sql-tests/results/regexp-functions.sql.out | 84 - 5 files changed, 188 insertions(+), 25 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated (970a6028 -> da18b51)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git. from 970a6028 Revert "[SPARK-33190][INFRA][TESTS] Set upper bound of PyArrow version in GitHub Actions" add da18b51 [SPARK-32247][INFRA] Install and test scipy with PyPy in GitHub Actions No new revisions were added by this update. Summary of changes: .github/workflows/build_and_test.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated (970a6028 -> da18b51)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git. from 970a6028 Revert "[SPARK-33190][INFRA][TESTS] Set upper bound of PyArrow version in GitHub Actions" add da18b51 [SPARK-32247][INFRA] Install and test scipy with PyPy in GitHub Actions No new revisions were added by this update. Summary of changes: .github/workflows/build_and_test.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
