commits  

Messages by Thread

• (spark-kubernetes-operator) branch main updated: [SPARK-55371] Increase `Gradle` retry setting to stablize CIs dongjoon
• (spark-kubernetes-operator) branch main updated: [SPARK-55374] Remove `vendor` requirement from Java toolchain dongjoon
• (spark-kubernetes-operator) branch main updated: MINOR: Add release version badge and link to `README.md` dongjoon
• (spark) branch master updated: [SPARK-55373][CONNECT] Improve noHandlerFoundForExtension error message hvanhovell
• (spark) branch master updated: [SPARK-55341][SQL] Add storage level flag for cached local relations hvanhovell
• (spark) branch master updated: [SPARK-55356][SQL] Support alias for PIVOT clause wenchen
• svn commit: r82303 - in dev/spark/v4.2.0-preview2-rc1-docs: . _site _site/api _site/api/R _site/api/R/articles _site/api/R/articles/sparkr-vignettes_files _site/api/R/articles/sparkr-vignettes_files/accessible-code-block-0.0.1 _site/api/R/deps _site/ap... gurwls223
• (spark-website) branch asf-site updated: Change Spark 4.2 release timeline (#668) dongjoon
• svn commit: r82302 - dev/spark/v4.2.0-preview2-rc1-bin gurwls223
• (spark) branch master updated: [SPARK-55365][PYTHON] Generalize the utils for arrow array conversion ruifengz
• (spark) branch master updated: [SPARK-55228][SPARK-55230][SQL][CONNECT] Implement Dataset.zipWithIndex in Scala API ruifengz
• (spark) branch master updated (7d3b32238bd6 -> c28d7ad006e0) ruifengz
• (spark) tag v4.2.0-preview2-rc1 created (now a2edb559299d) gurwls223
  • (spark) 02/02: Preparing Spark release v4.2.0-preview2-rc1 gurwls223
  • (spark) 01/02: Removing test jars and class files gurwls223
• (spark) branch master updated (b58fdcd4baf4 -> 7d3b32238bd6) ruifengz
• (spark) branch master updated (52b327fd3c19 -> b58fdcd4baf4) ruifengz
• (spark) branch master updated: [SPARK-55360][BUILD] Upgrade sbt to `1.12.2` dongjoon
• (spark) branch master updated: [SPARK-55359][CORE] Promote `TaskResourceRequest` to `Stable` dongjoon
• (spark) branch master updated: Revert "[SPARK-55313][PYTHON][FOLLOW-UP] Only add condabin to PATH for pip tests" ruifengz
• (spark) branch master updated (612ade46381b -> 214bf958757c) kabhwan
• (spark) branch master updated (481f9866f5f5 -> 612ade46381b) kabhwan
• (spark) branch master updated: [SPARK-55303][PYTHON][TESTS] Extract GoldenFileTestMixin for type coercion golden file tests ruifengz
• (spark) branch master updated: [SPARK-55335][PYTHON][TESTS] Use eventually instead of hard-coded wait for datasource test ruifengz
• (spark) branch master updated: [SPARK-55313][PYTHON][FOLLOW-UP] Only add condabin to PATH for pip tests ruifengz
• (spark) branch master updated: [SPARK-55363][PS][TESTS] Make ops tests with "decimal_nan" columns ignore NaN vs. None ruifengz
• (spark) branch master updated: [SPARK-55350][PYTHON][CONNECT] Fix row count loss when creating DataFrame from pandas with 0 columns ueshin
• (spark) branch master updated: [SPARK-46165][PS] Add support for DataFrame.all axis=None gurwls223
• (spark) branch master updated (0437a933a421 -> 508130f28099) gurwls223
• (spark) branch master updated (620b2f65f0da -> 0437a933a421) gurwls223
• [PR] change Spark 4.2 release timeline [spark-website] via GitHub
  • Re: [PR] Change Spark 4.2 release timeline [spark-website] via GitHub
  • Re: [PR] Change Spark 4.2 release timeline [spark-website] via GitHub
  • Re: [PR] Change Spark 4.2 release timeline [spark-website] via GitHub
  • Re: [PR] Change Spark 4.2 release timeline [spark-website] via GitHub
• (spark) branch master updated: [SPARK-55362][PYTHON][CONNECT] Don't wait for threadpool shutdown gurwls223
• (spark) branch master updated (fac11c494706 -> f9b712a35f80) gurwls223
• (spark) branch master updated: [SPARK-55291][CONNECT] Pre-process metadata headers at client interceptor construction time wenchen
• (spark) branch feature/crossjoin-array-contains-benchmark deleted (was 41c2b1c32fcf) yao
• (spark) branch feature/crossjoin-array-contains-benchmark updated (bd84f87b7e3a -> 41c2b1c32fcf) yao
• (spark) branch feature/crossjoin-array-contains-benchmark updated (8172b0502948 -> 9e3d4516673e) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch feature/crossjoin-array-contains-benchmark updated (e81100dec095 -> 8172b0502948) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch master updated: [SPARK-55354][CORE][DOCS] Fix `ExecutorAllocationClient` comment to include `Kubernetes` dongjoon
• (spark) branch feature/crossjoin-array-contains-benchmark updated (ee8d603257e2 -> e81100dec095) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch master updated: Revert "[SPARK-55351][PYTHON][SQL] PythonArrowInput encapsulate resource allocation inside `newWriter`" ruifengz
• (spark) branch feature/crossjoin-array-contains-benchmark updated (af9bfec8e91f -> ee8d603257e2) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch feature/crossjoin-array-contains-benchmark updated (b9c7217539ee -> af9bfec8e91f) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch master updated: [MINOR][PYTHON][TESTS] Fix `test_time_zone_against_map_in_arrow` for tzdata on ubuntu 24 ruifengz
• (spark) branch feature/crossjoin-array-contains-benchmark updated (25d66488a5c7 -> b9c7217539ee) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch feature/crossjoin-array-contains-benchmark updated (668dba331db3 -> 25d66488a5c7) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch feature/crossjoin-array-contains-benchmark updated (7da480e5ca16 -> 668dba331db3) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch feature/crossjoin-array-contains-benchmark updated (0e22ceb62099 -> 7da480e5ca16) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Add CrossJoinArrayContainsToInnerJoin optimizer rule yao
• (spark) branch feature/crossjoin-array-contains-benchmark created (now 0e22ceb62099) yao
  • (spark) 01/01: [SPARK-XXXX][SQL] Address review comments yao
• (spark) branch master updated: [SPARK-55346][INFRA][PYTHON] Upgrade pystack version to 1.6.0 and install it on all major images gurwls223
• (spark-kubernetes-operator) branch main updated: [SPARK-55352] Use K8s Garbage Collection to delete executor pods dongjoon
• (spark) branch master updated: [SPARK-55351][PYTHON][SQL] PythonArrowInput encapsulate resource allocation inside `newWriter` ruifengz
• (spark) branch master updated: [MINOR][PYTHON][TESTS] Skip the doctest of toJSON ruifengz
• (spark) branch master updated: [SPARK-54599][PYTHON] Refactor PythonException so it can take errorClass with sqlstate ruifengz
• (spark) branch master updated: [SPARK-55309][BUILD][FOLLOW-UP] Bump container protobuf version yangjie01
• (spark-kubernetes-operator) branch main updated: [SPARK-55344] Support `spark.kubernetes.operator.metrics.path` dongjoon
• (spark) branch master updated (b91d4079036a -> 5802a78b7a6c) gurwls223
• (spark) branch master updated (455ea6c0c717 -> b91d4079036a) gurwls223
• (spark) branch master updated (5648458deddb -> 455ea6c0c717) gurwls223
• (spark) branch master updated (a263a5e4ce87 -> 5648458deddb) dongjoon
• (spark-kubernetes-operator) branch main updated: [SPARK-55343] Simplify `HealthProbe` class dongjoon
• (spark) branch master updated: [SPARK-55280][CONNECT] Add GetStatus proto to support execution status monitoring hvanhovell
• (spark) branch master updated: [SPARK-55106][SS] Add Repartition Integration test for TransformWithState Operators ashrigondekar
• (spark) branch branch-4.1 updated: [SPARK-55258][DOCS] Document CLI parameters in declarative pipelines programming guide sandy
• (spark) branch master updated (a7bc39504035 -> 9788c52426df) sandy
  • (spark) branch master updated (a7bc39504035 -> 9788c52426df) sandy
• (spark) branch master updated (7b673d6fd61b -> a7bc39504035) wenchen
• (spark) branch memory-stream-compat deleted (was 3ca4eb6f5577) wenchen
• (spark) branch memory-stream-compat created (now 3ca4eb6f5577) wenchen
  • (spark) 01/01: [SPARK-53656][SS][FOLLOWUP] Remove confusing MemoryStream factory method wenchen
• (spark-connect-swift) branch main updated: [SPARK-55316] Upgrade `gRPC Swift NIO Transport` to 2.4.1 dongjoon
• (spark) branch master updated: [SPARK-55308][BUILD] Upgrade icu4j to 78.2 dongjoon
• (spark) branch branch-4.1 updated (20974ced5a78 -> e3de8690059b) dongjoon
• (spark-kubernetes-operator) branch main updated: [SPARK-55329] Upgrade `Apache DataFusion Comet` to 0.13.0 dongjoon
• (spark) branch master updated: [SPARK-55320][SQL][CONNECT] Use raise_error instead of divide by zero in Observation tests wenchen
• (spark) branch master updated: [SPARK-54969][PYTHON] Implement new arrow->pandas conversion ruifengz
• (spark) branch master updated: [SPARK-55328][SQL][PYTHON] Reuse PythonArrowInput.codec in GroupedPythonArrowInput gurwls223
• (spark) branch master updated: [SPARK-55327][K8S] Reduce Spark docker image sizes dongjoon
• (spark) branch master updated (11d3fec06d94 -> 14bc85248f24) ruifengz
• (spark) branch master updated: [SPARK-55315][PYTHON][TESTS] Allow eventually to take custom exceptions ruifengz
• (spark) branch master updated: [SPARK-55323][PYTHON] Move UDF metadata to EvalConf to simplify worker protocol ruifengz
• (spark) branch master updated: [SPARK-55319][PYTHON][INFRA] Add libjpeg-dev to pypy dockerfile ruifengz
• (spark) branch master updated: [SPARK-55293][PS][TESTS][FOLLOW-UP] Avoid more old offset aliases gurwls223
• (spark) branch dependabot/pip/dev/protobuf-6.33.5 deleted (was 01ea8fa0586d) github-bot
• (spark) branch master updated: [SPARK-55309][BUILD] Upgrade protobuf to 33.5 yangjie01
• (spark) branch master updated (60c8c3f30b4d -> 7b242f223467) wenchen
• (spark-kubernetes-operator) branch main updated: [SPARK-55292] Add discover latency metric to track operator processing delay dongjoon
• (spark) branch master updated: [SPARK-55176][PYTHON][FOLLOW-UP] Fix `_input_type` and `_arrow_cast` not defined in `ArrowStreamPandasSerializer` ruifengz
• (spark) branch master updated (22094fef57b2 -> 0c041c27c4a5) ruifengz
• (spark) branch master updated: [SPARK-55161][PYTHON] Support profilers on python data source gurwls223
• (spark) branch master updated: [SPARK-55302][SQL] Fix custom metrics in case of `KeyGroupedPartitioning` ptoth
• (spark) branch master updated: [SPARK-55285][SQL][PYTHON][FOLLOW-UP] Code clean up ruifengz
• (spark-kubernetes-operator) branch main updated: [SPARK-55310] Upgrade `Gradle` to 9.3.1 dongjoon
• [PR] Add Apache Iceberg to index.html, add alt text [spark-website] via GitHub
  • Re: [PR] Add Apache Iceberg to index.html, add alt text [spark-website] via GitHub
  • Re: [PR] Add Apache Iceberg to index.html, add alt text [spark-website] via GitHub
  • Re: [PR] Add Apache Iceberg to index.html, add alt text [spark-website] via GitHub
  • Re: [PR] Add Apache Iceberg to index.html, add alt text [spark-website] via GitHub
• (spark) branch master updated: [SPARK-55260][GEO][SQL] Implement Parquet write support for Geo types wenchen
• (spark) branch master updated: [SPARK-55289][SQL] Fix flaky test in-set-operations.sql by disabling broadcast join yao
• svn commit: r82244 - in dev/spark/v4.0.2-rc1-docs: . _site _site/api _site/api/R _site/api/R/articles _site/api/R/articles/sparkr-vignettes_files _site/api/R/articles/sparkr-vignettes_files/accessible-code-block-0.0.1 _site/api/R/deps _site/api/R/deps/... dongjoon
• svn commit: r82242 - dev/spark/v4.0.2-rc1-bin dongjoon
• (spark) branch branch-4.0 updated (c90c62751718 -> 9921bb62bf2b) dongjoon
  • (spark) 01/02: Revert "Removing test jars and class files" dongjoon
  • (spark) 02/02: Preparing development version 4.0.3-SNAPSHOT dongjoon
• (spark) tag v4.0.2-rc1 created (now 7cc3b9bcdaab) dongjoon
  • (spark) 02/02: Preparing Spark release v4.0.2-rc1 dongjoon
  • (spark) 01/02: Removing test jars and class files dongjoon
• (spark) branch master updated: [SPARK-55305][SQL][TESTS] Use `ParquetFooterReader.readFooter` uniformly in test code to read the footer yangjie01
• (spark) branch master updated: [SPARK-55307][K8S][INFRA] Update `setup-minikube` to v0.0.21 dongjoon
• (spark) branch master updated: [SPARK-55297][PYTHON][PS] Restore timedelta dtype based on the original dtype ruifengz
• (spark) branch master updated: [SPARK-55093][CORE] Handle TaskRunner construction failures in launchTask wenchen
• (spark-kubernetes-operator) branch main updated: [SPARK-55298] Fix "Argument list too long" error in `assertGeneratedCRDMatchesHelmChart` task dongjoon
• (spark) branch master updated: [SPARK-55293][PS][TESTS] Avoid using old offset aliases gurwls223
• (spark) branch master updated: [SPARK-55286][INFRA] Add test summary to GitHub Actions for better failure visibility gurwls223
• (spark) branch master updated: [SPARK-55283][PYTHON][PS][TESTS] Add a new argument ignore_null to assert_eq gurwls223
• (spark) branch master updated: [SPARK-55285][SQL][PYTHON] Fix the initialization of `PythonArrowInput` gurwls223
• (spark) branch master updated: [SPARK-55284][PYTHON][TEST] Move mypy-data related configs to the script gurwls223
• (spark) branch master updated: [SPARK-55176][PYTHON] Extract `arrow_to_pandas` converter into ArrowArrayToPandasConversion gurwls223
• (spark) branch master updated: [SPARK-55287][INFRA] Consolidate steps in `lint` gurwls223
• (spark) branch master updated (c5cb2435400e -> 1ce11023d758) ashrigondekar
• (spark) branch master updated (2d940919c0a1 -> c5cb2435400e) ashrigondekar
• (spark) branch master updated: [SPARK-55193][CORE][BUILD] Use `CompressionHandler` as a replacement for the deprecated `GzipHandler` in `JettyUtils` sarutak
• (spark) branch branch-4.1 updated: [SPARK-55290][NETWORK][TESTS] Fix testReloadMissingTrustStore cross-device link error with JDK 21 sarutak
• (spark) branch master updated: [SPARK-55290][NETWORK][TESTS] Fix testReloadMissingTrustStore cross-device link error with JDK 21 sarutak
• (spark) branch master updated (fbb4019d70d1 -> 65a6a55630f7) ashrigondekar
• (spark) branch master updated: [SPARK-55279][SQL] Add `sketch_funcs` group for DataSketches SQL functions yao
• (spark) branch branch-4.1 updated: [SPARK-55133][CONNECT] Fix race condition in IsolatedSessionState lifecycle management wenchen
• (spark) branch master updated (04b821c69e85 -> 76f6c784a0c4) sarutak
• [PR] website: add declarative pipeline support to Spark SQL highlights [spark-website] via GitHub
• (spark) branch master updated: [SPARK-55256][SQL] Support IGNORE NULLS / RESPECT NULLS for array_agg and collect_list yao
• (spark) branch master updated: [SPARK-55133][CONNECT] Fix race condition in IsolatedSessionState lifecycle management wenchen
• (spark) branch master updated: [SPARK-49110][SQL] Simplify SubqueryAlias.metadataOutput to always propagate metadata columns wenchen
• (spark) branch master updated (4a58b849484e -> 4e0537283336) yao
• (spark-kubernetes-operator) branch main updated: [SPARK-55288] Upgrade Netty to `4.2.9.Final` dongjoon
• (spark) branch master updated: Revert "[SPARK-55277][SQL] Add `protobuf_funcs` group for Protobuf SQL functions" yao
• (spark) branch master updated: [SPARK-55277][SQL] Add `protobuf_funcs` group for Protobuf SQL functions yao
• (spark-kubernetes-operator) branch main updated: [SPARK-55274] Set `-XX:+AlwaysPreTouch` by default dongjoon
• (spark) branch master updated (4729b9917107 -> 8e657692986b) wuyi
• (spark) branch master updated: [SPARK-55031][SQL] Add vector avg/sum aggregation function expressions wenchen
• (spark) branch master updated: [SPARK-55237][SQL] Suppress annoying messages when looking up nonexistent DBs yangjie01
• (spark) branch master updated (b3cbff35877c -> 4344f3fce786) yangjie01
• (spark) branch master updated (86f8b3fe6375 -> b3cbff35877c) yangjie01
• (spark) branch master updated (ea26cac89a07 -> 86f8b3fe6375) gurwls223
• (spark) branch master updated: [SPARK-55266][INFRA] Add pre-commit hooks for format/lint gurwls223
• (spark) branch master updated (9254e8912aec -> 66255917ca59) gurwls223
• (spark) branch master updated (23afba2e8e07 -> 9254e8912aec) gurwls223
• (spark) branch master updated (a1577253d88d -> 23afba2e8e07) gurwls223
• (spark) branch master updated: [SPARK-55011][DOCS] CURSORs docs dtenedor
• (spark) branch master updated: [SPARK-54887] Add previously removed legacy error class back in hvanhovell
• (spark) branch master updated (c32aee117b60 -> fe9e5c092b7c) wenchen
• (spark) branch master updated: [SPARK-55243][CONNECT] Allow setting binary headers via the -bin suffix in the Scala Connect client hvanhovell
• (spark) branch master updated: [SPARK-55259][GEO][SQL] Implement Parquet schema conversion for Geo types wenchen
• (spark) branch master updated: [SPARK-55114][PYTHON][TESTS][FOLLOW-UP] Update the result format to be more friendly to markdown ruifengz
• (spark-kubernetes-operator) branch main updated: [SPARK-55270] Disallow all HTTP methods except `GET` and `HEAD` dongjoon
• (spark) branch master updated: [SPARK-55064][SQL][CORE] Support query level indeterminate shuffle retry wenchen
• (spark-kubernetes-operator) branch main updated: [SPARK-55269] Fix a wrong comment of `testHandleSentinelResourceReconciliation` dongjoon
• (spark-kubernetes-operator) branch main updated: [SPARK-55268] Fix `ConfigOption.getValue` not to invoke `resolveValue` twice dongjoon
• (spark) branch master updated: [MINOR][PYTHON][TESTS] Consolidate DataStreamReader.name() tests into test_streaming.py gurwls223
• (spark) branch master updated: [SPARK-47996][PS] support cross merge in pandas API ruifengz
• (spark) branch master updated: [SPARK-55198][SQL] spark-sql should skip comment line with leading whitespaces wenchen
• (spark) branch master updated: [SPARK-54943][PYTHON][TESTS][FOLLOW-UP] Disable `test_pyarrow_array_cast` for now ruifengz
• (spark) branch master updated: Revert "[SPARK-54943][PYTHON][TESTS][FOLLOW-UP] Mistake Commit" ruifengz
• (spark) branch master updated: [SPARK-54943][PYTHON][TESTS][FOLLOW-UP] Disable `test_pyarrow_array_cast` ruifengz
• (spark) branch master updated (44f61d5117fc -> fe73cecbf4a9) wenchen
• (spark) branch master updated: [SPARK-41398][SQL][FOLLOWUP] Update runtime filtering javadoc to reflect relaxed partition constraints wenchen
• (spark) branch master updated: [SPARK-55150][CONNECT][SQL] Improve observation error handling wenchen
• (spark) branch master updated: [SPARK-54830][SPARK-48037][TESTS][FOLLOWUP] Disable shuffle checksum for the test case of to avoid memory issues wenchen
• (spark) branch master updated: [SPARK-55244][PYTHON][PS] Use np.nan as default value for pandas string types ruifengz
• (spark) branch master updated: [SPARK-55225][PYTHON][PS] Restore to the original dtype for Datetime ruifengz
• (spark) branch master updated (3de3d81d01f6 -> 0cba1c0a81a8) dongjoon
• (spark) branch master updated (7783f46f58ff -> 3de3d81d01f6) dtenedor
• (spark) branch master updated (9931781c4c8e -> 3092a7762658) gurwls223
• (spark) branch master updated (0dad3cb7bbde -> 9931781c4c8e) gurwls223
• (spark) branch master updated (8719c6c8dfd3 -> 0dad3cb7bbde) allisonwang
• (spark) branch master updated (778661a7e109 -> 8719c6c8dfd3) wenchen
• (spark) branch master updated: [SPARK-55255][BUILD] Upgrade `objenesis` to 3.5 dongjoon
• (spark) branch master updated (cecf758b4323 -> cf5898b4aa05) wenchen
• (spark) branch branch-4.1 updated: [SPARK-55119][SQL] Fix Continue Handler: prevent INTERNAL_ERROR and incorrect conditional statements interruption wenchen
• (spark) branch master updated: [SPARK-55119][SQL] Fix Continue Handler: prevent INTERNAL_ERROR and incorrect conditional statements interruption wenchen
• (spark) branch master updated (67bfa7b317d5 -> cbc0ecd41007) wenchen
• (spark) branch master updated: [SPARK-55254][BUILD] Upgrade `analyticsaccelerator-s3` to 1.3.1 dongjoon
• (spark) branch master updated (589fedc9b231 -> 135cc503aaeb) dongjoon
• (spark) branch master updated (71335da93af5 -> 589fedc9b231) ruifengz
• (spark) branch master updated: [SPARK-55248][SS] Clean up Jackson deprecated API usage in `streaming.checkpointing.Checksum` yangjie01
• (spark) branch master updated: [SPARK-55247][CONNECT] Clean up deprecated API usage related to `o.a.c.io.input.BoundedInputStream` yangjie01

• Earlier messages