Messages by Date
-
2026/11/13
[jira] [Updated] (SPARK-61672) SPIP (proposed): Add native AI Functions to Spark
Holden Karau (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61624) Make setParams and the params argument of the native ML classes as PySpark does
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61638) Pickle the Python UDFs, pandas UDFs and UDTFs as PySpark Connect's, so that a foreachBatch function can call one defined outside it, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61635) Raise PySpark's errors for the fields, names and data types of StructType, StructField and StructType.add, and keep the nullable flags as given, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61672) Add native AI Functions to Spark
Holden Karau (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61638) Pickle the Python UDFs, pandas UDFs and UDTFs as PySpark Connect's, so that a foreachBatch function can call one defined outside it, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61635) Raise PySpark's errors for the fields, names and data types of StructType, StructField and StructType.add, and keep the nullable flags as given, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61652) Fix missing runtime bloom filters when a join cannot broadcast its small side
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61652) Fix missing runtime bloom filters when a join cannot broadcast its small side
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61666) Avoid serializing the child plan into every task from SortExec and sort-merge join codegen
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61668) Avoid serializing the child plans into every task from ShuffledHashJoinExec codegen
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61503) Add tests for invalid input shape to the predict_batch_udf in pyspark/ml/functions.py
Tian Gao (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61503) Add tests for invalid input shape to the predict_batch_udf in pyspark/ml/functions.py
Tian Gao (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61671) Add SparkSession.listFiles and SparkSession.listArchives for Spark Connect
Kaicheng Zhou (Jira) via issues
-
2026/11/13
[jira] [Comment Edited] (SPARK-59132) FileType: A new data type to simplify handling of unstructured data in Spark
Dejan Krakovic (Jira) via issues
-
2026/11/13
[jira] [Commented] (SPARK-61301) Improve the FILE type error messages and their builders
Dejan Krakovic (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61657) Fix `ArrowColumnarPythonUDFSuite` limit checks when `LimitPushDown` moves the limit below the Python UDF
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61670) Document the trust assumptions of the operator in SECURITY.md
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61670) Document the trust assumptions of the operator in SECURITY.md
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61649) Preserve structured JVM linkage error details
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61657) Fix `ArrowColumnarPythonUDFSuite` limit checks when `LimitPushDown` moves the limit below the Python UDF
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61669) Fix CALL failing with INTERNAL_ERROR when an argument has a Python UDF or a nested IN subquery
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61669) Fix CALL failing with INTERNAL_ERROR when an argument has a Python UDF or a nested IN subquery
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61667) Use Expression.isTotal to fold the size of CreateArray in ConstantFolding
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61657) Fix `ArrowColumnarPythonUDFSuite` limit checks when `LimitPushDown` moves the limit below the Python UDF
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61656) Fix CALL failing with INTERNAL_ERROR when an argument has an IN subquery
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-60651) Allow decimal remainders with integral quotients wider than the math context
Szehon Ho (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61665) Allow creating backport PRs from the merge script
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61656) Fix CALL failing with INTERNAL_ERROR when an argument has an IN subquery
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61668) Avoid serializing the child plans into every task from ShuffledHashJoinExec codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61666) Avoid serializing the child plan into every task from SortExec and sort-merge join codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Commented] (SPARK-55216) Explore UDAFs
Jing Zhan (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61666) Avoid serializing the child plan into every task from SortExec and join operators codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61666) Avoid serializing the child plan into every task from SortExec and sort-merge join codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61668) Avoid serializing the child plans into every task from ShuffledHashJoinExec codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61668) Avoid serializing the child plans into every task from ShuffledHashJoinExec codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61667) Use Expression.isTotal to fold the size of CreateArray in ConstantFolding
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61667) Use Expression.isTotal to fold the size of CreateArray in ConstantFolding
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61666) Avoid serializing the child plan into every task from SortExec and join operators codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61665) Allow creating backport PRs from the merge script
Holden Karau (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61583) Batch setup inserts in enhanced runtime partition filter tests
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61583) Batch setup inserts in enhanced runtime partition filter tests
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61583) Batch setup inserts in enhanced runtime partition filter tests
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61584) Run PushablePredicateSuite without a Spark session
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61584) Run PushablePredicateSuite without a Spark session
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61584) Run PushablePredicateSuite without a Spark session
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61313) Allow forcing post-merge CI with the do-not-skip-ci label in merge_spark_pr.py
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-60651) Allow decimal remainders with integral quotients wider than the math context
Szehon Ho (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61313) Allow forcing post-merge CI with the do-not-skip-ci label in merge_spark_pr.py
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61664) Fix wrong results of sketch aggregates and schema_of_variant_agg over empty input in correlated scalar subqueries
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61653) Avoid allocating a column vector array per row in columnar Arrow Python UDF output
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61653) Avoid allocating a column vector array per row in columnar Arrow Python UDF output
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61663) Let each expression declare whether it is total instead of the allowlist in ExprUtils.isTotalOperator
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61651) Separate unresolved Rust client expression payloads from structural inputs
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61663) Let each expression declare whether it is total instead of the allowlist in ExprUtils.isTotalOperator
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61663) Let each expression declare whether it is total instead of the allowlist in ExprUtils.isTotalOperator
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61662) Do not push filter conditions written after a non-deterministic condition ahead of it
Kousuke Saruta (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61661) Avoid serializing the child plan into every task from HashAggregateExec codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61661) Avoid serializing the child plan into every task from HashAggregateExec codegen
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61661) Avoid serializing the child plan into every task from HashAggregateExec codegen
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Commented] (SPARK-60129) RECOVER PARTITIONS changes an existing partition's location after SET LOCATION
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-60129) RECOVER PARTITIONS changes an existing partition's location after SET LOCATION
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Commented] (SPARK-61640) Fix wrong results when a partially clustered storage-partitioned join replicates the MERGE target
Rakesh Kumar (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61660) Fix a Python UDF over aggregates in a single-aggregate correlated scalar subquery with the COUNT bug
Kousuke Saruta (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-60492) Keep Hive incompatible partition columns in `HiveExternalCatalog.alterTable/renameTable`
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-60348) Fix state data source change feed dropping list state values after the first
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-60515) Keep the existing table when Hive rejects the new table of `saveAsTable` overwrite
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61658) Stop treating duplicate Build check suites as overflow
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61659) Fix in-memory test catalogs losing column details when creating or altering tables
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61659) Fix in-memory test catalogs losing column details when creating or altering tables
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61658) Stop treating duplicate Build check suites as overflow
Nicholas Chammas (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61251) Sort-merge join and window frames leave spill file readers open when they stop reading a spilled buffer early
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-57670) Spark Pipelines table properties are not dropped on subsequent run
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-59268) [SDP] Preserve desired column metadata during additive schema evolution
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61657) Fix `ArrowColumnarPythonUDFSuite` limit checks when `LimitPushDown` moves the limit below the Python UDF
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61657) Fix `ArrowColumnarPythonUDFSuite` limit checks when `LimitPushDown` moves the limit below the Python UDF
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61656) Fix CALL failing with INTERNAL_ERROR when an argument has an IN subquery
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61652) Fix missing runtime bloom filters when a join cannot broadcast its small side
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61656) Fix CALL failing with INTERNAL_ERROR when an argument has an IN subquery
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61655) Let a Rust UDF read the session's Python worker environment
Kaicheng Zhou (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61654) Place a cogroup key stored like the layout's column by the column's own transform in the one-side shuffle
Anil K (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61653) Avoid allocating a column vector array per row in columnar Arrow Python UDF output
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61653) Avoid allocating a column vector array per row in columnar Arrow Python UDF output
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61652) Fix missing runtime bloom filters when a join cannot broadcast its small side
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61651) Separate unresolved Rust client expression payloads from structural inputs
DB Tsai (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61651) Separate unresolved Rust client expression payloads from structural inputs
DB Tsai (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61647) Drop a closed fast hash map's buckets and key offsets
Rakesh Kumar (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61502) Wait for GitHub to compute whether a PR is mergeable in merge_spark_pr.py
Gengliang Wang (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-60928) Upgrade Guava to 33.8.0-jre
Yang Jie (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-60928) Upgrade Guava to 33.8.0-jre
Yang Jie (Jira) via issues
-
2026/11/13
[jira] [Commented] (SPARK-60813) Storage-partitioned join evaluates a partition transform on a datetime or interval value of another unit
Anil K (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61650) Storage-partitioned join passes a datetime or interval value that its cast to the declared type would change by time zone or precision
Anil K (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61650) Storage-partitioned join passes a datetime or interval value that its cast to the declared type would change by time zone or precision
Anil K (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61563) Write the direction strategy of SortMergeAsOfJoinExec once
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61563) Write the direction strategy of SortMergeAsOfJoinExec once
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61649) Preserve structured JVM linkage error details
Marko Sisovic (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61625) Avoid allocating a column vector array per row in `RowToColumnarExec`
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61625) Avoid allocating a column vector array per row in `RowToColumnarExec`
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61614) Fix evaluating a Python UDF twice when a filter over a scan uses its result
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61614) Fix evaluating a Python UDF twice when a filter over a scan uses its result
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-59693) Avoid double evaluation of non-trivial projected elements for Python UDFs
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-59693) Avoid double evaluation of non-trivial projected elements for Python UDFs
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61647) Drop a fast hash map's buckets when it closes
Rakesh Kumar (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61647) Drop a fast hash map's buckets when it closes
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61646) Support write distribution and ordering in the Python Data Source API
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61648) Arrow Python UDF fails on naive TimestampType results on 9999-12-31 and 0001-01-01
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61648) Arrow Python UDF fails on naive TimestampType results on 9999-12-31 and 0001-01-01
Tim Meehan (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-60564) LEFT JOIN LATERAL with a generator or Python UDTF and no ON condition drops left rows (behaves as an inner join)
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61647) Drop a fast hash map's buckets when it closes
Rakesh Kumar (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61646) Support write distribution and ordering in the Python Data Source API
Ganesha S (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61643) Run the JavaScript linter only when its inputs change
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61643) Run the JavaScript linter only when its inputs change
Nicholas Chammas (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-60124) `LISTAGG(DISTINCT ...)` with `FILTER` fails during planning
Wenchen Fan (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-60124) `LISTAGG(DISTINCT ...)` with `FILTER` fails during planning
Wenchen Fan (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-60124) `LISTAGG(DISTINCT ...)` with `FILTER` fails during planning
Wenchen Fan (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-59244) Wrap foldable path-expression evaluation in JSON path function builders to emit a clean analysis error
Wenchen Fan (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61644) Create Java linter test module
Nicholas Chammas (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61643) Create JavaScript linter test module
Nicholas Chammas (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61645) Create R linter test module
Nicholas Chammas (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-59244) Wrap foldable path-expression evaluation in JSON path function builders to emit a clean analysis error
Wenchen Fan (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61615) Correlated subquery in the FILTER clause of an aggregate function in a correlated scalar subquery fails with INTERNAL_ERROR
Kousuke Saruta (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61641) Raise PySpark's errors at the action for a string schema of DataFrame.to and a non-bool allowMissingColumns of unionByName, in the Python client
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61642) Add all native Spark ML evaluators
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61642) Add all native Spark ML evaluators
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61637) Let a foreachBatch function capture the readers, the writers and GroupedData, and share the options of a copied reader, as PySpark does, in the Python client
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61642) Add all native Spark ML evaluators
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61623) Segment-tree window SUM fails with ARITHMETIC_OVERFLOW although no frame overflows
sepuri sai krishna (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61608) Fix arrays_zip field naming for map lookups
Yan Yan (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61627) Let copy, deepcopy and pickle copy GroupedData, the readers, the writers, the catalog and the other DataFrame helpers as PySpark does in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61625) Avoid allocating a column vector array per row in `RowToColumnarExec`
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61612) Support native FMClassifier, MultilayerPerceptronClassifier and OneVsRest
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61614) Fix evaluating a Python UDF twice when a filter over a scan uses its result
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61632) Replace the view in register_temp_table and raise PySpark's error for a classic SQLContext, as PySpark does, in Spark Connect Rust
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61630) Make the native LogisticRegression and evaluators follow pyspark.ml, not the deprecated pyspark.ml.connect, and stop converting array features implicitly, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61492) Upgrade PyO3 to 0.29 in the Python client, so that its PyO3 classes do not leak the instance dict that CPython 3.11 and 3.12 create
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61628) Match PySpark's SRID and type-string errors, and the bool arguments of checkpoint and unionByName, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61612) Support native FMClassifier, MultilayerPerceptronClassifier and OneVsRest
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61642) Add all native Spark ML evaluators
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61642) Add all native Spark ML evaluators
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-60692) Support lookup-side projections and filters, semi/anti joins and range predicates in external lookup join
Jungtaek Lim (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61616) Support native AFTSurvivalRegression, IsotonicRegression and FMRegressor
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61609) Raise PySpark's errors for a keyword option key that is not a str, the jdbc predicates and an out-of-range show count, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61641) Raise PySpark's errors at the action for a string schema of DataFrame.to and a non-bool allowMissingColumns of unionByName, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61641) Raise PySpark's errors at the action for a string schema of DataFrame.to and a non-bool allowMissingColumns of unionByName, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61621) Filtering a cached table with startswith on a collated string column drops matching rows
sepuri sai krishna (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61617) Match PySpark's registerTempTable, the bool and properties arguments of the catalog and writers, and createExternalTable's signature, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61640) Fix wrong results when a partially clustered storage-partitioned join replicates the MERGE target
Peter Toth (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61640) Fix wrong results when a partially clustered storage-partitioned join replicates the MERGE target
Peter Toth (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61620) Add a shuffled-query case to `ArrowCacheBenchmark`
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61620) Add a shuffled-query case to `ArrowCacheBenchmark`
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61639) Add native ALS recommendation to Spark Connect Rust
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61639) Add native ALS recommendation to Spark Connect Rust
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61635) Raise PySpark's errors for the fields, names and data types of StructType, StructField and StructType.add, and keep the nullable flags as given, in the Python client
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61639) Add native ALS recommendation to Spark Connect Rust
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61639) Add native ALS recommendation to Spark Connect Rust
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61632) Replace the view in register_temp_table and raise PySpark's error for a classic SQLContext, as PySpark does, in Spark Connect Rust
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61638) Pickle the Python UDFs, pandas UDFs and UDTFs as PySpark Connect's, so that a foreachBatch function can call one defined outside it, in the Python client
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61639) Add native ALS recommendation to Spark Connect Rust
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61636) Connect Rust: add native FPGrowth and PrefixSpan
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-60379) Push down shuffle exchange through union
XiDuo You (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61630) Make the native LogisticRegression and evaluators follow pyspark.ml, not the deprecated pyspark.ml.connect, and stop converting array features implicitly, in the Python client
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61613) Take bytes, as PySpark does, in the table names and paths of the catalog, the readers and the writers, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61619) Name the int columns of a pandas frame as createDataFrame does, evaluate only the read columns, and resolve the case of featuresCol, in the native ML of the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61616) Support native AFTSurvivalRegression, IsotonicRegression and FMRegressor
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61553) Fix CALL silently doing nothing when ProcedureCatalog.loadProcedure returns null
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61628) Match PySpark's SRID and type-string errors, and the bool arguments of checkpoint and unionByName, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61594) Build the operator jar once on `$BUILDPLATFORM` in multi-platform Docker builds
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61617) Match PySpark's registerTempTable, the bool and properties arguments of the catalog and writers, and createExternalTable's signature, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61598) Document every Spark SQL type of DataType in the types guide
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61419) from_avro/schema_of_avro can corrupt a shared options map when the plan is canonicalized concurrently
Jungtaek Lim (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61633) Support the remaining native clustering algorithms: LDA and PowerIterationClustering
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61626) Make clippy a blocking CI gate of spark-connect-rust
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61575) Fix `to_time` failing on a foldable format that no row evaluates
Dongjoon Hyun (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61636) Connect Rust: add native FPGrowth and PrefixSpan
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61619) Name the int columns of a pandas frame as createDataFrame does, evaluate only the read columns, and resolve the case of featuresCol, in the native ML of the Python client
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61625) Avoid allocating a column vector array per row in `RowToColumnarExec`
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61637) Let a foreachBatch function capture the readers, the writers and GroupedData, and share the options of a copied reader, as PySpark does, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61534) Estimate the size of an existence join from its left side
Peter Toth (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61636) Connect Rust: add native FPGrowth and PrefixSpan
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61589) Define NULL semantics for KLL quantile and rank getter arrays
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61619) Name the int columns of a pandas frame as createDataFrame does, evaluate only the read columns, and resolve the case of featuresCol, in the native ML of the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61534) Estimate the size of an existence join from its left side
Peter Toth (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61636) Connect Rust: add native FPGrowth and PrefixSpan
Ruifeng Zheng (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61613) Take bytes, as PySpark does, in the table names and paths of the catalog, the readers and the writers, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-60691) Add the EXTERNAL_LOOKUP join hint for external lookup join
Jungtaek Lim (Jira) via issues
-
2026/11/13
[jira] [Resolved] (SPARK-61576) Let copy.copy, copy.deepcopy and pickle copy an Observation in the Python client, as PySpark does
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61627) Let copy, deepcopy and pickle copy GroupedData, the readers, the writers, the catalog and the other DataFrame helpers as PySpark does in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61617) Match PySpark's registerTempTable, the bool and properties arguments of the catalog and writers, and createExternalTable's signature, in the Python client
ASF GitHub Bot (Jira) via issues
-
2026/11/13
[jira] [Assigned] (SPARK-61626) Make clippy a blocking CI gate of spark-connect-rust
L. C. Hsieh (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61549) Estimate the size of a single join from its left side
Peter Toth (Jira) via issues
-
2026/11/13
[jira] [Updated] (SPARK-61638) Pickle the Python UDFs, pandas UDFs and UDTFs as PySpark Connect's, so that a foreachBatch function can call one defined outside it, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61638) Pickle the Python UDFs, pandas UDFs and UDTFs as PySpark Connect's, so that a foreachBatch function can call one defined outside it, in the Python client
Hyukjin Kwon (Jira) via issues
-
2026/11/13
[jira] [Created] (SPARK-61637) Let a foreachBatch function capture the readers, the writers and GroupedData, and share the options of a copied reader, as PySpark does, in the Python client
Hyukjin Kwon (Jira) via issues