[jira] [Updated] (SPARK-43968) Improve error messages for Python UDTFs with wrong number of outputs

[ https://issues.apache.org/jira/browse/SPARK-43968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-43968: - Fix Version/s: 3.5.0 > Improve error messages for Python UDTFs with wrong number of outputs > --

[jira] [Resolved] (SPARK-44548) Add support for pandas DataFrame assertDataFrameEqual

[ https://issues.apache.org/jira/browse/SPARK-44548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44548. -- Resolution: Fixed Issue resolved by pull request 42158 [https://github.com/apache/spark/pull/4

[jira] [Created] (SPARK-44580) RocksDB crashed when testing in GitHub Actions

Yang Jie created SPARK-44580: Summary: RocksDB crashed when testing in GitHub Actions Key: SPARK-44580 URL: https://issues.apache.org/jira/browse/SPARK-44580 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-42098) ResolveInlineTables should handle RuntimeReplaceable

[ https://issues.apache.org/jira/browse/SPARK-42098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42098: --- Assignee: Jia Fan > ResolveInlineTables should handle RuntimeReplaceable >

[jira] [Resolved] (SPARK-42098) ResolveInlineTables should handle RuntimeReplaceable

[ https://issues.apache.org/jira/browse/SPARK-42098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42098. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 42110 [https://gith

[jira] [Commented] (SPARK-43242) diagnoseCorruption should not throw Unexpected type of BlockId for ShuffleBlockBatchId

[ https://issues.apache.org/jira/browse/SPARK-43242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748400#comment-17748400 ] Snoot.io commented on SPARK-43242: -- User 'CavemanIV' has created a pull request for thi

[jira] [Commented] (SPARK-44579) Support Interrupt On Cancel in SQLExecution

[ https://issues.apache.org/jira/browse/SPARK-44579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748398#comment-17748398 ] Snoot.io commented on SPARK-44579: -- User 'yaooqinn' has created a pull request for this

[jira] [Commented] (SPARK-44239) Free memory allocated by large vectors when vectors are reset

[ https://issues.apache.org/jira/browse/SPARK-44239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748396#comment-17748396 ] Snoot.io commented on SPARK-44239: -- User 'wankunde' has created a pull request for this

[jira] [Commented] (SPARK-44239) Free memory allocated by large vectors when vectors are reset

[ https://issues.apache.org/jira/browse/SPARK-44239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748395#comment-17748395 ] Snoot.io commented on SPARK-44239: -- User 'wankunde' has created a pull request for this

[jira] [Created] (SPARK-44579) Support Interrupt On Cancel in SQLExecution

Kent Yao created SPARK-44579: Summary: Support Interrupt On Cancel in SQLExecution Key: SPARK-44579 URL: https://issues.apache.org/jira/browse/SPARK-44579 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-44554) Install different Python linter dependencies for daily testing of different Spark versions

[ https://issues.apache.org/jira/browse/SPARK-44554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748390#comment-17748390 ] Snoot.io commented on SPARK-44554: -- User 'LuciferYang' has created a pull request for t

[jira] [Commented] (SPARK-44287) Define the computing logic through PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators.

[ https://issues.apache.org/jira/browse/SPARK-44287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748389#comment-17748389 ] Snoot.io commented on SPARK-44287: -- User 'cloud-fan' has created a pull request for thi

[jira] [Commented] (SPARK-44567) Daily GA for Maven testing

[ https://issues.apache.org/jira/browse/SPARK-44567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748387#comment-17748387 ] Snoot.io commented on SPARK-44567: -- User 'LuciferYang' has created a pull request for t

[jira] [Commented] (SPARK-44567) Daily GA for Maven testing

[ https://issues.apache.org/jira/browse/SPARK-44567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748388#comment-17748388 ] Snoot.io commented on SPARK-44567: -- User 'LuciferYang' has created a pull request for t

[jira] [Resolved] (SPARK-44558) Export Pyspark's Spark Connect Log Level

[ https://issues.apache.org/jira/browse/SPARK-44558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44558. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44558) Export Pyspark's Spark Connect Log Level

[ https://issues.apache.org/jira/browse/SPARK-44558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44558: Assignee: Alice Sayutina > Export Pyspark's Spark Connect Log Level > ---

[jira] [Updated] (SPARK-44542) eagerly load SparkExitCode class in SparkUncaughtExceptionHandler

[ https://issues.apache.org/jira/browse/SPARK-44542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YE updated SPARK-44542: --- Summary: eagerly load SparkExitCode class in SparkUncaughtExceptionHandler (was: easily load SparkExitCode class in

[jira] [Created] (SPARK-44578) Support pushing down UDFs in DSv2

Holden Karau created SPARK-44578: Summary: Support pushing down UDFs in DSv2 Key: SPARK-44578 URL: https://issues.apache.org/jira/browse/SPARK-44578 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-44198) Support propagation of the log level to the executors

[ https://issues.apache.org/jira/browse/SPARK-44198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-44198. Fix Version/s: 4.0.0 Assignee: Vinod KC Resolution: Fixed Issue re

[jira] [Created] (SPARK-44577) INSERT BY NAME returns non-sensical error message

Serge Rielau created SPARK-44577: Summary: INSERT BY NAME returns non-sensical error message Key: SPARK-44577 URL: https://issues.apache.org/jira/browse/SPARK-44577 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44425) Validate that session_id is an UUID

[ https://issues.apache.org/jira/browse/SPARK-44425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44425: - Fix Version/s: 4.0.0 > Validate that session_id is an UUID > ---

[jira] [Resolved] (SPARK-44425) Validate that session_id is an UUID

[ https://issues.apache.org/jira/browse/SPARK-44425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44425. -- Fix Version/s: 3.5.0 Assignee: Juliusz Sompolski Resolution: Fixed Fixed in ht

[jira] [Commented] (SPARK-44547) BlockManagerDecommissioner throws exceptions when migrating RDD cached blocks to fallback storage

[ https://issues.apache.org/jira/browse/SPARK-44547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748327#comment-17748327 ] Ignite TC Bot commented on SPARK-44547: --- User 'ukby1234' has created a pull reques

[jira] [Resolved] (SPARK-44560) Improve tests and documentation for Arrow Python UDF

[ https://issues.apache.org/jira/browse/SPARK-44560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-44560. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44560) Improve tests and documentation for Arrow Python UDF

[ https://issues.apache.org/jira/browse/SPARK-44560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-44560: Assignee: Xinrong Meng > Improve tests and documentation for Arrow Python UDF > -

[jira] [Created] (SPARK-44576) Session Artifact update breaks XXWithState methods in KVGDS

Zhen Li created SPARK-44576: --- Summary: Session Artifact update breaks XXWithState methods in KVGDS Key: SPARK-44576 URL: https://issues.apache.org/jira/browse/SPARK-44576 Project: Spark Issue Type

[jira] [Updated] (SPARK-44479) Support Python UDTFs with empty schema

[ https://issues.apache.org/jira/browse/SPARK-44479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-44479: -- Fix Version/s: 3.5.0 > Support Python UDTFs with empty schema > --

[jira] [Resolved] (SPARK-43968) Improve error messages for Python UDTFs with wrong number of outputs

[ https://issues.apache.org/jira/browse/SPARK-43968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-43968. --- Fix Version/s: 4.0.0 Assignee: Allison Wang Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-44575) Implement Error Translation

Yihong He created SPARK-44575: - Summary: Implement Error Translation Key: SPARK-44575 URL: https://issues.apache.org/jira/browse/SPARK-44575 Project: Spark Issue Type: New Feature Compo

[jira] [Updated] (SPARK-44559) Improve error messages for Python UDTF arrow type casts

[ https://issues.apache.org/jira/browse/SPARK-44559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-44559: - Summary: Improve error messages for Python UDTF arrow type casts (was: Improve error messages f

[jira] [Created] (SPARK-44574) Errors that moved into sq/api should also use Analysis

Rui Wang created SPARK-44574: Summary: Errors that moved into sq/api should also use Analysis Key: SPARK-44574 URL: https://issues.apache.org/jira/browse/SPARK-44574 Project: Spark Issue Type: Su

[jira] [Resolved] (SPARK-44507) SCSC does not depend on AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-44507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang resolved SPARK-44507. -- Resolution: Won't Fix > SCSC does not depend on AnalysisException > --

[jira] [Created] (SPARK-44573) Couldn't submit Spark application to Kubenetes in versions v1.27.3

Siddaraju G C created SPARK-44573: - Summary: Couldn't submit Spark application to Kubenetes in versions v1.27.3 Key: SPARK-44573 URL: https://issues.apache.org/jira/browse/SPARK-44573 Project: Spark

[jira] [Assigned] (SPARK-44505) DataSource v2 Scans should not require planning the input partitions on explain

[ https://issues.apache.org/jira/browse/SPARK-44505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44505: --- Assignee: Martin Grund > DataSource v2 Scans should not require planning the input partitio

[jira] [Resolved] (SPARK-44505) DataSource v2 Scans should not require planning the input partitions on explain

[ https://issues.apache.org/jira/browse/SPARK-44505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44505. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 42099 [https://gith

[jira] [Commented] (SPARK-44567) Daily GA for Maven testing

[ https://issues.apache.org/jira/browse/SPARK-44567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748109#comment-17748109 ] Yang Jie commented on SPARK-44567: -- I've tried this [before|https://github.com/apache/s

[jira] [Commented] (SPARK-44566) Spark CI Improvement

[ https://issues.apache.org/jira/browse/SPARK-44566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748101#comment-17748101 ] Ruifeng Zheng commented on SPARK-44566: --- also cc [~panbingkun] [~dongjoon] [~yikun

[jira] [Updated] (SPARK-44566) Spark CI Improvement

[ https://issues.apache.org/jira/browse/SPARK-44566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44566: -- Description: I have an offline discussion with [~gurwls223] and [~LuciferYang], and we think

[jira] [Created] (SPARK-44572) Clean up unused files ASAP

Ruifeng Zheng created SPARK-44572: - Summary: Clean up unused files ASAP Key: SPARK-44572 URL: https://issues.apache.org/jira/browse/SPARK-44572 Project: Spark Issue Type: Sub-task C

[jira] [Updated] (SPARK-44566) Spark CI Improvement

[ https://issues.apache.org/jira/browse/SPARK-44566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44566: -- Description: I have an offline discussion with [~gurwls223] and [~LuciferYang], and we think

[jira] [Updated] (SPARK-44571) Eliminate the Join by combine multiple Aggregates

[ https://issues.apache.org/jira/browse/SPARK-44571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-44571: --- Summary: Eliminate the Join by combine multiple Aggregates (was: Eliminate the Join by Combine mult

[jira] [Commented] (SPARK-44571) Eliminate the Join by Combine multiple Aggregates

[ https://issues.apache.org/jira/browse/SPARK-44571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748084#comment-17748084 ] jiaan.geng commented on SPARK-44571: I'm working on. > Eliminate the Join by Combin

[jira] [Created] (SPARK-44571) Eliminate the Join by Combine multiple Aggregates

jiaan.geng created SPARK-44571: -- Summary: Eliminate the Join by Combine multiple Aggregates Key: SPARK-44571 URL: https://issues.apache.org/jira/browse/SPARK-44571 Project: Spark Issue Type: New

[jira] [Created] (SPARK-44570) Reuse spark build among pyspark-* modules

Ruifeng Zheng created SPARK-44570: - Summary: Reuse spark build among pyspark-* modules Key: SPARK-44570 URL: https://issues.apache.org/jira/browse/SPARK-44570 Project: Spark Issue Type: Sub-t

[jira] [Created] (SPARK-44569) Daily GA for Python 3.11

Ruifeng Zheng created SPARK-44569: - Summary: Daily GA for Python 3.11 Key: SPARK-44569 URL: https://issues.apache.org/jira/browse/SPARK-44569 Project: Spark Issue Type: Sub-task Com

[jira] [Created] (SPARK-44568) Daily GA for Python 3.10

Ruifeng Zheng created SPARK-44568: - Summary: Daily GA for Python 3.10 Key: SPARK-44568 URL: https://issues.apache.org/jira/browse/SPARK-44568 Project: Spark Issue Type: Sub-task Com

[jira] [Created] (SPARK-44567) Daily GA for Maven testing

Ruifeng Zheng created SPARK-44567: - Summary: Daily GA for Maven testing Key: SPARK-44567 URL: https://issues.apache.org/jira/browse/SPARK-44567 Project: Spark Issue Type: Sub-task C

[jira] [Created] (SPARK-44566) Spark CI Improvement

Ruifeng Zheng created SPARK-44566: - Summary: Spark CI Improvement Key: SPARK-44566 URL: https://issues.apache.org/jira/browse/SPARK-44566 Project: Spark Issue Type: Umbrella Compone

[jira] [Assigned] (SPARK-44454) HiveShim getTablesByType support fallback

[ https://issues.apache.org/jira/browse/SPARK-44454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44454: --- Assignee: dzcxzl > HiveShim getTablesByType support fallback >

[jira] [Resolved] (SPARK-44454) HiveShim getTablesByType support fallback

[ https://issues.apache.org/jira/browse/SPARK-44454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44454. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42033 [https://gith

[jira] [Assigned] (SPARK-44536) Upgrade sbt to 1.9.3

[ https://issues.apache.org/jira/browse/SPARK-44536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-44536: Assignee: BingKun Pan > Upgrade sbt to 1.9.3 > > > Key: SPAR

[jira] [Resolved] (SPARK-44536) Upgrade sbt to 1.9.3

[ https://issues.apache.org/jira/browse/SPARK-44536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-44536. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42141 [https://github.com

[jira] [Assigned] (SPARK-44482) Connect server should can specify the bind address

[ https://issues.apache.org/jira/browse/SPARK-44482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44482: Assignee: BingKun Pan > Connect server should can specify the bind address >

[jira] [Resolved] (SPARK-44482) Connect server should can specify the bind address

[ https://issues.apache.org/jira/browse/SPARK-44482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44482. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42073 [https://gi

[jira] [Updated] (SPARK-44513) Upgrade snappy-java to 1.1.10.3

[ https://issues.apache.org/jira/browse/SPARK-44513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44513: Fix Version/s: 3.4.2 > Upgrade snappy-java to 1.1.10.3 > --- > >