[jira] [Updated] (SPARK-44780) Document SQL Session variables

[ https://issues.apache.org/jira/browse/SPARK-44780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Rielau updated SPARK-44780: - Attachment: Screenshot 2023-08-11 at 10.22.55 PM.png Screenshot 2023-08-11 at

[jira] [Resolved] (SPARK-44719) NoClassDefFoundError when using Hive UDF

[ https://issues.apache.org/jira/browse/SPARK-44719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44719. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42446

[jira] [Assigned] (SPARK-44719) NoClassDefFoundError when using Hive UDF

[ https://issues.apache.org/jira/browse/SPARK-44719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44719: Assignee: Yuming Wang > NoClassDefFoundError when using Hive UDF >

[jira] [Commented] (SPARK-44781) Runtime filter should supports reuse exchange if it can reduce the data size of application side

[ https://issues.apache.org/jira/browse/SPARK-44781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753458#comment-17753458 ] jiaan.geng commented on SPARK-44781: I'm working on. > Runtime filter should supports reuse

[jira] [Created] (SPARK-44781) Runtime filter should supports reuse exchange if it can reduce the data size of application side

jiaan.geng created SPARK-44781: -- Summary: Runtime filter should supports reuse exchange if it can reduce the data size of application side Key: SPARK-44781 URL: https://issues.apache.org/jira/browse/SPARK-44781

[jira] [Resolved] (SPARK-44242) Spark job submission failed because Xmx string is available on one parameter provided into spark.driver.extraJavaOptions

[ https://issues.apache.org/jira/browse/SPARK-44242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-44242. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-44242) Spark job submission failed because Xmx string is available on one parameter provided into spark.driver.extraJavaOptions

[ https://issues.apache.org/jira/browse/SPARK-44242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-44242: --- Assignee: Nicolas Fraison > Spark job submission failed because Xmx string

[jira] [Resolved] (SPARK-43987) Separate finalizeShuffleMerge Processing to Dedicated Thread Pools

[ https://issues.apache.org/jira/browse/SPARK-43987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-43987. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-43987) Separate finalizeShuffleMerge Processing to Dedicated Thread Pools

[ https://issues.apache.org/jira/browse/SPARK-43987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-43987: --- Assignee: SHU WANG > Separate finalizeShuffleMerge Processing to Dedicated

[jira] [Commented] (SPARK-44461) Enable Process Isolation for streaming python worker

[ https://issues.apache.org/jira/browse/SPARK-44461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753427#comment-17753427 ] Hyukjin Kwon commented on SPARK-44461: -- [~rangadi] can we switch the JIRA by switching the

[jira] [Updated] (SPARK-44780) Document SQL Session variables

[ https://issues.apache.org/jira/browse/SPARK-44780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Rielau updated SPARK-44780: - Summary: Document SQL Session variables (was: Docuement SQL Session variables) > Document SQL

[jira] [Created] (SPARK-44780) Docuement SQL Session variables

Serge Rielau created SPARK-44780: Summary: Docuement SQL Session variables Key: SPARK-44780 URL: https://issues.apache.org/jira/browse/SPARK-44780 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-44778) Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()`

[ https://issues.apache.org/jira/browse/SPARK-44778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44778: - Epic Link: (was: SPARK-38783) > Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()` >

[jira] [Updated] (SPARK-44778) Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()`

[ https://issues.apache.org/jira/browse/SPARK-44778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44778: - Description: Introduce the timediff() function, which takes three arguments: unit, and two datetime

[jira] [Updated] (SPARK-44778) Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()`

[ https://issues.apache.org/jira/browse/SPARK-44778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44778: - Affects Version/s: 4.0.0 (was: 3.3.0) > Add the `TIMEDIFF` aliases for

[jira] [Updated] (SPARK-44778) Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()`

[ https://issues.apache.org/jira/browse/SPARK-44778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44778: - Fix Version/s: (was: 3.3.0) > Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()` >

[jira] [Created] (SPARK-44778) Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()`

Max Gekk created SPARK-44778: Summary: Add the `TIMEDIFF` aliases for `TIMESTAMPDIFF()` Key: SPARK-44778 URL: https://issues.apache.org/jira/browse/SPARK-44778 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-44625) Spark Connect clean up abandoned executions

[ https://issues.apache.org/jira/browse/SPARK-44625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44625. --- Fix Version/s: 3.5.0 Assignee: Juliusz Sompolski Resolution: Fixed

[jira] [Created] (SPARK-44777) Allow to specify eagerness to RDD.checkpoint

Emil Ejbyfeldt created SPARK-44777: -- Summary: Allow to specify eagerness to RDD.checkpoint Key: SPARK-44777 URL: https://issues.apache.org/jira/browse/SPARK-44777 Project: Spark Issue Type:

[jira] [Created] (SPARK-44776) Add ProducedRowCount to SparkListenerConnectOperationFinished

Lingkai Kong created SPARK-44776: Summary: Add ProducedRowCount to SparkListenerConnectOperationFinished Key: SPARK-44776 URL: https://issues.apache.org/jira/browse/SPARK-44776 Project: Spark

[jira] [Comment Edited] (SPARK-43327) Trigger `committer.setupJob` before plan execute in `FileFormatWriter`

[ https://issues.apache.org/jira/browse/SPARK-43327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718060#comment-17718060 ] ming95 edited comment on SPARK-43327 at 8/11/23 12:58 PM: -- pr :

[jira] [Resolved] (SPARK-44761) Add DataStreamWriter.foreachBatch(org.apache.spark.api.java.function.VoidFunction2) signature

[ https://issues.apache.org/jira/browse/SPARK-44761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44761. --- Fix Version/s: 3.5.0 Resolution: Fixed > Add >

[jira] [Resolved] (SPARK-44760) Index Out Of Bound for JIRA resolution in merge_spark_pr

[ https://issues.apache.org/jira/browse/SPARK-44760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44760. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42429

[jira] [Created] (SPARK-44775) Add missing version information in DataFrame APIs

Ruifeng Zheng created SPARK-44775: - Summary: Add missing version information in DataFrame APIs Key: SPARK-44775 URL: https://issues.apache.org/jira/browse/SPARK-44775 Project: Spark Issue

[jira] [Updated] (SPARK-44774) SaveMode.ErrorIfExists does not work with kafka-sql

[ https://issues.apache.org/jira/browse/SPARK-44774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-44774: --- Description: I' trying to write batch dataframe to Kafka topic with {{mode="error"}}, but

[jira] [Updated] (SPARK-44774) SaveMode.ErrorIfExists does not work with kafka-sql

[ https://issues.apache.org/jira/browse/SPARK-44774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-44774: --- Description: I' trying to write batch dataframe to Kafka topic with {{mode="error"}}, but

[jira] [Updated] (SPARK-44774) SaveMode.ErrorIfExists does not work with kafka-sql

[ https://issues.apache.org/jira/browse/SPARK-44774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-44774: --- Description: I' trying to write batch dataframe to Kafka topic with {{mode="error"}}, but

[jira] [Updated] (SPARK-44774) SaveMode.ErrorIfExists does not work with kafka-sql

[ https://issues.apache.org/jira/browse/SPARK-44774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-44774: --- Description: I' trying to write batch dataframe to Kafka topic with {{mode="error"}}, but

[jira] [Assigned] (SPARK-43477) Enable SeriesStringTests.test_string_rsplit for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43477: - Assignee: Haejoon Lee > Enable SeriesStringTests.test_string_rsplit for pandas 2.0.0.

[jira] [Assigned] (SPARK-43478) Enable SeriesStringTests.test_string_split for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43478: - Assignee: Haejoon Lee > Enable SeriesStringTests.test_string_split for pandas 2.0.0. >

[jira] [Resolved] (SPARK-43478) Enable SeriesStringTests.test_string_split for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43478. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42312

[jira] [Assigned] (SPARK-43476) Enable SeriesStringTests.test_string_replace for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43476: - Assignee: Haejoon Lee > Enable SeriesStringTests.test_string_replace for pandas 2.0.0.

[jira] [Resolved] (SPARK-43477) Enable SeriesStringTests.test_string_rsplit for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43477. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42312

[jira] [Created] (SPARK-44774) SaveMode.ErrorIfExists does not work with kafka-sql

Maxim Martynov created SPARK-44774: -- Summary: SaveMode.ErrorIfExists does not work with kafka-sql Key: SPARK-44774 URL: https://issues.apache.org/jira/browse/SPARK-44774 Project: Spark

[jira] [Resolved] (SPARK-43476) Enable SeriesStringTests.test_string_replace for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43476. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42312

[jira] [Resolved] (SPARK-44731) Support 'spark.sql.timestampType' in Python Spark Connect client

[ https://issues.apache.org/jira/browse/SPARK-44731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44731. --- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-44731) Support 'spark.sql.timestampType' in Python Spark Connect client

[ https://issues.apache.org/jira/browse/SPARK-44731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44731: - Assignee: Hyukjin Kwon > Support 'spark.sql.timestampType' in Python Spark Connect

[jira] [Created] (SPARK-44773) Code-gen CodegenFallback expression in WholeStageCodegen if possible

Wan Kun created SPARK-44773: --- Summary: Code-gen CodegenFallback expression in WholeStageCodegen if possible Key: SPARK-44773 URL: https://issues.apache.org/jira/browse/SPARK-44773 Project: Spark

[jira] [Assigned] (SPARK-44770) Add a displayOrder variable to WebUITab to specify the order in which tabs appear

[ https://issues.apache.org/jira/browse/SPARK-44770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44770: Assignee: Jason Li > Add a displayOrder variable to WebUITab to specify the order in which tabs

[jira] [Resolved] (SPARK-44770) Add a displayOrder variable to WebUITab to specify the order in which tabs appear

[ https://issues.apache.org/jira/browse/SPARK-44770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44770. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42442

[jira] [Assigned] (SPARK-44727) Improve the error message for dynamic allocation conditions

[ https://issues.apache.org/jira/browse/SPARK-44727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44727: Assignee: Cheng Pan > Improve the error message for dynamic allocation conditions >

[jira] [Resolved] (SPARK-44727) Improve the error message for dynamic allocation conditions

[ https://issues.apache.org/jira/browse/SPARK-44727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44727. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-44737) Should not display json format errors on SQL page for non-SparkThrowables

[ https://issues.apache.org/jira/browse/SPARK-44737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44737. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-44737) Should not display json format errors on SQL page for non-SparkThrowables

[ https://issues.apache.org/jira/browse/SPARK-44737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44737: Assignee: Kent Yao > Should not display json format errors on SQL page for non-SparkThrowables >