[jira] [Resolved] (SPARK-47420) Fix CollationSupport test output

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47420. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46058

[jira] [Resolved] (SPARK-47769) Add schema_of_variant_agg expression.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47769. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45934

[jira] [Assigned] (SPARK-47769) Add schema_of_variant_agg expression.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47769: --- Assignee: Chenhao Li > Add schema_of_variant_agg expression. >

[jira] [Resolved] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47463. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45589

[jira] [Resolved] (SPARK-47862) Connect generated proots can't be pickled

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47862. -- Resolution: Fixed Issue resolved by pull request 46068

[jira] [Assigned] (SPARK-47862) Connect generated proots can't be pickled

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47862: Assignee: Martin Grund > Connect generated proots can't be pickled >

[jira] [Assigned] (SPARK-47840) Remove foldable propagation across Streaming Aggregate/Join nodes

2024-04-15 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47840: Assignee: Bhuwan Sahni > Remove foldable propagation across Streaming Aggregate/Join

[jira] [Resolved] (SPARK-47840) Remove foldable propagation across Streaming Aggregate/Join nodes

2024-04-15 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47840. -- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-47371) XML: Ignore row tags in CDATA Tokenizer

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47371: Assignee: Yousof Hosny > XML: Ignore row tags in CDATA Tokenizer >

[jira] [Resolved] (SPARK-47371) XML: Ignore row tags in CDATA Tokenizer

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47371. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45487

[jira] [Resolved] (SPARK-47866) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47866. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46070

[jira] [Assigned] (SPARK-47866) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47866: Assignee: Dongjoon Hyun > Deflaky PythonForeachWriterSuite >

[jira] [Resolved] (SPARK-46810) Clarify error class terminology

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46810. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44902

[jira] [Assigned] (SPARK-46810) Clarify error class terminology

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46810: --- Assignee: Nicholas Chammas > Clarify error class terminology >

[jira] [Updated] (SPARK-47867) Support Variant in JSON scan.

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47867: --- Labels: pull-request-available (was: ) > Support Variant in JSON scan. >

[jira] [Resolved] (SPARK-47673) [Arbitrary State Support] State TTL support - ListState

2024-04-15 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47673. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45932

[jira] [Created] (SPARK-47867) Support Variant in JSON scan.

2024-04-15 Thread Chenhao Li (Jira)
Chenhao Li created SPARK-47867: -- Summary: Support Variant in JSON scan. Key: SPARK-47867 URL: https://issues.apache.org/jira/browse/SPARK-47867 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-47804) Add Dataframe cache debug log

2024-04-15 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47804. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45990

[jira] [Updated] (SPARK-47866) Deflaky PythonForeachWriterSuite

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47866: --- Labels: pull-request-available (was: ) > Deflaky PythonForeachWriterSuite >

[jira] [Created] (SPARK-47865) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47865: - Summary: Deflaky PythonForeachWriterSuite Key: SPARK-47865 URL: https://issues.apache.org/jira/browse/SPARK-47865 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-47866) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47866: - Summary: Deflaky PythonForeachWriterSuite Key: SPARK-47866 URL: https://issues.apache.org/jira/browse/SPARK-47866 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43394) Upgrade maven to 3.8.8

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43394: -- Fix Version/s: 3.4.4 > Upgrade maven to 3.8.8 > -- > >

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47828: -- Fix Version/s: 3.5.2 > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Created] (SPARK-47864) Enhance "Installation" page to cover all installable options

2024-04-15 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-47864: --- Summary: Enhance "Installation" page to cover all installable options Key: SPARK-47864 URL: https://issues.apache.org/jira/browse/SPARK-47864 Project: Spark

[jira] [Assigned] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47855: - Assignee: Ruifeng Zheng > Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in

[jira] [Resolved] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47855. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46056

[jira] [Resolved] (SPARK-47860) Upgrade `kubernetes-client` to 6.12.0

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47860. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46066

[jira] [Updated] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-15 Thread Vladimir Golubev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Golubev updated SPARK-47863: - Parent: SPARK-46837 Issue Type: Sub-task (was: Bug) > endsWith and startsWith

[jira] [Created] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-15 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-47863: Summary: endsWith and startsWith don't work correctly for some collations Key: SPARK-47863 URL: https://issues.apache.org/jira/browse/SPARK-47863 Project:

[jira] [Updated] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-15 Thread Vladimir Golubev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Golubev updated SPARK-47863: - Description: *CollationSupport.EndsWIth* and *CollationSupport.StartsWith* use

[jira] [Updated] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-15 Thread Vladimir Golubev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Golubev updated SPARK-47863: - Description: *CollationSupport.EndsWIth* and *CollationSupport.StartsWith* use

[jira] [Updated] (SPARK-47862) Connect generated proots can't be pickled

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47862: --- Labels: pull-request-available (was: ) > Connect generated proots can't be pickled >

[jira] [Created] (SPARK-47862) Connect generated proots can't be pickled

2024-04-15 Thread Martin Grund (Jira)
Martin Grund created SPARK-47862: Summary: Connect generated proots can't be pickled Key: SPARK-47862 URL: https://issues.apache.org/jira/browse/SPARK-47862 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-47603) Resource managers: Migrate logWarn with variables to structured logging framework

2024-04-15 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47603. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45957

[jira] [Created] (SPARK-47860) Upgrade `kubernetes-client` to 6.12.0

2024-04-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47860: - Summary: Upgrade `kubernetes-client` to 6.12.0 Key: SPARK-47860 URL: https://issues.apache.org/jira/browse/SPARK-47860 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-47859) Why does javaRDD().mapPartitions lead to the memory leak in this case?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Description: Hello Spark community. I have an Java Spark Structured Streaming application:

[jira] [Comment Edited] (SPARK-47859) Why does javaRDD().mapPartitions lead to the memory leak in this case?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837420#comment-17837420 ] Leo Timofeyev edited comment on SPARK-47859 at 4/15/24 7:40 PM: This is

[jira] [Commented] (SPARK-47859) Why does javaRDD().mapPartitions lead to the memory leak in this case?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837420#comment-17837420 ] Leo Timofeyev commented on SPARK-47859: --- This is likely duplicate > Why does

[jira] [Comment Edited] (SPARK-47859) Why does javaRDD().mapPartitions lead to the memory leak in this case?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837420#comment-17837420 ] Leo Timofeyev edited comment on SPARK-47859 at 4/15/24 7:39 PM: This is

[jira] [Commented] (SPARK-35262) Memory leak when dataset is being persisted

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837418#comment-17837418 ] Leo Timofeyev commented on SPARK-35262: --- Hello [~dnskrv] [~iamelin] I can confirm that the issue

[jira] [Updated] (SPARK-47859) Why does javaRDD().mapPartitions lead to the memory leak in this case?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Summary: Why does javaRDD().mapPartitions lead to the memory leak in this case? (was: Why

[jira] [Updated] (SPARK-47859) Why does lead to the memory leak?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Summary: Why does lead to the memory leak? (was: Why does this lead to the memory leak?) >

[jira] [Updated] (SPARK-47859) Why does this lead to the memory leak?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Description: Hello Spark community. I have an Java Spark Structured Streaming application:

[jira] [Updated] (SPARK-47859) Why does this lead to the memory leak?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Description: Hello Spark community. I have an Java Spark Structured Streaming application:

[jira] [Updated] (SPARK-47859) Why does this lead to the memory leak?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Description: Hello Spark community. I have an Java Spark Structured Streaming application:

[jira] [Updated] (SPARK-47859) Why does this lead to the memory leak?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Description: Hello Spark community. I have an Java Spark Structured Streaming application:

[jira] [Created] (SPARK-47859) Why does this lead to the memory leak?

2024-04-15 Thread Leo Timofeyev (Jira)
Leo Timofeyev created SPARK-47859: - Summary: Why does this lead to the memory leak? Key: SPARK-47859 URL: https://issues.apache.org/jira/browse/SPARK-47859 Project: Spark Issue Type: IT Help

[jira] [Updated] (SPARK-47859) Why does this lead to the memory leak?

2024-04-15 Thread Leo Timofeyev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo Timofeyev updated SPARK-47859: -- Attachment: Screenshot 2024-04-15 at 20.43.22.png > Why does this lead to the memory leak? >

[jira] [Assigned] (SPARK-47694) Make max message size configurable on client side

2024-04-15 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-47694: - Assignee: Robert Dillitz (was: Martin Grund) > Make max message size

[jira] [Resolved] (SPARK-47694) Make max message size configurable on client side

2024-04-15 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-47694. --- Fix Version/s: 4.0.0 Resolution: Fixed > Make max message size configurable

[jira] [Resolved] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47856. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46059

[jira] [Assigned] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47856: - Assignee: Kent Yao > Oracle: Document Mapping Spark SQL Data Types from Oracle >

[jira] [Updated] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle and add tests

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47856: -- Summary: Oracle: Document Mapping Spark SQL Data Types from Oracle and add tests (was:

[jira] [Updated] (SPARK-47819) Use asynchronous callback for execution cleanup

2024-04-15 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-47819: -- Affects Version/s: 3.5.1 3.5.0 > Use asynchronous callback for

[jira] [Resolved] (SPARK-47357) Add support for Upper, Lower, InitCap (all collations)

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47357. - Fix Version/s: 4.0.0 Assignee: Mihailo Milosevic Resolution: Fixed > Add

[jira] [Updated] (SPARK-47429) Rename errorClass to errorCondition

2024-04-15 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-47429: - Description: We've agreed on the parent task to rename {{errorClass}} to align it more

[jira] [Commented] (SPARK-28024) Incorrect numeric values when out of range

2024-04-15 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837292#comment-17837292 ] Nicholas Chammas commented on SPARK-28024: -- [~cloud_fan] - Given the updated descriptions for

[jira] [Updated] (SPARK-47858) Refactoring the structure for DataFrame error context

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47858: --- Labels: pull-request-available (was: ) > Refactoring the structure for DataFrame error

[jira] [Created] (SPARK-47858) Refactoring the structure for DataFrame error context

2024-04-15 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-47858: --- Summary: Refactoring the structure for DataFrame error context Key: SPARK-47858 URL: https://issues.apache.org/jira/browse/SPARK-47858 Project: Spark Issue

[jira] [Updated] (SPARK-47356) Add support for ConcatWs & Elt (all collations)

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47356: --- Labels: pull-request-available (was: ) > Add support for ConcatWs & Elt (all collations) >

[jira] [Updated] (SPARK-47356) Add support for ConcatWs & Elt (all collations)

2024-04-15 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-47356: -- Summary: Add support for ConcatWs & Elt (all collations) (was: ConcatWs & Elt (all

[jira] [Updated] (SPARK-47420) Fix CollationSupport test output

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47420: --- Labels: pull-request-available (was: ) > Fix CollationSupport test output >

[jira] [Updated] (SPARK-47412) StringLPad, StringRPad (all collations)

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47412: --- Labels: pull-request-available (was: ) > StringLPad, StringRPad (all collations) >

[jira] [Updated] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47856: --- Labels: pull-request-available (was: ) > Oracle: Document Mapping Spark SQL Data Types

[jira] [Commented] (SPARK-47356) ConcatWs & Elt (all collations)

2024-04-15 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837145#comment-17837145 ] Mihailo Milosevic commented on SPARK-47356: --- Working on this. > ConcatWs & Elt (all

[jira] [Created] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle

2024-04-15 Thread Kent Yao (Jira)
Kent Yao created SPARK-47856: Summary: Oracle: Document Mapping Spark SQL Data Types from Oracle Key: SPARK-47856 URL: https://issues.apache.org/jira/browse/SPARK-47856 Project: Spark Issue

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47413: -- Assignee: Apache Spark > Substring, Right, Left (all collations) >

[jira] [Assigned] (SPARK-47594) Connector module: Migrate logInfo with variables to structured logging framework

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47594: -- Assignee: (was: Apache Spark) > Connector module: Migrate logInfo with variables

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47413: -- Assignee: (was: Apache Spark) > Substring, Right, Left (all collations) >

[jira] [Assigned] (SPARK-47566) SubstringIndex

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47566: -- Assignee: (was: Apache Spark) > SubstringIndex > -- > >

[jira] [Assigned] (SPARK-47566) SubstringIndex

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47566: -- Assignee: Apache Spark > SubstringIndex > -- > > Key:

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47413: -- Assignee: (was: Apache Spark) > Substring, Right, Left (all collations) >

[jira] [Assigned] (SPARK-47594) Connector module: Migrate logInfo with variables to structured logging framework

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47594: -- Assignee: (was: Apache Spark) > Connector module: Migrate logInfo with variables

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47413: -- Assignee: Apache Spark > Substring, Right, Left (all collations) >

[jira] [Updated] (SPARK-47420) Fix CollationSupport test output

2024-04-15 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-47420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-47420: - Summary: Fix CollationSupport test output (was: TBD) > Fix CollationSupport test output >

[jira] [Updated] (SPARK-47584) SQL core: Migrate logWarn with variables to structured logging framework

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47584: --- Labels: pull-request-available (was: ) > SQL core: Migrate logWarn with variables to

[jira] [Updated] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47855: --- Labels: pull-request-available (was: ) > Warn

[jira] [Updated] (SPARK-47854) [PYTHON] Avoid shadowing python built-ins in python function variable naming

2024-04-15 Thread Liu Cao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Cao updated SPARK-47854: Description: Given that spark 4.0.0 is upcoming I wonder if we should at least consider renaming certain

[jira] [Created] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

2024-04-15 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-47855: - Summary: Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect Key: SPARK-47855 URL: https://issues.apache.org/jira/browse/SPARK-47855 Project:

[jira] [Resolved] (SPARK-47851) Document pyspark-connect package

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47851. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46054

[jira] [Updated] (SPARK-47853) jdbc connect to duckdb with error Unrecognized configuration property "path"

2024-04-15 Thread WeiNan Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WeiNan Zhao updated SPARK-47853: Description: Link issue: _[https://github.com/duckdb/duckdb/issues/11651_] # reproduce python

[jira] [Assigned] (SPARK-47851) Document pyspark-connect package

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47851: Assignee: Hyukjin Kwon > Document pyspark-connect package >

[jira] [Updated] (SPARK-47853) jdbc connect to duckdb with error Unrecognized configuration property "path"

2024-04-15 Thread WeiNan Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WeiNan Zhao updated SPARK-47853: Description: Link issue: _https://github.com/duckdb/duckdb/issues/11651_ # reproduce python code 

[jira] [Updated] (SPARK-47853) jdbc connect to duckdb with error Unrecognized configuration property "path"

2024-04-15 Thread WeiNan Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WeiNan Zhao updated SPARK-47853: Description: # reproduce python code  ```python from pyspark.sql import SparkSession if

[jira] [Created] (SPARK-47853) jdbc connect to duckdb with error Unrecognized configuration property "path"

2024-04-15 Thread WeiNan Zhao (Jira)
WeiNan Zhao created SPARK-47853: --- Summary: jdbc connect to duckdb with error Unrecognized configuration property "path" Key: SPARK-47853 URL: https://issues.apache.org/jira/browse/SPARK-47853 Project:

[jira] [Updated] (SPARK-47760) Reeanble Avro function doctests

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47760: --- Labels: pull-request-available (was: ) > Reeanble Avro function doctests >

[jira] [Updated] (SPARK-47851) Document pyspark-connect package

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47851: --- Labels: pull-request-available (was: ) > Document pyspark-connect package >

[jira] [Updated] (SPARK-47852) Support DataFrameQueryContext for reverse operations

2024-04-15 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47852: --- Labels: pull-request-available (was: ) > Support DataFrameQueryContext for reverse

[jira] [Resolved] (SPARK-47788) Ensure the same hash partitioning scheme/hash function is used across batches

2024-04-15 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47788. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45971

[jira] [Assigned] (SPARK-47788) Ensure the same hash partitioning scheme/hash function is used across batches

2024-04-15 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47788: Assignee: Fanyue Xia > Ensure the same hash partitioning scheme/hash function is used

[jira] [Resolved] (SPARK-47848) Fix thread safe access for loadedMaps in close

2024-04-15 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47848. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46048