[jira] [Created] (SPARK-48935) Restrictions on`collatinId` should be added to the constructor of `StringType`

BingKun Pan created SPARK-48935: --- Summary: Restrictions on`collatinId` should be added to the constructor of `StringType` Key: SPARK-48935 URL: https://issues.apache.org/jira/browse/SPARK-48935 Project:

[jira] [Updated] (SPARK-48935) Restrictions on`collatinId` should be added to the constructor of `StringType`

[ https://issues.apache.org/jira/browse/SPARK-48935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48935: --- Labels: pull-request-available (was: ) > Restrictions on`collatinId` should be added to the

[jira] [Assigned] (SPARK-48935) Restrictions on`collatinId` should be added to the constructor of `StringType`

[ https://issues.apache.org/jira/browse/SPARK-48935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48935: -- Assignee: Apache Spark > Restrictions on`collatinId` should be added to the construct

[jira] [Assigned] (SPARK-48829) Upgrade `RoaringBitmap` to 1.2.0

[ https://issues.apache.org/jira/browse/SPARK-48829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48829: -- Assignee: (was: Apache Spark) > Upgrade `RoaringBitmap` to 1.2.0 > -

[jira] [Assigned] (SPARK-48388) [M0] Fix SET behavior for scripts

[ https://issues.apache.org/jira/browse/SPARK-48388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48388: -- Assignee: Apache Spark > [M0] Fix SET behavior for scripts >

[jira] [Assigned] (SPARK-48388) [M0] Fix SET behavior for scripts

[ https://issues.apache.org/jira/browse/SPARK-48388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48388: -- Assignee: (was: Apache Spark) > [M0] Fix SET behavior for scripts > -

[jira] [Commented] (SPARK-48292) Revert [SPARK-39195][SQL] Spark OutputCommitCoordinator should abort stage when committed file not consistent with task status

[ https://issues.apache.org/jira/browse/SPARK-48292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17866966#comment-17866966 ] Steve Loughran commented on SPARK-48292: what happens if a TA is authorized to c

[jira] [Created] (SPARK-48936) Makes spark-shell works with Spark connect

Hyukjin Kwon created SPARK-48936: Summary: Makes spark-shell works with Spark connect Key: SPARK-48936 URL: https://issues.apache.org/jira/browse/SPARK-48936 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-48936) Makes spark-shell works with Spark connect

[ https://issues.apache.org/jira/browse/SPARK-48936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48936: --- Labels: pull-request-available (was: ) > Makes spark-shell works with Spark connect > -

[jira] [Created] (SPARK-48937) Fix collation support for the StringToMap expression

Uroš Bojanić created SPARK-48937: Summary: Fix collation support for the StringToMap expression Key: SPARK-48937 URL: https://issues.apache.org/jira/browse/SPARK-48937 Project: Spark Issue Ty

[jira] [Updated] (SPARK-48937) Fix collation support for the StringToMap expression

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48937: - Description: Enable collation support for *StringToMap* built-in string function in Spark ({*}s

[jira] [Updated] (SPARK-48937) Fix collation support for the StringToMap expression

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48937: - Description: Enable collation support for *StringToMap* built-in string function in Spark ({*}s

[jira] [Updated] (SPARK-48937) Fix collation support for the StringToMap expression

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48937: - Description: Enable collation support for *StringToMap* built-in string function in Spark ({*}s

[jira] [Updated] (SPARK-48937) Fix collation support for the StringToMap expression

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48937: - Description: Enable collation support for *StringToMap* built-in string function in Spark ({*}s

[jira] [Updated] (SPARK-48937) Fix collation support for the StringToMap expression (binary & lowercase collation only)

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48937: - Summary: Fix collation support for the StringToMap expression (binary & lowercase collation only

[jira] [Commented] (SPARK-48937) Fix collation support for the StringToMap expression (binary & lowercase collation only)

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867003#comment-17867003 ] Uroš Bojanić commented on SPARK-48937: -- [~psyren99] Here is an open ticket within t

[jira] [Updated] (SPARK-48338) Sql Scripting support for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-48338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48338: --- Labels: pull-request-available (was: ) > Sql Scripting support for Spark SQL >

[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48791: -- Issue Type: Bug (was: Improvement) > Perf regression due to accumulator registration overhead

[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48791: -- Fix Version/s: 3.5.3 > Perf regression due to accumulator registration overhead using > CopyO

[jira] [Commented] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867038#comment-17867038 ] Dongjoon Hyun commented on SPARK-48791: --- I added a fix version, 3.5.3, for now bec

[jira] [Resolved] (SPARK-48890) Add Streaming related fields to log4j ThreadContext

[ https://issues.apache.org/jira/browse/SPARK-48890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48890. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47340 [https:

[jira] [Assigned] (SPARK-48890) Add Streaming related fields to log4j ThreadContext

[ https://issues.apache.org/jira/browse/SPARK-48890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-48890: -- Assignee: Wei Liu > Add Streaming related fields to log4j ThreadContext > ---

[jira] [Updated] (SPARK-48929) View fails with internal error after upgrade causes expected syntax error.

[ https://issues.apache.org/jira/browse/SPARK-48929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48929: --- Labels: pull-request-available (was: ) > View fails with internal error after upgrade cause

[jira] [Commented] (SPARK-48937) Fix collation support for the StringToMap expression (binary & lowercase collation only)

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867106#comment-17867106 ] psyren99 commented on SPARK-48937: -- Yes, I do it > Fix collation support for the Strin

[jira] [Comment Edited] (SPARK-48937) Fix collation support for the StringToMap expression (binary & lowercase collation only)

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867106#comment-17867106 ] psyren99 edited comment on SPARK-48937 at 7/18/24 8:30 PM: --- [~

[jira] [Comment Edited] (SPARK-48937) Fix collation support for the StringToMap expression (binary & lowercase collation only)

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867106#comment-17867106 ] psyren99 edited comment on SPARK-48937 at 7/18/24 8:31 PM: --- [~

[jira] [Commented] (SPARK-48937) Fix collation support for the StringToMap expression (binary & lowercase collation only)

[ https://issues.apache.org/jira/browse/SPARK-48937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867108#comment-17867108 ] Uroš Bojanić commented on SPARK-48937: -- Ack. Feel free to ping me for review when y

[jira] [Commented] (SPARK-48495) Document planned approach to shredding

[ https://issues.apache.org/jira/browse/SPARK-48495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867126#comment-17867126 ] Russell Spitzer commented on SPARK-48495: - This was merged with a bug in the tab

[jira] [Comment Edited] (SPARK-48495) Document planned approach to shredding

[ https://issues.apache.org/jira/browse/SPARK-48495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867126#comment-17867126 ] Russell Spitzer edited comment on SPARK-48495 at 7/18/24 10:05 PM: ---

[jira] [Resolved] (SPARK-48921) ScalaUDF in subquery should run through analyzer

[ https://issues.apache.org/jira/browse/SPARK-48921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48921. --- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pul

[jira] [Updated] (SPARK-48921) ScalaUDF in subquery should run through analyzer

[ https://issues.apache.org/jira/browse/SPARK-48921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48921: -- Fix Version/s: 3.5.3 (was: 3.5.2) > ScalaUDF in subquery should run thr

[jira] [Commented] (SPARK-46446) Correctness bug in correlated subquery with OFFSET

[ https://issues.apache.org/jira/browse/SPARK-46446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867132#comment-17867132 ] Andy Lam commented on SPARK-46446: -- [~cloud_fan] Could we unresolve this ticket or crea

[jira] [Created] (SPARK-48938) Improve error message when registering UDTFs

Allison Wang created SPARK-48938: Summary: Improve error message when registering UDTFs Key: SPARK-48938 URL: https://issues.apache.org/jira/browse/SPARK-48938 Project: Spark Issue Type: Sub-

[jira] [Updated] (SPARK-48938) Improve error message when registering UDTFs

[ https://issues.apache.org/jira/browse/SPARK-48938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48938: --- Labels: pull-request-available (was: ) > Improve error message when registering UDTFs > ---

[jira] [Created] (SPARK-48939) Support recursive reference of Avro schema

Yuchen Liu created SPARK-48939: -- Summary: Support recursive reference of Avro schema Key: SPARK-48939 URL: https://issues.apache.org/jira/browse/SPARK-48939 Project: Spark Issue Type: New Featur

[jira] [Updated] (SPARK-48940) Upgrade `Arrow` to 17.0.0

[ https://issues.apache.org/jira/browse/SPARK-48940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48940: --- Labels: pull-request-available (was: ) > Upgrade `Arrow` to 17.0.0 > --

[jira] [Assigned] (SPARK-48934) Python datetime types converted incorrectly for setting timeout in applyInPandasWithState

[ https://issues.apache.org/jira/browse/SPARK-48934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48934: Assignee: Siying Dong > Python datetime types converted incorrectly for setting timeout i

[jira] [Resolved] (SPARK-48934) Python datetime types converted incorrectly for setting timeout in applyInPandasWithState

[ https://issues.apache.org/jira/browse/SPARK-48934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48934. -- Fix Version/s: 4.0.0 3.4.4 3.5.3 Resolution: Fixed

[jira] [Assigned] (SPARK-48388) [M0] Fix SET behavior for scripts

[ https://issues.apache.org/jira/browse/SPARK-48388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48388: --- Assignee: David Milicevic > [M0] Fix SET behavior for scripts > ---

[jira] [Resolved] (SPARK-48388) [M0] Fix SET behavior for scripts

[ https://issues.apache.org/jira/browse/SPARK-48388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48388. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47272 [https://gith

[jira] [Updated] (SPARK-48934) Python datetime types converted incorrectly for setting timeout in applyInPandasWithState

[ https://issues.apache.org/jira/browse/SPARK-48934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48934: - Fix Version/s: (was: 3.4.4) > Python datetime types converted incorrectly for setting timeou

[jira] [Created] (SPARK-48941) PySparkML: Replace RDD read / write API invocation with Dataframe read / write API

Weichen Xu created SPARK-48941: -- Summary: PySparkML: Replace RDD read / write API invocation with Dataframe read / write API Key: SPARK-48941 URL: https://issues.apache.org/jira/browse/SPARK-48941 Proje

[jira] [Updated] (SPARK-48941) PySparkML: Replace RDD read / write API invocation with Dataframe read / write API

[ https://issues.apache.org/jira/browse/SPARK-48941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48941: --- Labels: pull-request-available (was: ) > PySparkML: Replace RDD read / write API invocation

[jira] [Resolved] (SPARK-48933) Upgrade `protobuf-java` to `3.25.3`

[ https://issues.apache.org/jira/browse/SPARK-48933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48933. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47397 [https://

[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48791: -- Fix Version/s: 3.4.4 > Perf regression due to accumulator registration overhead using > CopyO

[jira] [Created] (SPARK-48942) Reading parquet with Array of Structs of UDTs throws Exception

James Willis created SPARK-48942: Summary: Reading parquet with Array of Structs of UDTs throws Exception Key: SPARK-48942 URL: https://issues.apache.org/jira/browse/SPARK-48942 Project: Spark

[jira] [Commented] (SPARK-48934) Python datetime types converted incorrectly for setting timeout in applyInPandasWithState

[ https://issues.apache.org/jira/browse/SPARK-48934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867173#comment-17867173 ] Kent Yao commented on SPARK-48934: -- Collected this to 3.5.2 > Python datetime types co

[jira] [Commented] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867175#comment-17867175 ] Kent Yao commented on SPARK-48791: -- Collected to 3.5.2 > Perf regression due to accumu

[jira] [Updated] (SPARK-48791) Perf regression due to accumulator registration overhead using CopyOnWriteArrayList

[ https://issues.apache.org/jira/browse/SPARK-48791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48791: - Fix Version/s: 3.5.2 (was: 3.5.3) > Perf regression due to accumulator registrati

[jira] [Updated] (SPARK-48921) ScalaUDF in subquery should run through analyzer

[ https://issues.apache.org/jira/browse/SPARK-48921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48921: - Fix Version/s: 3.5.2 (was: 3.5.3) > ScalaUDF in subquery should run through analy