[jira] [Updated] (SPARK-48187) Run `docs` only in PR builders and Java 21 Daily CI

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48187: --- Labels: pull-request-available (was: ) > Run `docs` only in PR builders and Java 21 Daily

[jira] [Updated] (SPARK-48138) Disable a flaky `SparkSessionE2ESuite.interrupt tag` test

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48138: -- Fix Version/s: 3.5.2 > Disable a flaky `SparkSessionE2ESuite.interrupt tag` test >

[jira] [Updated] (SPARK-48139) Re-enable `SparkSessionE2ESuite.interrupt tag`

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48139: -- Affects Version/s: 3.5.2 > Re-enable `SparkSessionE2ESuite.interrupt tag` >

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Fix Version/s: 3.5.2 > SortShuffleWriter lacks shuffle write related metrics resulting in >

[jira] [Updated] (SPARK-48160) XPath expressions (all collations)

2024-05-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48160: - Component/s: SQL (was: Spark Core) > XPath expressions (all collations) >

[jira] [Updated] (SPARK-48158) XML expressions (all collations)

2024-05-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48158: - Component/s: SQL (was: Spark Core) > XML expressions (all collations) >

[jira] [Updated] (SPARK-48159) Datetime expressions (all collations)

2024-05-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48159: - Component/s: SQL (was: Spark Core) > Datetime expressions (all collations)

[jira] [Updated] (SPARK-48157) CSV expressions (all collations)

2024-05-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48157: - Component/s: SQL (was: Spark Core) > CSV expressions (all collations) >

[jira] [Updated] (SPARK-48161) JSON expressions (all collations)

2024-05-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48161: - Component/s: SQL (was: Spark Core) > JSON expressions (all collations) >

[jira] [Updated] (SPARK-48162) Miscellaneous expressions (all collations)

2024-05-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uroš Bojanić updated SPARK-48162: - Component/s: SQL (was: Spark Core) > Miscellaneous expressions (all

[jira] [Created] (SPARK-48186) Add support for AbstractMapType

2024-05-07 Thread Jira
Uroš Bojanić created SPARK-48186: Summary: Add support for AbstractMapType Key: SPARK-48186 URL: https://issues.apache.org/jira/browse/SPARK-48186 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-48183) Update error contribution guide to respect new error class file

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48183. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46455

[jira] [Assigned] (SPARK-48183) Update error contribution guide to respect new error class file

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48183: - Assignee: Haejoon Lee > Update error contribution guide to respect new error class

[jira] [Resolved] (SPARK-47914) Do not display the splits parameter in Rang

2024-05-07 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47914. -- Resolution: Fixed Issue resolved by pull request 46136 [https://github.com/apache/spark/pull/46136]

[jira] [Assigned] (SPARK-47914) Do not display the splits parameter in Rang

2024-05-07 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47914: Assignee: guihuawen > Do not display the splits parameter in Rang >

[jira] [Updated] (SPARK-48185) Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo'

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48185: --- Labels: pull-request-available (was: ) > Fix 'symbolic reference class is not accessible:

[jira] [Created] (SPARK-48185) Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo'

2024-05-07 Thread Kent Yao (Jira)
Kent Yao created SPARK-48185: Summary: Fix 'symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo' Key: SPARK-48185 URL: https://issues.apache.org/jira/browse/SPARK-48185 Project:

[jira] [Updated] (SPARK-48183) Update error contribution guide to respect new error class file

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48183: --- Labels: pull-request-available (was: ) > Update error contribution guide to respect new

[jira] [Created] (SPARK-48183) Update error contribution guide to respect new error class file

2024-05-07 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-48183: --- Summary: Update error contribution guide to respect new error class file Key: SPARK-48183 URL: https://issues.apache.org/jira/browse/SPARK-48183 Project: Spark

[jira] [Updated] (SPARK-47365) Add toArrowTable() DataFrame method to PySpark

2024-05-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Description: Over in the Apache Arrow community, we hear from a lot of users who want to return the

[jira] [Resolved] (SPARK-48126) Make spark.log.structuredLogging.enabled effective

2024-05-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48126. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46452

[jira] [Updated] (SPARK-47365) Add toArrowTable() DataFrame method to PySpark

2024-05-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Summary: Add toArrowTable() DataFrame method to PySpark (was: Add _toArrowTable() DataFrame method to

[jira] [Assigned] (SPARK-48045) Pandas API groupby with multi-agg-relabel ignores as_index=False

2024-05-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48045: Assignee: Saidatt Sinai Amonkar > Pandas API groupby with multi-agg-relabel ignores

[jira] [Resolved] (SPARK-48045) Pandas API groupby with multi-agg-relabel ignores as_index=False

2024-05-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48045. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46391

[jira] [Resolved] (SPARK-48152) Make spark-profiler as a part of release and publish to maven central repo

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48152. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46402

[jira] [Assigned] (SPARK-48152) Make spark-profiler as a part of release and publish to maven central repo

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48152: - Assignee: BingKun Pan > Make spark-profiler as a part of release and publish to maven

[jira] [Assigned] (SPARK-47960) Support Chaining Stateful Operators in TransformWithState

2024-05-07 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47960: Assignee: Bhuwan Sahni > Support Chaining Stateful Operators in TransformWithState >

[jira] [Resolved] (SPARK-47960) Support Chaining Stateful Operators in TransformWithState

2024-05-07 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-47960. -- Resolution: Fixed Issue resolved by pull request 45376

[jira] [Updated] (SPARK-48126) Make spark.log.structuredLogging.enabled effective

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48126: --- Labels: pull-request-available (was: ) > Make spark.log.structuredLogging.enabled

[jira] [Commented] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844471#comment-17844471 ] Daniel commented on SPARK-48180: Fix here: [https://github.com/apache/spark/pull/46451] > Analyzer bug

[jira] [Updated] (SPARK-48126) Make spark.log.structuredLogging.enabled effective

2024-05-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-48126: --- Summary: Make spark.log.structuredLogging.enabled effective (was: Make

[jira] [Updated] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48180: --- Labels: pull-request-available (was: ) > Analyzer bug with multiple ORDER BY items for

[jira] [Updated] (SPARK-48182) SQL (java side): Migrate `error/warn/info` with variables to structured logging framework

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48182: --- Labels: pull-request-available (was: ) > SQL (java side): Migrate `error/warn/info` with

[jira] [Created] (SPARK-48182) SQL (java side): Migrate `error/warn/info` with variables to structured logging framework

2024-05-07 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48182: --- Summary: SQL (java side): Migrate `error/warn/info` with variables to structured logging framework Key: SPARK-48182 URL: https://issues.apache.org/jira/browse/SPARK-48182

[jira] [Updated] (SPARK-48152) Make spark-profiler as a part of release and publish to maven central repo

2024-05-07 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-48152: Summary: Make spark-profiler as a part of release and publish to maven central repo (was:

[jira] [Resolved] (SPARK-48178) Run `build/scala-213/java-11-17` jobs of branch-3.5 only if needed

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48178. --- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 46449

[jira] [Commented] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844467#comment-17844467 ] Daniel commented on SPARK-48180: The bug is that parentheses are required around the two arguments in

[jira] [Assigned] (SPARK-48178) Run `build/scala-213/java-11-17` jobs of branch-3.5 only if needed

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48178: - Assignee: Dongjoon Hyun > Run `build/scala-213/java-11-17` jobs of branch-3.5 only if

[jira] [Created] (SPARK-48181) Unify StreamingPythonRunner and PythonPlannerRunner

2024-05-07 Thread Wei Liu (Jira)
Wei Liu created SPARK-48181: --- Summary: Unify StreamingPythonRunner and PythonPlannerRunner Key: SPARK-48181 URL: https://issues.apache.org/jira/browse/SPARK-48181 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-48178) Run `build/scala-213/java-11-17` jobs of branch-3.5 only if needed

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48178: -- Summary: Run `build/scala-213/java-11-17` jobs of branch-3.5 only if needed (was: Run

[jira] [Resolved] (SPARK-48179) Pin `nbsphinx` to `0.9.3`

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48179. --- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 46448

[jira] [Assigned] (SPARK-48179) Pin `nbsphinx` to `0.9.3`

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48179: - Assignee: Dongjoon Hyun > Pin `nbsphinx` to `0.9.3` > -- > >

[jira] [Updated] (SPARK-48179) Pin `nbsphinx` to `0.9.3`

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48179: --- Labels: pull-request-available (was: ) > Pin `nbsphinx` to `0.9.3` >

[jira] [Created] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
Daniel created SPARK-48180: -- Summary: Analyzer bug with multiple ORDER BY items for input table argument Key: SPARK-48180 URL: https://issues.apache.org/jira/browse/SPARK-48180 Project: Spark

[jira] [Created] (SPARK-48179) Pin `nbsphinx` to `0.9.3`

2024-05-07 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-48179: - Summary: Pin `nbsphinx` to `0.9.3` Key: SPARK-48179 URL: https://issues.apache.org/jira/browse/SPARK-48179 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-48178) Run `build/scala-211/java-11-17` jobs of branch-3.5 only if needed

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48178: --- Labels: pull-request-available (was: ) > Run `build/scala-211/java-11-17` jobs of

[jira] [Created] (SPARK-48178) Run `build/scala-211/java-11-17` jobs of branch-3.5 only if needed

2024-05-07 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-48178: - Summary: Run `build/scala-211/java-11-17` jobs of branch-3.5 only if needed Key: SPARK-48178 URL: https://issues.apache.org/jira/browse/SPARK-48178 Project: Spark

[jira] [Updated] (SPARK-48177) Upgrade `Parquet` to 1.14.0

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48177: -- Summary: Upgrade `Parquet` to 1.14.0 (was: Bump Parquet to 1.14.0) > Upgrade `Parquet` to

[jira] [Updated] (SPARK-48177) Bump Parquet to 1.14.0

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48177: -- Affects Version/s: 4.0.0 (was: 3.5.2) > Bump Parquet to 1.14.0 >

[jira] [Assigned] (SPARK-48177) Bump Parquet to 1.14.0

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48177: - Assignee: Fokko Driesprong > Bump Parquet to 1.14.0 > -- > >

[jira] [Updated] (SPARK-48177) Bump Parquet to 1.14.0

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48177: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Bump Parquet to 1.14.0

[jira] [Updated] (SPARK-48177) Bump Parquet to 1.14.0

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48177: -- Fix Version/s: (was: 4.0.0) > Bump Parquet to 1.14.0 > -- > >

[jira] [Updated] (SPARK-48177) Bump Parquet to 1.14.0

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48177: --- Labels: pull-request-available (was: ) > Bump Parquet to 1.14.0 > -- >

[jira] [Created] (SPARK-48177) Bump Parquet to 1.14.0

2024-05-07 Thread Fokko Driesprong (Jira)
Fokko Driesprong created SPARK-48177: Summary: Bump Parquet to 1.14.0 Key: SPARK-48177 URL: https://issues.apache.org/jira/browse/SPARK-48177 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-48134) Spark core (java side): Migrate `error/warn/info` with variables to structured logging framework

2024-05-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48134. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46390

[jira] [Assigned] (SPARK-48134) Spark core (java side): Migrate `error/warn/info` with variables to structured logging framework

2024-05-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-48134: -- Assignee: BingKun Pan > Spark core (java side): Migrate `error/warn/info` with

[jira] [Created] (SPARK-48176) Fix name of FIELD_ALREADY_EXISTS error condition

2024-05-07 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-48176: Summary: Fix name of FIELD_ALREADY_EXISTS error condition Key: SPARK-48176 URL: https://issues.apache.org/jira/browse/SPARK-48176 Project: Spark

[jira] [Resolved] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48037. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46273

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48037: --- Labels: correctness pull-request-available (was: correctness) > SortShuffleWriter lacks

[jira] [Commented] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844388#comment-17844388 ] Dongjoon Hyun commented on SPARK-48037: --- Thank you, [~dzcxzl]. I raised the priority to `Blocker`

[jira] [Assigned] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48037: - Assignee: dzcxzl > SortShuffleWriter lacks shuffle write related metrics resulting in

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Affects Version/s: 3.4.3 3.5.1 4.0.0 >

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Target Version/s: 4.0.0, 3.5.2, 3.4.4 > SortShuffleWriter lacks shuffle write related metrics

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Labels: correctness (was: pull-request-available) > SortShuffleWriter lacks shuffle write

[jira] [Updated] (SPARK-48037) SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48037: -- Priority: Blocker (was: Major) > SortShuffleWriter lacks shuffle write related metrics

[jira] [Updated] (SPARK-48146) Fix error with aggregate function in With child

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48146: --- Labels: pull-request-available (was: ) > Fix error with aggregate function in With child >

[jira] [Resolved] (SPARK-41547) Reenable ANSI mode in pyspark.sql.tests.connect.test_connect_functions

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41547. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46432

[jira] [Resolved] (SPARK-48169) Use lazy BadRecordException cause for StaxXmlParser and JacksonParser

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48169. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46438

[jira] [Created] (SPARK-48175) Store collation information in metadata and not in type for SER/DE

2024-05-07 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-48175: - Summary: Store collation information in metadata and not in type for SER/DE Key: SPARK-48175 URL: https://issues.apache.org/jira/browse/SPARK-48175 Project: Spark

[jira] [Resolved] (SPARK-48165) Update `ap-loader` to 3.0-9

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48165. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46427

[jira] [Resolved] (SPARK-48173) CheckAnalsis should see the entire query plan

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48173. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46439

[jira] [Assigned] (SPARK-48173) CheckAnalsis should see the entire query plan

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48173: - Assignee: Wenchen Fan > CheckAnalsis should see the entire query plan >

[jira] [Assigned] (SPARK-47297) Format expressions (all collations)

2024-05-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47297: --- Assignee: Uroš Bojanić > Format expressions (all collations) >

[jira] [Resolved] (SPARK-47297) Format expressions (all collations)

2024-05-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47297. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46423

[jira] [Assigned] (SPARK-48171) Clean up the use of deprecated APIs related to `o.rocksdb.Logger`

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48171: - Assignee: Yang Jie > Clean up the use of deprecated APIs related to `o.rocksdb.Logger`

[jira] [Resolved] (SPARK-48171) Clean up the use of deprecated APIs related to `o.rocksdb.Logger`

2024-05-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48171. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46436

[jira] [Updated] (SPARK-47465) Remove experimental tag from toArrowTable() PySpark DataFrame method

2024-05-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47465: - Description: As a follow-up to SPARK-47365: What is needed to consider making the *toArrowTable()*

[jira] [Updated] (SPARK-47466) Add PySpark DataFrame method to return iterator of PyArrow RecordBatches

2024-05-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47466: - Description: As a follow-up to SPARK-47365: *toArrowTable()* is useful when the data is relatively

[jira] [Updated] (SPARK-47465) Remove experimental tag from toArrowTable() PySpark DataFrame method

2024-05-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47465: - Summary: Remove experimental tag from toArrowTable() PySpark DataFrame method (was: Remove

[jira] [Updated] (SPARK-47365) Add _toArrowTable() DataFrame method to PySpark

2024-05-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Summary: Add _toArrowTable() DataFrame method to PySpark (was: Add _toArrow() DataFrame method to

[jira] [Updated] (SPARK-47365) Add _toArrowTable() DataFrame method to PySpark

2024-05-07 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-47365: - Description: Over in the Apache Arrow community, we hear from a lot of users who want to return the

[jira] [Updated] (SPARK-48173) CheckAnalsis should see the entire query plan

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48173: --- Labels: pull-request-available (was: ) > CheckAnalsis should see the entire query plan >

[jira] [Created] (SPARK-48173) CheckAnalsis should see the entire query plan

2024-05-07 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-48173: --- Summary: CheckAnalsis should see the entire query plan Key: SPARK-48173 URL: https://issues.apache.org/jira/browse/SPARK-48173 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-48169) Use lazy BadRecordException cause for StaxXmlParser and JacksonParser

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48169: --- Labels: pull-request-available (was: ) > Use lazy BadRecordException cause for

[jira] [Assigned] (SPARK-48143) UnivocityParser is slow when parsing partially-malformed CSV in PERMISSIVE mode

2024-05-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48143: --- Assignee: Vladimir Golubev > UnivocityParser is slow when parsing partially-malformed CSV

[jira] [Resolved] (SPARK-48143) UnivocityParser is slow when parsing partially-malformed CSV in PERMISSIVE mode

2024-05-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48143. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46400

[jira] [Updated] (SPARK-48172) Fix escaping issue for mysql

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48172: --- Labels: pull-request-available (was: ) > Fix escaping issue for mysql >

[jira] [Created] (SPARK-48172) Fix escaping issue for mysql

2024-05-07 Thread Mihailo Milosevic (Jira)
Mihailo Milosevic created SPARK-48172: - Summary: Fix escaping issue for mysql Key: SPARK-48172 URL: https://issues.apache.org/jira/browse/SPARK-48172 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-48171) Clean up the use of deprecated APIs related to `o.rocksdb.Logger`

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48171: --- Labels: pull-request-available (was: ) > Clean up the use of deprecated APIs related to

[jira] [Created] (SPARK-48171) Clean up the use of deprecated APIs related to `o.rocksdb.Logger`

2024-05-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-48171: Summary: Clean up the use of deprecated APIs related to `o.rocksdb.Logger` Key: SPARK-48171 URL: https://issues.apache.org/jira/browse/SPARK-48171 Project: Spark

[jira] [Updated] (SPARK-48000) Hash join support for strings with collation

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48000: --- Labels: pull-request-available (was: ) > Hash join support for strings with collation >

[jira] [Commented] (SPARK-48123) Provide a constant table schema for querying structured logs

2024-05-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844245#comment-17844245 ] Steve Loughran commented on SPARK-48123: this doesn't handle nested stack traces. I seem to have

[jira] [Updated] (SPARK-41794) Reenable ANSI mode in pyspark.sql.tests.connect.test_connect_column

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41794: --- Labels: pull-request-available (was: ) > Reenable ANSI mode in

[jira] [Resolved] (SPARK-48086) Different Arrow versions in client and server

2024-05-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48086. -- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 46431

[jira] [Assigned] (SPARK-48086) Different Arrow versions in client and server

2024-05-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48086: Assignee: Hyukjin Kwon > Different Arrow versions in client and server >

[jira] [Created] (SPARK-48169) Use lazy BadRecordException cause for StaxXmlParser and JacksonParser

2024-05-07 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-48169: Summary: Use lazy BadRecordException cause for StaxXmlParser and JacksonParser Key: SPARK-48169 URL: https://issues.apache.org/jira/browse/SPARK-48169

[jira] [Updated] (SPARK-41547) Reenable ANSI mode in pyspark.sql.tests.connect.test_connect_functions

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41547: --- Labels: pull-request-available (was: ) > Reenable ANSI mode in

[jira] [Created] (SPARK-48168) Add bitwise shifting operators support

2024-05-07 Thread Kent Yao (Jira)
Kent Yao created SPARK-48168: Summary: Add bitwise shifting operators support Key: SPARK-48168 URL: https://issues.apache.org/jira/browse/SPARK-48168 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-47267) Hash functions should respect collation

2024-05-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47267: --- Assignee: Uroš Bojanić > Hash functions should respect collation >

[jira] [Updated] (SPARK-47267) Hash functions should respect collation

2024-05-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47267: --- Labels: pull-request-available (was: ) > Hash functions should respect collation >

  1   2   >