[jira] [Created] (SPARK-45478) codegen sum(decimal_column / 2) computes div twice

2023-10-10 Thread Zhizhen Hou (Jira)
Zhizhen Hou created SPARK-45478: --- Summary: codegen sum(decimal_column / 2) computes div twice Key: SPARK-45478 URL: https://issues.apache.org/jira/browse/SPARK-45478 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45478) codegen sum(decimal_column / 2) computes div twice

2023-10-10 Thread Zhizhen Hou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhizhen Hou updated SPARK-45478: Description: *The SQL to reproduce*  {code:java} create table t_dec (c1 decimal(6,2)); insert into

[jira] [Commented] (SPARK-45478) codegen sum(decimal_column / 2) computes div twice

2023-10-10 Thread Zhizhen Hou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773579#comment-17773579 ] Zhizhen Hou commented on SPARK-45478: - The new compiled spark 4.0.0-SNAPSHOT from ma

[jira] [Updated] (SPARK-45478) codegen sum(decimal_column / 2) computes div twice

2023-10-10 Thread Zhizhen Hou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhizhen Hou updated SPARK-45478: Description: *The SQL to reproduce the result* {code:java} create table t_dec (c1 decimal(6,2)); i

[jira] [Assigned] (SPARK-45475) Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45475: Assignee: Hyukjin Kwon > Should use DataFrame.foreachPartition instead of RDD.foreachPart

[jira] [Resolved] (SPARK-45475) Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45475. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45479: Summary: Recover Python build in branch-3.3, 3.4 and 3.5 Key: SPARK-45479 URL: https://issues.apache.org/jira/browse/SPARK-45479 Project: Spark Issue Type: I

[jira] [Updated] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45479: --- Labels: pull-request-available (was: ) > Recover Python build in branch-3.3, 3.4 and 3.5 >

[jira] [Created] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread Kent Yao (Jira)
Kent Yao created SPARK-45480: Summary: Selectable SQL Plan Key: SPARK-45480 URL: https://issues.apache.org/jira/browse/SPARK-45480 Project: Spark Issue Type: Improvement Components: SQL

[jira] [Updated] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-10 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45481: --- Description: Currently, StorageLevel provides fromString to get the StorageLevel's instance with Sto

[jira] [Created] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-10 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45481: -- Summary: Introduce a mapper for parquet compression codecs Key: SPARK-45481 URL: https://issues.apache.org/jira/browse/SPARK-45481 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45480: --- Labels: pull-request-available (was: ) > Selectable SQL Plan > --- > >

[jira] [Assigned] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45480: -- Assignee: Apache Spark > Selectable SQL Plan > --- > >

[jira] [Assigned] (SPARK-45356) Optimize the Maven daily test configuration

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45356: -- Assignee: (was: Apache Spark) > Optimize the Maven daily test configuration > ---

[jira] [Assigned] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45480: -- Assignee: (was: Apache Spark) > Selectable SQL Plan > --- > >

[jira] [Updated] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-10 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-45481: --- Description: Currently, Spark supported most of all parquet compression codecs, the parquet support

[jira] [Assigned] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-10 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng reassigned SPARK-45481: -- Assignee: Jiaan Geng > Introduce a mapper for parquet compression codecs > --

[jira] [Assigned] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45480: -- Assignee: (was: Apache Spark) > Selectable SQL Plan > --- > >

[jira] [Assigned] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45480: -- Assignee: Apache Spark > Selectable SQL Plan > --- > >

[jira] [Assigned] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45480: -- Assignee: Apache Spark > Selectable SQL Plan > --- > >

[jira] [Assigned] (SPARK-43664) Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43664: -- Assignee: (was: Apache Spark) > Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests >

[jira] [Assigned] (SPARK-45480) Selectable SQL Plan

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45480: -- Assignee: (was: Apache Spark) > Selectable SQL Plan > --- > >

[jira] [Assigned] (SPARK-43664) Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43664: -- Assignee: Apache Spark > Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests > --

[jira] [Assigned] (SPARK-43664) Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43664: -- Assignee: Apache Spark > Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests > --

[jira] [Assigned] (SPARK-43664) Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-43664: -- Assignee: (was: Apache Spark) > Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests >

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45479: -- Assignee: Apache Spark > Recover Python build in branch-3.3, 3.4 and 3.5 > --

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45479: -- Assignee: (was: Apache Spark) > Recover Python build in branch-3.3, 3.4 and 3.5 >

[jira] [Created] (SPARK-45482) Clean up the usage of `AccessControlContext` and `AccessController`

2023-10-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-45482: Summary: Clean up the usage of `AccessControlContext` and `AccessController` Key: SPARK-45482 URL: https://issues.apache.org/jira/browse/SPARK-45482 Project: Spark

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45479: -- Assignee: Apache Spark > Recover Python build in branch-3.3, 3.4 and 3.5 > --

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45479: -- Assignee: (was: Apache Spark) > Recover Python build in branch-3.3, 3.4 and 3.5 >

[jira] [Updated] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45481: --- Labels: pull-request-available (was: ) > Introduce a mapper for parquet compression codecs

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45479: -- Assignee: Apache Spark > Recover Python build in branch-3.3, 3.4 and 3.5 > --

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45479: -- Assignee: (was: Apache Spark) > Recover Python build in branch-3.3, 3.4 and 3.5 >

[jira] [Assigned] (SPARK-45481) Introduce a mapper for parquet compression codecs

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45481: -- Assignee: Apache Spark (was: Jiaan Geng) > Introduce a mapper for parquet compressio

[jira] [Updated] (SPARK-45482) Clean up the usage of `AccessControlContext` and `AccessController`

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45482: - Description:     {code:java} * @deprecated This class is only useful in conjunction with * {@li

[jira] [Updated] (SPARK-45482) Clean up the usage of `AccessControlContext` and `AccessController`

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45482: - Description:     {code:java} * @deprecated This class is only useful in conjunction with * {@li

[jira] [Updated] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45482: - Summary: Handle the usage of AccessControlContext and AccessController. (was: Clean up the usage of `Ac

[jira] [Commented] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773616#comment-17773616 ] Yang Jie commented on SPARK-45482: -- For case 1, the business can be changed to  {code:j

[jira] [Commented] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773617#comment-17773617 ] Yang Jie commented on SPARK-45482: -- friendly ping [~dongjoon] Do you have any suggestio

[jira] [Comment Edited] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773616#comment-17773616 ] Yang Jie edited comment on SPARK-45482 at 10/10/23 9:38 AM:

[jira] [Created] (SPARK-45483) Correct the function groups in connect.functions

2023-10-10 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-45483: - Summary: Correct the function groups in connect.functions Key: SPARK-45483 URL: https://issues.apache.org/jira/browse/SPARK-45483 Project: Spark Issue Type

[jira] [Updated] (SPARK-45483) Correct the function groups in connect.functions

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45483: --- Labels: pull-request-available (was: ) > Correct the function groups in connect.functions >

[jira] [Updated] (SPARK-45204) Allow CommandPlugins to be trackable

2023-10-10 Thread Robert Dillitz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Dillitz updated SPARK-45204: --- Summary: Allow CommandPlugins to be trackable (was: Extend CommandPlugins to be trackable)

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45479: Assignee: Hyukjin Kwon > Recover Python build in branch-3.3, 3.4 and 3.5 > --

[jira] [Resolved] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45479. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43306 [https://gi

[jira] [Commented] (SPARK-32494) Null Aware Anti Join Optimize Support Multi-Column

2023-10-10 Thread Runkang He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773659#comment-17773659 ] Runkang He commented on SPARK-32494: Hi [~leanken], is this feature available in Spa

[jira] [Created] (SPARK-45484) Fix the bug that uses incorrect parquet compression codec lz4raw

2023-10-10 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45484: -- Summary: Fix the bug that uses incorrect parquet compression codec lz4raw Key: SPARK-45484 URL: https://issues.apache.org/jira/browse/SPARK-45484 Project: Spark

[jira] [Resolved] (SPARK-45213) Assign name to _LEGACY_ERROR_TEMP_2151

2023-10-10 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45213. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43029 [https://github.com

[jira] [Assigned] (SPARK-45213) Assign name to _LEGACY_ERROR_TEMP_2151

2023-10-10 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45213: Assignee: Deng Ziming (was: Haejoon Lee) > Assign name to _LEGACY_ERROR_TEMP_2151 >

[jira] [Updated] (SPARK-45484) Fix the bug that uses incorrect parquet compression codec lz4raw

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45484: --- Labels: pull-request-available (was: ) > Fix the bug that uses incorrect parquet compressio

[jira] [Updated] (SPARK-45204) Allow CommandPlugins to be trackable

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45204: --- Labels: connect pull-request-available scala (was: connect scala) > Allow CommandPlugins to

[jira] [Created] (SPARK-45485) make add_artifact idempotent

2023-10-10 Thread Alice Sayutina (Jira)
Alice Sayutina created SPARK-45485: -- Summary: make add_artifact idempotent Key: SPARK-45485 URL: https://issues.apache.org/jira/browse/SPARK-45485 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45486) [CONNECT][SCALA] Update user agent to recognize SPARK_CONNECT_USER_AGENT and include environment specific attributes

2023-10-10 Thread Robert Dillitz (Jira)
Robert Dillitz created SPARK-45486: -- Summary: [CONNECT][SCALA] Update user agent to recognize SPARK_CONNECT_USER_AGENT and include environment specific attributes Key: SPARK-45486 URL: https://issues.apache.org/j

[jira] [Created] (SPARK-45487) Replace: _LEGACY_ERROR_TEMP_3007

2023-10-10 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-45487: Summary: Replace: _LEGACY_ERROR_TEMP_3007 Key: SPARK-45487 URL: https://issues.apache.org/jira/browse/SPARK-45487 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-45477) Use `matrix.java/inputs.java` to replace the hardcoded Java version in `test results/unit tests log` naming

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45477: - Assignee: Yang Jie > Use `matrix.java/inputs.java` to replace the hardcoded Java versio

[jira] [Resolved] (SPARK-45477) Use `matrix.java/inputs.java` to replace the hardcoded Java version in `test results/unit tests log` naming

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45477. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43305 [https://

[jira] [Updated] (SPARK-45486) [CONNECT][SCALA] Update user agent to recognize SPARK_CONNECT_USER_AGENT and include environment specific attributes

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45486: --- Labels: pull-request-available (was: ) > [CONNECT][SCALA] Update user agent to recognize SP

[jira] [Updated] (SPARK-44076) SPIP: Python Data Source API

2023-10-10 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-44076: - Affects Version/s: 4.0.0 (was: 3.5.0) > SPIP: Python Data Source API

[jira] [Created] (SPARK-45488) XML: Add support for value in 'rowTag' element

2023-10-10 Thread Sandip Agarwala (Jira)
Sandip Agarwala created SPARK-45488: --- Summary: XML: Add support for value in 'rowTag' element Key: SPARK-45488 URL: https://issues.apache.org/jira/browse/SPARK-45488 Project: Spark Issue Ty

[jira] [Updated] (SPARK-45488) XML: Add support for value in 'rowTag' element

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45488: --- Labels: pull-request-available (was: ) > XML: Add support for value in 'rowTag' element > -

[jira] [Created] (SPARK-45489) Replace: _LEGACY_ERROR_TEMP_2134 with a regular error class

2023-10-10 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-45489: Summary: Replace: _LEGACY_ERROR_TEMP_2134 with a regular error class Key: SPARK-45489 URL: https://issues.apache.org/jira/browse/SPARK-45489 Project: Spark

[jira] [Created] (SPARK-45490) Replace: _LEGACY_ERROR_TEMP_2151 with a proper error class

2023-10-10 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-45490: Summary: Replace: _LEGACY_ERROR_TEMP_2151 with a proper error class Key: SPARK-45490 URL: https://issues.apache.org/jira/browse/SPARK-45490 Project: Spark Is

[jira] [Created] (SPARK-45491) Replace: _LEGACY_ERROR_TEMP_2196 with a better error class

2023-10-10 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-45491: Summary: Replace: _LEGACY_ERROR_TEMP_2196 with a better error class Key: SPARK-45491 URL: https://issues.apache.org/jira/browse/SPARK-45491 Project: Spark Is

[jira] [Created] (SPARK-45492) Replace: _LEGACY_ERROR_TEMP_2152 with a better error class

2023-10-10 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-45492: Summary: Replace: _LEGACY_ERROR_TEMP_2152 with a better error class Key: SPARK-45492 URL: https://issues.apache.org/jira/browse/SPARK-45492 Project: Spark Is

[jira] [Created] (SPARK-45493) Replace: _LEGACY_ERROR_TEMP_2187 with a better error message

2023-10-10 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-45493: Summary: Replace: _LEGACY_ERROR_TEMP_2187 with a better error message Key: SPARK-45493 URL: https://issues.apache.org/jira/browse/SPARK-45493 Project: Spark

[jira] [Created] (SPARK-45494) Introduce read/write a byte array util functions for PythonWorkerUtils

2023-10-10 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-45494: - Summary: Introduce read/write a byte array util functions for PythonWorkerUtils Key: SPARK-45494 URL: https://issues.apache.org/jira/browse/SPARK-45494 Project: Spa

[jira] [Updated] (SPARK-45494) Introduce read/write a byte array util functions for PythonWorkerUtils

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45494: --- Labels: pull-request-available (was: ) > Introduce read/write a byte array util functions f

[jira] [Updated] (SPARK-43828) Add config to control whether close idle connection

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43828: --- Labels: pull-request-available (was: ) > Add config to control whether close idle connectio

[jira] [Updated] (SPARK-45495) Support stage level task resource profile for k8s cluster when dynamic allocation disabled

2023-10-10 Thread Bobby Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bobby Wang updated SPARK-45495: --- Description: [https://github.com/apache/spark/pull/37268] has introduced a new feature that support

[jira] [Created] (SPARK-45495) Support stage level task resource profile for k8s cluster when dynamic allocation disabled

2023-10-10 Thread Bobby Wang (Jira)
Bobby Wang created SPARK-45495: -- Summary: Support stage level task resource profile for k8s cluster when dynamic allocation disabled Key: SPARK-45495 URL: https://issues.apache.org/jira/browse/SPARK-45495

[jira] [Resolved] (SPARK-45464) [CORE] Fix yarn distribution build

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45464. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43289 [https://github.com

[jira] [Assigned] (SPARK-45464) [CORE] Fix yarn distribution build

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-45464: Assignee: Hasnain Lakhani > [CORE] Fix yarn distribution build >

[jira] [Assigned] (SPARK-45473) Incorrect error message for RoundBase

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45473: - Assignee: L. C. Hsieh > Incorrect error message for RoundBase > ---

[jira] [Resolved] (SPARK-45473) Incorrect error message for RoundBase

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45473. --- Fix Version/s: 3.5.1 4.0.0 3.4.2 Resolution: Fix

[jira] [Resolved] (SPARK-45416) Sanity check that Spark Connect returns arrow batches in order

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45416. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43219 [https://gi

[jira] [Assigned] (SPARK-45416) Sanity check that Spark Connect returns arrow batches in order

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45416: Assignee: Juliusz Sompolski > Sanity check that Spark Connect returns arrow batches in or

[jira] [Created] (SPARK-45496) Fix the compilation warning related to other-pure-statement

2023-10-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-45496: Summary: Fix the compilation warning related to other-pure-statement Key: SPARK-45496 URL: https://issues.apache.org/jira/browse/SPARK-45496 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-45310) Mapstatus location type changed from external shuffle service to executor after decommission migration

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45310. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43112 [https://

[jira] [Resolved] (SPARK-45494) Introduce read/write a byte array util functions for PythonWorkerUtils

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45494. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43321 [https://

[jira] [Assigned] (SPARK-45494) Introduce read/write a byte array util functions for PythonWorkerUtils

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45494: - Assignee: Takuya Ueshin > Introduce read/write a byte array util functions for PythonWo

[jira] [Updated] (SPARK-45468) More transparent proxy handling for HTTP redirects

2023-10-10 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-45468: - Description: Currently, proxies can be made transparent for hyperlinks in Spark web UIs

[jira] [Comment Edited] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773892#comment-17773892 ] Dongjoon Hyun edited comment on SPARK-45482 at 10/11/23 4:15 AM: -

[jira] [Commented] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773892#comment-17773892 ] Dongjoon Hyun commented on SPARK-45482: --- Actually, I'm not sure about those three

[jira] [Updated] (SPARK-41730) `min` fails on the minimal timestamp

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41730: --- Labels: pull-request-available (was: ) > `min` fails on the minimal timestamp > ---

[jira] [Updated] (SPARK-42881) Codegen Support for get_json_object

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42881: --- Labels: pull-request-available (was: ) > Codegen Support for get_json_object >

[jira] [Created] (SPARK-45497) Add an symbolic link file `spark-examples.jar` in K8s Docker images

2023-10-10 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45497: - Summary: Add an symbolic link file `spark-examples.jar` in K8s Docker images Key: SPARK-45497 URL: https://issues.apache.org/jira/browse/SPARK-45497 Project: Spark

[jira] [Updated] (SPARK-45497) Add an symbolic link file `spark-examples.jar` in K8s Docker images

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45497: --- Labels: pull-request-available (was: ) > Add an symbolic link file `spark-examples.jar` in

[jira] [Updated] (SPARK-45497) Add a symbolic link file `spark-examples.jar` in K8s Docker images

2023-10-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45497: -- Summary: Add a symbolic link file `spark-examples.jar` in K8s Docker images (was: Add an symb

[jira] [Resolved] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45482. -- Resolution: Won't Fix > Handle the usage of AccessControlContext and AccessController. > -

[jira] [Commented] (SPARK-45482) Handle the usage of AccessControlContext and AccessController.

2023-10-10 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773897#comment-17773897 ] Yang Jie commented on SPARK-45482: -- OK, Let me close this one > Handle the usage of Ac

[jira] [Created] (SPARK-45499) Replace `Reference#isEnqueued` with `Reference#refersTo`

2023-10-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-45499: Summary: Replace `Reference#isEnqueued` with `Reference#refersTo` Key: SPARK-45499 URL: https://issues.apache.org/jira/browse/SPARK-45499 Project: Spark Issue Type:

[jira] [Created] (SPARK-45498) Followup: Ignore task completion from old stage after retrying indeterminate stages

2023-10-10 Thread Mayur Bhosale (Jira)
Mayur Bhosale created SPARK-45498: - Summary: Followup: Ignore task completion from old stage after retrying indeterminate stages Key: SPARK-45498 URL: https://issues.apache.org/jira/browse/SPARK-45498

[jira] [Updated] (SPARK-45499) Replace `Reference#isEnqueued` with `Reference#refersTo`

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45499: --- Labels: pull-request-available (was: ) > Replace `Reference#isEnqueued` with `Reference#ref

[jira] [Updated] (SPARK-14745) CEP support in Spark Streaming

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-14745: --- Labels: pull-request-available (was: ) > CEP support in Spark Streaming > -

[jira] [Updated] (SPARK-45498) Followup: Ignore task completion from old stage after retrying indeterminate stages

2023-10-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45498: --- Labels: pull-request-available (was: ) > Followup: Ignore task completion from old stage af

[jira] [Commented] (SPARK-45478) codegen sum(decimal_column / 2) computes div twice

2023-10-10 Thread Zhizhen Hou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773921#comment-17773921 ] Zhizhen Hou commented on SPARK-45478: - There are three children in If: predicate, tr

[jira] [Created] (SPARK-45500) Show the number of abnormally completed drivers in MasterPage

2023-10-10 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45500: - Summary: Show the number of abnormally completed drivers in MasterPage Key: SPARK-45500 URL: https://issues.apache.org/jira/browse/SPARK-45500 Project: Spark