[jira] [Created] (SPARK-48549) Restrict the number of parameters for function `sentences` to 1 or 3

2024-06-05 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48549: --- Summary: Restrict the number of parameters for function `sentences` to 1 or 3 Key: SPARK-48549 URL: https://issues.apache.org/jira/browse/SPARK-48549 Project: Spark

[jira] [Resolved] (SPARK-47857) Utilize java.sql.RowId.getBytes API directly for UTF8String

2024-06-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47857. -- Resolution: Not A Problem > Utilize java.sql.RowId.getBytes API directly for UTF8String >

[jira] [Resolved] (SPARK-48540) Avoid ivy output loading settings to stdout

2024-06-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48540. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46882 [https://github.com

[jira] [Assigned] (SPARK-48540) Avoid ivy output loading settings to stdout

2024-06-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48540: Assignee: dzcxzl > Avoid ivy output loading settings to stdout >

[jira] [Comment Edited] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2024-06-05 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852555#comment-17852555 ] Yeachan Park edited comment on SPARK-41469 at 6/6/24 6:28 AM:

[jira] [Updated] (SPARK-48539) Upgrade docker-java to 3.3.6

2024-06-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48539: - Parent: SPARK-47361 Issue Type: Sub-task (was: Improvement) > Upgrade docker-java to 3.3.6 > --

[jira] [Updated] (SPARK-48548) Update LICENSE/NOTICE for spark-core with shaded dependencies

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48548: --- Labels: pull-request-available (was: ) > Update LICENSE/NOTICE for spark-core with shaded d

[jira] [Created] (SPARK-48548) Update LICENSE/NOTICE for spark-core with shaded dependencies

2024-06-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-48548: Summary: Update LICENSE/NOTICE for spark-core with shaded dependencies Key: SPARK-48548 URL: https://issues.apache.org/jira/browse/SPARK-48548 Project: Spark Issue

[jira] [Updated] (SPARK-48547) Add opt-in flag to have SparkSubmit automatically call System.exit after user code main method exits

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48547: --- Labels: pull-request-available (was: ) > Add opt-in flag to have SparkSubmit automatically

[jira] [Updated] (SPARK-48546) Fix ExpressionEncoder after replacing NullPointerExceptions with proper error classes in AssertNotNull expression

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48546: --- Labels: pull-request-available (was: ) > Fix ExpressionEncoder after replacing NullPointerE

[jira] [Updated] (SPARK-48547) Add opt-in flag to have SparkSubmit automatically call System.exit after user code main method exits

2024-06-05 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-48547: --- Description: This PR proposes to add a new flag, `spark.submit.callSystemExitOnMainExit` (default f

[jira] [Created] (SPARK-48547) Add opt-in flag to have SparkSubmit automatically call System.exit after user code main method exits

2024-06-05 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-48547: -- Summary: Add opt-in flag to have SparkSubmit automatically call System.exit after user code main method exits Key: SPARK-48547 URL: https://issues.apache.org/jira/browse/SPARK-48547

[jira] [Created] (SPARK-48546) Fix ExpressionEncoder after replacing NullPointerExceptions with proper error classes in AssertNotNull expression

2024-06-05 Thread Daniel (Jira)
Daniel created SPARK-48546: -- Summary: Fix ExpressionEncoder after replacing NullPointerExceptions with proper error classes in AssertNotNull expression Key: SPARK-48546 URL: https://issues.apache.org/jira/browse/SPARK-48

[jira] [Commented] (SPARK-48545) Create from_avro SQL function to match PySpark equivalent

2024-06-05 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852593#comment-17852593 ] Gengliang Wang commented on SPARK-48545: +1 for having such functions (from_avro

[jira] [Created] (SPARK-48545) Create from_avro SQL function to match PySpark equivalent

2024-06-05 Thread Daniel (Jira)
Daniel created SPARK-48545: -- Summary: Create from_avro SQL function to match PySpark equivalent Key: SPARK-48545 URL: https://issues.apache.org/jira/browse/SPARK-48545 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2024-06-05 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852555#comment-17852555 ] Yeachan Park commented on SPARK-41469: -- Maybe a stupid question, but I thought the

[jira] [Updated] (SPARK-48543) Track invalid unsafe row exception explicitly as error class

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48543: --- Labels: pull-request-available (was: ) > Track invalid unsafe row exception explicitly as e

[jira] [Created] (SPARK-48543) Track invalid unsafe row exception explicitly as error class

2024-06-05 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-48543: -- Summary: Track invalid unsafe row exception explicitly as error class Key: SPARK-48543 URL: https://issues.apache.org/jira/browse/SPARK-48543 Project: Spa

[jira] [Created] (SPARK-48542) Give snapshotStartBatchId and snapshotPartitionId to the state data source

2024-06-05 Thread Yuchen Liu (Jira)
Yuchen Liu created SPARK-48542: -- Summary: Give snapshotStartBatchId and snapshotPartitionId to the state data source Key: SPARK-48542 URL: https://issues.apache.org/jira/browse/SPARK-48542 Project: Spark

[jira] [Updated] (SPARK-48541) Assign specific exit code for executors killed by TaskReaper

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48541: --- Labels: pull-request-available (was: ) > Assign specific exit code for executors killed by

[jira] [Updated] (SPARK-48540) Avoid ivy output loading settings to stdout

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48540: --- Labels: pull-request-available (was: ) > Avoid ivy output loading settings to stdout >

[jira] [Created] (SPARK-48540) Avoid ivy output loading settings to stdout

2024-06-05 Thread dzcxzl (Jira)
dzcxzl created SPARK-48540: -- Summary: Avoid ivy output loading settings to stdout Key: SPARK-48540 URL: https://issues.apache.org/jira/browse/SPARK-48540 Project: Spark Issue Type: Bug Com

[jira] [Resolved] (SPARK-48536) Cache user specified schema in applyInPandas and applyInArrow

2024-06-05 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48536. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46877 [https://

[jira] [Created] (SPARK-48539) Upgrade docker-java to 3.3.6

2024-06-05 Thread Wei Guo (Jira)
Wei Guo created SPARK-48539: --- Summary: Upgrade docker-java to 3.3.6 Key: SPARK-48539 URL: https://issues.apache.org/jira/browse/SPARK-48539 Project: Spark Issue Type: Improvement Componen

[jira] [Updated] (SPARK-48538) Avoid memory leak of bonecp as described in HIVE-15551

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48538: --- Labels: pull-request-available (was: ) > Avoid memory leak of bonecp as described in HIVE-1

[jira] [Created] (SPARK-48538) Avoid memory leak of bonecp as described in HIVE-15551

2024-06-05 Thread Kent Yao (Jira)
Kent Yao created SPARK-48538: Summary: Avoid memory leak of bonecp as described in HIVE-15551 Key: SPARK-48538 URL: https://issues.apache.org/jira/browse/SPARK-48538 Project: Spark Issue Type: Bu

[jira] [Assigned] (SPARK-48537) Support `Hive UDAF` inherited from `GenericUDAFResolver` or `GenericUDAFResolver2`

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48537: -- Assignee: (was: Apache Spark) > Support `Hive UDAF` inherited from `GenericUDAFRe

[jira] [Updated] (SPARK-48537) Support `Hive UDAF` inherited from `GenericUDAFResolver` or `GenericUDAFResolver2`

2024-06-05 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-48537: Summary: Support `Hive UDAF` inherited from `GenericUDAFResolver` or `GenericUDAFResolver2` (was:

[jira] [Resolved] (SPARK-48535) Update doc to log warning for join null related config usage

2024-06-05 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48535. -- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-48537) Support Hive UDAF inherit from `GenericUDAFResolver` and `GenericUDAFResolver2`

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48537: --- Labels: pull-request-available (was: ) > Support Hive UDAF inherit from `GenericUDAFResolve

[jira] [Created] (SPARK-48537) Support Hive UDAF inherit from `GenericUDAFResolver` and `GenericUDAFResolver2`

2024-06-05 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48537: --- Summary: Support Hive UDAF inherit from `GenericUDAFResolver` and `GenericUDAFResolver2` Key: SPARK-48537 URL: https://issues.apache.org/jira/browse/SPARK-48537 Project

[jira] [Updated] (SPARK-48508) Client Side RPC optimization for Spark Connect

2024-06-05 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48508: -- Labels: (was: pull-request-available) > Client Side RPC optimization for Spark Connect > ---

[jira] [Updated] (SPARK-48536) Cache user specified schema in applyInPandas and applyInArrow

2024-06-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48536: --- Labels: pull-request-available (was: ) > Cache user specified schema in applyInPandas and a