[jira] [Resolved] (SPARK-44390) Rename SparkSerDerseUtils to SparkSerDeUtils

2023-07-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-44390. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41960

[jira] [Assigned] (SPARK-44390) Rename SparkSerDerseUtils to SparkSerDeUtils

2023-07-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-44390: Assignee: Yang Jie > Rename SparkSerDerseUtils to SparkSerDeUtils >

[jira] [Updated] (SPARK-44407) Prohibit using `enum` as a variable or function name

2023-07-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44407: - Description: {code:java} [warn]

[jira] [Created] (SPARK-44407) Prohibit using `enum` as a variable or function name

2023-07-12 Thread Yang Jie (Jira)
Yang Jie created SPARK-44407: Summary: Prohibit using `enum` as a variable or function name Key: SPARK-44407 URL: https://issues.apache.org/jira/browse/SPARK-44407 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43964) Support arrow-optimized Python UDTFs

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43964. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41867

[jira] [Assigned] (SPARK-43964) Support arrow-optimized Python UDTFs

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43964: Assignee: Allison Wang > Support arrow-optimized Python UDTFs >

[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec

2023-07-12 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Summary: Define the computing logic through PartitionEvaluator API and use it in

[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators

2023-07-12 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate

[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, DeserializeToObjectExec

2023-07-12 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Description: Use  PartitionEvaluator API in AggregateInPandasExec EvalPythonExec

[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, DeserializeToObjectExec

2023-07-12 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Summary: Use PartitionEvaluator API in

[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, SortExec

2023-07-12 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Summary: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec,

[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec, SortExec

2023-07-12 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Description: Use PartitionEvaluator API in  CollectMetricsExec GenerateExec ExpandExec DebugExec

[jira] [Updated] (SPARK-44406) DataFrame depending on temp view fail after the view is dropped

2023-07-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44406: -- Summary: DataFrame depending on temp view fail after the view is dropped (was: DataFrame

[jira] [Created] (SPARK-44406) DataFrame depending on temp view fail after the view is drop

2023-07-12 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-44406: - Summary: DataFrame depending on temp view fail after the view is drop Key: SPARK-44406 URL: https://issues.apache.org/jira/browse/SPARK-44406 Project: Spark

[jira] [Assigned] (SPARK-44387) The execution time of "Test encryption" is too long after SPARK-44275

2023-07-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-44387: Assignee: Yang Jie > The execution time of "Test encryption" is too long after SPARK-44275 >

[jira] [Resolved] (SPARK-44387) The execution time of "Test encryption" is too long after SPARK-44275

2023-07-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-44387. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41956

[jira] [Resolved] (SPARK-44393) Upgrade H2 from 2.1.214 to 2.2.220

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44393. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41963

[jira] [Assigned] (SPARK-44393) Upgrade H2 from 2.1.214 to 2.2.220

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44393: Assignee: Bjørn Jørgensen > Upgrade H2 from 2.1.214 to 2.2.220 >

[jira] [Resolved] (SPARK-44399) Import SparkSession in Python UDF only when useArrow is None

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44399. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41970

[jira] [Assigned] (SPARK-44399) Import SparkSession in Python UDF only when useArrow is None

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44399: Assignee: Xinrong Meng > Import SparkSession in Python UDF only when useArrow is None >

[jira] [Updated] (SPARK-44405) Reduce code duplication in group-based DELETE and MERGE tests

2023-07-12 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-44405: - Summary: Reduce code duplication in group-based DELETE and MERGE tests (was: Reduce

[jira] [Created] (SPARK-44405) Reduce code duplication in row-level tests

2023-07-12 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-44405: Summary: Reduce code duplication in row-level tests Key: SPARK-44405 URL: https://issues.apache.org/jira/browse/SPARK-44405 Project: Spark Issue

[jira] [Created] (SPARK-44404) Assign names to the error class _LEGACY_ERROR_TEMP_[1009,1010,1013,1015,1016,1278]

2023-07-12 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-44404: --- Summary: Assign names to the error class _LEGACY_ERROR_TEMP_[1009,1010,1013,1015,1016,1278] Key: SPARK-44404 URL: https://issues.apache.org/jira/browse/SPARK-44404

[jira] [Assigned] (SPARK-44397) Expose assertDataFrameEqual in pyspark.testing.utils

2023-07-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44397: - Assignee: Amanda Liu > Expose assertDataFrameEqual in pyspark.testing.utils >

[jira] [Resolved] (SPARK-44397) Expose assertDataFrameEqual in pyspark.testing.utils

2023-07-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44397. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41967

[jira] [Created] (SPARK-44403) Add docs for MergeRows node

2023-07-12 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-44403: Summary: Add docs for MergeRows node Key: SPARK-44403 URL: https://issues.apache.org/jira/browse/SPARK-44403 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44401) Arrow Python UDF Use Guide

2023-07-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-44401: - Component/s: Documentation > Arrow Python UDF Use Guide > -- > >

[jira] [Assigned] (SPARK-44348) Reenable Session-based artifact test cases

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44348: Assignee: Hyukjin Kwon > Reenable Session-based artifact test cases >

[jira] [Resolved] (SPARK-44348) Reenable Session-based artifact test cases

2023-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44348. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41942

[jira] [Created] (SPARK-44402) Add tests for schema pruning in delta-based DELETEs

2023-07-12 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-44402: Summary: Add tests for schema pruning in delta-based DELETEs Key: SPARK-44402 URL: https://issues.apache.org/jira/browse/SPARK-44402 Project: Spark

[jira] [Created] (SPARK-44401) Arrow Python UDF Use Guide

2023-07-12 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-44401: Summary: Arrow Python UDF Use Guide Key: SPARK-44401 URL: https://issues.apache.org/jira/browse/SPARK-44401 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44400) Improve Scala StreamingQueryListener to provide users a way to access the Spark session for Spark Connect

2023-07-12 Thread Bo Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Gao updated SPARK-44400: --- Description: Improve the Listener to provide users a way to access the Spark session and perform arbitrary

[jira] [Created] (SPARK-44400) Improve Scala StreamingQueryListener to provide users a way to access the Spark session for Spark Connect

2023-07-12 Thread Bo Gao (Jira)
Bo Gao created SPARK-44400: -- Summary: Improve Scala StreamingQueryListener to provide users a way to access the Spark session for Spark Connect Key: SPARK-44400 URL: https://issues.apache.org/jira/browse/SPARK-44400

[jira] [Resolved] (SPARK-42981) Add direct arrow serialization

2023-07-12 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42981. --- Fix Version/s: 3.5.0 Resolution: Fixed > Add direct arrow serialization >

[jira] [Created] (SPARK-44399) Import SparkSession in Python UDF only when useArrow is None

2023-07-12 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-44399: Summary: Import SparkSession in Python UDF only when useArrow is None Key: SPARK-44399 URL: https://issues.apache.org/jira/browse/SPARK-44399 Project: Spark

[jira] [Created] (SPARK-44398) Scala foreachBatch API in Streaming Spark Connect

2023-07-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44398: Summary: Scala foreachBatch API in Streaming Spark Connect Key: SPARK-44398 URL: https://issues.apache.org/jira/browse/SPARK-44398 Project: Spark Issue

[jira] [Updated] (SPARK-44279) Upgrade optionator to ^0.9.3

2023-07-12 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-44279: Summary: Upgrade optionator to ^0.9.3 (was: Upgrade Eslint from 7.25.0 to 8.44.0) >

[jira] [Created] (SPARK-44397) Expose assertDataFrameEqual in pyspark.testing.utils

2023-07-12 Thread Amanda Liu (Jira)
Amanda Liu created SPARK-44397: -- Summary: Expose assertDataFrameEqual in pyspark.testing.utils Key: SPARK-44397 URL: https://issues.apache.org/jira/browse/SPARK-44397 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44378) Jobs that have join & have .rdd calls get executed 2x when AQE is enabled.

2023-07-12 Thread Priyanka Raju (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanka Raju updated SPARK-44378: -- Issue Type: Bug (was: Question) > Jobs that have join & have .rdd calls get executed 2x when

[jira] [Created] (SPARK-44396) Add direct Arrow deserialization

2023-07-12 Thread Jira
Herman van Hövell created SPARK-44396: - Summary: Add direct Arrow deserialization Key: SPARK-44396 URL: https://issues.apache.org/jira/browse/SPARK-44396 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-44392) Add tests for schema pruning in delta-based UPDATEs

2023-07-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44392. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41962

[jira] [Assigned] (SPARK-44392) Add tests for schema pruning in delta-based UPDATEs

2023-07-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44392: - Assignee: Anton Okolnychyi > Add tests for schema pruning in delta-based UPDATEs >

[jira] [Updated] (SPARK-44389) ExecutorDeadException when using decommissioning without external shuffle service

2023-07-12 Thread Volodymyr Kot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Kot updated SPARK-44389: -- Description: Hey, we are trying to use executor decommissioning without external shuffle

[jira] [Commented] (SPARK-44059) Add named argument support for SQL functions

2023-07-12 Thread ci-cassandra.apache.org (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742499#comment-17742499 ] ci-cassandra.apache.org commented on SPARK-44059: - User 'learningchess2003' has created

[jira] [Commented] (SPARK-44217) Allow custom precision for fp approx equality

2023-07-12 Thread ci-cassandra.apache.org (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742495#comment-17742495 ] ci-cassandra.apache.org commented on SPARK-44217: - User 'asl3' has created a pull

[jira] [Commented] (SPARK-44395) Update table function arguments to require parentheses around identifier after the TABLE keyword

2023-07-12 Thread ci-cassandra.apache.org (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742496#comment-17742496 ] ci-cassandra.apache.org commented on SPARK-44395: - User 'dtenedor' has created a pull

[jira] [Created] (SPARK-44395) Update table function arguments to require parentheses around identifier after the TABLE keyword

2023-07-12 Thread Daniel (Jira)
Daniel created SPARK-44395: -- Summary: Update table function arguments to require parentheses around identifier after the TABLE keyword Key: SPARK-44395 URL: https://issues.apache.org/jira/browse/SPARK-44395

[jira] [Created] (SPARK-44394) Add a new Spark UI page for Spark Connect

2023-07-12 Thread Jason Li (Jira)
Jason Li created SPARK-44394: Summary: Add a new Spark UI page for Spark Connect Key: SPARK-44394 URL: https://issues.apache.org/jira/browse/SPARK-44394 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-44393) Upgrade H2 from 2.1.214 to 2.2.220

2023-07-12 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-44393: Description: [CVE-2022-45868|https://nvd.nist.gov/vuln/detail/CVE-2022-45868] > Upgrade

[jira] [Updated] (SPARK-44279) Upgrade Eslint from 7.25.0 to 8.44.0

2023-07-12 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-44279: Summary: Upgrade Eslint from 7.25.0 to 8.44.0 (was: Upgrade word-wrap) > Upgrade Eslint

[jira] [Created] (SPARK-44393) Upgrade H2 from 2.1.214 to 2.2.220

2023-07-12 Thread Jira
Bjørn Jørgensen created SPARK-44393: --- Summary: Upgrade H2 from 2.1.214 to 2.2.220 Key: SPARK-44393 URL: https://issues.apache.org/jira/browse/SPARK-44393 Project: Spark Issue Type:

[jira] [Commented] (SPARK-43995) Implement UDFRegistration

2023-07-12 Thread ci-cassandra.apache.org (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742490#comment-17742490 ] ci-cassandra.apache.org commented on SPARK-43995: - User 'vicennial' has created a pull

[jira] [Created] (SPARK-44392) Add tests for schema pruning in delta-based UPDATEs

2023-07-12 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-44392: Summary: Add tests for schema pruning in delta-based UPDATEs Key: SPARK-44392 URL: https://issues.apache.org/jira/browse/SPARK-44392 Project: Spark

[jira] [Created] (SPARK-44391) `url_decode` can fail w/ an internal error

2023-07-12 Thread Max Gekk (Jira)
Max Gekk created SPARK-44391: Summary: `url_decode` can fail w/ an internal error Key: SPARK-44391 URL: https://issues.apache.org/jira/browse/SPARK-44391 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-44389) ExecutorDeadException when using decommissioning without external shuffle service

2023-07-12 Thread Volodymyr Kot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742473#comment-17742473 ] Volodymyr Kot commented on SPARK-44389: --- [~dongjoon] [~Ngone51] [~warrenzhu25] [~mridulm80]

[jira] [Updated] (SPARK-44390) Rename SparkSerDerseUtils to SparkSerDeUtils

2023-07-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44390: - Component/s: (was: Connect) > Rename SparkSerDerseUtils to SparkSerDeUtils >

[jira] [Created] (SPARK-44390) Rename SparkSerDerseUtils to SparkSerDeUtils

2023-07-12 Thread Yang Jie (Jira)
Yang Jie created SPARK-44390: Summary: Rename SparkSerDerseUtils to SparkSerDeUtils Key: SPARK-44390 URL: https://issues.apache.org/jira/browse/SPARK-44390 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-44389) ExecutorDeadException when using decommissioning without external shuffle service

2023-07-12 Thread Volodymyr Kot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742468#comment-17742468 ] Volodymyr Kot commented on SPARK-44389: --- Please let me know if there any other information that

[jira] [Created] (SPARK-44389) ExecutorDeadException when using decommissioning without external shuffle service

2023-07-12 Thread Volodymyr Kot (Jira)
Volodymyr Kot created SPARK-44389: - Summary: ExecutorDeadException when using decommissioning without external shuffle service Key: SPARK-44389 URL: https://issues.apache.org/jira/browse/SPARK-44389

[jira] [Updated] (SPARK-44388) Using an updated instance of ScalarUserDefinedFunction causes protobuf cast failures on server

2023-07-12 Thread Venkata Sai Akhil Gudesa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Sai Akhil Gudesa updated SPARK-44388: - Epic Link: SPARK-42554 > Using an updated instance of

[jira] [Created] (SPARK-44388) Using an updated instance of ScalarUserDefinedFunction causes protobuf cast failures on server

2023-07-12 Thread Venkata Sai Akhil Gudesa (Jira)
Venkata Sai Akhil Gudesa created SPARK-44388: Summary: Using an updated instance of ScalarUserDefinedFunction causes protobuf cast failures on server Key: SPARK-44388 URL:

[jira] [Created] (SPARK-44387) The execution time of "Test encryption" is too long after SPARK-44275

2023-07-12 Thread Yang Jie (Jira)
Yang Jie created SPARK-44387: Summary: The execution time of "Test encryption" is too long after SPARK-44275 Key: SPARK-44387 URL: https://issues.apache.org/jira/browse/SPARK-44387 Project: Spark

[jira] [Commented] (SPARK-44385) Use PartitionEvaluator API in MergingSessionsExec & UpdatingSessionsExec

2023-07-12 Thread Jia Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742402#comment-17742402 ] Jia Fan commented on SPARK-44385: - I'm working on it. > Use PartitionEvaluator API in

[jira] [Commented] (SPARK-44386) Use PartitionEvaluator API in HashAggregateExec, ObjectHashAggregateExec, SortAggregateExec

2023-07-12 Thread Jia Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742403#comment-17742403 ] Jia Fan commented on SPARK-44386: - I'm working on it. > Use PartitionEvaluator API in

[jira] [Created] (SPARK-44386) Use PartitionEvaluator API in HashAggregateExec, ObjectHashAggregateExec, SortAggregateExec

2023-07-12 Thread Jia Fan (Jira)
Jia Fan created SPARK-44386: --- Summary: Use PartitionEvaluator API in HashAggregateExec, ObjectHashAggregateExec, SortAggregateExec Key: SPARK-44386 URL: https://issues.apache.org/jira/browse/SPARK-44386

[jira] [Created] (SPARK-44385) Use PartitionEvaluator API in MergingSessionsExec & UpdatingSessionsExec

2023-07-12 Thread Jia Fan (Jira)
Jia Fan created SPARK-44385: --- Summary: Use PartitionEvaluator API in MergingSessionsExec & UpdatingSessionsExec Key: SPARK-44385 URL: https://issues.apache.org/jira/browse/SPARK-44385 Project: Spark

[jira] [Created] (SPARK-44384) Use checkError() to check Exception in *View*Suite, *Namespace*Suite, *DataSource*Suite

2023-07-12 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-44384: --- Summary: Use checkError() to check Exception in *View*Suite, *Namespace*Suite, *DataSource*Suite Key: SPARK-44384 URL: https://issues.apache.org/jira/browse/SPARK-44384

[jira] [Commented] (SPARK-43974) Upgrade buf to v1.23.1

2023-07-12 Thread GridGain Integration (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742352#comment-17742352 ] GridGain Integration commented on SPARK-43974: -- User 'panbingkun' has created a pull

[jira] [Updated] (SPARK-44383) Fix the trim logic did't handle ASCII control characters correctly

2023-07-12 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-44383: - Fix Version/s: 3.4.1 (was: 3.4.2) > Fix the trim logic did't handle ASCII

[jira] [Resolved] (SPARK-44383) Fix the trim logic did't handle ASCII control characters correctly

2023-07-12 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44383. -- Fix Version/s: 3.5.0 3.4.2 Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-44383) Fix the trim logic did't handle ASCII control characters correctly

2023-07-12 Thread Jira
王俊博 created SPARK-44383: --- Summary: Fix the trim logic did't handle ASCII control characters correctly Key: SPARK-44383 URL: https://issues.apache.org/jira/browse/SPARK-44383 Project: Spark Issue

[jira] [Commented] (SPARK-42981) Add direct arrow serialization

2023-07-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742334#comment-17742334 ] ASF GitHub Bot commented on SPARK-42981: User 'hvanhovell' has created a pull request for this

[jira] [Commented] (SPARK-42981) Add direct arrow serialization

2023-07-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742333#comment-17742333 ] ASF GitHub Bot commented on SPARK-42981: User 'hvanhovell' has created a pull request for this

[jira] [Commented] (SPARK-44375) Use PartitionEvaluator API in DebugExec

2023-07-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742326#comment-17742326 ] ASF GitHub Bot commented on SPARK-44375: User 'Hisoka-X' has created a pull request for this

[jira] [Commented] (SPARK-43781) IllegalStateException when cogrouping two datasets derived from the same source

2023-07-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742325#comment-17742325 ] ASF GitHub Bot commented on SPARK-43781: User 'Hisoka-X' has created a pull request for this

[jira] [Comment Edited] (SPARK-43907) Add SQL functions into Scala, Python and R API

2023-07-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742320#comment-17742320 ] Ruifeng Zheng edited comment on SPARK-43907 at 7/12/23 8:45 AM: I think

[jira] [Commented] (SPARK-43907) Add SQL functions into Scala, Python and R API

2023-07-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742320#comment-17742320 ] Ruifeng Zheng commented on SPARK-43907: --- I think we have finish the work to add missing SQL

[jira] [Assigned] (SPARK-44349) Add math functions to SparkR

2023-07-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44349: - Assignee: Ruifeng Zheng > Add math functions to SparkR >

[jira] [Resolved] (SPARK-44349) Add math functions to SparkR

2023-07-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44349. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41914

[jira] [Created] (SPARK-44382) Upgrade protobuf-java to 3.23.4

2023-07-12 Thread Yang Jie (Jira)
Yang Jie created SPARK-44382: Summary: Upgrade protobuf-java to 3.23.4 Key: SPARK-44382 URL: https://issues.apache.org/jira/browse/SPARK-44382 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2023-07-12 Thread Pratik Malani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742006#comment-17742006 ] Pratik Malani edited comment on SPARK-33782 at 7/12/23 7:01 AM: Hi

[jira] [Updated] (SPARK-44381) How to specify parameters in spark-sumbit to make HiveDelegationTokenProvider refresh token regularly

2023-07-12 Thread qingbo jiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qingbo jiao updated SPARK-44381: Description: export KRB5CCNAME=FILE:/tmp/krb5cc_1001 ./bin/spark-submit -{-}master yarn

[jira] [Commented] (SPARK-44381) How to specify parameters in spark-sumbit to make HiveDelegationTokenProvider refresh token regularly

2023-07-12 Thread qingbo jiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742293#comment-17742293 ] qingbo jiao commented on SPARK-44381: - [~jshao] please cc ,thanks > How to specify parameters in

[jira] [Resolved] (SPARK-44353) Remove toAttributes from StructType

2023-07-12 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44353. --- Fix Version/s: 3.5.0 Resolution: Fixed > Remove toAttributes from StructType