[jira] [Created] (SPARK-47910) Memory leak when interrupting shuffle write using zstd compression

2024-04-18 Thread JacobZheng (Jira)
JacobZheng created SPARK-47910: -- Summary: Memory leak when interrupting shuffle write using zstd compression Key: SPARK-47910 URL: https://issues.apache.org/jira/browse/SPARK-47910 Project: Spark

[jira] [Created] (SPARK-47146) Possible thread leak when doing sort merge join

2024-02-23 Thread JacobZheng (Jira)
JacobZheng created SPARK-47146: -- Summary: Possible thread leak when doing sort merge join Key: SPARK-47146 URL: https://issues.apache.org/jira/browse/SPARK-47146 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-46588) Interrupt when executing ANALYSIS phase

2024-01-03 Thread JacobZheng (Jira)
JacobZheng created SPARK-46588: -- Summary: Interrupt when executing ANALYSIS phase Key: SPARK-46588 URL: https://issues.apache.org/jira/browse/SPARK-46588 Project: Spark Issue Type: Question

[jira] [Created] (SPARK-46071) TreeNode.toJSON may result in OOM when there are multiple levels of nesting of expressions.

2023-11-23 Thread JacobZheng (Jira)
JacobZheng created SPARK-46071: -- Summary: TreeNode.toJSON may result in OOM when there are multiple levels of nesting of expressions. Key: SPARK-46071 URL: https://issues.apache.org/jira/browse/SPARK-46071

[jira] [Created] (SPARK-45929) support grouping set operation in dataframe api

2023-11-14 Thread JacobZheng (Jira)
JacobZheng created SPARK-45929: -- Summary: support grouping set operation in dataframe api Key: SPARK-45929 URL: https://issues.apache.org/jira/browse/SPARK-45929 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-19 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng resolved SPARK-45601. Fix Version/s: 3.3.0 Resolution: Resolved > stackoverflow when executing rule

[jira] [Commented] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-19 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1536#comment-1536 ] JacobZheng commented on SPARK-45601: Got it, Thanks [~bersprockets] > stackoverflow when executing

[jira] [Updated] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-18 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-45601: --- Description: I am encountering stackoverflow errors while executing the following test case. I

[jira] [Created] (SPARK-45601) stackoverflow when executing rule ExtractWindowExpressions

2023-10-18 Thread JacobZheng (Jira)
JacobZheng created SPARK-45601: -- Summary: stackoverflow when executing rule ExtractWindowExpressions Key: SPARK-45601 URL: https://issues.apache.org/jira/browse/SPARK-45601 Project: Spark Issue

[jira] [Commented] (SPARK-41129) When multiple SQLs are concurrent, the driver subquery thread is permanently locked

2023-08-14 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754048#comment-17754048 ] JacobZheng commented on SPARK-41129: I'm experiencing the exact same thing, not sure if it has

[jira] [Created] (SPARK-42902) CVE-2020-13936 request for upgrading version of Velocity

2023-03-22 Thread JacobZheng (Jira)
JacobZheng created SPARK-42902: -- Summary: CVE-2020-13936 request for upgrading version of Velocity Key: SPARK-42902 URL: https://issues.apache.org/jira/browse/SPARK-42902 Project: Spark Issue

[jira] [Resolved] (SPARK-41500) auto generate concat as Double when string minus an INTERVAL type

2023-03-22 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng resolved SPARK-41500. Resolution: Won't Fix > auto generate concat as Double when string minus an INTERVAL type >

[jira] [Closed] (SPARK-41501) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng closed SPARK-41501. -- Network issues lead to duplicate creation > auto generate concat as Double when string minus an INTERVAL

[jira] [Resolved] (SPARK-41501) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng resolved SPARK-41501. Resolution: Invalid > auto generate concat as Double when string minus an INTERVAL type >

[jira] [Created] (SPARK-41501) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
JacobZheng created SPARK-41501: -- Summary: auto generate concat as Double when string minus an INTERVAL type Key: SPARK-41501 URL: https://issues.apache.org/jira/browse/SPARK-41501 Project: Spark

[jira] [Created] (SPARK-41500) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
JacobZheng created SPARK-41500: -- Summary: auto generate concat as Double when string minus an INTERVAL type Key: SPARK-41500 URL: https://issues.apache.org/jira/browse/SPARK-41500 Project: Spark

[jira] [Resolved] (SPARK-41336) BroadcastExchange does not support the execute() code path. when AQE enabled

2022-12-01 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng resolved SPARK-41336. Fix Version/s: 3.2.2 Resolution: Fixed > BroadcastExchange does not support the execute()

[jira] [Commented] (SPARK-41336) BroadcastExchange does not support the execute() code path. when AQE enabled

2022-12-01 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641843#comment-17641843 ] JacobZheng commented on SPARK-41336: [SPARK-39551][SQL][3.2] Add AQE invalid plan check solved this

[jira] [Commented] (SPARK-41336) BroadcastExchange does not support the execute() code path. when AQE enabled

2022-11-30 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641679#comment-17641679 ] JacobZheng commented on SPARK-41336: This case runs successfully on spark3.3.1. I will check the

[jira] [Created] (SPARK-41336) BroadcastExchange does not support the execute() code path. when AQE enabled

2022-11-30 Thread JacobZheng (Jira)
JacobZheng created SPARK-41336: -- Summary: BroadcastExchange does not support the execute() code path. when AQE enabled Key: SPARK-41336 URL: https://issues.apache.org/jira/browse/SPARK-41336 Project:

[jira] [Updated] (SPARK-38682) Complex calculations with lead to driver oom

2022-03-29 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38682: --- Description: My spark job is working fine in version 3.0.1. After I upgraded to 3.2, the driver

[jira] [Updated] (SPARK-38682) Complex calculations with lead to driver oom

2022-03-29 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38682: --- Attachment: (was: 20220329164645.jpg) > Complex calculations with lead to driver oom >

[jira] [Updated] (SPARK-38682) Complex calculations with lead to driver oom

2022-03-29 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38682: --- Description: My spark job is working fine in version 3.0.1. After I upgraded to 3.2, the driver

[jira] [Updated] (SPARK-38682) Complex calculations with lead to driver oom

2022-03-29 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38682: --- Attachment: 20220329164645.jpg > Complex calculations with lead to driver oom >

[jira] [Updated] (SPARK-38682) Complex calculations with lead to driver oom

2022-03-29 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38682: --- Attachment: screenshot-1.png > Complex calculations with lead to driver oom >

[jira] [Created] (SPARK-38682) Complex calculations with lead to driver oom

2022-03-29 Thread JacobZheng (Jira)
JacobZheng created SPARK-38682: -- Summary: Complex calculations with lead to driver oom Key: SPARK-38682 URL: https://issues.apache.org/jira/browse/SPARK-38682 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

2022-03-06 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502071#comment-17502071 ] JacobZheng commented on SPARK-38429: The hadoop jars inside package is version 3.3.1 > Why the

[jira] [Updated] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

2022-03-06 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38429: --- Description: [https://spark.apache.org/downloads.html]  !11271646623853_.pic.jpg! was:

[jira] [Updated] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

2022-03-06 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38429: --- Issue Type: Question (was: Improvement) > Why the latest release named

[jira] [Updated] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

2022-03-06 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-38429: --- Attachment: 11271646623853_.pic.jpg > Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz

[jira] [Created] (SPARK-38429) Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1

2022-03-06 Thread JacobZheng (Jira)
JacobZheng created SPARK-38429: -- Summary: Why the latest release named spark-3.2.1-bin-hadoop3.2.tgz when I choose hadoop 3.3.1 Key: SPARK-38429 URL: https://issues.apache.org/jira/browse/SPARK-38429

[jira] [Updated] (SPARK-37692) sql-migration-guide wrong description

2021-12-20 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-37692: --- Description: Description in the documentation: {code:java} In Spark 3.2, the unit list interval

[jira] [Created] (SPARK-37692) sql-migration-guide wrong description

2021-12-20 Thread JacobZheng (Jira)
JacobZheng created SPARK-37692: -- Summary: sql-migration-guide wrong description Key: SPARK-37692 URL: https://issues.apache.org/jira/browse/SPARK-37692 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-37128) Application has been removed by master but driver still running

2021-10-28 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435189#comment-17435189 ] JacobZheng commented on SPARK-37128: My Spark version is 3.0.1 and I run Spark standalone on k8s.I

[jira] [Updated] (SPARK-37128) Application has been removed by master but driver still running

2021-10-28 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-37128: --- Description: {code:java} 21/08/30 10:27:31 INFO Master: Removing executor app-20210827190502-0030/1

[jira] [Updated] (SPARK-37128) Application has been removed by master but driver still running

2021-10-28 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-37128: --- Description: {code:java} 21/08/30 10:27:31 INFO Master: Removing executor app-20210827190502-0030/1

[jira] [Created] (SPARK-37128) Application has been removed by master but driver still running

2021-10-27 Thread JacobZheng (Jira)
JacobZheng created SPARK-37128: -- Summary: Application has been removed by master but driver still running Key: SPARK-37128 URL: https://issues.apache.org/jira/browse/SPARK-37128 Project: Spark

[jira] [Commented] (SPARK-36903) oom exception occurred during code generation due to a large number of case when branches

2021-10-08 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426075#comment-17426075 ] JacobZheng commented on SPARK-36903: Yes. The same issue occurs on Spark 3.1. Actually, I'm facing a

[jira] [Updated] (SPARK-36903) oom exception occurred during code generation due to a large number of case when branches

2021-10-07 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-36903: --- Description: I have a spark task that contains many case when branches. When I run it, the driver

[jira] [Updated] (SPARK-36903) oom exception occurred during code generation due to a large number of case when branches

2021-10-07 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng updated SPARK-36903: --- Description: I have a spark task that contains many case when branches. When I run it, the driver

[jira] [Created] (SPARK-36903) oom exception occurred during code generation due to a large number of case when branches

2021-09-30 Thread JacobZheng (Jira)
JacobZheng created SPARK-36903: -- Summary: oom exception occurred during code generation due to a large number of case when branches Key: SPARK-36903 URL: https://issues.apache.org/jira/browse/SPARK-36903