[jira] [Updated] (SPARK-48396) Support configuring limit control for SQL to use maximum cores

[ https://issues.apache.org/jira/browse/SPARK-48396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-48396: - Description: When there is a long-running shared Spark SQL cluster, there may be a situation where a large SQL

[jira] [Created] (SPARK-48396) Support configuring limit control for SQL to use maximum cores

Mars created SPARK-48396: Summary: Support configuring limit control for SQL to use maximum cores Key: SPARK-48396 URL: https://issues.apache.org/jira/browse/SPARK-48396 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46710) Clean up the broadcast data generated when sql execution ends

[ https://issues.apache.org/jira/browse/SPARK-46710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-46710: - Description: Broadcast data cleaning can only rely on cleaning when GC is triggered, which may lead to a lot

[jira] [Updated] (SPARK-46710) Clean up the broadcast data generated when sql execution ends

[ https://issues.apache.org/jira/browse/SPARK-46710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-46710: - Description: Broadcast data cleaning can only rely on cleaning when GC is triggered, which may lead to a lot

[jira] [Updated] (SPARK-46710) Clean up the broadcast data generated when sql execution ends

[ https://issues.apache.org/jira/browse/SPARK-46710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-46710: - Summary: Clean up the broadcast data generated when sql execution ends (was: Clean up the broadcast data

[jira] [Updated] (SPARK-42388) Avoid unnecessary parquet footer reads when no filters in vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-42388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-42388: - Description: Parquet footer is now read twice even if there are no filters requiring pushdown in vectorized

[jira] [Updated] (SPARK-42388) Avoid unnecessary parquet footer reads when no filters in vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-42388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-42388: - Summary: Avoid unnecessary parquet footer reads when no filters in vectorized reader (was: Avoid unnecessary

[jira] [Updated] (SPARK-42388) Avoid unnecessary parquet footer reads when no filters in vectorized parquet reader

[ https://issues.apache.org/jira/browse/SPARK-42388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-42388: - Summary: Avoid unnecessary parquet footer reads when no filters in vectorized parquet reader (was: Avoid

[jira] [Created] (SPARK-42388) Avoid unnecessary parquet footer reads when no filters

Mars created SPARK-42388: Summary: Avoid unnecessary parquet footer reads when no filters Key: SPARK-42388 URL: https://issues.apache.org/jira/browse/SPARK-42388 Project: Spark Issue Type:

[jira] [Created] (SPARK-42387) Avoid unnecessary parquet footer reads when no filters

Mars created SPARK-42387: Summary: Avoid unnecessary parquet footer reads when no filters Key: SPARK-42387 URL: https://issues.apache.org/jira/browse/SPARK-42387 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38005) Support cleaning up merged shuffle files and state from external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-38005: - Fix Version/s: 3.4.0 > Support cleaning up merged shuffle files and state from external shuffle > service >

[jira] [Resolved] (SPARK-38005) Support cleaning up merged shuffle files and state from external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars resolved SPARK-38005. -- Resolution: Fixed > Support cleaning up merged shuffle files and state from external shuffle > service >

[jira] (SPARK-41470) SPJ: Spark shouldn't assume InternalRow implements equals and hashCode

[ https://issues.apache.org/jira/browse/SPARK-41470 ] Mars deleted comment on SPARK-41470: -- was (Author: JIRAUSER290821): [~csun] I want to fix it ~ > SPJ: Spark shouldn't assume InternalRow implements equals and hashCode >

[jira] [Comment Edited] (SPARK-41470) SPJ: Spark shouldn't assume InternalRow implements equals and hashCode

[ https://issues.apache.org/jira/browse/SPARK-41470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17679059#comment-17679059 ] Mars edited comment on SPARK-41470 at 1/20/23 8:43 AM: --- [~csun] I want to fix it ~

[jira] [Commented] (SPARK-41470) SPJ: Spark shouldn't assume InternalRow implements equals and hashCode

[ https://issues.apache.org/jira/browse/SPARK-41470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17679059#comment-17679059 ] Mars commented on SPARK-41470: -- [~csun] I want to take it ~ > SPJ: Spark shouldn't assume InternalRow

[jira] (SPARK-41471) SPJ: Reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning

[ https://issues.apache.org/jira/browse/SPARK-41471 ] Mars deleted comment on SPARK-41471: -- was (Author: JIRAUSER290821): [~csun] Hi, I want to take it :) > SPJ: Reduce Spark shuffle when only one side of a join is > KeyGroupedPartitioning >

[jira] [Commented] (SPARK-41471) SPJ: Reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning

[ https://issues.apache.org/jira/browse/SPARK-41471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17652484#comment-17652484 ] Mars commented on SPARK-41471: -- [~csun] Hi, I want to take it :) > SPJ: Reduce Spark shuffle when only one

[jira] [Updated] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-41365: - Description: My environment CDH 5.8 , click to enter the spark UI from the yarn interface when visit the stage

[jira] [Updated] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-41365: - Description: My environment CDH 5.8 , click to enter the spark UI from the yarn interface when visit the stage

[jira] [Updated] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

[ https://issues.apache.org/jira/browse/SPARK-41365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-41365: - Attachment: image-2022-12-02-17-53-03-003.png > Stages UI page fails to load for proxy in some yarn versions >

[jira] [Created] (SPARK-41365) Stages UI page fails to load for proxy in some yarn versions

Mars created SPARK-41365: Summary: Stages UI page fails to load for proxy in some yarn versions Key: SPARK-41365 URL: https://issues.apache.org/jira/browse/SPARK-41365 Project: Spark Issue Type:

[jira] (SPARK-37313) Child stage using merged output or not should be based on the availability of merged output from parent stage

[ https://issues.apache.org/jira/browse/SPARK-37313 ] Mars deleted comment on SPARK-37313: -- was (Author: JIRAUSER290821): as comment said [https://github.com/apache/spark/pull/34461#issuecomment-964557253] I'm working on this Issue and trying to

[jira] [Commented] (SPARK-38093) Set shuffleMergeAllowed to false for a determinate stage after the stage is finalized

[ https://issues.apache.org/jira/browse/SPARK-38093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635786#comment-17635786 ] Mars commented on SPARK-38093: -- comment https://github.com/apache/spark/pull/34122#discussion_r796929787 >

[jira] [Commented] (SPARK-37313) Child stage using merged output or not should be based on the availability of merged output from parent stage

[ https://issues.apache.org/jira/browse/SPARK-37313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635314#comment-17635314 ] Mars commented on SPARK-37313: -- as comment said

[jira] [Commented] (SPARK-38005) Support cleaning up merged shuffle files and state from external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-38005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625006#comment-17625006 ] Mars commented on SPARK-38005: -- [~mridulm80] [~csingh] Hi~ I want to take this issue. I had understood some

[jira] [Comment Edited] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601543#comment-17601543 ] Mars edited comment on SPARK-40320 at 9/7/22 10:50 PM: --- [~Ngone51]  Shouldn't it

[jira] [Commented] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601543#comment-17601543 ] Mars commented on SPARK-40320: -- [~Ngone51]  Shouldn't it bring up a new `receiveLoop()` to serve RPC

[jira] [Updated] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40320: - Description: *Reproduce step:* set `spark.plugins=ErrorSparkPlugin` `ErrorSparkPlugin` && `ErrorExecutorPlugin`

[jira] [Updated] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40320: - Description: *Reproduce step:* set `spark.plugins=ErrorSparkPlugin` `ErrorSparkPlugin` && `ErrorExecutorPlugin`

[jira] [Updated] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40320: - Description: Reproduce step: set `spark.plugins=ErrorSparkPlugin` `ErrorSparkPlugin` && `ErrorExecutorPlugin`

[jira] [Updated] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40320: - Description: *Reproduce step:* set `spark.plugins=ErrorSparkPlugin` `ErrorSparkPlugin` && `ErrorExecutorPlugin`

[jira] [Updated] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40320: - Description: *Reproduce step:* set `spark.plugins=ErrorSparkPlugin` `ErrorSparkPlugin` && `ErrorExecutorPlugin`

[jira] [Updated] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

[ https://issues.apache.org/jira/browse/SPARK-40320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40320: - Description: Reproduce step: set `spark.plugins=ErrorSparkPlugin` `ErrorSparkPlugin` && `ErrorExecutorPlugin`

[jira] [Created] (SPARK-40320) When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung

Mars created SPARK-40320: Summary: When the Executor plugin fails to initialize, the Executor shows active but does not accept tasks forever, just like being hung Key: SPARK-40320 URL:

[jira] [Updated] (SPARK-40113) Reactor ParquetScanBuilder DataSourceV2 interface implementation

[ https://issues.apache.org/jira/browse/SPARK-40113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40113: - Summary: Reactor ParquetScanBuilder DataSourceV2 interface implementation (was: Unify ParquetScanBuilder

[jira] [Updated] (SPARK-40113) Unify ParquetScanBuilder DataSourceV2 interface implementation

[ https://issues.apache.org/jira/browse/SPARK-40113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40113: - Priority: Minor (was: Major) > Unify ParquetScanBuilder DataSourceV2 interface implementation >

[jira] [Updated] (SPARK-40113) Unify ParquetScanBuilder DataSourceV2 interface implementation

[ https://issues.apache.org/jira/browse/SPARK-40113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mars updated SPARK-40113: - Summary: Unify ParquetScanBuilder DataSourceV2 interface implementation (was: Unified ParquetScanBuilder

[jira] [Created] (SPARK-40113) Unified ParquetScanBuilder DataSourceV2 interface implementation

Mars created SPARK-40113: Summary: Unified ParquetScanBuilder DataSourceV2 interface implementation Key: SPARK-40113 URL: https://issues.apache.org/jira/browse/SPARK-40113 Project: Spark Issue

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573197#comment-17573197 ] Mars commented on SPARK-39909: -- [~huaxingao] This is me :D [~miracle]  , my Jira id: miracle > Organize

[jira] [Commented] (SPARK-39909) Organize the check of push down information for JDBCV2Suite

[ https://issues.apache.org/jira/browse/SPARK-39909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572362#comment-17572362 ] Mars commented on SPARK-39909: -- assign [~miracle] > Organize the check of push down information for