[jira] [Commented] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646976#comment-17646976 ] Apache Spark commented on SPARK-41506: -- User 'zhengruifeng' has created a pull request for this

[jira] [Comment Edited] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646971#comment-17646971 ] wuyi edited comment on SPARK-41497 at 12/14/22 7:31 AM: I'm thinking if we could

[jira] [Commented] (SPARK-41510) Support easy way for user defined PYTHONPATH in workers

[ https://issues.apache.org/jira/browse/SPARK-41510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646973#comment-17646973 ] Ohad Raviv commented on SPARK-41510: ok.. after diving into the code I think I found what I was

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646972#comment-17646972 ] Mridul Muralidharan commented on SPARK-41497: - [~Ngone51] Agree, that is what I was not sure

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646971#comment-17646971 ] wuyi commented on SPARK-41497: -- I'm thinking if we could improve the improved Option 4 by changing the rdd

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646969#comment-17646969 ] wuyi commented on SPARK-41497: -- > do we have a way to do that ?   [~mridulm80]  Currently, we only have

[jira] [Comment Edited] (SPARK-38719) Test the error class: CANNOT_CAST_DATATYPE

[ https://issues.apache.org/jira/browse/SPARK-38719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646963#comment-17646963 ] Jayadeep Jayaraman edited comment on SPARK-38719 at 12/14/22 6:56 AM:

[jira] [Comment Edited] (SPARK-38719) Test the error class: CANNOT_CAST_DATATYPE

[ https://issues.apache.org/jira/browse/SPARK-38719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646963#comment-17646963 ] Jayadeep Jayaraman edited comment on SPARK-38719 at 12/14/22 6:56 AM:

[jira] [Comment Edited] (SPARK-38719) Test the error class: CANNOT_CAST_DATATYPE

[ https://issues.apache.org/jira/browse/SPARK-38719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646963#comment-17646963 ] Jayadeep Jayaraman edited comment on SPARK-38719 at 12/14/22 6:55 AM:

[jira] [Commented] (SPARK-38719) Test the error class: CANNOT_CAST_DATATYPE

[ https://issues.apache.org/jira/browse/SPARK-38719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646963#comment-17646963 ] Jayadeep Jayaraman commented on SPARK-38719: I tried creating the failure as shown below,

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646962#comment-17646962 ] Mridul Muralidharan commented on SPARK-41497: - Agree, if we can determine that - do we have

[jira] [Commented] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646960#comment-17646960 ] Apache Spark commented on SPARK-41506: -- User 'dengziming' has created a pull request for this

[jira] [Updated] (SPARK-41515) PVC-oriented executor pod allocation

[ https://issues.apache.org/jira/browse/SPARK-41515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41515: -- Labels: releasenotes (was: ) > PVC-oriented executor pod allocation >

[jira] [Updated] (SPARK-39324) Log ExecutorDecommission as INFO level in TaskSchedulerImpl

[ https://issues.apache.org/jira/browse/SPARK-39324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39324: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Improvement) > Log

[jira] [Updated] (SPARK-39450) Reuse PVCs by default

[ https://issues.apache.org/jira/browse/SPARK-39450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39450: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Improvement) > Reuse PVCs by default

[jira] [Updated] (SPARK-39688) getReusablePVCs should handle accounts with no PVC permission

[ https://issues.apache.org/jira/browse/SPARK-39688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39688: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Bug) > getReusablePVCs should handle

[jira] [Updated] (SPARK-39898) Upgrade kubernetes-client to 5.12.3

[ https://issues.apache.org/jira/browse/SPARK-39898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39898: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Bug) > Upgrade kubernetes-client to

[jira] [Updated] (SPARK-39846) Enable spark.dynamicAllocation.shuffleTracking.enabled by default

[ https://issues.apache.org/jira/browse/SPARK-39846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39846: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Improvement) > Enable

[jira] [Commented] (SPARK-38719) Test the error class: CANNOT_CAST_DATATYPE

[ https://issues.apache.org/jira/browse/SPARK-38719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646942#comment-17646942 ] Max Gekk commented on SPARK-38719: -- [~jjayadeep] Sure, go ahead. > Test the error class:

[jira] [Updated] (SPARK-39965) Skip PVC cleanup when driver doesn't own PVCs

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39965: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Bug) > Skip PVC cleanup when driver

[jira] [Updated] (SPARK-40198) Enable spark.storage.decommission.(rdd|shuffle)Blocks.enabled by default

[ https://issues.apache.org/jira/browse/SPARK-40198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40198: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Improvement) > Enable

[jira] [Updated] (SPARK-40304) Add decomTestTag to K8s Integration Test

[ https://issues.apache.org/jira/browse/SPARK-40304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40304: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Test) > Add decomTestTag to K8s

[jira] [Updated] (SPARK-40459) recoverDiskStore should not stop by existing recomputed files

[ https://issues.apache.org/jira/browse/SPARK-40459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40459: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Bug) > recoverDiskStore should not

[jira] [Updated] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41388: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Bug) > getReusablePVCs should ignore

[jira] [Assigned] (SPARK-41514) Add `PVC-oriented executor pod allocation` section and revise config name

[ https://issues.apache.org/jira/browse/SPARK-41514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41514: - Assignee: Dongjoon Hyun > Add `PVC-oriented executor pod allocation` section and

[jira] [Updated] (SPARK-41410) Support PVC-oriented executor pod allocation

[ https://issues.apache.org/jira/browse/SPARK-41410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41410: -- Parent: SPARK-41515 Issue Type: Sub-task (was: New Feature) > Support PVC-oriented

[jira] [Updated] (SPARK-41514) Add `PVC-oriented executor pod allocation` section and revise config name

[ https://issues.apache.org/jira/browse/SPARK-41514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41514: -- Parent: SPARK-41515 Issue Type: Sub-task (was: Documentation) > Add `PVC-oriented

[jira] [Created] (SPARK-41515) PVC-oriented executor pod allocation

Dongjoon Hyun created SPARK-41515: - Summary: PVC-oriented executor pod allocation Key: SPARK-41515 URL: https://issues.apache.org/jira/browse/SPARK-41515 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-38719) Test the error class: CANNOT_CAST_DATATYPE

[ https://issues.apache.org/jira/browse/SPARK-38719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646938#comment-17646938 ] Jayadeep Jayaraman commented on SPARK-38719: Hi [~maxgekk] - I would like to work on this

[jira] [Assigned] (SPARK-41248) Add config flag to control before of JSON partial results parsing in SPARK-40646

[ https://issues.apache.org/jira/browse/SPARK-41248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41248: Assignee: Ivan Sadikov > Add config flag to control before of JSON partial results parsing in >

[jira] [Resolved] (SPARK-41248) Add config flag to control before of JSON partial results parsing in SPARK-40646

[ https://issues.apache.org/jira/browse/SPARK-41248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41248. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38784

[jira] [Assigned] (SPARK-41514) Add `PVC-oriented executor pod allocation` section and revise config name

[ https://issues.apache.org/jira/browse/SPARK-41514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41514: Assignee: Apache Spark > Add `PVC-oriented executor pod allocation` section and revise

[jira] [Assigned] (SPARK-41514) Add `PVC-oriented executor pod allocation` section and revise config name

[ https://issues.apache.org/jira/browse/SPARK-41514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41514: Assignee: (was: Apache Spark) > Add `PVC-oriented executor pod allocation` section

[jira] [Commented] (SPARK-41514) Add `PVC-oriented executor pod allocation` section and revise config name

[ https://issues.apache.org/jira/browse/SPARK-41514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646931#comment-17646931 ] Apache Spark commented on SPARK-41514: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-41514) Add `PVC-oriented executor pod allocation` section and revise config name

Dongjoon Hyun created SPARK-41514: - Summary: Add `PVC-oriented executor pod allocation` section and revise config name Key: SPARK-41514 URL: https://issues.apache.org/jira/browse/SPARK-41514 Project:

[jira] [Assigned] (SPARK-41409) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043`

[ https://issues.apache.org/jira/browse/SPARK-41409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41409: Assignee: Yang Jie > Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043` >

[jira] [Resolved] (SPARK-41409) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043`

[ https://issues.apache.org/jira/browse/SPARK-41409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41409. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38940

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646919#comment-17646919 ] wuyi commented on SPARK-41497: -- [~mridulm80]  For b) and c), shouldn't we allow T2 to use the result of

[jira] [Updated] (SPARK-41512) Row count based shuffle read to optimize global limit after a single partition shuffle (optionally with input partition sorted)

[ https://issues.apache.org/jira/browse/SPARK-41512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-41512: - Description: h3. Problem Statement In current Spark optimizer, a single partition shuffle might be

[jira] [Assigned] (SPARK-41513) Implement a Accumulator to collect per mapper row count metrics

[ https://issues.apache.org/jira/browse/SPARK-41513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41513: Assignee: Rui Wang (was: Apache Spark) > Implement a Accumulator to collect per mapper

[jira] [Assigned] (SPARK-41513) Implement a Accumulator to collect per mapper row count metrics

[ https://issues.apache.org/jira/browse/SPARK-41513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41513: Assignee: Apache Spark (was: Rui Wang) > Implement a Accumulator to collect per mapper

[jira] [Commented] (SPARK-41513) Implement a Accumulator to collect per mapper row count metrics

[ https://issues.apache.org/jira/browse/SPARK-41513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646901#comment-17646901 ] Apache Spark commented on SPARK-41513: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Updated] (SPARK-41512) Row count based shuffle read to optimize global limit after a single partition shuffle (optionally with input partition sorted)

[ https://issues.apache.org/jira/browse/SPARK-41512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-41512: - Description: h3. Problem Statement In current Spark optimizer, a single partition shuffle might be

[jira] [Updated] (SPARK-41512) Row count based shuffle read to optimize global limit after a single partition shuffle (optionally with input partition sorted)

[ https://issues.apache.org/jira/browse/SPARK-41512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-41512: - Description: h3. Problem Statement In current Spark optimizer, a single partition shuffle might be

[jira] [Created] (SPARK-41513) Implement a Accumulator to collect per mapper row count metrics

Rui Wang created SPARK-41513: Summary: Implement a Accumulator to collect per mapper row count metrics Key: SPARK-41513 URL: https://issues.apache.org/jira/browse/SPARK-41513 Project: Spark

[jira] [Commented] (SPARK-41512) Row count based shuffle read to optimize global limit after a single partition shuffle (optionally with input partition sorted)

[ https://issues.apache.org/jira/browse/SPARK-41512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646898#comment-17646898 ] Rui Wang commented on SPARK-41512: -- cc [~cloud_fan] > Row count based shuffle read to optimize global

[jira] [Created] (SPARK-41512) Row count based shuffle read to optimize global limit after a single partition shuffle (optionally with input partition sorted)

Rui Wang created SPARK-41512: Summary: Row count based shuffle read to optimize global limit after a single partition shuffle (optionally with input partition sorted) Key: SPARK-41512 URL:

[jira] [Commented] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646896#comment-17646896 ] Apache Spark commented on SPARK-41506: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646895#comment-17646895 ] Apache Spark commented on SPARK-41506: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646890#comment-17646890 ] Apache Spark commented on SPARK-41506: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646889#comment-17646889 ] Apache Spark commented on SPARK-41506: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41506: Assignee: Ruifeng Zheng > Refactor LiteralExpression to support DataType >

[jira] [Resolved] (SPARK-41506) Refactor LiteralExpression to support DataType

[ https://issues.apache.org/jira/browse/SPARK-41506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41506. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39047

[jira] [Comment Edited] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646825#comment-17646825 ] Mridul Muralidharan edited comment on SPARK-41497 at 12/13/22 9:20 PM:

[jira] [Comment Edited] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646825#comment-17646825 ] Mridul Muralidharan edited comment on SPARK-41497 at 12/13/22 9:20 PM:

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646825#comment-17646825 ] Mridul Muralidharan commented on SPARK-41497: - > For example, a task is constructed by

[jira] [Commented] (SPARK-27561) Support "lateral column alias references" to allow column aliases to be used within SELECT clauses

[ https://issues.apache.org/jira/browse/SPARK-27561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646823#comment-17646823 ] Apache Spark commented on SPARK-27561: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-27561) Support "lateral column alias references" to allow column aliases to be used within SELECT clauses

[ https://issues.apache.org/jira/browse/SPARK-27561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646824#comment-17646824 ] Apache Spark commented on SPARK-27561: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-41062) Rename UNSUPPORTED_CORRELATED_REFERENCE to CORRELATED_REFERENCE

[ https://issues.apache.org/jira/browse/SPARK-41062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41062: Assignee: Haejoon Lee > Rename UNSUPPORTED_CORRELATED_REFERENCE to CORRELATED_REFERENCE >

[jira] [Resolved] (SPARK-41062) Rename UNSUPPORTED_CORRELATED_REFERENCE to CORRELATED_REFERENCE

[ https://issues.apache.org/jira/browse/SPARK-41062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41062. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38576

[jira] [Resolved] (SPARK-41482) Upgrade dropwizard metrics 4.2.13

[ https://issues.apache.org/jira/browse/SPARK-41482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41482. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39026

[jira] [Assigned] (SPARK-41482) Upgrade dropwizard metrics 4.2.13

[ https://issues.apache.org/jira/browse/SPARK-41482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41482: - Assignee: Yang Jie > Upgrade dropwizard metrics 4.2.13 >

[jira] [Commented] (SPARK-41510) Support easy way for user defined PYTHONPATH in workers

[ https://issues.apache.org/jira/browse/SPARK-41510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646734#comment-17646734 ] Ohad Raviv commented on SPARK-41510: the conda solution is more for a "static" packages. the

[jira] [Assigned] (SPARK-27561) Support "lateral column alias references" to allow column aliases to be used within SELECT clauses

[ https://issues.apache.org/jira/browse/SPARK-27561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27561: --- Assignee: Xinyi Yu > Support "lateral column alias references" to allow column aliases to

[jira] [Resolved] (SPARK-27561) Support "lateral column alias references" to allow column aliases to be used within SELECT clauses

[ https://issues.apache.org/jira/browse/SPARK-27561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27561. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38776

[jira] [Commented] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646682#comment-17646682 ] Apache Spark commented on SPARK-39601: -- User 'pan3793' has created a pull request for this issue:

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646658#comment-17646658 ] huangtengfei commented on SPARK-41497: -- I also think that option3/4(include the improved proposal

[jira] [Resolved] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-39601. --- Fix Version/s: 3.4.0 Assignee: Cheng Pan Resolution: Fixed >

[jira] [Resolved] (SPARK-41478) Assign a name to the error class _LEGACY_ERROR_TEMP_1234

[ https://issues.apache.org/jira/browse/SPARK-41478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41478. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39018

[jira] [Assigned] (SPARK-41478) Assign a name to the error class _LEGACY_ERROR_TEMP_1234

[ https://issues.apache.org/jira/browse/SPARK-41478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41478: Assignee: BingKun Pan > Assign a name to the error class _LEGACY_ERROR_TEMP_1234 >

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646639#comment-17646639 ] wuyi commented on SPARK-41497: -- [~mridulm80] Sounds like a better idea than option 4. But I think it still

[jira] [Updated] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-41497: - Description: Accumulator could be undercounted when the retried task has rdd cache.  See the example below and

[jira] [Commented] (SPARK-41510) Support easy way for user defined PYTHONPATH in workers

[ https://issues.apache.org/jira/browse/SPARK-41510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646620#comment-17646620 ] Hyukjin Kwon commented on SPARK-41510: -- What about using Conda

[jira] [Comment Edited] (SPARK-41510) Support easy way for user defined PYTHONPATH in workers

[ https://issues.apache.org/jira/browse/SPARK-41510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646606#comment-17646606 ] Ohad Raviv edited comment on SPARK-41510 at 12/13/22 12:23 PM: ---

[jira] [Commented] (SPARK-41510) Support easy way for user defined PYTHONPATH in workers

[ https://issues.apache.org/jira/browse/SPARK-41510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646606#comment-17646606 ] Ohad Raviv commented on SPARK-41510: [~hvanhovell] - can you please refer that to someone? >

[jira] [Commented] (SPARK-41360) Avoid BlockManager re-registration if the executor has been lost

[ https://issues.apache.org/jira/browse/SPARK-41360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646605#comment-17646605 ] Apache Spark commented on SPARK-41360: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-41360) Avoid BlockManager re-registration if the executor has been lost

[ https://issues.apache.org/jira/browse/SPARK-41360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646603#comment-17646603 ] Apache Spark commented on SPARK-41360: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-41511) LongToUnsafeRowMap support ignoresDuplicatedKey

[ https://issues.apache.org/jira/browse/SPARK-41511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41511: Assignee: (was: Apache Spark) > LongToUnsafeRowMap support ignoresDuplicatedKey >

[jira] [Updated] (SPARK-41510) Support easy way for user defined PYTHONPATH in workers

[ https://issues.apache.org/jira/browse/SPARK-41510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Raviv updated SPARK-41510: --- Description: When working interactively with Spark through notebooks in various envs -

[jira] [Commented] (SPARK-41511) LongToUnsafeRowMap support ignoresDuplicatedKey

[ https://issues.apache.org/jira/browse/SPARK-41511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646600#comment-17646600 ] Apache Spark commented on SPARK-41511: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-41511) LongToUnsafeRowMap support ignoresDuplicatedKey

[ https://issues.apache.org/jira/browse/SPARK-41511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41511: Assignee: Apache Spark > LongToUnsafeRowMap support ignoresDuplicatedKey >

[jira] [Created] (SPARK-41510) Support easy way for user defined PYTHONPATH in workers

Ohad Raviv created SPARK-41510: -- Summary: Support easy way for user defined PYTHONPATH in workers Key: SPARK-41510 URL: https://issues.apache.org/jira/browse/SPARK-41510 Project: Spark Issue

[jira] [Created] (SPARK-41511) LongToUnsafeRowMap support ignoresDuplicatedKey

XiDuo You created SPARK-41511: - Summary: LongToUnsafeRowMap support ignoresDuplicatedKey Key: SPARK-41511 URL: https://issues.apache.org/jira/browse/SPARK-41511 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41412) Implement `Cast`

[ https://issues.apache.org/jira/browse/SPARK-41412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646594#comment-17646594 ] Apache Spark commented on SPARK-41412: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-41412) Implement `Cast`

[ https://issues.apache.org/jira/browse/SPARK-41412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646593#comment-17646593 ] Apache Spark commented on SPARK-41412: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-41509) Delay execution hash until after aggregation for semi-join runtime filter.

[ https://issues.apache.org/jira/browse/SPARK-41509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-41509: --- Description: Currently, Spark runtime filter supports bloom filter and in subquery filter. The in

[jira] [Commented] (SPARK-41509) Delay execution hash until after aggregation for semi-join runtime filter.

[ https://issues.apache.org/jira/browse/SPARK-41509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646581#comment-17646581 ] Apache Spark commented on SPARK-41509: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41509) Delay execution hash until after aggregation for semi-join runtime filter.

[ https://issues.apache.org/jira/browse/SPARK-41509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41509: Assignee: (was: Apache Spark) > Delay execution hash until after aggregation for

[jira] [Assigned] (SPARK-41509) Delay execution hash until after aggregation for semi-join runtime filter.

[ https://issues.apache.org/jira/browse/SPARK-41509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41509: Assignee: Apache Spark > Delay execution hash until after aggregation for semi-join

[jira] [Commented] (SPARK-41509) Delay execution hash until after aggregation for semi-join runtime filter.

[ https://issues.apache.org/jira/browse/SPARK-41509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646579#comment-17646579 ] Apache Spark commented on SPARK-41509: -- User 'beliefer' has created a pull request for this issue:

[jira] [Created] (SPARK-41509) Delay execution hash until after aggregation for semi-join runtime filter.

jiaan.geng created SPARK-41509: -- Summary: Delay execution hash until after aggregation for semi-join runtime filter. Key: SPARK-41509 URL: https://issues.apache.org/jira/browse/SPARK-41509 Project:

[jira] [Commented] (SPARK-41319) when-otherwise support

[ https://issues.apache.org/jira/browse/SPARK-41319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646575#comment-17646575 ] Apache Spark commented on SPARK-41319: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41319) when-otherwise support

[ https://issues.apache.org/jira/browse/SPARK-41319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646574#comment-17646574 ] Apache Spark commented on SPARK-41319: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-41508) assign name to _LEGACY_ERROR_TEMP_1179 and unwrap the existing SparkThrowable

Yang Jie created SPARK-41508: Summary: assign name to _LEGACY_ERROR_TEMP_1179 and unwrap the existing SparkThrowable Key: SPARK-41508 URL: https://issues.apache.org/jira/browse/SPARK-41508 Project:

[jira] [Updated] (SPARK-41508) Assign name to _LEGACY_ERROR_TEMP_1179 and unwrap the existing SparkThrowable

[ https://issues.apache.org/jira/browse/SPARK-41508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-41508: - Summary: Assign name to _LEGACY_ERROR_TEMP_1179 and unwrap the existing SparkThrowable (was: assign

[jira] [Assigned] (SPARK-41406) Refactor error message for `NUM_COLUMNS_MISMATCH` to make it more generic

[ https://issues.apache.org/jira/browse/SPARK-41406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41406: Assignee: BingKun Pan > Refactor error message for `NUM_COLUMNS_MISMATCH` to make it more

[jira] [Resolved] (SPARK-41406) Refactor error message for `NUM_COLUMNS_MISMATCH` to make it more generic

[ https://issues.apache.org/jira/browse/SPARK-41406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41406. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38937

[jira] [Commented] (SPARK-41507) Correct group of collection_funcs

[ https://issues.apache.org/jira/browse/SPARK-41507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646509#comment-17646509 ] Apache Spark commented on SPARK-41507: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-41424) Protobuf serializer for TaskDataWrapper

[ https://issues.apache.org/jira/browse/SPARK-41424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646508#comment-17646508 ] Apache Spark commented on SPARK-41424: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-41507) Correct group of collection_funcs

[ https://issues.apache.org/jira/browse/SPARK-41507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41507: Assignee: (was: Apache Spark) > Correct group of collection_funcs >

  1   2   >