[jira] [Created] (SPARK-48599) OIDC support for Spark

2024-06-12 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-48599: Summary: OIDC support for Spark Key: SPARK-48599 URL: https://issues.apache.org/jira/browse/SPARK-48599 Project: Spark Issue Type: New Feature Comp

[jira] [Comment Edited] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2024-06-05 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852555#comment-17852555 ] Yeachan Park edited comment on SPARK-41469 at 6/6/24 6:28 AM:

[jira] [Commented] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2024-06-05 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852555#comment-17852555 ] Yeachan Park commented on SPARK-41469: -- Maybe a stupid question, but I thought the

[jira] [Commented] (SPARK-30292) Throw Exception when invalid string is cast to decimal in ANSI mode

2023-04-05 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709002#comment-17709002 ] Yeachan Park commented on SPARK-30292: -- Does anyone know why this behaviour is coup

[jira] [Updated] (SPARK-42737) Shuffle files lost with graceful decommission fallback storage enabled

2023-03-10 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-42737: - Description: During testing of graceful decommissioning, the driver logs indicate that shuffle

[jira] [Updated] (SPARK-42737) Shuffle files lost with graceful decommission fallback storage enabled

2023-03-10 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-42737: - Description: During testing of graceful decommissioning, the driver logs indicate that shuffle

[jira] [Created] (SPARK-42738) BlockManagerMaster requests removal of non-existent executor after executor has already gracefully decommissioned

2023-03-09 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-42738: Summary: BlockManagerMaster requests removal of non-existent executor after executor has already gracefully decommissioned Key: SPARK-42738 URL: https://issues.apache.org/jira/bro

[jira] [Created] (SPARK-42737) Shuffle files lost with graceful decommission fallback storage enabled

2023-03-09 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-42737: Summary: Shuffle files lost with graceful decommission fallback storage enabled Key: SPARK-42737 URL: https://issues.apache.org/jira/browse/SPARK-42737 Project: Spark

[jira] [Updated] (SPARK-42736) Display executor remove reason statistics for clusters which don't have dynamic allocation

2023-03-09 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-42736: - Description: The remove reason statistics do not seem to be enabled for clusters which do not h

[jira] [Updated] (SPARK-42736) Display executor remove reason statistics for clusters which don't have dynamic allocation

2023-03-09 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-42736: - Description: The remove reason statistics do not seem to be enabled for clusters which do not h

[jira] [Created] (SPARK-42736) Display executor remove reason statistics for clusters which don't have dynamic allocation

2023-03-09 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-42736: Summary: Display executor remove reason statistics for clusters which don't have dynamic allocation Key: SPARK-42736 URL: https://issues.apache.org/jira/browse/SPARK-42736

[jira] [Updated] (SPARK-42291) Allow drop column operation on non V2 tables

2023-02-07 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-42291: - Affects Version/s: 3.3.1 (was: 3.2.0) > Allow drop column operation o

[jira] [Updated] (SPARK-42291) Allow drop column operation on non V2 tables

2023-02-02 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-42291: - Description: Drop column operations do not seem to be supported in non-V2 tables: https://githu

[jira] [Commented] (SPARK-42291) Allow drop column operation on non V2 tables

2023-02-02 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17683411#comment-17683411 ] Yeachan Park commented on SPARK-42291: -- Created a PR: https://github.com/apache/spa

[jira] [Created] (SPARK-42291) Allow drop column operation on non V2 tables

2023-02-02 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-42291: Summary: Allow drop column operation on non V2 tables Key: SPARK-42291 URL: https://issues.apache.org/jira/browse/SPARK-42291 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-26209) Allow for dataframe bucketization without Hive

2022-12-20 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17649824#comment-17649824 ] Yeachan Park commented on SPARK-26209: -- Is there an update for this? We'd also be i

[jira] [Commented] (SPARK-40140) REST API for SQL level information does not show information on running queries

2022-08-19 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581708#comment-17581708 ] Yeachan Park commented on SPARK-40140: -- Please feel free to pick it up :) > REST A

[jira] [Commented] (SPARK-27142) Provide REST API for SQL level information

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581424#comment-17581424 ] Yeachan Park commented on SPARK-27142: -- Hi all, thanks a lot for working on this. I

[jira] [Comment Edited] (SPARK-27142) Provide REST API for SQL level information

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581424#comment-17581424 ] Yeachan Park edited comment on SPARK-27142 at 8/18/22 4:14 PM: ---

[jira] [Created] (SPARK-40140) REST API for SQL level information does not show information on running queries

2022-08-18 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-40140: Summary: REST API for SQL level information does not show information on running queries Key: SPARK-40140 URL: https://issues.apache.org/jira/browse/SPARK-40140 Proje

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Updated] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-40139: - Description: Hi all, We have some cases where it would be useful to be able to have a list of a

[jira] [Created] (SPARK-40139) Filter jobs by Job Group in UI & API

2022-08-18 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-40139: Summary: Filter jobs by Job Group in UI & API Key: SPARK-40139 URL: https://issues.apache.org/jira/browse/SPARK-40139 Project: Spark Issue Type: New Feature

[jira] [Comment Edited] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572308#comment-17572308 ] Yeachan Park edited comment on SPARK-39743 at 7/28/22 8:29 AM: ---

[jira] [Comment Edited] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572308#comment-17572308 ] Yeachan Park edited comment on SPARK-39743 at 7/28/22 8:28 AM: ---

[jira] [Commented] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-28 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572308#comment-17572308 ] Yeachan Park commented on SPARK-39743: -- OK, thanks for the clarification. Should we

[jira] [Updated] (SPARK-39763) Executor memory footprint substantially increases while reading zstd compressed parquet files

2022-07-13 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-39763: - Description: Hi all,   While transitioning from the default snappy compression to zstd, we not

[jira] [Updated] (SPARK-39763) Executor memory footprint substantially increases while reading zstd compressed parquet files

2022-07-13 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-39763: - Description: Hi all,   While transitioning from the default snappy compression to zstd, we not

[jira] [Created] (SPARK-39763) Executor memory footprint substantially increases while reading zstd compressed parquet files

2022-07-13 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-39763: Summary: Executor memory footprint substantially increases while reading zstd compressed parquet files Key: SPARK-39763 URL: https://issues.apache.org/jira/browse/SPARK-39763

[jira] [Updated] (SPARK-39743) Unable to set zstd compression level while writing parquet files

2022-07-11 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yeachan Park updated SPARK-39743: - Summary: Unable to set zstd compression level while writing parquet files (was: Unable to set z

[jira] [Created] (SPARK-39743) Unable to set zstd compression level while writing parquet

2022-07-11 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-39743: Summary: Unable to set zstd compression level while writing parquet Key: SPARK-39743 URL: https://issues.apache.org/jira/browse/SPARK-39743 Project: Spark Is

[jira] [Commented] (SPARK-38969) Graceful decomissionning on Kubernetes fails / decom script error

2022-05-02 Thread Yeachan Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17530931#comment-17530931 ] Yeachan Park commented on SPARK-38969: -- Hi Holden, thanks for responding. Triggerin

[jira] [Created] (SPARK-38969) Graceful decomissionning on Kubernetes fails / decom script error

2022-04-20 Thread Yeachan Park (Jira)
Yeachan Park created SPARK-38969: Summary: Graceful decomissionning on Kubernetes fails / decom script error Key: SPARK-38969 URL: https://issues.apache.org/jira/browse/SPARK-38969 Project: Spark