[jira] [Resolved] (SPARK-38548) New SQL function: try_sum

2022-03-20 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38548. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35848 [https:

[jira] [Updated] (SPARK-38608) [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`

2022-03-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Summary: [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.

[jira] [Updated] (SPARK-38608) [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`

2022-03-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Description: Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any` to include on

[jira] [Updated] (SPARK-38608) [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`

2022-03-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Description: [SPARK-38608][PYTHON] Implement `bool_only` parameter of `DataFrame.all` and`DataFr

[jira] [Updated] (SPARK-35781) Support Spark on Apple Silicon on macOS natively on Java 17

2022-03-20 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-35781: Labels: release-notes (was: ) > Support Spark on Apple Silicon on macOS natively on Java 17 > ---

[jira] [Updated] (SPARK-38610) UI for Pandas API on Spark

2022-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38610: - Priority: Critical (was: Major) > UI for Pandas API on Spark > -- > >

[jira] [Updated] (SPARK-38610) UI for Pandas API on Spark

2022-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38610: - Component/s: Web UI > UI for Pandas API on Spark > -- > >

[jira] [Created] (SPARK-38610) UI for Pandas API on Spark

2022-03-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38610: Summary: UI for Pandas API on Spark Key: SPARK-38610 URL: https://issues.apache.org/jira/browse/SPARK-38610 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-38607) Test result report for ANSI mode

2022-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38607. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35916 [https://gi

[jira] [Assigned] (SPARK-38607) Test result report for ANSI mode

2022-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38607: Assignee: Hyukjin Kwon > Test result report for ANSI mode > -

[jira] [Commented] (SPARK-38488) Spark doc build not work on Mac OS M1

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509585#comment-17509585 ] Apache Spark commented on SPARK-38488: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-38488) Spark doc build not work on Mac OS M1

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38488: Assignee: Apache Spark > Spark doc build not work on Mac OS M1 >

[jira] [Assigned] (SPARK-38488) Spark doc build not work on Mac OS M1

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38488: Assignee: (was: Apache Spark) > Spark doc build not work on Mac OS M1 > -

[jira] [Updated] (SPARK-38488) Spark doc build not work on Mac OS M1

2022-03-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38488: Affects Version/s: 3.4.0 > Spark doc build not work on Mac OS M1 > ---

[jira] [Commented] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509565#comment-17509565 ] Apache Spark commented on SPARK-38609: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38609: Assignee: Apache Spark > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias

[jira] [Assigned] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38609: Assignee: (was: Apache Spark) > Add PYSPARK_PANDAS_USAGE_LOGGER environment variable

[jira] [Commented] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509566#comment-17509566 ] Apache Spark commented on SPARK-38609: -- User 'HyukjinKwon' has created a pull reque

[jira] [Updated] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER

2022-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38609: - Summary: Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER

[jira] [Updated] (SPARK-38609) Add PYSPARK_PANDAS_USAGE_LOGGER environment variable as an alias of KOALAS_USAGE_LOGGER

2022-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38609: - Description: There's one place missing about the renaming Koalas to pandas API on Spark (POS). W

[jira] [Assigned] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38556: Assignee: Apache Spark > Disable Pandas usage logging for method calls inside @contextman

[jira] [Assigned] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38556: Assignee: (was: Apache Spark) > Disable Pandas usage logging for method calls inside

[jira] [Created] (SPARK-38609) Rename KOALAS_USAGE_LOGGER to POS_USAGE_LOGGER

2022-03-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38609: Summary: Rename KOALAS_USAGE_LOGGER to POS_USAGE_LOGGER Key: SPARK-38609 URL: https://issues.apache.org/jira/browse/SPARK-38609 Project: Spark Issue Type: Im

[jira] [Reopened] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions

2022-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-38556: -- Reverted at https://github.com/apache/spark/commit/dcc66e4b4933c3493be3c6f2bb0118cd1680b530 and

[jira] [Assigned] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38608: Assignee: (was: Apache Spark) > Implement `bool_only` parameter of `DataFrame.all` to

[jira] [Commented] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509550#comment-17509550 ] Apache Spark commented on SPARK-38608: -- User 'xinrong-databricks' has created a pul

[jira] [Commented] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509549#comment-17509549 ] Apache Spark commented on SPARK-38608: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38608: Assignee: Apache Spark > Implement `bool_only` parameter of `DataFrame.all` to include on

[jira] [Created] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns

2022-03-20 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38608: Summary: Implement `bool_only` parameter of `DataFrame.all` to include only boolean columns Key: SPARK-38608 URL: https://issues.apache.org/jira/browse/SPARK-38608 Pr

[jira] [Assigned] (SPARK-38607) Test result report for ANSI mode

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38607: Assignee: (was: Apache Spark) > Test result report for ANSI mode > --

[jira] [Commented] (SPARK-38607) Test result report for ANSI mode

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509545#comment-17509545 ] Apache Spark commented on SPARK-38607: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-38607) Test result report for ANSI mode

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38607: Assignee: Apache Spark > Test result report for ANSI mode > -

[jira] [Created] (SPARK-38607) Test result report for ANSI mode

2022-03-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38607: Summary: Test result report for ANSI mode Key: SPARK-38607 URL: https://issues.apache.org/jira/browse/SPARK-38607 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38456: Assignee: Apache Spark > Improve error messages of no viable alternative, extraneous inpu

[jira] [Assigned] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38456: Assignee: (was: Apache Spark) > Improve error messages of no viable alternative, extr

[jira] [Commented] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509534#comment-17509534 ] Apache Spark commented on SPARK-38456: -- User 'anchovYu' has created a pull request

[jira] [Updated] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token

2022-03-20 Thread Xinyi Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyi Yu updated SPARK-38456: - Description: Please view the parent task description for the general idea:  https://issues.apache.org/ji

[jira] [Updated] (SPARK-38456) Improve error messages of no viable alternative, extraneous input and missing token

2022-03-20 Thread Xinyi Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyi Yu updated SPARK-38456: - Description: Please view the parent task description for the general idea:  https://issues.apache.org/ji

[jira] [Assigned] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38606: Assignee: (was: Apache Spark) > Update document to make a good guide of multiple vers

[jira] [Assigned] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38606: Assignee: Apache Spark > Update document to make a good guide of multiple versions of the

[jira] [Commented] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service

2022-03-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509437#comment-17509437 ] Apache Spark commented on SPARK-38606: -- User 'TonyDoen' has created a pull request

[jira] [Comment Edited] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service

2022-03-20 Thread tonydoen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509432#comment-17509432 ] tonydoen edited comment on SPARK-38606 at 3/20/22, 11:01 AM: -

[jira] [Commented] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service

2022-03-20 Thread tonydoen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509432#comment-17509432 ] tonydoen commented on SPARK-38606: -- [https://github.com/apache/spark/pull/35914] > Upd

[jira] [Updated] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service

2022-03-20 Thread tonydoen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tonydoen updated SPARK-38606: - Description: We have tried to run multiple versions of the Spark Shuffle Service  according [https://gi

[jira] [Created] (SPARK-38606) Update document to make a good guide of multiple versions of the Spark Shuffle Service

2022-03-20 Thread tonydoen (Jira)
tonydoen created SPARK-38606: Summary: Update document to make a good guide of multiple versions of the Spark Shuffle Service Key: SPARK-38606 URL: https://issues.apache.org/jira/browse/SPARK-38606 Proje

[jira] [Commented] (SPARK-38605) Retrying on file manager operation in HDFSMetadataLog

2022-03-20 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509394#comment-17509394 ] Jungtaek Lim commented on SPARK-38605: -- I don't have a strong idea about this. It s