[jira] [Resolved] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39205. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36574 [https://gi

[jira] [Assigned] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39205: Assignee: Yikun Jiang > Add `PANDAS API ON SPARK` label > ---

[jira] [Resolved] (SPARK-39206) Add PS label for Pandas API on Spark in PRs

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39206. -- Resolution: Duplicate > Add PS label for Pandas API on Spark in PRs >

[jira] [Assigned] (SPARK-39206) Add PS label for Pandas API on Spark in PRs

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39206: Assignee: Apache Spark > Add PS label for Pandas API on Spark in PRs > --

[jira] [Assigned] (SPARK-39206) Add PS label for Pandas API on Spark in PRs

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39206: Assignee: (was: Apache Spark) > Add PS label for Pandas API on Spark in PRs > ---

[jira] [Commented] (SPARK-39206) Add PS label for Pandas API on Spark in PRs

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537936#comment-17537936 ] Apache Spark commented on SPARK-39206: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537934#comment-17537934 ] Apache Spark commented on SPARK-39205: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-38829) New configuration for controlling timestamp inference of Parquet

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537931#comment-17537931 ] Apache Spark commented on SPARK-38829: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537932#comment-17537932 ] Apache Spark commented on SPARK-39205: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-36718) only collapse projects if we don't duplicate expensive expressions

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537933#comment-17537933 ] Apache Spark commented on SPARK-36718: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39205: Assignee: Apache Spark > Add `PANDAS API ON SPARK` label > --

[jira] [Assigned] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39205: Assignee: (was: Apache Spark) > Add `PANDAS API ON SPARK` label > ---

[jira] [Updated] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-39205: Component/s: Pandas API on Spark > Add `PANDAS API ON SPARK` label > -

[jira] [Created] (SPARK-39206) Add PS label for Pandas API on Spark in PRs

2022-05-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-39206: Summary: Add PS label for Pandas API on Spark in PRs Key: SPARK-39206 URL: https://issues.apache.org/jira/browse/SPARK-39206 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-39205) Add `PANDAS API ON SPARK` label

2022-05-16 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-39205: --- Summary: Add `PANDAS API ON SPARK` label Key: SPARK-39205 URL: https://issues.apache.org/jira/browse/SPARK-39205 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-27380) Get and install GPU cards to Jenkins machines

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27380. -- Resolution: Invalid We don't have the Jenkins anymore. Closing > Get and install GPU cards to

[jira] [Resolved] (SPARK-27984) Jenkins job spark-master-package fails due to invalid gpg option

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27984. -- Resolution: Invalid We don't have the Jenkins anymore. Closing > Jenkins job spark-master-pac

[jira] [Resolved] (SPARK-27381) Design: Spark Jenkins supports GPU integration tests

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27381. -- Resolution: Invalid We don't have the Jenkins anymore. Closing > Design: Spark Jenkins suppor

[jira] [Resolved] (SPARK-27058) Support mounting host dirs for K8s tests

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27058. -- Resolution: Invalid > Support mounting host dirs for K8s tests > -

[jira] [Resolved] (SPARK-27365) Spark Jenkins supports testing GPU-aware scheduling features

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27365. -- Resolution: Invalid We don't have the Jenkins anymore. Closing > Spark Jenkins supports testi

[jira] [Commented] (SPARK-27058) Support mounting host dirs for K8s tests

2022-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537920#comment-17537920 ] Hyukjin Kwon commented on SPARK-27058: -- We don't have the Jenkins anymore. Closing

[jira] [Created] (SPARK-39204) Replace `Utils.createTempDir` related methods with JavaUtils

2022-05-16 Thread Yang Jie (Jira)
Yang Jie created SPARK-39204: Summary: Replace `Utils.createTempDir` related methods with JavaUtils Key: SPARK-39204 URL: https://issues.apache.org/jira/browse/SPARK-39204 Project: Spark Issue T

[jira] [Created] (SPARK-39203) Fix remote table location based on database location

2022-05-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39203: --- Summary: Fix remote table location based on database location Key: SPARK-39203 URL: https://issues.apache.org/jira/browse/SPARK-39203 Project: Spark Issue Type

[jira] [Assigned] (SPARK-39202) Introduce a `putByteArrays` method to `WritableColumnVector` to support setting multiple duplicate `byte[]`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39202: Assignee: (was: Apache Spark) > Introduce a `putByteArrays` method to `WritableColumn

[jira] [Assigned] (SPARK-39202) Introduce a `putByteArrays` method to `WritableColumnVector` to support setting multiple duplicate `byte[]`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39202: Assignee: Apache Spark > Introduce a `putByteArrays` method to `WritableColumnVector` to

[jira] [Commented] (SPARK-39202) Introduce a `putByteArrays` method to `WritableColumnVector` to support setting multiple duplicate `byte[]`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537893#comment-17537893 ] Apache Spark commented on SPARK-39202: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-39202) Introduce a `putByteArrays` method to `WritableColumnVector` to support setting multiple duplicate `byte[]`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537892#comment-17537892 ] Apache Spark commented on SPARK-39202: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-39096) Support MERGE commands with DEFAULT values

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537891#comment-17537891 ] Apache Spark commented on SPARK-39096: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-39202) Introduce a `putByteArrays` method to `WritableColumnVector` to support setting multiple duplicate `byte[]`

2022-05-16 Thread Yang Jie (Jira)
Yang Jie created SPARK-39202: Summary: Introduce a `putByteArrays` method to `WritableColumnVector` to support setting multiple duplicate `byte[]` Key: SPARK-39202 URL: https://issues.apache.org/jira/browse/SPARK-3920

[jira] [Resolved] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39120. - Resolution: Won't Do > Prunes the duplicate columns from child of UnaryNode > --

[jira] [Assigned] (SPARK-39201) Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39201: Assignee: (was: Apache Spark) > Implement `ignore_index` of `DataFrame.explode` and

[jira] [Commented] (SPARK-39201) Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537849#comment-17537849 ] Apache Spark commented on SPARK-39201: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-39201) Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39201: Assignee: Apache Spark > Implement `ignore_index` of `DataFrame.explode` and > `DataFram

[jira] [Created] (SPARK-39201) Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates`

2022-05-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-39201: Summary: Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates` Key: SPARK-39201 URL: https://issues.apache.org/jira/browse/SPARK-39201 Proje

[jira] [Resolved] (SPARK-39183) Upgrade Apache Xerces Java to 2.12.2

2022-05-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39183. -- Assignee: Bjørn Jørgensen Resolution: Fixed Resolved by https://github.com/apache/spark/

[jira] [Updated] (SPARK-39183) Upgrade Apache Xerces Java to 2.12.2

2022-05-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-39183: - Fix Version/s: 3.2.2 3.4.0 3.3.1 Affects Ve

[jira] [Commented] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537829#comment-17537829 ] Xiao Li commented on SPARK-36837: - Since we will bump to Kafka 3.2.0 in Spark 3.4, we ca

[jira] [Resolved] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-36837. - Resolution: Done > Upgrade Kafka to 3.1.0 > -- > > Key: SPARK-36837

[jira] [Updated] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-36837: Labels: (was: releasenotes) > Upgrade Kafka to 3.1.0 > -- > > Key: S

[jira] [Resolved] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-36837. - Resolution: Not A Problem > Upgrade Kafka to 3.1.0 > -- > > Key: SPA

[jira] [Reopened] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-36837: - > Upgrade Kafka to 3.1.0 > -- > > Key: SPARK-36837 > URL

[jira] [Reopened] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-36837: - > Upgrade Kafka to 3.1.0 > -- > > Key: SPARK-36837 > URL

[jira] [Updated] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-36837: Fix Version/s: (was: 3.3.0) > Upgrade Kafka to 3.1.0 > -- > > Key:

[jira] [Updated] (SPARK-39200) Stream is corrupted Exception while fetching the blocks from fallback storage system

2022-05-16 Thread Rajendra Gujja (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajendra Gujja updated SPARK-39200: --- Description: When executor decommissioning and fallback storage is enabled - the shuffle re

[jira] [Created] (SPARK-39200) Stream is corrupted Exception while fetching the blocks from fallback storage system

2022-05-16 Thread Rajendra Gujja (Jira)
Rajendra Gujja created SPARK-39200: -- Summary: Stream is corrupted Exception while fetching the blocks from fallback storage system Key: SPARK-39200 URL: https://issues.apache.org/jira/browse/SPARK-39200

[jira] [Updated] (SPARK-39199) Implement pandas API missing parameters

2022-05-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39199: - Description: pandas API on Spark aims to achieve full pandas API coverage. Currently, most pand

[jira] [Updated] (SPARK-39199) Implement pandas API missing parameters

2022-05-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39199: - Description: pandas API on Spark aims to achieve full pandas API coverage. Currently, most pand

[jira] [Updated] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`

2022-05-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Summary: Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any` (was: [SPARK-386

[jira] [Created] (SPARK-39199) Implement pandas API missing parameters

2022-05-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-39199: Summary: Implement pandas API missing parameters Key: SPARK-39199 URL: https://issues.apache.org/jira/browse/SPARK-39199 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

2022-05-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-37525: - Description: Since DayTimeIntervalType is supported in PySpark, we may add TimedeltaIndex suppo

[jira] [Updated] (SPARK-39198) Cannot refer to nested CTE within a nested CTE in a subquery.

2022-05-16 Thread Jarno Rajala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarno Rajala updated SPARK-39198: - Environment: Tested on * Databricks runtime 10.4 * Spark 3.2.1 from [https://spark.apache.org/

[jira] [Updated] (SPARK-39198) Cannot refer to nested CTE within a nested CTE in a subquery.

2022-05-16 Thread Jarno Rajala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarno Rajala updated SPARK-39198: - Description: The following query fails with {color:#ff}Table or view not found: cte1;{color

[jira] [Updated] (SPARK-39198) Cannot refer to nested CTE within a nested CTE in a subquery.

2022-05-16 Thread Jarno Rajala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarno Rajala updated SPARK-39198: - Description: The following query fails with {color:#ff}Table or view not found: cte1;{color

[jira] [Updated] (SPARK-39198) Cannot refer to nested CTE within a nested CTE in a subquery.

2022-05-16 Thread Jarno Rajala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarno Rajala updated SPARK-39198: - Description: The following query fails with {color:#ff}Table or view not found: cte1;{color

[jira] [Created] (SPARK-39198) Cannot refer to nested CTE within a nested CTE in a subquery.

2022-05-16 Thread Jarno Rajala (Jira)
Jarno Rajala created SPARK-39198: Summary: Cannot refer to nested CTE within a nested CTE in a subquery. Key: SPARK-39198 URL: https://issues.apache.org/jira/browse/SPARK-39198 Project: Spark

[jira] [Commented] (SPARK-39197) Implement `skipna` parameter of `GroupBy.all`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537674#comment-17537674 ] Apache Spark commented on SPARK-39197: -- User 'xinrong-databricks' has created a pul

[jira] [Commented] (SPARK-39197) Implement `skipna` parameter of `GroupBy.all`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537673#comment-17537673 ] Apache Spark commented on SPARK-39197: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-39197) Implement `skipna` parameter of `GroupBy.all`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39197: Assignee: Apache Spark > Implement `skipna` parameter of `GroupBy.all` >

[jira] [Assigned] (SPARK-39197) Implement `skipna` parameter of `GroupBy.all`

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39197: Assignee: (was: Apache Spark) > Implement `skipna` parameter of `GroupBy.all` > -

[jira] [Created] (SPARK-39197) Implement `skipna` parameter of `GroupBy.all`

2022-05-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-39197: Summary: Implement `skipna` parameter of `GroupBy.all` Key: SPARK-39197 URL: https://issues.apache.org/jira/browse/SPARK-39197 Project: Spark Issue Type: Sub

[jira] [Assigned] (SPARK-39180) Simplify the planning of limit and offset

2022-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39180: --- Assignee: Wenchen Fan > Simplify the planning of limit and offset > ---

[jira] [Resolved] (SPARK-39180) Simplify the planning of limit and offset

2022-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39180. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36541 [https://gith

[jira] [Assigned] (SPARK-39172) Remove outer join if all output come from streamed side and buffered side keys exist unique key

2022-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39172: --- Assignee: XiDuo You > Remove outer join if all output come from streamed side and buffered

[jira] [Resolved] (SPARK-39172) Remove outer join if all output come from streamed side and buffered side keys exist unique key

2022-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39172. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36530 [https://gith

[jira] [Commented] (SPARK-37219) support AS OF syntax

2022-05-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537627#comment-17537627 ] Huaxin Gao commented on SPARK-37219: Correct. > support AS OF syntax >

[jira] (SPARK-37219) support AS OF syntax

2022-05-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37219 ] Huaxin Gao deleted comment on SPARK-37219: was (Author: JIRAUSER284812): h2. Bagaimana Dengan Sistem Keposlot? [Keposlot|https://165.22.216.152/] Menjalankan sistem dengan 1 user id dapat b

[jira] (SPARK-37219) support AS OF syntax

2022-05-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37219 ] Huaxin Gao deleted comment on SPARK-37219: was (Author: JIRAUSER284812): Situs Resmi Judi Togel Online Terlengkap [Acctoto|https://katie-cassidy.us/]  adalah situs togel online terlengkap di

[jira] [Commented] (SPARK-37219) support AS OF syntax

2022-05-16 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537589#comment-17537589 ] Nicholas Chammas commented on SPARK-37219: -- This change will enable not just De

[jira] [Assigned] (SPARK-39196) Replace getOrElse(null) with orNull

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39196: Assignee: (was: Apache Spark) > Replace getOrElse(null) with orNull > ---

[jira] [Commented] (SPARK-39196) Replace getOrElse(null) with orNull

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537564#comment-17537564 ] Apache Spark commented on SPARK-39196: -- User 'dcoliversun' has created a pull reque

[jira] [Assigned] (SPARK-39196) Replace getOrElse(null) with orNull

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39196: Assignee: Apache Spark > Replace getOrElse(null) with orNull > --

[jira] [Created] (SPARK-39196) Replace getOrElse(null) with orNull

2022-05-16 Thread qian (Jira)
qian created SPARK-39196: Summary: Replace getOrElse(null) with orNull Key: SPARK-39196 URL: https://issues.apache.org/jira/browse/SPARK-39196 Project: Spark Issue Type: Improvement Compone

[jira] [Updated] (SPARK-39190) Provide query context for decimal precision overflow error when WSCG is off

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39190: - Fix Version/s: 3.3.1 (was: 3.3.0) > Provide query context for decimal precision o

[jira] [Updated] (SPARK-39186) make skew consistent with pandas

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39186: - Fix Version/s: 3.3.0 > make skew consistent with pandas > > >

[jira] [Updated] (SPARK-36837) Upgrade Kafka to 3.1.0

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-36837: - Fix Version/s: 3.3.0 > Upgrade Kafka to 3.1.0 > -- > > Key: SPARK-36

[jira] [Updated] (SPARK-39121) Fix doc format/syntax error

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39121: - Fix Version/s: 3.3.0 (was: 3.3.1) > Fix doc format/syntax error > ---

[jira] [Updated] (SPARK-38786) Test Bug in StatisticsSuite "change stats after add/drop partition command"

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38786: - Fix Version/s: 3.3.0 (was: 3.3.1) > Test Bug in StatisticsSuite "change stats aft

[jira] [Updated] (SPARK-35912) [SQL] JSON read behavior is different depending on the cache setting when nullable is false.

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35912: - Fix Version/s: 3.3.0 (was: 3.3.1) > [SQL] JSON read behavior is different dependi

[jira] [Updated] (SPARK-39060) Typo in error messages of decimal overflow

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39060: - Fix Version/s: 3.3.0 (was: 3.3.1) > Typo in error messages of decimal overflow >

[jira] [Updated] (SPARK-39099) Add dependencies to Dockerfile for building Spark releases

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39099: - Fix Version/s: 3.3.0 (was: 3.3.1) > Add dependencies to Dockerfile for building S

[jira] [Updated] (SPARK-39012) SparkSQL parse partition value does not support all data types

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39012: - Fix Version/s: 3.3.0 (was: 3.3.1) > SparkSQL parse partition value does not suppo

[jira] [Updated] (SPARK-39105) Add ConditionalExpression trait

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39105: - Fix Version/s: 3.3.0 (was: 3.3.1) > Add ConditionalExpression trait > ---

[jira] [Updated] (SPARK-38891) Skipping allocating vector for repetition & definition levels when possible

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38891: - Fix Version/s: 3.3.0 (was: 3.3.1) > Skipping allocating vector for repetition & d

[jira] [Updated] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39093: - Fix Version/s: 3.3.0 (was: 3.3.1) > Dividing interval by integral can result in c

[jira] [Updated] (SPARK-39087) Improve messages of error classes

2022-05-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39087: - Fix Version/s: 3.3.0 (was: 3.3.1) > Improve messages of error classes > -

[jira] [Commented] (SPARK-39176) Pyspark failed to serialize dates before 1970 in windows

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537446#comment-17537446 ] Apache Spark commented on SPARK-39176: -- User 'AnywalkerGiser' has created a pull re

[jira] [Resolved] (SPARK-39190) Provide query context for decimal precision overflow error when WSCG is off

2022-05-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-39190. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36557 [https:

[jira] [Assigned] (SPARK-39096) Support MERGE commands with DEFAULT values

2022-05-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-39096: -- Assignee: Daniel > Support MERGE commands with DEFAULT values > -

[jira] [Resolved] (SPARK-39096) Support MERGE commands with DEFAULT values

2022-05-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-39096. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36445 [https:

[jira] [Commented] (SPARK-39176) Pyspark failed to serialize dates before 1970 in windows

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537417#comment-17537417 ] Apache Spark commented on SPARK-39176: -- User 'AnywalkerGiser' has created a pull re

[jira] [Commented] (SPARK-39176) Pyspark failed to serialize dates before 1970 in windows

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537416#comment-17537416 ] Apache Spark commented on SPARK-39176: -- User 'AnywalkerGiser' has created a pull re

[jira] [Commented] (SPARK-39195) Spark should use two step update of outputCommitCoordinator

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537414#comment-17537414 ] Apache Spark commented on SPARK-39195: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-39192) make pandas-on-spark's kurt consistent with pandas

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39192: Assignee: (was: Apache Spark) > make pandas-on-spark's kurt consistent with pandas >

[jira] [Assigned] (SPARK-39195) Spark should use two step update of outputCommitCoordinator

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39195: Assignee: Apache Spark > Spark should use two step update of outputCommitCoordinator > --

[jira] [Commented] (SPARK-39192) make pandas-on-spark's kurt consistent with pandas

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537413#comment-17537413 ] Apache Spark commented on SPARK-39192: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-39192) make pandas-on-spark's kurt consistent with pandas

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39192: Assignee: Apache Spark > make pandas-on-spark's kurt consistent with pandas > ---

[jira] [Assigned] (SPARK-39195) Spark should use two step update of outputCommitCoordinator

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39195: Assignee: (was: Apache Spark) > Spark should use two step update of outputCommitCoord

[jira] [Assigned] (SPARK-39194) Add a pre resolution builder for spark session extensions

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39194: Assignee: Apache Spark > Add a pre resolution builder for spark session extensions >

[jira] [Commented] (SPARK-37939) Use error classes in the parsing errors of properties

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537412#comment-17537412 ] Apache Spark commented on SPARK-37939: -- User 'panbingkun' has created a pull reques

[jira] [Commented] (SPARK-39194) Add a pre resolution builder for spark session extensions

2022-05-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537415#comment-17537415 ] Apache Spark commented on SPARK-39194: -- User 'yaooqinn' has created a pull request

  1   2   >