[jira] [Commented] (SPARK-37681) Support ANSI Aggregate Function: regr_sxy

2021-12-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461806#comment-17461806 ] jiaan.geng commented on SPARK-37681: I'm working on. > Support ANSI Aggregate Function: regr_sxy >

[jira] [Created] (SPARK-37681) Support ANSI Aggregate Function: regr_sxy

2021-12-17 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-37681: -- Summary: Support ANSI Aggregate Function: regr_sxy Key: SPARK-37681 URL: https://issues.apache.org/jira/browse/SPARK-37681 Project: Spark Issue Type: Sub-task

[jira] (SPARK-37672) Support ANSI Aggregate Function: regr_sxx

2021-12-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37672 ] jiaan.geng deleted comment on SPARK-37672: was (Author: beliefer): I'm working on. > Support ANSI Aggregate Function: regr_sxx > - > >

[jira] [Commented] (SPARK-37672) Support ANSI Aggregate Function: regr_sxx

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461805#comment-17461805 ] Apache Spark commented on SPARK-37672: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37672) Support ANSI Aggregate Function: regr_sxx

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37672: Assignee: (was: Apache Spark) > Support ANSI Aggregate Function: regr_sxx >

[jira] [Assigned] (SPARK-37672) Support ANSI Aggregate Function: regr_sxx

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37672: Assignee: Apache Spark > Support ANSI Aggregate Function: regr_sxx >

[jira] [Commented] (SPARK-37680) Support RocksDB backend in Spark History Server

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461804#comment-17461804 ] Apache Spark commented on SPARK-37680: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-37680) Support RocksDB backend in Spark History Server

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37680: Assignee: (was: Apache Spark) > Support RocksDB backend in Spark History Server >

[jira] [Assigned] (SPARK-37680) Support RocksDB backend in Spark History Server

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37680: Assignee: Apache Spark > Support RocksDB backend in Spark History Server >

[jira] [Created] (SPARK-37680) Support RocksDB backend in Spark History Server

2021-12-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-37680: - Summary: Support RocksDB backend in Spark History Server Key: SPARK-37680 URL: https://issues.apache.org/jira/browse/SPARK-37680 Project: Spark Issue

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461803#comment-17461803 ] Apache Spark commented on SPARK-6305: - User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461800#comment-17461800 ] jingxiong zhong commented on SPARK-37677: - :D > spark on k8s, when the user want to push

[jira] [Commented] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461799#comment-17461799 ] jingxiong zhong commented on SPARK-37677: - Let me have a try. > spark on k8s, when the user

[jira] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677 ] jingxiong zhong deleted comment on SPARK-37677: - was (Author: JIRAUSER281124): :D > spark on k8s, when the user want to push python3.6.6.zip to the pod , but no > permission to execute

[jira] [Commented] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461798#comment-17461798 ] Hyukjin Kwon commented on SPARK-37677: -- I know that some implementation of zip's do that via

[jira] [Commented] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461797#comment-17461797 ] jingxiong zhong commented on SPARK-37677: - can we fix it throuth modify file permissions when

[jira] [Resolved] (SPARK-37673) Implement `ps.timedelta_range` method

2021-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37673. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34932

[jira] [Assigned] (SPARK-37673) Implement `ps.timedelta_range` method

2021-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37673: Assignee: Xinrong Meng > Implement `ps.timedelta_range` method >

[jira] [Assigned] (SPARK-6305) Add support for log4j 2.x to Spark

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-6305: Assignee: L. C. Hsieh > Add support for log4j 2.x to Spark >

[jira] [Resolved] (SPARK-6305) Add support for log4j 2.x to Spark

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-6305. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34895

[jira] [Assigned] (SPARK-37258) Upgrade kubernetes-client to 5.11.0 to support Volcano

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37258: Assignee: (was: Apache Spark) > Upgrade kubernetes-client to 5.11.0 to support

[jira] [Commented] (SPARK-37258) Upgrade kubernetes-client to 5.11.0 to support Volcano

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461769#comment-17461769 ] Apache Spark commented on SPARK-37258: -- User 'Yikun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37258) Upgrade kubernetes-client to 5.11.0 to support Volcano

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37258: Assignee: Apache Spark > Upgrade kubernetes-client to 5.11.0 to support Volcano >

[jira] [Resolved] (SPARK-37604) Change emptyValueInRead's effect to that any fields matching this string will be set as "" when reading csv files

2021-12-17 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo resolved SPARK-37604. - Resolution: Not A Problem > Change emptyValueInRead's effect to that any fields matching this string

[jira] [Assigned] (SPARK-37331) Add the ability to creating resources before driver pod

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37331: - Assignee: Yikun Jiang > Add the ability to creating resources before driver pod >

[jira] [Resolved] (SPARK-37331) Add the ability to creating resources before driver pod

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37331. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34599

[jira] [Assigned] (SPARK-37679) Add a new executor roll policy, FAILED_TASKS

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37679: - Assignee: William Hyun > Add a new executor roll policy, FAILED_TASKS >

[jira] [Resolved] (SPARK-37679) Add a new executor roll policy, FAILED_TASKS

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37679. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34938

[jira] [Commented] (SPARK-37679) Add a new executor roll policy, FAILED_TASKS

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461728#comment-17461728 ] Apache Spark commented on SPARK-37679: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-37679) Add a new executor roll policy, FAILED_TASKS

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37679: Assignee: Apache Spark > Add a new executor roll policy, FAILED_TASKS >

[jira] [Assigned] (SPARK-37679) Add a new executor roll policy, FAILED_TASKS

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37679: Assignee: (was: Apache Spark) > Add a new executor roll policy, FAILED_TASKS >

[jira] [Commented] (SPARK-37679) Add a new executor roll policy, FAILED_TASKS

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461727#comment-17461727 ] Apache Spark commented on SPARK-37679: -- User 'williamhyun' has created a pull request for this

[jira] [Created] (SPARK-37679) Add a new executor roll policy, FAILED_TASKS

2021-12-17 Thread William Hyun (Jira)
William Hyun created SPARK-37679: Summary: Add a new executor roll policy, FAILED_TASKS Key: SPARK-37679 URL: https://issues.apache.org/jira/browse/SPARK-37679 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

2021-12-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461681#comment-17461681 ] Takuya Ueshin commented on SPARK-37678: --- Yes! > Incorrect annotations in

[jira] [Commented] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

2021-12-17 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461679#comment-17461679 ] Maciej Szymkiewicz commented on SPARK-37678: ??It's an actual implementation of an abstract

[jira] [Comment Edited] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

2021-12-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461677#comment-17461677 ] Takuya Ueshin edited comment on SPARK-37678 at 12/17/21, 9:30 PM: -- Good

[jira] [Commented] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

2021-12-17 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461677#comment-17461677 ] Takuya Ueshin commented on SPARK-37678: --- Good catch! It must be {{{}_cleanup_and_return(self,

[jira] [Resolved] (SPARK-37663) Mitigate ConcurrentModificationException thrown from tests in SparkContextSuite

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37663. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34922

[jira] [Resolved] (SPARK-37620) Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm)

2021-12-17 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37620. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37620) Use more precise types for SparkContext Optional fields (i.e. _gateway, _jvm)

2021-12-17 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37620: -- Assignee: Byron Hsu > Use more precise types for SparkContext Optional

[jira] [Resolved] (SPARK-37655) Add RocksDB Implementation for KVStore

2021-12-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37655. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34913

[jira] [Commented] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

2021-12-17 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461609#comment-17461609 ] Maciej Szymkiewicz commented on SPARK-37678: cc [~ueshin] Do you have any thoughts on this

[jira] [Created] (SPARK-37678) Incorrect annotations in SeriesGroupBy._cleanup_and_return

2021-12-17 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-37678: -- Summary: Incorrect annotations in SeriesGroupBy._cleanup_and_return Key: SPARK-37678 URL: https://issues.apache.org/jira/browse/SPARK-37678 Project:

[jira] [Commented] (SPARK-37604) Change emptyValueInRead's effect to that any fields matching this string will be set as "" when reading csv files

2021-12-17 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461525#comment-17461525 ] Wei Guo commented on SPARK-37604: - Well, I think your explanation is clearly and reasonable and it

[jira] [Commented] (SPARK-37604) Change emptyValueInRead's effect to that any fields matching this string will be set as "" when reading csv files

2021-12-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461495#comment-17461495 ] Max Gekk commented on SPARK-37604: -- 1. We don't declare that the behavior with the options *emptyValue*

[jira] [Updated] (SPARK-37675) Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block

2021-12-17 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-37675: -- Summary: Return PushMergedRemoteMetaFailedFetchResult if no available push-merged block (was: Fix

[jira] [Commented] (SPARK-32307) Aggression that use map type input UDF as group expression can fail

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461452#comment-17461452 ] Apache Spark commented on SPARK-32307: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Resolved] (SPARK-37343) Implement createIndex and IndexExists in JDBC (Postgres dialect)

2021-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37343. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34673

[jira] [Assigned] (SPARK-37343) Implement createIndex and IndexExists in JDBC (Postgres dialect)

2021-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37343: --- Assignee: dch nguyen > Implement createIndex and IndexExists in JDBC (Postgres dialect) >

[jira] [Commented] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461407#comment-17461407 ] Hyukjin Kwon commented on SPARK-37677: -- zip compression algorithm by design doesn't keep the

[jira] [Updated] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37677: - Environment: (was: spark-3.2.0) > spark on k8s, when the user want to push python3.6.6.zip

[jira] [Commented] (SPARK-36088) 'spark.archives' does not extract the archive file into the driver under client mode

2021-12-17 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461405#comment-17461405 ] jingxiong zhong commented on SPARK-36088: - @hyukjin.kwon I make an issue at

[jira] [Resolved] (SPARK-37564) Support sort aggregate code-gen without grouping keys

2021-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37564. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34826

[jira] [Assigned] (SPARK-37564) Support sort aggregate code-gen without grouping keys

2021-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37564: --- Assignee: Cheng Su > Support sort aggregate code-gen without grouping keys >

[jira] [Created] (SPARK-37677) spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute

2021-12-17 Thread jingxiong zhong (Jira)
jingxiong zhong created SPARK-37677: --- Summary: spark on k8s, when the user want to push python3.6.6.zip to the pod , but no permission to execute Key: SPARK-37677 URL:

[jira] [Assigned] (SPARK-37676) Support ANSI Aggregation Function: percentile_cont

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37676: Assignee: Apache Spark > Support ANSI Aggregation Function: percentile_cont >

[jira] [Commented] (SPARK-37676) Support ANSI Aggregation Function: percentile_cont

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461394#comment-17461394 ] Apache Spark commented on SPARK-37676: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37676) Support ANSI Aggregation Function: percentile_cont

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37676: Assignee: (was: Apache Spark) > Support ANSI Aggregation Function: percentile_cont >

[jira] [Commented] (SPARK-37604) Change emptyValueInRead's effect to that any fields matching this string will be set as "" when reading csv files

2021-12-17 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461393#comment-17461393 ] Wei Guo commented on SPARK-37604: - As the consideration of Hyukjin Kwon in the PR related, if we worry

[jira] [Commented] (SPARK-37604) Change emptyValueInRead's effect to that any fields matching this string will be set as "" when reading csv files

2021-12-17 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461390#comment-17461390 ] Wei Guo commented on SPARK-37604: - In short, for null values, we can save null values in dataframe as

[jira] [Resolved] (SPARK-27974) Support ANSI Aggregate Function: array_agg

2021-12-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27974. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34866

[jira] [Comment Edited] (SPARK-37521) insert overwrite table but the partition information stored in Metastore was not changed

2021-12-17 Thread jingxiong zhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452521#comment-17452521 ] jingxiong zhong edited comment on SPARK-37521 at 12/17/21, 10:45 AM: -

[jira] [Commented] (SPARK-37675) Fix calculate approxChunkSize when bitmaps isEmpty

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461330#comment-17461330 ] Apache Spark commented on SPARK-37675: -- User 'pan3793' has created a pull request for this issue:

[jira] [Commented] (SPARK-37675) Fix calculate approxChunkSize when bitmaps isEmpty

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461329#comment-17461329 ] Apache Spark commented on SPARK-37675: -- User 'pan3793' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37675) Fix calculate approxChunkSize when bitmaps isEmpty

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37675: Assignee: Apache Spark > Fix calculate approxChunkSize when bitmaps isEmpty >

[jira] [Assigned] (SPARK-37675) Fix calculate approxChunkSize when bitmaps isEmpty

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37675: Assignee: (was: Apache Spark) > Fix calculate approxChunkSize when bitmaps isEmpty >

[jira] [Updated] (SPARK-37676) Support ANSI Aggregation Function: percentile_cont

2021-12-17 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37676: --- Description: PERCENTILE_CONT is an ANSI aggregate function. many database support it. (was:

[jira] [Created] (SPARK-37676) Support ANSI Aggregation Function: percentile_cont

2021-12-17 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-37676: -- Summary: Support ANSI Aggregation Function: percentile_cont Key: SPARK-37676 URL: https://issues.apache.org/jira/browse/SPARK-37676 Project: Spark Issue Type:

[jira] [Created] (SPARK-37675) Fix calculate approxChunkSize when bitmaps isEmpty

2021-12-17 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-37675: - Summary: Fix calculate approxChunkSize when bitmaps isEmpty Key: SPARK-37675 URL: https://issues.apache.org/jira/browse/SPARK-37675 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36088) 'spark.archives' does not extract the archive file into the driver under client mode

2021-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461305#comment-17461305 ] Hyukjin Kwon commented on SPARK-36088: -- [~rickcheng] can you try a fix? we can maybe try to

[jira] [Commented] (SPARK-36088) 'spark.archives' does not extract the archive file into the driver under client mode

2021-12-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461298#comment-17461298 ] Hyukjin Kwon commented on SPARK-36088: -- [~zhongjingxiong] can you try with tar.gz? zip doesn't keep

[jira] [Commented] (SPARK-37674) Reduce the output partition of output stage to avoid producing small files.

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461286#comment-17461286 ] Apache Spark commented on SPARK-37674: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37674) Reduce the output partition of output stage to avoid producing small files.

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37674: Assignee: Apache Spark > Reduce the output partition of output stage to avoid producing

[jira] [Commented] (SPARK-37674) Reduce the output partition of output stage to avoid producing small files.

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461285#comment-17461285 ] Apache Spark commented on SPARK-37674: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37674) Reduce the output partition of output stage to avoid producing small files.

2021-12-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37674: Assignee: (was: Apache Spark) > Reduce the output partition of output stage to avoid

[jira] [Updated] (SPARK-37258) Upgrade kubernetes-client to 5.11.0 (add Volcano support)

2021-12-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Summary: Upgrade kubernetes-client to 5.11.0 (add Volcano support) (was: Upgrade

[jira] [Updated] (SPARK-37258) Upgrade kubernetes-client to 5.11.0 to support Volcano

2021-12-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Summary: Upgrade kubernetes-client to 5.11.0 to support Volcano (was: Upgrade kubernetes-client

[jira] [Updated] (SPARK-37258) Upgrade kubernetes-client to 5.11.0

2021-12-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Summary: Upgrade kubernetes-client to 5.11.0 (was: Add Volcano support in kubernetes-client) >

[jira] [Created] (SPARK-37674) Reduce the output partition of output stage to avoid producing small files.

2021-12-17 Thread weixiuli (Jira)
weixiuli created SPARK-37674: Summary: Reduce the output partition of output stage to avoid producing small files. Key: SPARK-37674 URL: https://issues.apache.org/jira/browse/SPARK-37674 Project: Spark