[jira] [Created] (SPARK-42879) Spark SQL reads unnecessary nested fields

2023-03-21 Thread Jiri Humpolicek (Jira)
Jiri Humpolicek created SPARK-42879: --- Summary: Spark SQL reads unnecessary nested fields Key: SPARK-42879 URL: https://issues.apache.org/jira/browse/SPARK-42879 Project: Spark Issue Type: I

[jira] [Created] (SPARK-42880) Improve the yarn document for lo4j2 configuration

2023-03-21 Thread Zhifang Li (Jira)
Zhifang Li created SPARK-42880: -- Summary: Improve the yarn document for lo4j2 configuration Key: SPARK-42880 URL: https://issues.apache.org/jira/browse/SPARK-42880 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-33307) Refactor GROUPING ANALYTICS

2023-03-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33307. - Fix Version/s: 3.2.0 Assignee: angerszhu Resolution: Fixed > Refactor GROUPING A

[jira] [Assigned] (SPARK-42880) Improve the yarn document for lo4j2 configuration

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42880: Assignee: Apache Spark > Improve the yarn document for lo4j2 configuration >

[jira] [Assigned] (SPARK-42880) Improve the yarn document for lo4j2 configuration

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42880: Assignee: (was: Apache Spark) > Improve the yarn document for lo4j2 configuration > -

[jira] [Commented] (SPARK-42880) Improve the yarn document for lo4j2 configuration

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703071#comment-17703071 ] Apache Spark commented on SPARK-42880: -- User 'frankliee' has created a pull request

[jira] [Updated] (SPARK-42340) Implement GroupedData.applyInPandas

2023-03-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42340: - Fix Version/s: 3.4.1 (was: 3.5.0) > Implement GroupedData.applyInPandas >

[jira] [Created] (SPARK-42881) get_json_object Codegen Support

2023-03-21 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-42881: --- Summary: get_json_object Codegen Support Key: SPARK-42881 URL: https://issues.apache.org/jira/browse/SPARK-42881 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-42881) get_json_object Codegen Support

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42881: Assignee: (was: Apache Spark) > get_json_object Codegen Support > ---

[jira] [Assigned] (SPARK-42881) get_json_object Codegen Support

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42881: Assignee: Apache Spark > get_json_object Codegen Support > --

[jira] [Commented] (SPARK-42881) get_json_object Codegen Support

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703077#comment-17703077 ] Apache Spark commented on SPARK-42881: -- User 'panbingkun' has created a pull reques

[jira] [Commented] (SPARK-42881) get_json_object Codegen Support

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703078#comment-17703078 ] Apache Spark commented on SPARK-42881: -- User 'panbingkun' has created a pull reques

[jira] [Updated] (SPARK-42881) Codegen Support for get_json_object

2023-03-21 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-42881: Summary: Codegen Support for get_json_object (was: get_json_object Codegen Support) > Codegen Su

[jira] [Resolved] (SPARK-42876) DataType's physicalDataType should be private[sql]

2023-03-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42876. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40499 [https://gith

[jira] [Updated] (SPARK-42662) Add `distributed_sequence_id` as an internal function.

2023-03-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42662: Summary: Add `distributed_sequence_id` as an internal function. (was: Support `withSequenceColumn

[jira] [Updated] (SPARK-42662) Add `_distributed_sequence_id` for distributed-sequence index.

2023-03-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42662: Summary: Add `_distributed_sequence_id` for distributed-sequence index. (was: Add `distributed_se

[jira] [Commented] (SPARK-42662) Add `_distributed_sequence_id` for distributed-sequence index.

2023-03-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703112#comment-17703112 ] Apache Spark commented on SPARK-42662: -- User 'itholic' has created a pull request f

[jira] [Created] (SPARK-42882) Implement missing Pandas API and incomplete parameters

2023-03-21 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42882: Summary: Implement missing Pandas API and incomplete parameters Key: SPARK-42882 URL: https://issues.apache.org/jira/browse/SPARK-42882 Project: Spark Issue

[jira] [Updated] (SPARK-40345) Implement `ExpandingGroupby.quantile`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40345: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `ExpandingGroupby.quantile`. > -

[jira] [Updated] (SPARK-40498) Implement `kendall` and `min_periods` in `Series.corr`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40498: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `kendall` and `min_periods` in `Series.corr`

[jira] [Updated] (SPARK-40579) `GroupBy.first` should skip nulls

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40579: - Parent: SPARK-42882 (was: SPARK-40327) > `GroupBy.first` should skip nulls > --

[jira] [Updated] (SPARK-40621) Implement `numeric_only` and `min_count` in `GroupBy.sum`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40621: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `numeric_only` and `min_count` in `GroupBy.s

[jira] [Updated] (SPARK-40386) Implement `ddof` in `DataFrame.cov`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40386: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `ddof` in `DataFrame.cov` >

[jira] [Updated] (SPARK-40305) Implement Groupby.sem

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40305: - Parent: SPARK-42882 (was: SPARK-40327) > Implement Groupby.sem > - > >

[jira] [Updated] (SPARK-40445) Refactor Resampler

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40445: - Parent: SPARK-42882 (was: SPARK-40327) > Refactor Resampler > -- > >

[jira] [Updated] (SPARK-40542) Make `ddof` in `DataFrame.std` and `Series.std` accept arbitary integers

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40542: - Parent: SPARK-42882 (was: SPARK-40327) > Make `ddof` in `DataFrame.std` and `Series.std` accept

[jira] [Updated] (SPARK-40332) Implement `GroupBy.quantile`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40332: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `GroupBy.quantile`. > --

[jira] [Updated] (SPARK-40447) Implement `kendall` correlation in `DataFrame.corr`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40447: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `kendall` correlation in `DataFrame.corr` >

[jira] [Updated] (SPARK-40698) Improve the precision of `product` for intergral inputs

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40698: - Parent: SPARK-42882 (was: SPARK-40327) > Improve the precision of `product` for intergral input

[jira] [Updated] (SPARK-40348) Implement `RollingGroupby.quantile`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40348: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `RollingGroupby.quantile`. > ---

[jira] [Updated] (SPARK-40510) Implement `ddof` in `Series.cov`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40510: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `ddof` in `Series.cov` > ---

[jira] [Updated] (SPARK-40341) Implement `Rolling.median`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40341: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `Rolling.median`. >

[jira] [Updated] (SPARK-40744) Make `_reduce_for_stat_function` in `groupby` accept `min_count`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40744: - Parent: SPARK-42882 (was: SPARK-40327) > Make `_reduce_for_stat_function` in `groupby` accept `

[jira] [Updated] (SPARK-40631) Implement `min_count` in `GroupBy.first`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40631: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `min_count` in `GroupBy.first` > ---

[jira] [Updated] (SPARK-40339) Implement `Expanding.quantile`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40339: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `Expanding.quantile`. >

[jira] [Updated] (SPARK-40643) Implement `min_count` in `GroupBy.last`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40643: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `min_count` in `GroupBy.last` >

[jira] [Updated] (SPARK-40161) Make Series.mode apply PandasMode

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40161: - Parent: SPARK-42882 (was: SPARK-40327) > Make Series.mode apply PandasMode > --

[jira] [Updated] (SPARK-40592) Implement `min_count` in `GroupBy.max`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40592: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `min_count` in `GroupBy.max` > -

[jira] [Updated] (SPARK-40573) Make `ddof` in `GroupBy.std`, `GroupBy.var` and `GroupBy.sem` accept arbitary integers

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40573: - Parent: SPARK-42882 (was: SPARK-40327) > Make `ddof` in `GroupBy.std`, `GroupBy.var` and `Group

[jira] [Updated] (SPARK-40340) Implement `Expanding.sem`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40340: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `Expanding.sem`. > -

[jira] [Updated] (SPARK-40421) Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40421: - Parent: SPARK-42882 (was: SPARK-40327) > Make `spearman` correlation in `DataFrame.corr` suppor

[jira] [Updated] (SPARK-40503) Add resampling to API references

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40503: - Parent: SPARK-42882 (was: SPARK-40327) > Add resampling to API references > ---

[jira] [Updated] (SPARK-40399) Make `pearson` correlation in `DataFrame.corr` support missing values and `min_periods`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40399: - Parent: SPARK-42882 (was: SPARK-40327) > Make `pearson` correlation in `DataFrame.corr` support

[jira] [Updated] (SPARK-40543) Make `ddof` in `DataFrame.var` and `Series.var` accept arbitary integers

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40543: - Parent: SPARK-42882 (was: SPARK-40327) > Make `ddof` in `DataFrame.var` and `Series.var` accept

[jira] [Updated] (SPARK-40486) Implement `spearman` and `kendall` in `DataFrame.corrwith`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40486: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `spearman` and `kendall` in `DataFrame.corrw

[jira] [Updated] (SPARK-40561) Implement `min_count` in GroupBy.min

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40561: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `min_count` in GroupBy.min > ---

[jira] [Updated] (SPARK-40446) Rename `_MissingPandasXXX` as `MissingPandasXXX`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40446: - Parent: SPARK-42882 (was: SPARK-40327) > Rename `_MissingPandasXXX` as `MissingPandasXXX` > ---

[jira] [Updated] (SPARK-40529) Remove `pyspark.pandas.ml`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40529: - Parent: SPARK-42882 (was: SPARK-40327) > Remove `pyspark.pandas.ml` > -

[jira] [Updated] (SPARK-40330) Implement `Series.searchsorted`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40330: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `Series.searchsorted`. > ---

[jira] [Updated] (SPARK-40554) Make `ddof` in `DataFrame.sem` and `Series.sem` accept arbitary integers

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40554: - Parent: SPARK-42882 (was: SPARK-40327) > Make `ddof` in `DataFrame.sem` and `Series.sem` accept

[jira] [Updated] (SPARK-40138) Implement DataFrame.mode

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40138: - Parent: SPARK-42882 (was: SPARK-40327) > Implement DataFrame.mode > >

[jira] [Updated] (SPARK-40342) Implement `Rolling.quantile`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40342: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `Rolling.quantile`. > --

[jira] [Updated] (SPARK-40333) Implement `GroupBy.nth`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40333: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `GroupBy.nth`. > >

[jira] [Updated] (SPARK-40135) Support ps.Index in DataFrame creation

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40135: - Parent: SPARK-42882 (was: SPARK-40327) > Support ps.Index in DataFrame creation > -

[jira] [Updated] (SPARK-40313) ps.DataFrame(data, index) should support the same anchor

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40313: - Parent: SPARK-42882 (was: SPARK-40327) > ps.DataFrame(data, index) should support the same anch

[jira] [Updated] (SPARK-40393) Refactor expanding and rolling test for function with input

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40393: - Parent: SPARK-42882 (was: SPARK-40327) > Refactor expanding and rolling test for function with

[jira] [Updated] (SPARK-42882) Pandas API Coverage Improvements

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42882: - Summary: Pandas API Coverage Improvements (was: Implement missing Pandas API and incomplete par

[jira] [Updated] (SPARK-40334) Implement `GroupBy.prod`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40334: - Parent: SPARK-42882 (was: SPARK-40327) > Implement `GroupBy.prod`. > -

[jira] [Created] (SPARK-42883) Implement Pandas API Missing Parameters

2023-03-21 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42883: Summary: Implement Pandas API Missing Parameters Key: SPARK-42883 URL: https://issues.apache.org/jira/browse/SPARK-42883 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-39199) Implement pandas API missing parameters

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39199: - Description: (was: pandas API on Spark aims to make pandas code work on Spark clusters witho

[jira] [Updated] (SPARK-42194) Allow `columns` parameter when creating DataFrame with Series.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42194: - Parent: SPARK-42883 (was: SPARK-39199) > Allow `columns` parameter when creating DataFrame with

[jira] [Updated] (SPARK-38837) Implement `dropna` parameter of `SeriesGroupBy.value_counts`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38837: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `dropna` parameter of `SeriesGroupBy.value_c

[jira] [Updated] (SPARK-39228) Implement `skipna` of `Series.argmax`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39228: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `skipna` of `Series.argmax` > --

[jira] [Assigned] (SPARK-42883) Implement Pandas API Missing Parameters

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42883: Assignee: Xinrong Meng > Implement Pandas API Missing Parameters > --

[jira] [Updated] (SPARK-38890) Implement `ignore_index` of `DataFrame.sort_index`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38890: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `ignore_index` of `DataFrame.sort_index`. >

[jira] [Updated] (SPARK-38400) Enable Series.rename to change index labels

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38400: - Parent: SPARK-42883 (was: SPARK-39199) > Enable Series.rename to change index labels >

[jira] [Updated] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38763: - Parent: SPARK-42883 (was: SPARK-39199) > Pandas API on spark Can`t apply lamda to columns. >

[jira] [Updated] (SPARK-38937) interpolate support param `limit_direction`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38937: - Parent: SPARK-42883 (was: SPARK-39199) > interpolate support param `limit_direction` >

[jira] [Updated] (SPARK-38863) Implement `skipna` parameter of `DataFrame.all`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38863: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `skipna` parameter of `DataFrame.all` >

[jira] [Updated] (SPARK-38491) Support `ignore_index` of `Series.sort_values`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38491: - Parent: SPARK-42883 (was: SPARK-39199) > Support `ignore_index` of `Series.sort_values` > -

[jira] [Updated] (SPARK-38793) Support `return_indexer` parameter of `Index/MultiIndex.sort_values`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38793: - Parent: SPARK-42883 (was: SPARK-39199) > Support `return_indexer` parameter of `Index/MultiInde

[jira] [Updated] (SPARK-38441) Support string and bool `regex` in `Series.replace`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38441: - Parent: SPARK-42883 (was: SPARK-39199) > Support string and bool `regex` in `Series.replace` >

[jira] [Updated] (SPARK-38989) Implement `ignore_index` of `DataFrame/Series.sample`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38989: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `ignore_index` of `DataFrame/Series.sample`

[jira] [Updated] (SPARK-38726) Support `how` parameter of `MultiIndex.dropna`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38726: - Parent: SPARK-42883 (was: SPARK-39199) > Support `how` parameter of `MultiIndex.dropna` > -

[jira] [Updated] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `bool_only` parameter of `DataFrame.all` and

[jira] [Updated] (SPARK-38576) Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38576: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `numeric_only` parameter for `DataFrame/Seri

[jira] [Updated] (SPARK-38387) Support `na_action` and Series input correspondence in `Series.map`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38387: - Parent: SPARK-42883 (was: SPARK-39199) > Support `na_action` and Series input correspondence in

[jira] [Updated] (SPARK-39201) Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39201: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `ignore_index` of `DataFrame.explode` and >

[jira] [Updated] (SPARK-38704) Support string `inclusive` parameter of `Series.between`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38704: - Parent: SPARK-42883 (was: SPARK-39199) > Support string `inclusive` parameter of `Series.betwee

[jira] [Updated] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38765: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `inplace` parameter of `Series.clip` > -

[jira] [Updated] (SPARK-38686) Implement `keep` parameter of `(Index/MultiIndex).drop_duplicates`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38686: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `keep` parameter of `(Index/MultiIndex).drop

[jira] [Updated] (SPARK-39907) Implement axis and skipna of Series.argmin

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39907: - Parent: SPARK-42883 (was: SPARK-39199) > Implement axis and skipna of Series.argmin > -

[jira] [Updated] (SPARK-38903) Implement `ignore_index` of `Series.sort_values` and `Series.sort_index`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38903: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `ignore_index` of `Series.sort_values` and `

[jira] [Updated] (SPARK-38943) EWM support ignore_na

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38943: - Parent: SPARK-42883 (was: SPARK-39199) > EWM support ignore_na > - > >

[jira] [Updated] (SPARK-39189) interpolate supports limit_area

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39189: - Parent: SPARK-42883 (was: SPARK-39199) > interpolate supports limit_area >

[jira] [Updated] (SPARK-38518) Implement `skipna` of `Series.all/Index.all` to exclude NA/null values

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38518: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `skipna` of `Series.all/Index.all` to exclud

[jira] [Updated] (SPARK-38479) Add `Series.duplicated` to indicate duplicate Series values.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38479: - Parent: SPARK-42883 (was: SPARK-39199) > Add `Series.duplicated` to indicate duplicate Series v

[jira] [Updated] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38938: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `inplace` and `columns` parameters of `Serie

[jira] [Resolved] (SPARK-42882) Pandas API Coverage Improvements

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42882. -- Resolution: Resolved > Pandas API Coverage Improvements > > >

[jira] [Updated] (SPARK-38552) Implement `keep` parameter of `frame.nlargest/nsmallest` to decide how to resolve ties

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38552: - Parent: SPARK-42883 (was: SPARK-39199) > Implement `keep` parameter of `frame.nlargest/nsmalles

[jira] [Resolved] (SPARK-42883) Implement Pandas API Missing Parameters

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42883. -- Resolution: Resolved > Implement Pandas API Missing Parameters > -

[jira] [Commented] (SPARK-39199) Implement pandas API missing parameters

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703121#comment-17703121 ] Xinrong Meng commented on SPARK-39199: -- Please see https://issues.apache.org/jira/b

[jira] [Updated] (SPARK-40341) Implement `Rolling.median`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40341: - Parent: SPARK-40327 (was: SPARK-42882) > Implement `Rolling.median`. >

[jira] [Updated] (SPARK-40340) Implement `Expanding.sem`.

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40340: - Parent: SPARK-40327 (was: SPARK-42882) > Implement `Expanding.sem`. > -

[jira] [Updated] (SPARK-40327) Increase pandas API coverage for pandas API on Spark

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40327: - Fix Version/s: (was: 3.4.0) > Increase pandas API coverage for pandas API on Spark > ---

[jira] [Updated] (SPARK-40327) Increase pandas API coverage for pandas API on Spark

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40327: - Affects Version/s: 3.5.0 (was: 3.4.0) > Increase pandas API coverage

[jira] [Commented] (SPARK-40327) Increase pandas API coverage for pandas API on Spark

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703123#comment-17703123 ] Xinrong Meng commented on SPARK-40327: -- Hi, all resolved issues are moved to https

[jira] [Comment Edited] (SPARK-40327) Increase pandas API coverage for pandas API on Spark

2023-03-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703123#comment-17703123 ] Xinrong Meng edited comment on SPARK-40327 at 3/21/23 9:48 AM: ---

[jira] [Commented] (SPARK-41006) ConfigMap has the same name when launching two pods on the same namespace

2023-03-21 Thread Cedric van Eetvelde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703145#comment-17703145 ] Cedric van Eetvelde commented on SPARK-41006: - As mentionned above, I create

[jira] [Comment Edited] (SPARK-41006) ConfigMap has the same name when launching two pods on the same namespace

2023-03-21 Thread Cedric van Eetvelde (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703145#comment-17703145 ] Cedric van Eetvelde edited comment on SPARK-41006 at 3/21/23 11:03 AM: ---

  1   2   >