[jira] [Created] (SPARK-46287) DataFrame.isEmpty should work with all datatypes

2023-12-06 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46287: - Summary: DataFrame.isEmpty should work with all datatypes Key: SPARK-46287 URL: https://issues.apache.org/jira/browse/SPARK-46287 Project: Spark Issue Type

[jira] [Commented] (SPARK-40939) Release a shaded version of Apache Spark / shade jars on main jar

2023-12-06 Thread Will Boulter (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793632#comment-17793632 ] Will Boulter commented on SPARK-40939: -- This would make a huge difference to anyone

[jira] [Updated] (SPARK-46286) Document spark.io.compression.zstd.bufferPool.enabled

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46286: --- Labels: pull-request-available (was: ) > Document spark.io.compression.zstd.bufferPool.enab

[jira] [Updated] (SPARK-46278) Re-organize `GroupByTests`

2023-12-06 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46278: -- Priority: Minor (was: Major) > Re-organize `GroupByTests` > -- > >

[jira] [Updated] (SPARK-46278) Re-organize `GroupByTests`

2023-12-06 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46278: -- Issue Type: Test (was: Bug) > Re-organize `GroupByTests` > -- > >

[jira] [Created] (SPARK-46286) Document spark.io.compression.zstd.bufferPool.enabled

2023-12-06 Thread Kent Yao (Jira)
Kent Yao created SPARK-46286: Summary: Document spark.io.compression.zstd.bufferPool.enabled Key: SPARK-46286 URL: https://issues.apache.org/jira/browse/SPARK-46286 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-46285) Add foreachWithSubqueries

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46285: --- Labels: pull-request-available (was: ) > Add foreachWithSubqueries > --

[jira] [Created] (SPARK-46285) Add foreachWithSubqueries

2023-12-06 Thread Rui Wang (Jira)
Rui Wang created SPARK-46285: Summary: Add foreachWithSubqueries Key: SPARK-46285 URL: https://issues.apache.org/jira/browse/SPARK-46285 Project: Spark Issue Type: Improvement Component

[jira] [Updated] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46283: - Description: After the merge of https://github.com/apache/spark/pull/43736, the master branch began tes

[jira] [Assigned] (SPARK-46228) Insert window group limit node for cumulative aggregation with limit

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46228: -- Assignee: (was: Apache Spark) > Insert window group limit node for cumulative agg

[jira] [Assigned] (SPARK-46228) Insert window group limit node for cumulative aggregation with limit

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46228: -- Assignee: Apache Spark > Insert window group limit node for cumulative aggregation wi

[jira] [Assigned] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46283: -- Assignee: Apache Spark > Avoid testing the `streaming-kinesis-asl` module in the dail

[jira] [Assigned] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46283: -- Assignee: Apache Spark > Avoid testing the `streaming-kinesis-asl` module in the dail

[jira] [Assigned] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46283: -- Assignee: (was: Apache Spark) > Avoid testing the `streaming-kinesis-asl` module

[jira] [Assigned] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46283: -- Assignee: Apache Spark > Avoid testing the `streaming-kinesis-asl` module in the dail

[jira] [Assigned] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46283: -- Assignee: (was: Apache Spark) > Avoid testing the `streaming-kinesis-asl` module

[jira] [Assigned] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46283: -- Assignee: Apache Spark > Avoid testing the `streaming-kinesis-asl` module in the dail

[jira] [Assigned] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46283: -- Assignee: (was: Apache Spark) > Avoid testing the `streaming-kinesis-asl` module

[jira] [Resolved] (SPARK-46281) Remove unused code in `pyspark.pandas.tests.computation.*`

2023-12-06 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46281. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44200 [https://

[jira] [Assigned] (SPARK-46228) Insert window group limit node for cumulative aggregation with limit

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46228: -- Assignee: (was: Apache Spark) > Insert window group limit node for cumulative agg

[jira] [Assigned] (SPARK-46228) Insert window group limit node for cumulative aggregation with limit

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46228: -- Assignee: Apache Spark > Insert window group limit node for cumulative aggregation wi

[jira] [Updated] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46283: --- Labels: pull-request-available (was: ) > Avoid testing the `streaming-kinesis-asl` module i

[jira] [Updated] (SPARK-46283) Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x.

2023-12-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46283: - Summary: Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x. (was: "Avoi

[jira] [Created] (SPARK-46283) "Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x."

2023-12-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-46283: Summary: "Avoid testing the `streaming-kinesis-asl` module in the daily tests of branch-3.x." Key: SPARK-46283 URL: https://issues.apache.org/jira/browse/SPARK-46283 Project:

[jira] [Resolved] (SPARK-46280) Move test_parity_frame_resample and test_parity_series_resample to pyspark.pandas.tests.connect.resample

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46280. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44199 [https://gi

[jira] [Updated] (SPARK-46279) Support write partition values to data files

2023-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46279: --- Labels: pull-request-available (was: ) > Support write partition values to data files > ---

[jira] [Assigned] (SPARK-46280) Move test_parity_frame_resample and test_parity_series_resample to pyspark.pandas.tests.connect.resample

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46280: Assignee: Ruifeng Zheng > Move test_parity_frame_resample and test_parity_series_resample

[jira] [Resolved] (SPARK-46276) Improve test coverage of pyspark utils

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46276. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44192 [https://gi

[jira] [Assigned] (SPARK-46276) Improve test coverage of pyspark utils

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46276: Assignee: Xinrong Meng > Improve test coverage of pyspark utils > ---

[jira] [Resolved] (SPARK-46260) DataFrame.withColumnsRenamed should respect the dict ordering

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46260. -- Assignee: Ruifeng Zheng Resolution: Fixed Fixed in https://github.com/apache/spark/pull/

[jira] [Assigned] (SPARK-46213) Add PySparkImportError for error framework

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46213: Assignee: Haejoon Lee > Add PySparkImportError for error framework >

[jira] [Resolved] (SPARK-46252) Improve test coverage of memory_profiler.py

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46252. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44167 [https://gi

[jira] [Resolved] (SPARK-46213) Add PySparkImportError for error framework

2023-12-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46213. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44176 [https://gi

<    1   2