[jira] [Assigned] (SPARK-41648) Deduplicate docstrings in pyspark.sql.connect.readwriter

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41648: Assignee: Apache Spark > Deduplicate docstrings in pyspark.sql.connect.readwriter >

[jira] [Assigned] (SPARK-41648) Deduplicate docstrings in pyspark.sql.connect.readwriter

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41648: Assignee: (was: Apache Spark) > Deduplicate docstrings in

[jira] [Commented] (SPARK-41648) Deduplicate docstrings in pyspark.sql.connect.readwriter

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650178#comment-17650178 ] Apache Spark commented on SPARK-41648: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-41648) Deduplicate docstrings in pyspark.sql.connect.readwriter

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650174#comment-17650174 ] Apache Spark commented on SPARK-41648: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-41660) only propagate metadata columns if they are used

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650134#comment-17650134 ] Apache Spark commented on SPARK-41660: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41660) only propagate metadata columns if they are used

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41660: Assignee: (was: Apache Spark) > only propagate metadata columns if they are used >

[jira] [Assigned] (SPARK-41660) only propagate metadata columns if they are used

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41660: Assignee: Apache Spark > only propagate metadata columns if they are used >

[jira] [Created] (SPARK-41660) only propagate metadata columns if they are used

2022-12-20 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-41660: --- Summary: only propagate metadata columns if they are used Key: SPARK-41660 URL: https://issues.apache.org/jira/browse/SPARK-41660 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-41653) Test parity: enable doctests in Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41653: Assignee: Hyukjin Kwon > Test parity: enable doctests in Spark Connect >

[jira] [Assigned] (SPARK-41651) Test parity: pyspark.sql.tests.test_dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41651: Assignee: Hyukjin Kwon > Test parity: pyspark.sql.tests.test_dataframe >

[jira] [Assigned] (SPARK-41652) Test parity: pyspark.sql.tests.test_functions

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41652: Assignee: Hyukjin Kwon > Test parity: pyspark.sql.tests.test_functions >

[jira] [Commented] (SPARK-41653) Test parity: enable doctests in Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650122#comment-17650122 ] Hyukjin Kwon commented on SPARK-41653: -- cc jiaan.geng and Deng Ziming in case you guys are

[jira] [Commented] (SPARK-41653) Test parity: enable doctests in Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650121#comment-17650121 ] Hyukjin Kwon commented on SPARK-41653: -- If the files is too big, feel free to split the JIRA or

[jira] [Created] (SPARK-41659) Enable doctests in pyspark.sql.connect.readwriter

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41659: Summary: Enable doctests in pyspark.sql.connect.readwriter Key: SPARK-41659 URL: https://issues.apache.org/jira/browse/SPARK-41659 Project: Spark Issue

[jira] [Created] (SPARK-41657) Enable doctests in pyspark.sql.connect.session

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41657: Summary: Enable doctests in pyspark.sql.connect.session Key: SPARK-41657 URL: https://issues.apache.org/jira/browse/SPARK-41657 Project: Spark Issue Type:

[jira] [Created] (SPARK-41658) Enable doctests in pyspark.sql.connect.functions

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41658: Summary: Enable doctests in pyspark.sql.connect.functions Key: SPARK-41658 URL: https://issues.apache.org/jira/browse/SPARK-41658 Project: Spark Issue Type:

[jira] [Created] (SPARK-41655) Enable doctests in pyspark.sql.connect.column

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41655: Summary: Enable doctests in pyspark.sql.connect.column Key: SPARK-41655 URL: https://issues.apache.org/jira/browse/SPARK-41655 Project: Spark Issue Type:

[jira] [Created] (SPARK-41654) Enable doctests in pyspark.sql.connect.window

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41654: Summary: Enable doctests in pyspark.sql.connect.window Key: SPARK-41654 URL: https://issues.apache.org/jira/browse/SPARK-41654 Project: Spark Issue Type:

[jira] [Created] (SPARK-41656) Enable doctests in pyspark.sql.connect.dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41656: Summary: Enable doctests in pyspark.sql.connect.dataframe Key: SPARK-41656 URL: https://issues.apache.org/jira/browse/SPARK-41656 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41653) Test parity: enable doctests in Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41653: - Epic Link: SPARK-39375 Issue Type: Umbrella (was: Bug) > Test parity: enable doctests in

[jira] [Updated] (SPARK-41653) Test parity: enable doctests in Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41653: - Parent: (was: SPARK-39375) Issue Type: Bug (was: Sub-task) > Test parity: enable

[jira] [Created] (SPARK-41653) Test parity: enable doctests in Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41653: Summary: Test parity: enable doctests in Spark Connect Key: SPARK-41653 URL: https://issues.apache.org/jira/browse/SPARK-41653 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41651) Test parity: pyspark.sql.tests.test_dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41651: - Description: After https://github.com/apache/spark/pull/39041 (SPARK-41528), we now reuses the

[jira] [Updated] (SPARK-41652) Test parity: pyspark.sql.tests.test_functions

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41652: - Description: After https://github.com/apache/spark/pull/39041 (SPARK-41528), we now reuses the

[jira] [Commented] (SPARK-41651) Test parity: pyspark.sql.tests.test_dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650118#comment-17650118 ] Hyukjin Kwon commented on SPARK-41651: -- cc [~beliefer] and [~dengziming] in case you guys are

[jira] [Commented] (SPARK-41652) Test parity: pyspark.sql.tests.test_functions

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650119#comment-17650119 ] Hyukjin Kwon commented on SPARK-41652: -- cc [~beliefer] and [~dengziming] in case you guys are

[jira] [Commented] (SPARK-41642) Deduplicate docstrings in Python Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650117#comment-17650117 ] Hyukjin Kwon commented on SPARK-41642: -- cc [~beliefer] and [~dengziming] in case you guys are

[jira] [Updated] (SPARK-41652) Test parity: pyspark.sql.tests.test_functions

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41652: - Epic Link: SPARK-39375 > Test parity: pyspark.sql.tests.test_functions >

[jira] [Created] (SPARK-41652) Test parity: pyspark.sql.tests.test_functions

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41652: Summary: Test parity: pyspark.sql.tests.test_functions Key: SPARK-41652 URL: https://issues.apache.org/jira/browse/SPARK-41652 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41652) Test parity: pyspark.sql.tests.test_functions

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650115#comment-17650115 ] Hyukjin Kwon commented on SPARK-41652: -- Please create a subtask and go ahead. > Test parity:

[jira] [Commented] (SPARK-41650) json expressions much slower in optimized mode

2022-12-20 Thread Yi Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650114#comment-17650114 ] Yi Zhang commented on SPARK-41650: -- [~gurwls223] , [~viirya]  can you help look into this?  > json

[jira] [Updated] (SPARK-41651) Test parity: pyspark.sql.tests.test_dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41651: - Issue Type: Umbrella (was: Improvement) > Test parity: pyspark.sql.tests.test_dataframe >

[jira] [Commented] (SPARK-41651) Test parity: pyspark.sql.tests.test_dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650112#comment-17650112 ] Hyukjin Kwon commented on SPARK-41651: -- Please create a subtask and work on it. > Test parity:

[jira] [Created] (SPARK-41651) Test parity: pyspark.sql.tests.test_dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41651: Summary: Test parity: pyspark.sql.tests.test_dataframe Key: SPARK-41651 URL: https://issues.apache.org/jira/browse/SPARK-41651 Project: Spark Issue Type:

[jira] [Created] (SPARK-41650) json expressions much slower in optimized mode

2022-12-20 Thread Yi Zhang (Jira)
Yi Zhang created SPARK-41650: Summary: json expressions much slower in optimized mode Key: SPARK-41650 URL: https://issues.apache.org/jira/browse/SPARK-41650 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-41649) Deduplicate docstrings in pyspark.sql.connect.window

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41649: Summary: Deduplicate docstrings in pyspark.sql.connect.window Key: SPARK-41649 URL: https://issues.apache.org/jira/browse/SPARK-41649 Project: Spark Issue

[jira] [Commented] (SPARK-41642) Deduplicate docstrings in Python Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650111#comment-17650111 ] Hyukjin Kwon commented on SPARK-41642: -- If the files is too big, feel free to split the JIRA or

[jira] [Created] (SPARK-41648) Deduplicate docstrings in pyspark.sql.connect.readwriter

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41648: Summary: Deduplicate docstrings in pyspark.sql.connect.readwriter Key: SPARK-41648 URL: https://issues.apache.org/jira/browse/SPARK-41648 Project: Spark

[jira] [Created] (SPARK-41646) Deduplicate docstrings in pyspark.sql.connect.session

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41646: Summary: Deduplicate docstrings in pyspark.sql.connect.session Key: SPARK-41646 URL: https://issues.apache.org/jira/browse/SPARK-41646 Project: Spark Issue

[jira] [Created] (SPARK-41647) Deduplicate docstrings in pyspark.sql.connect.functions

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41647: Summary: Deduplicate docstrings in pyspark.sql.connect.functions Key: SPARK-41647 URL: https://issues.apache.org/jira/browse/SPARK-41647 Project: Spark

[jira] [Created] (SPARK-41645) Deduplicate docstrings in pyspark.sql.connect.dataframe

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41645: Summary: Deduplicate docstrings in pyspark.sql.connect.dataframe Key: SPARK-41645 URL: https://issues.apache.org/jira/browse/SPARK-41645 Project: Spark

[jira] [Commented] (SPARK-41643) Deduplicate docstrings in pyspark.sql.connect.column

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650108#comment-17650108 ] Apache Spark commented on SPARK-41643: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-41643) Deduplicate docstrings in pyspark.sql.connect.column

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41643: Assignee: Apache Spark > Deduplicate docstrings in pyspark.sql.connect.column >

[jira] [Assigned] (SPARK-41643) Deduplicate docstrings in pyspark.sql.connect.column

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41643: Assignee: (was: Apache Spark) > Deduplicate docstrings in pyspark.sql.connect.column

[jira] [Assigned] (SPARK-41644) Introducing SPI mechanism to make it easy for other modules to register ProtoBufSerializer

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41644: Assignee: (was: Apache Spark) > Introducing SPI mechanism to make it easy for other

[jira] [Assigned] (SPARK-41644) Introducing SPI mechanism to make it easy for other modules to register ProtoBufSerializer

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41644: Assignee: Apache Spark > Introducing SPI mechanism to make it easy for other modules to

[jira] [Commented] (SPARK-41644) Introducing SPI mechanism to make it easy for other modules to register ProtoBufSerializer

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650100#comment-17650100 ] Apache Spark commented on SPARK-41644: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-41644) Introducing SPI mechanism to make it easy for other modules to register ProtoBufSerializer

2022-12-20 Thread Yang Jie (Jira)
Yang Jie created SPARK-41644: Summary: Introducing SPI mechanism to make it easy for other modules to register ProtoBufSerializer Key: SPARK-41644 URL: https://issues.apache.org/jira/browse/SPARK-41644

[jira] [Updated] (SPARK-41597) Improve PySpark errors

2022-12-20 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-41597: Component/s: Connect Tests > Improve PySpark errors > -- > >

[jira] [Resolved] (SPARK-41539) stats and constraints in LogicalRDD may not be in sync with output attributes

2022-12-20 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-41539. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39082

[jira] [Assigned] (SPARK-41539) stats and constraints in LogicalRDD may not be in sync with output attributes

2022-12-20 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-41539: Assignee: Jungtaek Lim > stats and constraints in LogicalRDD may not be in sync with

[jira] [Resolved] (SPARK-41634) Upgrade minimatch to 3.1.2

2022-12-20 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-41634. Fix Version/s: 3.4.0 Assignee: Bjørn Jørgensen Resolution: Fixed Issue

[jira] [Resolved] (SPARK-41587) Upgrade org.scalatestplus:selenium-4-4 to org.scalatestplus:selenium-4-7

2022-12-20 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-41587. Fix Version/s: 3.4.0 Assignee: Yang Jie Resolution: Fixed Issue resolved

[jira] [Commented] (SPARK-41292) Window-function support

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650073#comment-17650073 ] Apache Spark commented on SPARK-41292: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41292) Window-function support

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41292: Assignee: (was: Apache Spark) > Window-function support > --- >

[jira] [Assigned] (SPARK-41292) Window-function support

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41292: Assignee: Apache Spark > Window-function support > --- > >

[jira] [Created] (SPARK-41643) Deduplicate docstrings in pyspark.sql.connect.column

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41643: Summary: Deduplicate docstrings in pyspark.sql.connect.column Key: SPARK-41643 URL: https://issues.apache.org/jira/browse/SPARK-41643 Project: Spark Issue

[jira] [Commented] (SPARK-41643) Deduplicate docstrings in pyspark.sql.connect.column

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650071#comment-17650071 ] Hyukjin Kwon commented on SPARK-41643: -- I am working on this > Deduplicate docstrings in

[jira] [Assigned] (SPARK-41642) Deduplicate docstrings in Python Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41642: Assignee: Hyukjin Kwon > Deduplicate docstrings in Python Spark Connect >

[jira] [Updated] (SPARK-41642) Deduplicate docstrings in Python Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41642: - Description: There are a lot of duplications in the current docstrings in PySpark Spark

[jira] [Created] (SPARK-41642) Deduplicate docstrings in Python Spark Connect

2022-12-20 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41642: Summary: Deduplicate docstrings in Python Spark Connect Key: SPARK-41642 URL: https://issues.apache.org/jira/browse/SPARK-41642 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-41426) Protobuf serializer for ResourceProfileWrapper

2022-12-20 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41426. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39105

[jira] [Assigned] (SPARK-41426) Protobuf serializer for ResourceProfileWrapper

2022-12-20 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41426: -- Assignee: Sandeep Singh > Protobuf serializer for ResourceProfileWrapper >

[jira] [Resolved] (SPARK-41434) Support LambdaFunction expresssion

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41434. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39068

[jira] [Assigned] (SPARK-41434) Support LambdaFunction expresssion

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41434: Assignee: Ruifeng Zheng > Support LambdaFunction expresssion >

[jira] [Assigned] (SPARK-41192) Task finished before speculative task scheduled leads to holding idle executors

2022-12-20 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-41192: --- Assignee: Yazhi Wang > Task finished before speculative task scheduled

[jira] [Resolved] (SPARK-41192) Task finished before speculative task scheduled leads to holding idle executors

2022-12-20 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-41192. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-41641) Implement `Column.over`

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650065#comment-17650065 ] Apache Spark commented on SPARK-41641: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41641) Implement `Column.over`

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41641: Assignee: (was: Apache Spark) > Implement `Column.over` > --- >

[jira] [Assigned] (SPARK-41641) Implement `Column.over`

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41641: Assignee: Apache Spark > Implement `Column.over` > --- > >

[jira] [Assigned] (SPARK-41640) implement `Window` functions

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41640: Assignee: (was: Apache Spark) > implement `Window` functions >

[jira] [Commented] (SPARK-41640) implement `Window` functions

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650064#comment-17650064 ] Apache Spark commented on SPARK-41640: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41640) implement `Window` functions

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41640: Assignee: Apache Spark > implement `Window` functions > > >

[jira] [Created] (SPARK-41641) Implement `Column.over`

2022-12-20 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41641: - Summary: Implement `Column.over` Key: SPARK-41641 URL: https://issues.apache.org/jira/browse/SPARK-41641 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41640) implement `Window` functions

2022-12-20 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41640: - Summary: implement `Window` functions Key: SPARK-41640 URL: https://issues.apache.org/jira/browse/SPARK-41640 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-41631) Support lateral column alias in Aggregate code path

2022-12-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41631. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39040

[jira] [Assigned] (SPARK-41631) Support lateral column alias in Aggregate code path

2022-12-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41631: --- Assignee: Xinyi Yu > Support lateral column alias in Aggregate code path >

[jira] [Assigned] (SPARK-41639) Remove ScalaReflectionLock

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41639: Assignee: (was: Apache Spark) > Remove ScalaReflectionLock >

[jira] [Assigned] (SPARK-41639) Remove ScalaReflectionLock

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41639: Assignee: Apache Spark > Remove ScalaReflectionLock > --- > >

[jira] [Commented] (SPARK-41639) Remove ScalaReflectionLock

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650060#comment-17650060 ] Apache Spark commented on SPARK-41639: -- User 'SandishKumarHN' has created a pull request for this

[jira] [Commented] (SPARK-41639) Remove ScalaReflectionLock

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650059#comment-17650059 ] Apache Spark commented on SPARK-41639: -- User 'SandishKumarHN' has created a pull request for this

[jira] [Created] (SPARK-41639) Remove ScalaReflectionLock

2022-12-20 Thread Sandish Kumar HN (Jira)
Sandish Kumar HN created SPARK-41639: Summary: Remove ScalaReflectionLock Key: SPARK-41639 URL: https://issues.apache.org/jira/browse/SPARK-41639 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-41440) Implement DataFrame.randomSplit

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41440: Assignee: jiaan.geng (was: Ruifeng Zheng) > Implement DataFrame.randomSplit >

[jira] [Resolved] (SPARK-41440) Implement DataFrame.randomSplit

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41440. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39017

[jira] [Assigned] (SPARK-41440) Implement DataFrame.randomSplit

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41440: Assignee: Ruifeng Zheng > Implement DataFrame.randomSplit >

[jira] [Resolved] (SPARK-41566) Upgrade netty from 4.1.84.Final to 4.1.86.Final

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41566. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39116

[jira] [Assigned] (SPARK-41566) Upgrade netty from 4.1.84.Final to 4.1.86.Final

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41566: Assignee: Bjørn Jørgensen > Upgrade netty from 4.1.84.Final to 4.1.86.Final >

[jira] [Resolved] (SPARK-40520) Add a script to generate DOI mainifest

2022-12-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-40520. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 27

[jira] [Assigned] (SPARK-41396) Oneof field support and recursive fields

2022-12-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41396: --- Assignee: Sandish Kumar HN > Oneof field support and recursive fields >

[jira] [Resolved] (SPARK-41396) Oneof field support and recursive fields

2022-12-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41396. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38922

[jira] [Resolved] (SPARK-41584) Upgrade RoaringBitmap to 0.9.36

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41584. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39125

[jira] [Assigned] (SPARK-41584) Upgrade RoaringBitmap to 0.9.36

2022-12-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41584: Assignee: Yang Jie > Upgrade RoaringBitmap to 0.9.36 > --- >

[jira] [Assigned] (SPARK-41589) PyTorch Distributor

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41589: Assignee: (was: Apache Spark) > PyTorch Distributor > --- > >

[jira] [Commented] (SPARK-41589) PyTorch Distributor

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650043#comment-17650043 ] Apache Spark commented on SPARK-41589: -- User 'rithwik-db' has created a pull request for this

[jira] [Assigned] (SPARK-41589) PyTorch Distributor

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41589: Assignee: Apache Spark > PyTorch Distributor > --- > >

[jira] [Updated] (SPARK-41589) PyTorch Distributor

2022-12-20 Thread Rithwik Ediga Lakhamsani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rithwik Ediga Lakhamsani updated SPARK-41589: - Description: This is a project to make it easier for PySpark users to

[jira] [Created] (SPARK-41638) Move most tests to .sql files

2022-12-20 Thread Xinyi Yu (Jira)
Xinyi Yu created SPARK-41638: Summary: Move most tests to .sql files Key: SPARK-41638 URL: https://issues.apache.org/jira/browse/SPARK-41638 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41637) ORDER BY ALL

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650036#comment-17650036 ] Apache Spark commented on SPARK-41637: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-41637) ORDER BY ALL

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650035#comment-17650035 ] Apache Spark commented on SPARK-41637: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41637) ORDER BY ALL

2022-12-20 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41637: Assignee: Apache Spark (was: Reynold Xin) > ORDER BY ALL > > >

  1   2   3   >