[jira] [Assigned] (SPARK-39285) Spark should not check filed name when read data

[ https://issues.apache.org/jira/browse/SPARK-39285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39285: Assignee: (was: Apache Spark) > Spark should not check filed name when read data > --

[jira] [Commented] (SPARK-39285) Spark should not check filed name when read data

[ https://issues.apache.org/jira/browse/SPARK-39285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541850#comment-17541850 ] Apache Spark commented on SPARK-39285: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-39285) Spark should not check filed name when read data

[ https://issues.apache.org/jira/browse/SPARK-39285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39285: Assignee: Apache Spark > Spark should not check filed name when read data > -

[jira] [Created] (SPARK-39285) Spark should not check filed name when read data

angerszhu created SPARK-39285: - Summary: Spark should not check filed name when read data Key: SPARK-39285 URL: https://issues.apache.org/jira/browse/SPARK-39285 Project: Spark Issue Type: Sub-ta

[jira] [Commented] (SPARK-39284) Implement Groupby.mad

[ https://issues.apache.org/jira/browse/SPARK-39284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541835#comment-17541835 ] Apache Spark commented on SPARK-39284: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-39284) Implement Groupby.mad

[ https://issues.apache.org/jira/browse/SPARK-39284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39284: Assignee: (was: Apache Spark) > Implement Groupby.mad > - > >

[jira] [Assigned] (SPARK-39284) Implement Groupby.mad

[ https://issues.apache.org/jira/browse/SPARK-39284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39284: Assignee: Apache Spark > Implement Groupby.mad > - > >

[jira] [Created] (SPARK-39284) Implement Groupby.mad

zhengruifeng created SPARK-39284: Summary: Implement Groupby.mad Key: SPARK-39284 URL: https://issues.apache.org/jira/browse/SPARK-39284 Project: Spark Issue Type: Sub-task Componen

[jira] [Commented] (SPARK-39282) Replace If-Else branch with bitwise operators in roundNumberOfBytesToNearestWord

[ https://issues.apache.org/jira/browse/SPARK-39282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541805#comment-17541805 ] xiangxiang Shen commented on SPARK-39282: - CC [~ueshin] ,[~cloud_fan] , Thanks!

[jira] [Commented] (SPARK-39282) Replace If-Else branch with bitwise operators in roundNumberOfBytesToNearestWord

[ https://issues.apache.org/jira/browse/SPARK-39282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541804#comment-17541804 ] Apache Spark commented on SPARK-39282: -- User 'zhixingheyi-tian' has created a pull

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Description: We are seems this deadlock between {{TaskMemoryManager}} and {{UnsafeExternalSorter}

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Attachment: DeadlockSparkTasks.png > Spark tasks stuck forever due to deadlock between TaskMemoryM

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Affects Version/s: 3.1.2 (was: 3.0.0) > Spark tasks stuck forever due t

[jira] [Commented] (SPARK-39282) Replace If-Else branch with bitwise operators in roundNumberOfBytesToNearestWord

[ https://issues.apache.org/jira/browse/SPARK-39282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541803#comment-17541803 ] Apache Spark commented on SPARK-39282: -- User 'zhixingheyi-tian' has created a pull

[jira] [Assigned] (SPARK-39282) Replace If-Else branch with bitwise operators in roundNumberOfBytesToNearestWord

[ https://issues.apache.org/jira/browse/SPARK-39282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39282: Assignee: Apache Spark > Replace If-Else branch with bitwise operators in > roundNumberO

[jira] [Assigned] (SPARK-39282) Replace If-Else branch with bitwise operators in roundNumberOfBytesToNearestWord

[ https://issues.apache.org/jira/browse/SPARK-39282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39282: Assignee: (was: Apache Spark) > Replace If-Else branch with bitwise operators in > r

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Description: We are seems this deadlock between {{TaskMemoryManager}} and {{UnsafeExternalSorter}

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Attachment: (was: DeadlockSparkTasks.png) > Spark tasks stuck forever due to deadlock between

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Labels: Deadlock spark3.0 (was: ) > Spark tasks stuck forever due to deadlock between TaskMemoryM

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Description: We are seems this deadlock between {{TaskMemoryManager}} and {{UnsafeExternalSorter}

[jira] [Created] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

Sandeep Pal created SPARK-39283: --- Summary: Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter Key: SPARK-39283 URL: https://issues.apache.org/jira/browse/SPARK-39283

[jira] [Updated] (SPARK-39283) Spark tasks stuck forever due to deadlock between TaskMemoryManager and UnsafeExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-39283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Pal updated SPARK-39283: Attachment: DeadlockSparkTasks.png > Spark tasks stuck forever due to deadlock between TaskMemoryM

[jira] [Created] (SPARK-39282) Replace If-Else branch with bitwise operators in roundNumberOfBytesToNearestWord

xiangxiang Shen created SPARK-39282: --- Summary: Replace If-Else branch with bitwise operators in roundNumberOfBytesToNearestWord Key: SPARK-39282 URL: https://issues.apache.org/jira/browse/SPARK-39282

[jira] [Created] (SPARK-39281) Fasten Timestamp type inference of legacy format in JSON/CSV data source

Gengliang Wang created SPARK-39281: -- Summary: Fasten Timestamp type inference of legacy format in JSON/CSV data source Key: SPARK-39281 URL: https://issues.apache.org/jira/browse/SPARK-39281 Project:

[jira] [Created] (SPARK-39280) Fasten Timestamp type inference with user-provided format in JSON/CSV data source

Gengliang Wang created SPARK-39280: -- Summary: Fasten Timestamp type inference with user-provided format in JSON/CSV data source Key: SPARK-39280 URL: https://issues.apache.org/jira/browse/SPARK-39280

[jira] [Updated] (SPARK-39193) Fasten Timestamp type inference of default format in JSON/CSV data source

[ https://issues.apache.org/jira/browse/SPARK-39193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-39193: --- Parent: SPARK-39279 Issue Type: Sub-task (was: Improvement) > Fasten Timestamp type

[jira] [Updated] (SPARK-39193) Fasten Timestamp type inference of default format in JSON/CSV data source

[ https://issues.apache.org/jira/browse/SPARK-39193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-39193: --- Summary: Fasten Timestamp type inference of default format in JSON/CSV data source (was: Im

[jira] [Created] (SPARK-39279) Fasten the schema inference of CSV/JSON data source

Gengliang Wang created SPARK-39279: -- Summary: Fasten the schema inference of CSV/JSON data source Key: SPARK-39279 URL: https://issues.apache.org/jira/browse/SPARK-39279 Project: Spark Issue

[jira] [Resolved] (SPARK-39252) Flaky Test: pyspark.sql.tests.test_dataframe.DataFrameTests test_df_is_empty

[ https://issues.apache.org/jira/browse/SPARK-39252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39252. -- Fix Version/s: 3.1.3 3.2.2 3.3.1 Resolution: Fixed

[jira] [Assigned] (SPARK-39252) Flaky Test: pyspark.sql.tests.test_dataframe.DataFrameTests test_df_is_empty

[ https://issues.apache.org/jira/browse/SPARK-39252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39252: Assignee: Ivan Sadikov > Flaky Test: pyspark.sql.tests.test_dataframe.DataFrameTests test

[jira] [Assigned] (SPARK-39278) Alternative configs of Hadoop Filesystems to access break backward compatibility

[ https://issues.apache.org/jira/browse/SPARK-39278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39278: Assignee: Apache Spark > Alternative configs of Hadoop Filesystems to access break backwa

[jira] [Assigned] (SPARK-39278) Alternative configs of Hadoop Filesystems to access break backward compatibility

[ https://issues.apache.org/jira/browse/SPARK-39278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39278: Assignee: (was: Apache Spark) > Alternative configs of Hadoop Filesystems to access b

[jira] [Commented] (SPARK-39278) Alternative configs of Hadoop Filesystems to access break backward compatibility

[ https://issues.apache.org/jira/browse/SPARK-39278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541779#comment-17541779 ] Apache Spark commented on SPARK-39278: -- User 'manuzhang' has created a pull request

[jira] [Commented] (SPARK-39278) Alternative configs of Hadoop Filesystems to access break backward compatibility

[ https://issues.apache.org/jira/browse/SPARK-39278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541781#comment-17541781 ] Apache Spark commented on SPARK-39278: -- User 'manuzhang' has created a pull request

[jira] [Resolved] (SPARK-39220) codegen cause NullPointException

[ https://issues.apache.org/jira/browse/SPARK-39220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39220. -- Resolution: Cannot Reproduce > codegen cause NullPointException >

[jira] [Commented] (SPARK-39274) AttributeError: 'datetime.time' object has no attribute 'timetuple'

[ https://issues.apache.org/jira/browse/SPARK-39274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541773#comment-17541773 ] Hyukjin Kwon commented on SPARK-39274: -- We don;t current'y corresponding mapping of

[jira] [Updated] (SPARK-39278) Alternative configs of Hadoop Filesystems to access break backward compatibility

[ https://issues.apache.org/jira/browse/SPARK-39278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated SPARK-39278: --- Description: Before [https://github.com/apache/spark/pull/23698,] The precedence of configuring Had

[jira] [Created] (SPARK-39278) Alternative configs of Hadoop Filesystems to access break backward compatibility

Manu Zhang created SPARK-39278: -- Summary: Alternative configs of Hadoop Filesystems to access break backward compatibility Key: SPARK-39278 URL: https://issues.apache.org/jira/browse/SPARK-39278 Project:

[jira] [Assigned] (SPARK-39277) Make Optimizer extends SQLConfHelper

[ https://issues.apache.org/jira/browse/SPARK-39277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39277: Assignee: (was: Apache Spark) > Make Optimizer extends SQLConfHelper > --

[jira] [Assigned] (SPARK-39277) Make Optimizer extends SQLConfHelper

[ https://issues.apache.org/jira/browse/SPARK-39277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39277: Assignee: Apache Spark > Make Optimizer extends SQLConfHelper > -

[jira] [Commented] (SPARK-39277) Make Optimizer extends SQLConfHelper

[ https://issues.apache.org/jira/browse/SPARK-39277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541770#comment-17541770 ] Apache Spark commented on SPARK-39277: -- User 'wangyum' has created a pull request f

[jira] [Created] (SPARK-39277) Make Optimizer extends SQLConfHelper

Yuming Wang created SPARK-39277: --- Summary: Make Optimizer extends SQLConfHelper Key: SPARK-39277 URL: https://issues.apache.org/jira/browse/SPARK-39277 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-39273) Make PandasOnSparkTestCase inherit ReusedSQLTestCase

[ https://issues.apache.org/jira/browse/SPARK-39273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39273. -- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-39273) Make PandasOnSparkTestCase inherit ReusedSQLTestCase

[ https://issues.apache.org/jira/browse/SPARK-39273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39273: Assignee: Hyukjin Kwon > Make PandasOnSparkTestCase inherit ReusedSQLTestCase > -

[jira] [Resolved] (SPARK-39053) test_multi_index_dtypes failed due to index mismatch

[ https://issues.apache.org/jira/browse/SPARK-39053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39053. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36391 [https://gi

[jira] [Assigned] (SPARK-39053) test_multi_index_dtypes failed due to index mismatch

[ https://issues.apache.org/jira/browse/SPARK-39053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39053: Assignee: Yikun Jiang > test_multi_index_dtypes failed due to index mismatch > --

[jira] [Assigned] (SPARK-39252) Flaky Test: pyspark.sql.tests.test_dataframe.DataFrameTests test_df_is_empty

[ https://issues.apache.org/jira/browse/SPARK-39252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39252: Assignee: Apache Spark > Flaky Test: pyspark.sql.tests.test_dataframe.DataFrameTests test

[jira] [Assigned] (SPARK-39252) Flaky Test: pyspark.sql.tests.test_dataframe.DataFrameTests test_df_is_empty

[ https://issues.apache.org/jira/browse/SPARK-39252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39252: Assignee: (was: Apache Spark) > Flaky Test: pyspark.sql.tests.test_dataframe.DataFram

[jira] [Commented] (SPARK-39252) Flaky Test: pyspark.sql.tests.test_dataframe.DataFrameTests test_df_is_empty

[ https://issues.apache.org/jira/browse/SPARK-39252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541751#comment-17541751 ] Apache Spark commented on SPARK-39252: -- User 'sadikovi' has created a pull request

[jira] [Created] (SPARK-39276) grouping_id() behavior changed between 3.1.x and 3.2.x

Martin Price created SPARK-39276: Summary: grouping_id() behavior changed between 3.1.x and 3.2.x Key: SPARK-39276 URL: https://issues.apache.org/jira/browse/SPARK-39276 Project: Spark Issue

[jira] [Updated] (SPARK-39048) Refactor `GroupBy._reduce_for_stat_function` on accepted data types

[ https://issues.apache.org/jira/browse/SPARK-39048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39048: - Parent: SPARK-39076 Issue Type: Sub-task (was: Improvement) > Refactor `GroupBy._reduce

[jira] [Updated] (SPARK-38880) Implement `numeric_only` parameter of `GroupBy.max/min`

[ https://issues.apache.org/jira/browse/SPARK-38880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38880: - Parent: SPARK-39076 Issue Type: Sub-task (was: Improvement) > Implement `numeric_only`

[jira] [Updated] (SPARK-39000) Convert bools to ints in basic statistical functions of GroupBy objects

[ https://issues.apache.org/jira/browse/SPARK-39000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39000: - Parent: SPARK-39076 Issue Type: Sub-task (was: Improvement) > Convert bools to ints in

[jira] [Updated] (SPARK-39227) Reach parity with pandas boolean cast

[ https://issues.apache.org/jira/browse/SPARK-39227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39227: - Parent: SPARK-39076 Issue Type: Sub-task (was: Improvement) > Reach parity with pandas

[jira] [Updated] (SPARK-38952) Implement `numeric_only` of `GroupBy.first` and `GroupBy.last`

[ https://issues.apache.org/jira/browse/SPARK-38952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38952: - Parent: SPARK-39076 Issue Type: Sub-task (was: Improvement) > Implement `numeric_only`

[jira] [Updated] (SPARK-38763) Pandas API on spark Can`t apply lamda to columns.

[ https://issues.apache.org/jira/browse/SPARK-38763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38763: - Parent: SPARK-39199 Issue Type: Sub-task (was: Bug) > Pandas API on spark Can`t apply l

[jira] [Updated] (SPARK-38766) Support lambda `column` parameter of `DataFrame.rename`

[ https://issues.apache.org/jira/browse/SPARK-38766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38766: - Parent: (was: SPARK-39199) Issue Type: Improvement (was: Sub-task) > Support lambda

[jira] [Updated] (SPARK-38387) Support `na_action` and Series input correspondence in `Series.map`

[ https://issues.apache.org/jira/browse/SPARK-38387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38387: - Parent: SPARK-39199 Issue Type: Sub-task (was: New Feature) > Support `na_action` and S

[jira] [Updated] (SPARK-38766) Support lambda `column` parameter of `DataFrame.rename`

[ https://issues.apache.org/jira/browse/SPARK-38766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38766: - Parent: SPARK-39199 Issue Type: Sub-task (was: Bug) > Support lambda `column` parameter

[jira] [Updated] (SPARK-38400) Enable Series.rename to change index labels

[ https://issues.apache.org/jira/browse/SPARK-38400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38400: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Enable Series.rename to c

[jira] [Updated] (SPARK-38491) Support `ignore_index` of `Series.sort_values`

[ https://issues.apache.org/jira/browse/SPARK-38491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38491: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Support `ignore_index` of

[jira] [Updated] (SPARK-38518) Implement `skipna` of `Series.all/Index.all` to exclude NA/null values

[ https://issues.apache.org/jira/browse/SPARK-38518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38518: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `skipna` of `Se

[jira] [Updated] (SPARK-38441) Support string and bool `regex` in `Series.replace`

[ https://issues.apache.org/jira/browse/SPARK-38441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38441: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Support string and bool `

[jira] [Updated] (SPARK-38479) Add `Series.duplicated` to indicate duplicate Series values.

[ https://issues.apache.org/jira/browse/SPARK-38479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38479: - Parent: SPARK-39199 Issue Type: Sub-task (was: New Feature) > Add `Series.duplicated` t

[jira] [Updated] (SPARK-38576) Implement `numeric_only` parameter for `DataFrame/Series.rank` to rank numeric columns only

[ https://issues.apache.org/jira/browse/SPARK-38576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38576: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `numeric_only`

[jira] [Updated] (SPARK-38608) Implement `bool_only` parameter of `DataFrame.all` and`DataFrame.any`

[ https://issues.apache.org/jira/browse/SPARK-38608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38608: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `bool_only` par

[jira] [Updated] (SPARK-38552) Implement `keep` parameter of `frame.nlargest/nsmallest` to decide how to resolve ties

[ https://issues.apache.org/jira/browse/SPARK-38552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38552: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `keep` paramete

[jira] [Updated] (SPARK-38686) Implement `keep` parameter of `(Index/MultiIndex).drop_duplicates`

[ https://issues.apache.org/jira/browse/SPARK-38686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38686: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `keep` paramete

[jira] [Commented] (SPARK-39275) Pass SQL config values as parameters of error classes

[ https://issues.apache.org/jira/browse/SPARK-39275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541659#comment-17541659 ] Apache Spark commented on SPARK-39275: -- User 'MaxGekk' has created a pull request f

[jira] [Updated] (SPARK-38704) Support string `inclusive` parameter of `Series.between`

[ https://issues.apache.org/jira/browse/SPARK-38704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38704: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Support string `inclusive

[jira] [Commented] (SPARK-39275) Pass SQL config values as parameters of error classes

[ https://issues.apache.org/jira/browse/SPARK-39275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541660#comment-17541660 ] Apache Spark commented on SPARK-39275: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-39275) Pass SQL config values as parameters of error classes

[ https://issues.apache.org/jira/browse/SPARK-39275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39275: Assignee: Apache Spark (was: Max Gekk) > Pass SQL config values as parameters of error c

[jira] [Assigned] (SPARK-39275) Pass SQL config values as parameters of error classes

[ https://issues.apache.org/jira/browse/SPARK-39275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39275: Assignee: Max Gekk (was: Apache Spark) > Pass SQL config values as parameters of error c

[jira] [Commented] (SPARK-39255) Improve error messages

[ https://issues.apache.org/jira/browse/SPARK-39255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541658#comment-17541658 ] Apache Spark commented on SPARK-39255: -- User 'MaxGekk' has created a pull request f

[jira] [Created] (SPARK-39275) Pass SQL config values as parameters of error classes

Max Gekk created SPARK-39275: Summary: Pass SQL config values as parameters of error classes Key: SPARK-39275 URL: https://issues.apache.org/jira/browse/SPARK-39275 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-38726) Support `how` parameter of `MultiIndex.dropna`

[ https://issues.apache.org/jira/browse/SPARK-38726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38726: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Support `how` parameter o

[jira] [Updated] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38765: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `inplace` param

[jira] [Updated] (SPARK-38837) Implement `dropna` parameter of `SeriesGroupBy.value_counts`

[ https://issues.apache.org/jira/browse/SPARK-38837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38837: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `dropna` parame

[jira] [Updated] (SPARK-38863) Implement `skipna` parameter of `DataFrame.all`

[ https://issues.apache.org/jira/browse/SPARK-38863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38863: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `skipna` parame

[jira] [Updated] (SPARK-38793) Support `return_indexer` parameter of `Index/MultiIndex.sort_values`

[ https://issues.apache.org/jira/browse/SPARK-38793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38793: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Support `return_indexer`

[jira] [Updated] (SPARK-38903) Implement `ignore_index` of `Series.sort_values` and `Series.sort_index`

[ https://issues.apache.org/jira/browse/SPARK-38903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38903: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `ignore_index`

[jira] [Updated] (SPARK-38890) Implement `ignore_index` of `DataFrame.sort_index`.

[ https://issues.apache.org/jira/browse/SPARK-38890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38890: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `ignore_index`

[jira] [Updated] (SPARK-38938) Implement `inplace` and `columns` parameters of `Series.drop`

[ https://issues.apache.org/jira/browse/SPARK-38938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38938: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `inplace` and `

[jira] [Updated] (SPARK-38989) Implement `ignore_index` of `DataFrame/Series.sample`

[ https://issues.apache.org/jira/browse/SPARK-38989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38989: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `ignore_index`

[jira] [Updated] (SPARK-39201) Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates`

[ https://issues.apache.org/jira/browse/SPARK-39201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39201: - Parent: SPARK-39199 Issue Type: Sub-task (was: Improvement) > Implement `ignore_index`

[jira] [Updated] (SPARK-39201) Implement `ignore_index` of `DataFrame.explode` and `DataFrame.drop_duplicates`

[ https://issues.apache.org/jira/browse/SPARK-39201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39201: - Issue Type: Improvement (was: Umbrella) > Implement `ignore_index` of `DataFrame.explode` and

[jira] [Updated] (SPARK-39104) Null Pointer Exeption on unpersist call

[ https://issues.apache.org/jira/browse/SPARK-39104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39104: - Fix Version/s: 3.3.0 (was: 3.3.1) > Null Pointer Exeption on unpersist call > ---

[jira] [Updated] (SPARK-38681) Support nested generic case classes

[ https://issues.apache.org/jira/browse/SPARK-38681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38681: - Fix Version/s: 3.3.0 (was: 3.3.1) > Support nested generic case classes > ---

[jira] [Updated] (SPARK-39187) Remove SparkIllegalStateException

[ https://issues.apache.org/jira/browse/SPARK-39187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39187: - Fix Version/s: 3.3.0 (was: 3.3.1) > Remove SparkIllegalStateException > -

[jira] [Updated] (SPARK-39190) Provide query context for decimal precision overflow error when WSCG is off

[ https://issues.apache.org/jira/browse/SPARK-39190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39190: - Fix Version/s: 3.3.0 (was: 3.3.1) > Provide query context for decimal precision o

[jira] [Updated] (SPARK-39183) Upgrade Apache Xerces Java to 2.12.2

[ https://issues.apache.org/jira/browse/SPARK-39183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39183: - Fix Version/s: 3.3.0 (was: 3.3.1) > Upgrade Apache Xerces Java to 2.12.2 > --

[jira] [Updated] (SPARK-39193) Improve the performance of inferring Timestamp type in JSON/CSV data source

[ https://issues.apache.org/jira/browse/SPARK-39193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39193: - Fix Version/s: 3.3.0 (was: 3.3.1) > Improve the performance of inferring Timestam

[jira] [Updated] (SPARK-39240) Source and binary releases using different tool to generates hashes for integrity

[ https://issues.apache.org/jira/browse/SPARK-39240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39240: - Fix Version/s: 3.3.0 (was: 3.3.1) > Source and binary releases using different to

[jira] [Updated] (SPARK-39216) Do not collapse projects in CombineUnions if it hasCorrelatedSubquery

[ https://issues.apache.org/jira/browse/SPARK-39216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39216: - Fix Version/s: 3.3.0 (was: 3.3.1) > Do not collapse projects in CombineUnions if

[jira] [Updated] (SPARK-39214) Improve errors related to CAST

[ https://issues.apache.org/jira/browse/SPARK-39214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39214: - Fix Version/s: 3.3.0 (was: 3.3.1) > Improve errors related to CAST >

[jira] [Comment Edited] (SPARK-38506) Push partial aggregation through join

[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541486#comment-17541486 ] Yuming Wang edited comment on SPARK-38506 at 5/24/22 2:40 PM:

[jira] [Created] (SPARK-39274) AttributeError: 'datetime.time' object has no attribute 'timetuple'

Andreas Fried created SPARK-39274: - Summary: AttributeError: 'datetime.time' object has no attribute 'timetuple' Key: SPARK-39274 URL: https://issues.apache.org/jira/browse/SPARK-39274 Project: Spark

[jira] [Resolved] (SPARK-39256) Reduce multiple file attribute calls of JavaUtils#deleteRecursivelyUsingJavaIO

[ https://issues.apache.org/jira/browse/SPARK-39256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39256. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36636 [https://gi

[jira] [Assigned] (SPARK-39256) Reduce multiple file attribute calls of JavaUtils#deleteRecursivelyUsingJavaIO

[ https://issues.apache.org/jira/browse/SPARK-39256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-39256: Assignee: Yang Jie > Reduce multiple file attribute calls of JavaUtils#deleteRecursivelyU

[jira] [Commented] (SPARK-38506) Push partial aggregation through join

[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541486#comment-17541486 ] Yuming Wang commented on SPARK-38506: - Benchmark result: |SQL|Before(ms)|With Parti

  1   2   >