[jira] [Commented] (SPARK-41875) Throw proper errors in Dataset.to()

2023-01-04 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654845#comment-17654845 ] jiaan.geng commented on SPARK-41875: It seems this is't the issue of connect. > Throw proper errors

[jira] [Assigned] (SPARK-41889) Attach root cause to invalidPatternError

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41889: Assignee: (was: Apache Spark) > Attach root cause to invalidPatternError >

[jira] [Assigned] (SPARK-41889) Attach root cause to invalidPatternError

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41889: Assignee: Apache Spark > Attach root cause to invalidPatternError >

[jira] [Commented] (SPARK-41889) Attach root cause to invalidPatternError

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654826#comment-17654826 ] Apache Spark commented on SPARK-41889: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-41893) Publish SBOM artifacts

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41893: Assignee: (was: Apache Spark) > Publish SBOM artifacts > -- > >

[jira] [Commented] (SPARK-41893) Publish SBOM artifacts

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654824#comment-17654824 ] Apache Spark commented on SPARK-41893: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41893) Publish SBOM artifacts

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41893: Assignee: Apache Spark > Publish SBOM artifacts > -- > >

[jira] [Commented] (SPARK-41894) sql/core module mvn clean failed

2023-01-04 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654823#comment-17654823 ] Yang Jie commented on SPARK-41894: -- The running environment is linux,  I haven't found the specific

[jira] [Created] (SPARK-41894) sql/core module mvn clean failed

2023-01-04 Thread Yang Jie (Jira)
Yang Jie created SPARK-41894: Summary: sql/core module mvn clean failed Key: SPARK-41894 URL: https://issues.apache.org/jira/browse/SPARK-41894 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-41829) Implement Dataframe.sort,sortWithinPartitions Ordering

2023-01-04 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41829. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39398

[jira] [Assigned] (SPARK-41829) Implement Dataframe.sort,sortWithinPartitions Ordering

2023-01-04 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41829: - Assignee: Ruifeng Zheng > Implement Dataframe.sort,sortWithinPartitions Ordering >

[jira] [Created] (SPARK-41893) Publish SBOM artifacts

2023-01-04 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41893: - Summary: Publish SBOM artifacts Key: SPARK-41893 URL: https://issues.apache.org/jira/browse/SPARK-41893 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

2023-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654818#comment-17654818 ] Dongjoon Hyun commented on SPARK-33772: --- FYI, Apache Spark has been Java 17 SBT test coverage,

[jira] [Updated] (SPARK-41162) Anti-join must not be pushed below aggregation with ambiguous predicates

2023-01-04 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-41162: -- Affects Version/s: 3.0.3 > Anti-join must not be pushed below aggregation with ambiguous

[jira] [Resolved] (SPARK-41580) Assign name to _LEGACY_ERROR_TEMP_2137

2023-01-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41580. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39305

[jira] [Assigned] (SPARK-41580) Assign name to _LEGACY_ERROR_TEMP_2137

2023-01-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41580: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2137 >

[jira] [Resolved] (SPARK-41576) Assign name to _LEGACY_ERROR_TEMP_2051

2023-01-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41576. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39281

[jira] [Assigned] (SPARK-41576) Assign name to _LEGACY_ERROR_TEMP_2051

2023-01-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41576: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2051 >

[jira] [Assigned] (SPARK-41821) Fix DataFrame.describe

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41821: Assignee: jiaan.geng > Fix DataFrame.describe > -- > >

[jira] [Resolved] (SPARK-41821) Fix DataFrame.describe

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41821. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39378

[jira] [Assigned] (SPARK-41871) DataFrame hint parameter can be str, float or int

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41871: Assignee: Sandeep Singh > DataFrame hint parameter can be str, float or int >

[jira] [Resolved] (SPARK-41871) DataFrame hint parameter can be str, float or int

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41871. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39393

[jira] [Commented] (SPARK-41891) Enable test_add_months_function, test_array_repeat, test_dayofweek, test_first_last_ignorenulls, test_function_parity, test_inline, test_window_time, test_reciprocal_t

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654734#comment-17654734 ] Apache Spark commented on SPARK-41891: -- User 'techaddict' has created a pull request for this

[jira] [Assigned] (SPARK-41891) Enable test_add_months_function, test_array_repeat, test_dayofweek, test_first_last_ignorenulls, test_function_parity, test_inline, test_window_time, test_reciprocal_tr

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41891: Assignee: Sandeep Singh (was: Apache Spark) > Enable test_add_months_function,

[jira] [Assigned] (SPARK-41891) Enable test_add_months_function, test_array_repeat, test_dayofweek, test_first_last_ignorenulls, test_function_parity, test_inline, test_window_time, test_reciprocal_tr

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41891: Assignee: Apache Spark (was: Sandeep Singh) > Enable test_add_months_function,

[jira] [Assigned] (SPARK-39318) Remove tpch-plan-stability WithStats golden files

2023-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39318: --- Assignee: XiDuo You > Remove tpch-plan-stability WithStats golden files >

[jira] [Resolved] (SPARK-39318) Remove tpch-plan-stability WithStats golden files

2023-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39318. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36700

[jira] [Updated] (SPARK-41891) Enable test_add_months_function, test_array_repeat, test_dayofweek, test_first_last_ignorenulls, test_function_parity, test_inline, test_window_time, test_reciprocal_tri

2023-01-04 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41891: -- Summary: Enable test_add_months_function, test_array_repeat, test_dayofweek,

[jira] [Created] (SPARK-41892) Add JIRAs or messages for skipped messages

2023-01-04 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41892: - Summary: Add JIRAs or messages for skipped messages Key: SPARK-41892 URL: https://issues.apache.org/jira/browse/SPARK-41892 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41878) Add JIRAs or messages for skipped tests

2023-01-04 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41878: -- Summary: Add JIRAs or messages for skipped tests (was: Add JIRAs or messages for skipped

[jira] [Created] (SPARK-41891) Enable 8 tests

2023-01-04 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41891: - Summary: Enable 8 tests Key: SPARK-41891 URL: https://issues.apache.org/jira/browse/SPARK-41891 Project: Spark Issue Type: Sub-task Components:

[jira] [Assigned] (SPARK-41694) Add new config to clean up `spark.ui.store.path` directory when SparkContext.stop()

2023-01-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41694: -- Assignee: Yang Jie > Add new config to clean up `spark.ui.store.path` directory when

[jira] [Resolved] (SPARK-41694) Add new config to clean up `spark.ui.store.path` directory when SparkContext.stop()

2023-01-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41694. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39226

[jira] [Commented] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for Scala 2.13

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654723#comment-17654723 ] Apache Spark commented on SPARK-41890: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for Scala 2.13

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41890: Assignee: (was: Apache Spark) > Reduce `toSeq` in >

[jira] [Assigned] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for Scala 2.13

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41890: Assignee: Apache Spark > Reduce `toSeq` in >

[jira] [Updated] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for Scala 2.13

2023-01-04 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-41890: - Description: Similar work to SPARK-41709 (was: Similar work as SPARK-41709) > Reduce `toSeq` in >

[jira] [Updated] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for Scala 2.13

2023-01-04 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-41890: - Summary: Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/SparkPlanGraphWrapperSerializer` for

[jira] [Created] (SPARK-41890) Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/`sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/SparkPlanGraphWrapperSerializer` for Scala 2.13

2023-01-04 Thread Yang Jie (Jira)
Yang Jie created SPARK-41890: Summary: Reduce `toSeq` in `RDDOperationGraphWrapperSerializer`/`sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/SparkPlanGraphWrapperSerializer` for Scala 2.13 Key: SPARK-41890

[jira] [Assigned] (SPARK-41829) Implement Dataframe.sort,sortWithinPartitions Ordering

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41829: Assignee: (was: Apache Spark) > Implement Dataframe.sort,sortWithinPartitions

[jira] [Assigned] (SPARK-41829) Implement Dataframe.sort,sortWithinPartitions Ordering

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41829: Assignee: Apache Spark > Implement Dataframe.sort,sortWithinPartitions Ordering >

[jira] [Commented] (SPARK-41829) Implement Dataframe.sort,sortWithinPartitions Ordering

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654718#comment-17654718 ] Apache Spark commented on SPARK-41829: -- User 'zhengruifeng' has created a pull request for this

[jira] [Updated] (SPARK-41889) Attach root cause to invalidPatternError

2023-01-04 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-41889: Summary: Attach root cause to invalidPatternError (was: Attach root cause to

[jira] [Commented] (SPARK-41889) Attach root cause to INVALID_PARAMETER_VALUE

2023-01-04 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654717#comment-17654717 ] BingKun Pan commented on SPARK-41889: - I work on it. > Attach root cause to INVALID_PARAMETER_VALUE

[jira] [Assigned] (SPARK-41825) DataFrame.show formatting int as double

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41825: Assignee: Ruifeng Zheng > DataFrame.show formatting int as double >

[jira] [Resolved] (SPARK-41825) DataFrame.show formatting int as double

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41825. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39396

[jira] [Created] (SPARK-41889) Attach root cause to INVALID_PARAMETER_VALUE

2023-01-04 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-41889: --- Summary: Attach root cause to INVALID_PARAMETER_VALUE Key: SPARK-41889 URL: https://issues.apache.org/jira/browse/SPARK-41889 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41888) Support StreamingQueryListener for DataFrame.observe

2023-01-04 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-41888: --- Summary: Support StreamingQueryListener for DataFrame.observe (was: Support StreamingQueryListener

[jira] [Updated] (SPARK-41888) Support StreamingQueryListener for connect

2023-01-04 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-41888: --- Description: {code:java} **

[jira] [Created] (SPARK-41888) Support StreamingQueryListener for connect

2023-01-04 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-41888: -- Summary: Support StreamingQueryListener for connect Key: SPARK-41888 URL: https://issues.apache.org/jira/browse/SPARK-41888 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41887) Support DataFrame hint parameter to be list

2023-01-04 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41887: - Summary: Support DataFrame hint parameter to be list Key: SPARK-41887 URL: https://issues.apache.org/jira/browse/SPARK-41887 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41887) Support DataFrame hint parameter to be list

2023-01-04 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41887: -- Description: {code:java} df = self.spark.range(10e10).toDF("id") such_a_nice_list =

[jira] [Updated] (SPARK-41871) DataFrame hint parameter can be str, float or int

2023-01-04 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41871: -- Summary: DataFrame hint parameter can be str, float or int (was: DataFrame hint parameter

[jira] [Commented] (SPARK-41825) DataFrame.show formatting int as double

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654697#comment-17654697 ] Apache Spark commented on SPARK-41825: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41825) DataFrame.show formatting int as double

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41825: Assignee: Apache Spark > DataFrame.show formatting int as double >

[jira] [Commented] (SPARK-41825) DataFrame.show formatting int as double

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654698#comment-17654698 ] Apache Spark commented on SPARK-41825: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41825) DataFrame.show formatting int as double

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41825: Assignee: (was: Apache Spark) > DataFrame.show formatting int as double >

[jira] [Created] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-04 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41886: - Summary: `DataFrame.intersect` doctest output has different order Key: SPARK-41886 URL: https://issues.apache.org/jira/browse/SPARK-41886 Project: Spark

[jira] [Updated] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

2023-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41053: -- Labels: releasenotes (was: release-notes) > Better Spark UI scalability and Driver stability

[jira] [Resolved] (SPARK-41286) Build, package and infrastructure for Spark Connect

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41286. -- Resolution: Done I am going to mark it as done for now. > Build, package and infrastructure

[jira] [Resolved] (SPARK-41841) Support PyPI packaging without JVM

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41841. -- Resolution: Later > Support PyPI packaging without JVM > -- >

[jira] [Resolved] (SPARK-41878) Add JIRAs or messages for skipped messages

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41878. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39382

[jira] [Assigned] (SPARK-41881) `DataFrame.collect` should handle None/NaN properly

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41881: Assignee: Ruifeng Zheng > `DataFrame.collect` should handle None/NaN properly >

[jira] [Assigned] (SPARK-41815) Column.isNull returns nan instead of None

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41815: Assignee: Ruifeng Zheng > Column.isNull returns nan instead of None >

[jira] [Resolved] (SPARK-41815) Column.isNull returns nan instead of None

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41815. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39386

[jira] [Resolved] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41833. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39386

[jira] [Assigned] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41833: Assignee: Ruifeng Zheng > DataFrame.collect() output parity with pyspark >

[jira] [Resolved] (SPARK-41881) `DataFrame.collect` should handle None/NaN properly

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41881. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39386

[jira] [Resolved] (SPARK-41846) DataFrame windowspec functions : unresolved columns

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41846. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39392

[jira] [Assigned] (SPARK-41846) DataFrame windowspec functions : unresolved columns

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41846: Assignee: Ruifeng Zheng (was: Sandeep Singh) > DataFrame windowspec functions :

[jira] [Assigned] (SPARK-41846) DataFrame windowspec functions : unresolved columns

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41846: Assignee: Sandeep Singh > DataFrame windowspec functions : unresolved columns >

[jira] [Assigned] (SPARK-41840) DataFrame.show(): 'Column' object is not callable

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41840: Assignee: Ruifeng Zheng > DataFrame.show(): 'Column' object is not callable >

[jira] [Resolved] (SPARK-41840) DataFrame.show(): 'Column' object is not callable

2023-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41840. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39390

[jira] [Assigned] (SPARK-41677) Protobuf serializer for StreamingQueryProgressWrapper

2023-01-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41677: -- Assignee: Yang Jie > Protobuf serializer for StreamingQueryProgressWrapper >

[jira] [Resolved] (SPARK-41677) Protobuf serializer for StreamingQueryProgressWrapper

2023-01-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41677. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39357

[jira] [Resolved] (SPARK-41768) Refactor the definition of enum - `JobExecutionStatus` to follow with the code style

2023-01-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41768. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39286

[jira] [Assigned] (SPARK-41768) Refactor the definition of enum - `JobExecutionStatus` to follow with the code style

2023-01-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41768: -- Assignee: BingKun Pan > Refactor the definition of enum - `JobExecutionStatus` to

[jira] [Assigned] (SPARK-41573) Assign name to _LEGACY_ERROR_TEMP_2136

2023-01-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41573: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2136 >

[jira] [Resolved] (SPARK-41573) Assign name to _LEGACY_ERROR_TEMP_2136

2023-01-04 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41573. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39284

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

2023-01-04 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654590#comment-17654590 ] Mridul Muralidharan commented on SPARK-41497: - Sounds good [~Ngone51], thanks ! >

[jira] [Updated] (SPARK-40497) Upgrade Scala to 2.13.11

2023-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40497: -- Description: We tested and decided to skip the following releases. This issue aims to use

[jira] [Updated] (SPARK-40497) Upgrade Scala to 2.13.11

2023-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40497: -- Description: We tested and decided to skip the following releases. This issue aims to use

[jira] [Created] (SPARK-41885) --packages may not work on Windows 11

2023-01-04 Thread Shixiong Zhu (Jira)
Shixiong Zhu created SPARK-41885: Summary: --packages may not work on Windows 11 Key: SPARK-41885 URL: https://issues.apache.org/jira/browse/SPARK-41885 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-41575) Assign name to _LEGACY_ERROR_TEMP_2054

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41575: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_2054 >

[jira] [Commented] (SPARK-41575) Assign name to _LEGACY_ERROR_TEMP_2054

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654532#comment-17654532 ] Apache Spark commented on SPARK-41575: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-41575) Assign name to _LEGACY_ERROR_TEMP_2054

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654533#comment-17654533 ] Apache Spark commented on SPARK-41575: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41575) Assign name to _LEGACY_ERROR_TEMP_2054

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41575: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_2054 >

[jira] [Commented] (SPARK-41871) DataFrame hint parameter can be str, list, float or int

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654516#comment-17654516 ] Apache Spark commented on SPARK-41871: -- User 'techaddict' has created a pull request for this

[jira] [Assigned] (SPARK-41871) DataFrame hint parameter can be str, list, float or int

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41871: Assignee: Apache Spark > DataFrame hint parameter can be str, list, float or int >

[jira] [Assigned] (SPARK-41871) DataFrame hint parameter can be str, list, float or int

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41871: Assignee: (was: Apache Spark) > DataFrame hint parameter can be str, list, float or

[jira] [Commented] (SPARK-41871) DataFrame hint parameter can be str, list, float or int

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654515#comment-17654515 ] Apache Spark commented on SPARK-41871: -- User 'techaddict' has created a pull request for this

[jira] [Updated] (SPARK-41884) DataFrame `toPandas` parity in return types

2023-01-04 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41884: -- Description: {code:java} import numpy as np import pandas as pd df =

[jira] [Created] (SPARK-41884) DataFrame `toPandas` parity in return types

2023-01-04 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41884: - Summary: DataFrame `toPandas` parity in return types Key: SPARK-41884 URL: https://issues.apache.org/jira/browse/SPARK-41884 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-39304) ps.read_csv ignore double quotes.

2023-01-04 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-39304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen resolved SPARK-39304. - Resolution: Won't Fix > ps.read_csv ignore double quotes. >

[jira] [Commented] (SPARK-41846) DataFrame windowspec functions : unresolved columns

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654446#comment-17654446 ] Apache Spark commented on SPARK-41846: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41846) DataFrame windowspec functions : unresolved columns

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41846: Assignee: (was: Apache Spark) > DataFrame windowspec functions : unresolved columns

[jira] [Assigned] (SPARK-41846) DataFrame windowspec functions : unresolved columns

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41846: Assignee: Apache Spark > DataFrame windowspec functions : unresolved columns >

[jira] [Commented] (SPARK-41825) DataFrame.show formatting int as double

2023-01-04 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654440#comment-17654440 ] Ruifeng Zheng commented on SPARK-41825: --- I'll take this one > DataFrame.show formatting int as

[jira] [Assigned] (SPARK-41883) Upgrade dropwizard metrics 4.2.15

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41883: Assignee: (was: Apache Spark) > Upgrade dropwizard metrics 4.2.15 >

[jira] [Assigned] (SPARK-41883) Upgrade dropwizard metrics 4.2.15

2023-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41883: Assignee: Apache Spark > Upgrade dropwizard metrics 4.2.15 >

  1   2   >