[jira] [Commented] (SPARK-41961) Support table-valued functions with LATERAL

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656493#comment-17656493 ] Apache Spark commented on SPARK-41961: -- User 'allisonwang-db' has created a pull re

[jira] [Assigned] (SPARK-41961) Support table-valued functions with LATERAL

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41961: Assignee: Apache Spark > Support table-valued functions with LATERAL > --

[jira] [Assigned] (SPARK-41961) Support table-valued functions with LATERAL

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41961: Assignee: (was: Apache Spark) > Support table-valued functions with LATERAL > ---

[jira] [Commented] (SPARK-41961) Support table-valued functions with LATERAL

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656494#comment-17656494 ] Apache Spark commented on SPARK-41961: -- User 'allisonwang-db' has created a pull re

[jira] [Created] (SPARK-41962) Update the import order of class SpecificParquetRecordReaderBase

2023-01-10 Thread shuyouZZ (Jira)
shuyouZZ created SPARK-41962: Summary: Update the import order of class SpecificParquetRecordReaderBase Key: SPARK-41962 URL: https://issues.apache.org/jira/browse/SPARK-41962 Project: Spark Iss

[jira] [Updated] (SPARK-41962) Update the import order of scala package in class SpecificParquetRecordReaderBase

2023-01-10 Thread shuyouZZ (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shuyouZZ updated SPARK-41962: - Summary: Update the import order of scala package in class SpecificParquetRecordReaderBase (was: Update

[jira] [Assigned] (SPARK-41960) Assign name to _LEGACY_ERROR_TEMP_1056

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41960: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_1056 > ---

[jira] [Assigned] (SPARK-41960) Assign name to _LEGACY_ERROR_TEMP_1056

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41960: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_1056 >

[jira] [Commented] (SPARK-41960) Assign name to _LEGACY_ERROR_TEMP_1056

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656498#comment-17656498 ] Apache Spark commented on SPARK-41960: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-41872) Fix DataFrame createDataframe handling of None

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41872: Assignee: Ruifeng Zheng > Fix DataFrame createDataframe handling of None > --

[jira] [Resolved] (SPARK-41872) Fix DataFrame createDataframe handling of None

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41872. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39477 [https://gi

[jira] [Assigned] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41958: Assignee: wuyi > Disallow arbitrary custom classpath with proxy user in cluster mode > --

[jira] [Resolved] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41958. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39474 [https://gi

[jira] [Commented] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656529#comment-17656529 ] jiaan.geng commented on SPARK-41886: I want take a look! > `DataFrame.intersect` do

[jira] [Created] (SPARK-41963) Different exception in DataFrame.unpivot

2023-01-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41963: Summary: Different exception in DataFrame.unpivot Key: SPARK-41963 URL: https://issues.apache.org/jira/browse/SPARK-41963 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41963) Different exception message in DataFrame.unpivot

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41963: - Summary: Different exception message in DataFrame.unpivot (was: Different exception in DataFram

[jira] [Assigned] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41877: Assignee: Apache Spark > SparkSession.createDataFrame error parity >

[jira] [Assigned] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41877: Assignee: (was: Apache Spark) > SparkSession.createDataFrame error parity > -

[jira] [Commented] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656541#comment-17656541 ] Apache Spark commented on SPARK-41877: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656543#comment-17656543 ] Apache Spark commented on SPARK-41877: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-41964) Add the unsupported function list

2023-01-10 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41964: - Summary: Add the unsupported function list Key: SPARK-41964 URL: https://issues.apache.org/jira/browse/SPARK-41964 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41886: Assignee: Apache Spark > `DataFrame.intersect` doctest output has different order > -

[jira] [Commented] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656581#comment-17656581 ] Apache Spark commented on SPARK-41886: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41886: Assignee: (was: Apache Spark) > `DataFrame.intersect` doctest output has different or

[jira] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41886 ] jiaan.geng deleted comment on SPARK-41886: was (Author: beliefer): I want take a look! > `DataFrame.intersect` doctest output has different order > --

[jira] [Assigned] (SPARK-41964) Add the unsupported function list

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41964: Assignee: (was: Apache Spark) > Add the unsupported function list > -

[jira] [Commented] (SPARK-41964) Add the unsupported function list

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656592#comment-17656592 ] Apache Spark commented on SPARK-41964: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41964) Add the unsupported function list

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41964: Assignee: Apache Spark > Add the unsupported function list >

[jira] [Created] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41965: - Summary: Add DataFrameWriterV2 to PySpark API references Key: SPARK-41965 URL: https://issues.apache.org/jira/browse/SPARK-41965 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41965: Assignee: Apache Spark > Add DataFrameWriterV2 to PySpark API references > --

[jira] [Assigned] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41965: Assignee: (was: Apache Spark) > Add DataFrameWriterV2 to PySpark API references > ---

[jira] [Commented] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656599#comment-17656599 ] Apache Spark commented on SPARK-41965: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656598#comment-17656598 ] Apache Spark commented on SPARK-41965: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-41966) Add `CharType` and `TimestampNTZType` to PySpark API references

2023-01-10 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41966: - Summary: Add `CharType` and `TimestampNTZType` to PySpark API references Key: SPARK-41966 URL: https://issues.apache.org/jira/browse/SPARK-41966 Project: Spark

[jira] [Assigned] (SPARK-41966) Add `CharType` and `TimestampNTZType` to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41966: Assignee: Apache Spark > Add `CharType` and `TimestampNTZType` to PySpark API references

[jira] [Commented] (SPARK-41966) Add `CharType` and `TimestampNTZType` to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656601#comment-17656601 ] Apache Spark commented on SPARK-41966: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41966) Add `CharType` and `TimestampNTZType` to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41966: Assignee: (was: Apache Spark) > Add `CharType` and `TimestampNTZType` to PySpark API

[jira] [Assigned] (SPARK-41966) Add `CharType` and `TimestampNTZType` to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41966: Assignee: Apache Spark > Add `CharType` and `TimestampNTZType` to PySpark API references

[jira] [Resolved] (SPARK-41907) Function `sampleby` return parity

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41907. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39476 [https://

[jira] [Assigned] (SPARK-41907) Function `sampleby` return parity

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41907: - Assignee: jiaan.geng > Function `sampleby` return parity >

[jira] [Assigned] (SPARK-41752) UI improvement for nested SQL executions

2023-01-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41752: --- Assignee: Linhong Liu > UI improvement for nested SQL executions >

[jira] [Resolved] (SPARK-41752) UI improvement for nested SQL executions

2023-01-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41752. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39268 [https://gith

[jira] [Assigned] (SPARK-41949) Make stage scheduling support local-cluster mode

2023-01-10 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-41949: -- Assignee: Weichen Xu > Make stage scheduling support local-cluster mode > ---

[jira] [Resolved] (SPARK-41949) Make stage scheduling support local-cluster mode

2023-01-10 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-41949. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39424 [https://github

[jira] [Assigned] (SPARK-41575) Assign name to _LEGACY_ERROR_TEMP_2054

2023-01-10 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41575: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2054 >

[jira] [Resolved] (SPARK-41575) Assign name to _LEGACY_ERROR_TEMP_2054

2023-01-10 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41575. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39394 [https://github.com

[jira] [Created] (SPARK-41967) SBT unable to resolve particular packages from the imported maven build

2023-01-10 Thread Venkata Sai Akhil Gudesa (Jira)
Venkata Sai Akhil Gudesa created SPARK-41967: Summary: SBT unable to resolve particular packages from the imported maven build Key: SPARK-41967 URL: https://issues.apache.org/jira/browse/SPARK-41967

[jira] [Assigned] (SPARK-41822) Setup Scala/JVM Client Connection

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41822: - Assignee: Venkata Sai Akhil Gudesa > Setup Scala/JVM Client Connection > --

[jira] [Resolved] (SPARK-41822) Setup Scala/JVM Client Connection

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41822. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39361 [https://

[jira] [Resolved] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41877. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39482 [https://

[jira] [Assigned] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41877: - Assignee: Hyukjin Kwon > SparkSession.createDataFrame error parity > --

[jira] [Resolved] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41886. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39483 [https://

[jira] [Assigned] (SPARK-41886) `DataFrame.intersect` doctest output has different order

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41886: - Assignee: jiaan.geng > `DataFrame.intersect` doctest output has different order > -

[jira] [Created] (SPARK-41968) Refactor ProtobufSerDe to ProtobufSerDe[T]

2023-01-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-41968: Summary: Refactor ProtobufSerDe to ProtobufSerDe[T] Key: SPARK-41968 URL: https://issues.apache.org/jira/browse/SPARK-41968 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-41968) Refactor ProtobufSerDe to ProtobufSerDe[T]

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41968: Assignee: Apache Spark > Refactor ProtobufSerDe to ProtobufSerDe[T] > ---

[jira] [Commented] (SPARK-41968) Refactor ProtobufSerDe to ProtobufSerDe[T]

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656701#comment-17656701 ] Apache Spark commented on SPARK-41968: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-41968) Refactor ProtobufSerDe to ProtobufSerDe[T]

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656700#comment-17656700 ] Apache Spark commented on SPARK-41968: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-41968) Refactor ProtobufSerDe to ProtobufSerDe[T]

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41968: Assignee: (was: Apache Spark) > Refactor ProtobufSerDe to ProtobufSerDe[T] >

[jira] [Updated] (SPARK-40588) Sorting issue with partitioned-writing and AQE turned on

2023-01-10 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-40588: Labels: correctness (was: ) > Sorting issue with partitioned-writing and AQE turned on >

[jira] [Commented] (SPARK-40588) Sorting issue with partitioned-writing and AQE turned on

2023-01-10 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656705#comment-17656705 ] Erik Krogen commented on SPARK-40588: - Labeling with 'correctness' since this breaks

[jira] [Created] (SPARK-41969) Flaky Test: StreamingQueryStatusListenerSuite.test small retained queries

2023-01-10 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41969: - Summary: Flaky Test: StreamingQueryStatusListenerSuite.test small retained queries Key: SPARK-41969 URL: https://issues.apache.org/jira/browse/SPARK-41969 Project:

[jira] [Updated] (SPARK-41969) Flaky Test: StreamingQueryStatusListenerSuite.test small retained queries

2023-01-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41969: -- Description: I saw this failures on master branch frequently. https://github.com/apache/spark

[jira] [Updated] (SPARK-41969) Flaky Test: StreamingQueryStatusListenerSuite.test small retained queries

2023-01-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41969: -- Description: I saw this failures on master branch frequently. https://github.com/apache/spark

[jira] [Updated] (SPARK-41969) Flaky Test: StreamingQueryStatusListenerSuite.test small retained queries

2023-01-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41969: -- Description: I saw this failures on master branch frequently. https://github.com/apache/spark

[jira] [Updated] (SPARK-38173) Quoted column cannot be recognized correctly when quotedRegexColumnNames is true

2023-01-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38173: -- Fix Version/s: 3.2.4 > Quoted column cannot be recognized correctly when quotedRegexColumnName

[jira] [Created] (SPARK-41970) SparkPath

2023-01-10 Thread David Lewis (Jira)
David Lewis created SPARK-41970: --- Summary: SparkPath Key: SPARK-41970 URL: https://issues.apache.org/jira/browse/SPARK-41970 Project: Spark Issue Type: Improvement Components: Spark C

[jira] [Commented] (SPARK-41970) SparkPath

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656986#comment-17656986 ] Apache Spark commented on SPARK-41970: -- User 'databricks-david-lewis' has created a

[jira] [Assigned] (SPARK-41970) SparkPath

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41970: Assignee: (was: Apache Spark) > SparkPath > - > > Key: SPARK-

[jira] [Assigned] (SPARK-41970) SparkPath

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41970: Assignee: Apache Spark > SparkPath > - > > Key: SPARK-41970 >

[jira] [Commented] (SPARK-41970) SparkPath

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656988#comment-17656988 ] Apache Spark commented on SPARK-41970: -- User 'databricks-david-lewis' has created a

[jira] [Commented] (SPARK-41752) UI improvement for nested SQL executions

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657015#comment-17657015 ] Apache Spark commented on SPARK-41752: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-41964) Add the unsupported function list

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41964: Assignee: Ruifeng Zheng > Add the unsupported function list > ---

[jira] [Resolved] (SPARK-41964) Add the unsupported function list

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41964. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39484 [https://gi

[jira] [Resolved] (SPARK-41966) Add `CharType` and `TimestampNTZType` to PySpark API references

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41966. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39486 [https://gi

[jira] [Assigned] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41965: Assignee: Ruifeng Zheng > Add DataFrameWriterV2 to PySpark API references > -

[jira] [Resolved] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41965. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39485 [https://gi

[jira] [Commented] (SPARK-41876) Implement DataFrame `toLocalIterator`

2023-01-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657028#comment-17657028 ] jiaan.geng commented on SPARK-41876: I will take a look! > Implement DataFrame `toL

[jira] [Commented] (SPARK-41589) PyTorch Distributor

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657029#comment-17657029 ] Apache Spark commented on SPARK-41589: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41887) Support DataFrame hint parameter to be list

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657031#comment-17657031 ] Apache Spark commented on SPARK-41887: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41887) Support DataFrame hint parameter to be list

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41887: Assignee: Apache Spark > Support DataFrame hint parameter to be list > --

[jira] [Commented] (SPARK-41887) Support DataFrame hint parameter to be list

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657032#comment-17657032 ] Apache Spark commented on SPARK-41887: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41887) Support DataFrame hint parameter to be list

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41887: Assignee: (was: Apache Spark) > Support DataFrame hint parameter to be list > ---

[jira] [Commented] (SPARK-41876) Implement DataFrame `toLocalIterator`

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657040#comment-17657040 ] Apache Spark commented on SPARK-41876: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-41876) Implement DataFrame `toLocalIterator`

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41876: Assignee: Apache Spark > Implement DataFrame `toLocalIterator` >

[jira] (SPARK-41876) Implement DataFrame `toLocalIterator`

2023-01-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41876 ] jiaan.geng deleted comment on SPARK-41876: was (Author: beliefer): I will take a look! > Implement DataFrame `toLocalIterator` > - > > Key

[jira] [Assigned] (SPARK-41876) Implement DataFrame `toLocalIterator`

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41876: Assignee: (was: Apache Spark) > Implement DataFrame `toLocalIterator` > -

[jira] [Commented] (SPARK-41838) DataFrame.show() fix map printing

2023-01-10 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657041#comment-17657041 ] jiaan.geng commented on SPARK-41838: I want take a look! > DataFrame.show() fix map

[jira] [Reopened] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-41965: -- Assignee: (was: Ruifeng Zheng) Reverted in https://github.com/apache/spark/commit/d8ea5

[jira] [Assigned] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41965: Assignee: Apache Spark > Add DataFrameWriterV2 to PySpark API references > --

[jira] [Updated] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41965: - Fix Version/s: (was: 3.4.0) > Add DataFrameWriterV2 to PySpark API references >

[jira] [Assigned] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41965: Assignee: (was: Apache Spark) > Add DataFrameWriterV2 to PySpark API references > ---

[jira] [Created] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

2023-01-10 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41971: - Summary: `toPandas` should support duplicate filed names when arrow-optimization is on Key: SPARK-41971 URL: https://issues.apache.org/jira/browse/SPARK-41971 Proje

[jira] [Commented] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657059#comment-17657059 ] Ruifeng Zheng commented on SPARK-41971: --- I think that is due to something is wrong

[jira] [Updated] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-41971: -- Priority: Minor (was: Major) > `toPandas` should support duplicate filed names when arrow-opt

[jira] [Updated] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-41971: -- Issue Type: Bug (was: Improvement) > `toPandas` should support duplicate filed names when arr

[jira] [Comment Edited] (SPARK-41971) `toPandas` should support duplicate filed names when arrow-optimization is on

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657059#comment-17657059 ] Ruifeng Zheng edited comment on SPARK-41971 at 1/11/23 3:17 AM: --

[jira] [Resolved] (SPARK-41879) `DataFrame.collect` should support nested types

2023-01-10 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41879. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39462 [https://

[jira] [Commented] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657066#comment-17657066 ] Apache Spark commented on SPARK-41965: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41965) Add DataFrameWriterV2 to PySpark API references

2023-01-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657067#comment-17657067 ] Apache Spark commented on SPARK-41965: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-41972) Fix flaky test in StreamingQueryStatusListenerSuite

2023-01-10 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-41972: -- Summary: Fix flaky test in StreamingQueryStatusListenerSuite Key: SPARK-41972 URL: https://issues.apache.org/jira/browse/SPARK-41972 Project: Spark Issue

  1   2   >