[jira] [Created] (SPARK-44888) Need to update the golden files of SQLQueryTestSuite for Java 21.

Yang Jie created SPARK-44888: Summary: Need to update the golden files of SQLQueryTestSuite for Java 21. Key: SPARK-44888 URL: https://issues.apache.org/jira/browse/SPARK-44888 Project: Spark

[jira] [Assigned] (SPARK-44887) Fix wildcard import `from pyspark.sql.functions import *`

[ https://issues.apache.org/jira/browse/SPARK-44887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44887: Assignee: Ruifeng Zheng > Fix wildcard import `from pyspark.sql.functions import *` >

[jira] [Resolved] (SPARK-44887) Fix wildcard import `from pyspark.sql.functions import *`

[ https://issues.apache.org/jira/browse/SPARK-44887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44887. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42579

[jira] [Commented] (SPARK-44751) XML: Implement FIleFormat Interface

[ https://issues.apache.org/jira/browse/SPARK-44751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756654#comment-17756654 ] Snoot.io commented on SPARK-44751: -- User 'sandip-db' has created a pull request for this issue:

[jira] [Commented] (SPARK-44845) spark job copies jars repeatedly if fs.defaultFS and application jar are same url

[ https://issues.apache.org/jira/browse/SPARK-44845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756652#comment-17756652 ] Snoot.io commented on SPARK-44845: -- User 'zekai-li' has created a pull request for this issue:

[jira] [Commented] (SPARK-44845) spark job copies jars repeatedly if fs.defaultFS and application jar are same url

[ https://issues.apache.org/jira/browse/SPARK-44845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756651#comment-17756651 ] Snoot.io commented on SPARK-44845: -- User 'zekai-li' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44844) Exclude `python/build/*` for local `lint-python` testing

[ https://issues.apache.org/jira/browse/SPARK-44844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-44844. - Resolution: Won't Fix It's not a general issue. > Exclude `python/build/*` for local

[jira] [Commented] (SPARK-44622) Implement error enrichment and JVM stacktrace

[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756649#comment-17756649 ] Snoot.io commented on SPARK-44622: -- User 'heyihong' has created a pull request for this issue:

[jira] [Commented] (SPARK-44622) Implement error enrichment and JVM stacktrace

[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756648#comment-17756648 ] Snoot.io commented on SPARK-44622: -- User 'heyihong' has created a pull request for this issue:

[jira] [Commented] (SPARK-43205) Add an IDENTIFIER(stringLiteral) clause that maps a string to an identifier

[ https://issues.apache.org/jira/browse/SPARK-43205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756647#comment-17756647 ] Snoot.io commented on SPARK-43205: -- User 'srielau' has created a pull request for this issue:

[jira] [Commented] (SPARK-43420) Make DisableUnnecessaryBucketedScan smart with table cache

[ https://issues.apache.org/jira/browse/SPARK-43420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756645#comment-17756645 ] Snoot.io commented on SPARK-43420: -- User 'ulysses-you' has created a pull request for this issue:

[jira] [Commented] (SPARK-43420) Make DisableUnnecessaryBucketedScan smart with table cache

[ https://issues.apache.org/jira/browse/SPARK-43420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756643#comment-17756643 ] Snoot.io commented on SPARK-43420: -- User 'ulysses-you' has created a pull request for this issue:

[jira] [Commented] (SPARK-44773) Code-gen CodegenFallback expression in WholeStageCodegen if possible

[ https://issues.apache.org/jira/browse/SPARK-44773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756642#comment-17756642 ] Snoot.io commented on SPARK-44773: -- User 'wankunde' has created a pull request for this issue:

[jira] [Commented] (SPARK-44864) Align streaming statistics link format with other page links

[ https://issues.apache.org/jira/browse/SPARK-44864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756641#comment-17756641 ] Snoot.io commented on SPARK-44864: -- User 'leesf' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44785) Convert common alreadyExistsExceptions and noSuchExceptions

[ https://issues.apache.org/jira/browse/SPARK-44785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44785. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44785) Convert common alreadyExistsExceptions and noSuchExceptions

[ https://issues.apache.org/jira/browse/SPARK-44785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44785: Assignee: Yihong He > Convert common alreadyExistsExceptions and noSuchExceptions >

[jira] [Resolved] (SPARK-44868) Convert datetime to string by `to_char`/`to_varchar`

[ https://issues.apache.org/jira/browse/SPARK-44868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44868. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42534

[jira] [Updated] (SPARK-44887) Fix wildcard import `from pyspark.sql.functions import *`

[ https://issues.apache.org/jira/browse/SPARK-44887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44887: -- Summary: Fix wildcard import `from pyspark.sql.functions import *` (was: Avoid wildcard

[jira] [Created] (SPARK-44887) Avoid wildcard import `from pyspark.sql.functions import *`

Ruifeng Zheng created SPARK-44887: - Summary: Avoid wildcard import `from pyspark.sql.functions import *` Key: SPARK-44887 URL: https://issues.apache.org/jira/browse/SPARK-44887 Project: Spark

[jira] [Updated] (SPARK-44886) Introduce CLUSTER BY clause to CREATE/REPLACE table

[ https://issues.apache.org/jira/browse/SPARK-44886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-44886: -- Description: This proposes to introduce CLUSTER BY clause to CREATE/REPLACE SQL syntax: {code:java}

[jira] [Created] (SPARK-44886) Introduce CLUSTER BY clause to CREATE/REPLACE table

Terry Kim created SPARK-44886: - Summary: Introduce CLUSTER BY clause to CREATE/REPLACE table Key: SPARK-44886 URL: https://issues.apache.org/jira/browse/SPARK-44886 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-44877) Support python protobuf functions for Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-44877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44877. --- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-44877) Support python protobuf functions for Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-44877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44877: - Assignee: Bo Gao > Support python protobuf functions for Spark Connect >

[jira] [Resolved] (SPARK-44842) Support stat functions

[ https://issues.apache.org/jira/browse/SPARK-44842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44842. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42526

[jira] [Resolved] (SPARK-43812) Enable DataFrameTests.test_all for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43812. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42526

[jira] [Assigned] (SPARK-43812) Enable DataFrameTests.test_all for pandas 2.0.0.

[ https://issues.apache.org/jira/browse/SPARK-43812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43812: - Assignee: Haejoon Lee > Enable DataFrameTests.test_all for pandas 2.0.0. >

[jira] [Assigned] (SPARK-44842) Support stat functions

[ https://issues.apache.org/jira/browse/SPARK-44842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44842: - Assignee: Haejoon Lee > Support stat functions > -- > >

[jira] [Commented] (SPARK-44817) Incremental Stats Collection

[ https://issues.apache.org/jira/browse/SPARK-44817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756625#comment-17756625 ] Hyukjin Kwon commented on SPARK-44817: -- I think this would need an SPIP preferably with a design

[jira] [Assigned] (SPARK-44858) Refine docstring of `DataFrame.isEmpty`

[ https://issues.apache.org/jira/browse/SPARK-44858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44858: - Assignee: Allison Wang > Refine docstring of `DataFrame.isEmpty` >

[jira] [Resolved] (SPARK-44858) Refine docstring of `DataFrame.isEmpty`

[ https://issues.apache.org/jira/browse/SPARK-44858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44858. --- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-40178) Rebalance/Repartition Hints Not Working in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40178. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42255

[jira] [Assigned] (SPARK-40178) Rebalance/Repartition Hints Not Working in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40178: Assignee: YE > Rebalance/Repartition Hints Not Working in PySpark >

[jira] [Resolved] (SPARK-44879) Refine docstring of `createDataFrame`

[ https://issues.apache.org/jira/browse/SPARK-44879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44879. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42569

[jira] [Assigned] (SPARK-44879) Refine docstring of `createDataFrame`

[ https://issues.apache.org/jira/browse/SPARK-44879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44879: Assignee: Allison Wang > Refine docstring of `createDataFrame` >

[jira] [Resolved] (SPARK-44876) Enable and fix test_parity_arrow_python_udf

[ https://issues.apache.org/jira/browse/SPARK-44876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44876. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44876) Enable and fix test_parity_arrow_python_udf

[ https://issues.apache.org/jira/browse/SPARK-44876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44876: Assignee: Takuya Ueshin > Enable and fix test_parity_arrow_python_udf >

[jira] [Resolved] (SPARK-44882) Remove function uuid/random/chr from PySpark

[ https://issues.apache.org/jira/browse/SPARK-44882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44882. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-44885) NullPointerException is thrown when column with ROWID type contains NULL values

[ https://issues.apache.org/jira/browse/SPARK-44885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Nieradzik updated SPARK-44885: -- Description: A row ID may be NULL in an Oracle table. When this is the case, the following

[jira] [Commented] (SPARK-44885) NullPointerException is thrown when column with ROWID type contains NULL values

[ https://issues.apache.org/jira/browse/SPARK-44885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756577#comment-17756577 ] Tim Nieradzik commented on SPARK-44885: --- PR submitted: https://github.com/apache/spark/pull/42576

[jira] [Created] (SPARK-44885) NullPointerException is thrown when column with ROWID type contains NULL values

Tim Nieradzik created SPARK-44885: - Summary: NullPointerException is thrown when column with ROWID type contains NULL values Key: SPARK-44885 URL: https://issues.apache.org/jira/browse/SPARK-44885

[jira] [Comment Edited] (SPARK-44840) array_insert() give wrong results for ngative index

[ https://issues.apache.org/jira/browse/SPARK-44840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756574#comment-17756574 ] Serge Rielau edited comment on SPARK-44840 at 8/20/23 7:41 PM: --- [~srowen]

[jira] [Commented] (SPARK-44840) array_insert() give wrong results for ngative index

[ https://issues.apache.org/jira/browse/SPARK-44840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756574#comment-17756574 ] Serge Rielau commented on SPARK-44840: -- [~srowen] There is no standard as such. However, there are

[jira] [Commented] (SPARK-44840) array_insert() give wrong results for ngative index

[ https://issues.apache.org/jira/browse/SPARK-44840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756549#comment-17756549 ] Sean R. Owen commented on SPARK-44840: -- What is the desired behavior - is Snowflake's behavior

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Attachment: image-2023-08-20-18-46-53-342.png > Spark doesn't create SUCCESS file when external

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Summary: Spark doesn't create SUCCESS file when external path is passed (was: Spark doesn't

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Attachment: image-2023-08-20-18-08-38-531.png > Spark doesn't create SUCCESS file when

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Created] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

Dipayan Dev created SPARK-44884: --- Summary: Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic Key: SPARK-44884 URL: https://issues.apache.org/jira/browse/SPARK-44884 Project:

[jira] [Updated] (SPARK-44883) Spark insertInto with location GCS bucket root causes NPE

[ https://issues.apache.org/jira/browse/SPARK-44883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44883: Description: In our Organisation, we are using GCS bucket root location to point to our Hive

[jira] [Created] (SPARK-44883) Spark insertInto with location GCS bucket root causes NPE

Dipayan Dev created SPARK-44883: --- Summary: Spark insertInto with location GCS bucket root causes NPE Key: SPARK-44883 URL: https://issues.apache.org/jira/browse/SPARK-44883 Project: Spark

[jira] [Assigned] (SPARK-44882) Remove function uuid/random/chr from PySpark

[ https://issues.apache.org/jira/browse/SPARK-44882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44882: - Assignee: Ruifeng Zheng > Remove function uuid/random/chr from PySpark >

[jira] [Updated] (SPARK-44882) Remove function uuid/random/chr from PySpark

[ https://issues.apache.org/jira/browse/SPARK-44882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44882: -- Parent: SPARK-43907 Issue Type: Sub-task (was: Improvement) > Remove function

[jira] [Created] (SPARK-44882) Remove function uuid/random/chr from PySpark

Ruifeng Zheng created SPARK-44882: - Summary: Remove function uuid/random/chr from PySpark Key: SPARK-44882 URL: https://issues.apache.org/jira/browse/SPARK-44882 Project: Spark Issue Type: