[jira] [Updated] (SPARK-41873) Implement DataFrame `pandas_api`

2023-01-03 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41873: -- Summary: Implement DataFrame `pandas_api` (was: Implement DataFrameReader `pandas_api`) > Im

[jira] [Assigned] (SPARK-41821) Fix DataFrame.describe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41821: Assignee: Apache Spark > Fix DataFrame.describe > -- > >

[jira] [Commented] (SPARK-41821) Fix DataFrame.describe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654272#comment-17654272 ] Apache Spark commented on SPARK-41821: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-41821) Fix DataFrame.describe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41821: Assignee: (was: Apache Spark) > Fix DataFrame.describe > -- > >

[jira] [Commented] (SPARK-41821) Fix DataFrame.describe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654273#comment-17654273 ] Apache Spark commented on SPARK-41821: -- User 'beliefer' has created a pull request

[jira] [Updated] (SPARK-41872) Fix DataFrame createDataframe handling of None

2023-01-03 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41872: -- Description: {code:java} row = self.spark.createDataFrame([("Alice", None, None, None)], sche

[jira] [Updated] (SPARK-41872) Fix DataFrame createDataframe handling of None

2023-01-03 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41872: -- Summary: Fix DataFrame createDataframe handling of None (was: Fix DataFrame fillna with bool)

[jira] [Created] (SPARK-41874) Implement DataFrame `sameSemantics`

2023-01-03 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41874: - Summary: Implement DataFrame `sameSemantics` Key: SPARK-41874 URL: https://issues.apache.org/jira/browse/SPARK-41874 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41875) Throw proper errors in Dataset.to()

2023-01-03 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41875: - Summary: Throw proper errors in Dataset.to() Key: SPARK-41875 URL: https://issues.apache.org/jira/browse/SPARK-41875 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41875) Throw proper errors in Dataset.to()

2023-01-03 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41875: -- Description: {code:java} schema = StructType( [StructField("i", StringType(), True), Struc

[jira] [Resolved] (SPARK-41850) Fix `isnan` function

2023-01-03 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41850. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39376 [https://

[jira] [Assigned] (SPARK-41850) Fix `isnan` function

2023-01-03 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41850: - Assignee: Ruifeng Zheng > Fix `isnan` function > > >

[jira] [Commented] (SPARK-41828) Implement creating empty Dataframe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654284#comment-17654284 ] Apache Spark commented on SPARK-41828: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41828) Implement creating empty Dataframe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41828: Assignee: (was: Apache Spark) > Implement creating empty Dataframe >

[jira] [Commented] (SPARK-41828) Implement creating empty Dataframe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654285#comment-17654285 ] Apache Spark commented on SPARK-41828: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41828) Implement creating empty Dataframe

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41828: Assignee: Apache Spark > Implement creating empty Dataframe > ---

[jira] [Commented] (SPARK-41862) Fix a correctness bug in existence DEFAULT value lookups for the Orc data source

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654286#comment-17654286 ] Apache Spark commented on SPARK-41862: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-41862) Fix a correctness bug in existence DEFAULT value lookups for the Orc data source

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654287#comment-17654287 ] Apache Spark commented on SPARK-41862: -- User 'dongjoon-hyun' has created a pull req

[jira] [Updated] (SPARK-36939) Add orphan migration page into list in PySpark documentation

2023-01-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36939: -- Affects Version/s: 3.3.0 (was: 3.2.0) > Add orphan migration page i

[jira] [Commented] (SPARK-36939) Add orphan migration page into list in PySpark documentation

2023-01-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654292#comment-17654292 ] Dongjoon Hyun commented on SPARK-36939: --- Due to https://github.com/apache/spark/pu

[jira] [Resolved] (SPARK-41859) CreateHiveTableAsSelectCommand should set the overwrite flag correctly

2023-01-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41859. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39365 [https://gith

[jira] [Assigned] (SPARK-41859) CreateHiveTableAsSelectCommand should set the overwrite flag correctly

2023-01-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41859: --- Assignee: Wenchen Fan > CreateHiveTableAsSelectCommand should set the overwrite flag correc

[jira] [Created] (SPARK-41876) Implement DataFrame `toLocalIterator`

2023-01-03 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41876: - Summary: Implement DataFrame `toLocalIterator` Key: SPARK-41876 URL: https://issues.apache.org/jira/browse/SPARK-41876 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41554) Decimal.changePrecision produces ArrayIndexOutOfBoundsException

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654294#comment-17654294 ] Apache Spark commented on SPARK-41554: -- User 'fe2s' has created a pull request for

[jira] [Created] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-03 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41877: - Summary: SparkSession.createDataFrame error parity Key: SPARK-41877 URL: https://issues.apache.org/jira/browse/SPARK-41877 Project: Spark Issue Type: Sub-t

[jira] [Updated] (SPARK-41877) SparkSession.createDataFrame error parity

2023-01-03 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41877: -- Description: {code:java} df = self.spark.createDataFrame( [ (1, 10, 1.0, "one"),

[jira] [Created] (SPARK-41878) Add JIRAs or messages for skipped messages

2023-01-03 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41878: - Summary: Add JIRAs or messages for skipped messages Key: SPARK-41878 URL: https://issues.apache.org/jira/browse/SPARK-41878 Project: Spark Issue Type: Sub-

[jira] [Updated] (SPARK-41878) Add JIRAs or messages for skipped messages

2023-01-03 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41878: -- Description: Add JIRAs or Messages for all the skipped messages. (was: 5 tests pass now. Shou

[jira] [Commented] (SPARK-41878) Add JIRAs or messages for skipped messages

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654298#comment-17654298 ] Apache Spark commented on SPARK-41878: -- User 'techaddict' has created a pull reques

[jira] [Assigned] (SPARK-41878) Add JIRAs or messages for skipped messages

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41878: Assignee: Apache Spark (was: Hyukjin Kwon) > Add JIRAs or messages for skipped messages

[jira] [Commented] (SPARK-41878) Add JIRAs or messages for skipped messages

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654297#comment-17654297 ] Apache Spark commented on SPARK-41878: -- User 'techaddict' has created a pull reques

[jira] [Assigned] (SPARK-41878) Add JIRAs or messages for skipped messages

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41878: Assignee: Hyukjin Kwon (was: Apache Spark) > Add JIRAs or messages for skipped messages

[jira] [Assigned] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41780: Assignee: (was: Apache Spark) > `regexp_replace('', '[ad]{0, 2}', 'x')` causes an

[jira] [Commented] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654300#comment-17654300 ] Apache Spark commented on SPARK-41780: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41780: Assignee: Apache Spark > `regexp_replace('', '[ad]{0, 2}', 'x')` causes an internal e

[jira] [Updated] (SPARK-38261) Sync missing R packages with CI

2023-01-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38261: -- Fix Version/s: 3.2.4 > Sync missing R packages with CI > --- > >

[jira] [Updated] (SPARK-39596) Run `Linters, licenses, dependencies and documentation generation ` GitHub Actions failed

2023-01-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39596: -- Fix Version/s: 3.2.4 > Run `Linters, licenses, dependencies and documentation generation ` Git

[jira] [Updated] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-41780: Attachment: image-2023-01-04-14-12-26-126.png > `regexp_replace('', '[ad]{0, 2}', 'x')` causes

[jira] [Commented] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654305#comment-17654305 ] BingKun Pan commented on SPARK-41780: - This is not bug, only the error prompt is not

[jira] [Commented] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654311#comment-17654311 ] Remzi Yang commented on SPARK-41780: It is a bug I guess, because an internal error

[jira] [Created] (SPARK-41879) `DataFrame.collect` should support nested types

2023-01-03 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41879: - Summary: `DataFrame.collect` should support nested types Key: SPARK-41879 URL: https://issues.apache.org/jira/browse/SPARK-41879 Project: Spark Issue Type:

[jira] [Created] (SPARK-41880) Function `from_json` should support non-literal expression

2023-01-03 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41880: - Summary: Function `from_json` should support non-literal expression Key: SPARK-41880 URL: https://issues.apache.org/jira/browse/SPARK-41880 Project: Spark

[jira] [Updated] (SPARK-40307) Optimize (De)Serialization of Python UDFs by Arrow

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Summary: Optimize (De)Serialization of Python UDFs by Arrow (was: Optimize (De)Serialization of

[jira] [Updated] (SPARK-40307) Optimize (De)Serialization of Python UDFs by Arrow

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against PyS

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Summary: Introduce Arrow-optimized Python UDFs (was: Optimize (De)Serialization of Python UDFs

[jira] [Assigned] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40307: Assignee: (was: Apache Spark) > Introduce Arrow-optimized Python UDFs > -

[jira] [Created] (SPARK-41881) `DataFrame.collect` should handle None/NaN properly

2023-01-03 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41881: - Summary: `DataFrame.collect` should handle None/NaN properly Key: SPARK-41881 URL: https://issues.apache.org/jira/browse/SPARK-41881 Project: Spark Issue T

[jira] [Assigned] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40307: Assignee: Apache Spark > Introduce Arrow-optimized Python UDFs >

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against PyS

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against PyS

[jira] [Commented] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654322#comment-17654322 ] Apache Spark commented on SPARK-40307: -- User 'xinrong-meng' has created a pull requ

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against PyS

[jira] [Commented] (SPARK-41432) Protobuf serializer for SparkPlanGraphWrapper

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654323#comment-17654323 ] Apache Spark commented on SPARK-41432: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654326#comment-17654326 ] Apache Spark commented on SPARK-41833: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41833: Assignee: (was: Apache Spark) > DataFrame.collect() output parity with pyspark >

[jira] [Assigned] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41833: Assignee: Apache Spark > DataFrame.collect() output parity with pyspark > ---

[jira] [Commented] (SPARK-41833) DataFrame.collect() output parity with pyspark

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654327#comment-17654327 ] Apache Spark commented on SPARK-41833: -- User 'zhengruifeng' has created a pull requ

[jira] [Updated] (SPARK-41053) Better Spark UI scalability and Driver stability for large applications

2023-01-03 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-41053: --- Labels: release-notes (was: ) > Better Spark UI scalability and Driver stability for large

[jira] [Resolved] (SPARK-41857) Enable test_between_function, test_datetime_functions, test_expr, test_math_functions, test_window_functions_cumulative_sum, test_corr, test_cov, test_crosstab, test_ap

2023-01-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41857. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/39359 > Enable test_between_f

[jira] [Assigned] (SPARK-41857) Enable test_between_function, test_datetime_functions, test_expr, test_math_functions, test_window_functions_cumulative_sum, test_corr, test_cov, test_crosstab, test_ap

2023-01-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41857: Assignee: Sandeep Singh (was: Hyukjin Kwon) > Enable test_between_function, test_datetim

[jira] [Created] (SPARK-41858) Fix ORC reader perf regression due to DEFAULT value feature

2023-01-03 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41858: - Summary: Fix ORC reader perf regression due to DEFAULT value feature Key: SPARK-41858 URL: https://issues.apache.org/jira/browse/SPARK-41858 Project: Spark

[jira] [Updated] (SPARK-41858) Fix ORC reader perf regression due to DEFAULT value feature

2023-01-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41858: -- Description: A huge ORC reader perf regression is detected by SPARK-41782. The root cause was

[jira] [Commented] (SPARK-41855) `createDataFrame` doesn't handle None/NaN properly

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653915#comment-17653915 ] Apache Spark commented on SPARK-41855: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41855) `createDataFrame` doesn't handle None/NaN properly

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41855: Assignee: Apache Spark > `createDataFrame` doesn't handle None/NaN properly > ---

[jira] [Commented] (SPARK-41855) `createDataFrame` doesn't handle None/NaN properly

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653917#comment-17653917 ] Apache Spark commented on SPARK-41855: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41855) `createDataFrame` doesn't handle None/NaN properly

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41855: Assignee: (was: Apache Spark) > `createDataFrame` doesn't handle None/NaN properly >

[jira] [Assigned] (SPARK-41822) Setup Scala/JVM Client Connection

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41822: Assignee: Apache Spark > Setup Scala/JVM Client Connection >

[jira] [Assigned] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41814: Assignee: (was: Apache Spark) > Column.eqNullSafe fails on NaN comparison > -

[jira] [Commented] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653919#comment-17653919 ] Apache Spark commented on SPARK-41814: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41814: Assignee: Apache Spark > Column.eqNullSafe fails on NaN comparison >

[jira] [Commented] (SPARK-41822) Setup Scala/JVM Client Connection

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653918#comment-17653918 ] Apache Spark commented on SPARK-41822: -- User 'vicennial' has created a pull request

[jira] [Assigned] (SPARK-41822) Setup Scala/JVM Client Connection

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41822: Assignee: (was: Apache Spark) > Setup Scala/JVM Client Connection > -

[jira] [Commented] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653922#comment-17653922 ] Apache Spark commented on SPARK-41851: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653920#comment-17653920 ] Apache Spark commented on SPARK-41814: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41822) Setup Scala/JVM Client Connection

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653921#comment-17653921 ] Apache Spark commented on SPARK-41822: -- User 'vicennial' has created a pull request

[jira] [Assigned] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41851: Assignee: (was: Apache Spark) > Fix `nanvl` function > > >

[jira] [Assigned] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41851: Assignee: Apache Spark > Fix `nanvl` function > > >

[jira] [Commented] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653923#comment-17653923 ] Apache Spark commented on SPARK-41851: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41852: Assignee: (was: Apache Spark) > Fix `pmod` function > --- > >

[jira] [Assigned] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41852: Assignee: Apache Spark > Fix `pmod` function > --- > > Ke

[jira] [Commented] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653925#comment-17653925 ] Apache Spark commented on SPARK-41852: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653924#comment-17653924 ] Apache Spark commented on SPARK-41852: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41858) Fix ORC reader perf regression due to DEFAULT value feature

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41858: Assignee: Apache Spark > Fix ORC reader perf regression due to DEFAULT value feature > --

[jira] [Assigned] (SPARK-41858) Fix ORC reader perf regression due to DEFAULT value feature

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41858: Assignee: (was: Apache Spark) > Fix ORC reader perf regression due to DEFAULT value f

[jira] [Commented] (SPARK-41858) Fix ORC reader perf regression due to DEFAULT value feature

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653935#comment-17653935 ] Apache Spark commented on SPARK-41858: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

2023-01-03 Thread Jorge Machado (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653941#comment-17653941 ] Jorge Machado commented on SPARK-33772: --- I still have an issue with this. Running

[jira] [Commented] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2023-01-03 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653952#comment-17653952 ] BingKun Pan commented on SPARK-41780: - I will fix it. > `regexp_replace('', '[a

[jira] [Resolved] (SPARK-41855) `createDataFrame` doesn't handle None/NaN properly

2023-01-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41855. -- Fix Version/s: 3.4.0 Assignee: Ruifeng Zheng Resolution: Fixed Fixed in https:

[jira] [Resolved] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41814. -- Assignee: Ruifeng Zheng Resolution: Fixed Fixed in https://github.com/apache/spark/pull/

[jira] [Resolved] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41851. -- Assignee: Ruifeng Zheng Resolution: Fixed Fixed in https://github.com/apache/spark/pull/

[jira] [Resolved] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41852. -- Assignee: Ruifeng Zheng Resolution: Fixed Fixed in https://github.com/apache/spark/pull/

[jira] [Commented] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653961#comment-17653961 ] Apache Spark commented on SPARK-41814: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653963#comment-17653963 ] Apache Spark commented on SPARK-41851: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41814) Column.eqNullSafe fails on NaN comparison

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653962#comment-17653962 ] Apache Spark commented on SPARK-41814: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653964#comment-17653964 ] Apache Spark commented on SPARK-41851: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653967#comment-17653967 ] Apache Spark commented on SPARK-41852: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41851) Fix `nanvl` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653965#comment-17653965 ] Apache Spark commented on SPARK-41851: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653966#comment-17653966 ] Apache Spark commented on SPARK-41852: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653969#comment-17653969 ] Apache Spark commented on SPARK-41852: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41852) Fix `pmod` function

2023-01-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653968#comment-17653968 ] Apache Spark commented on SPARK-41852: -- User 'HyukjinKwon' has created a pull reque

  1   2   3   >