[jira] [Created] (SPARK-42274) Upgrade `compress-lzf` to 1.1.2

2023-01-31 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-42274: - Summary: Upgrade `compress-lzf` to 1.1.2 Key: SPARK-42274 URL: https://issues.apache.org/jira/browse/SPARK-42274 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-42052) Codegen Support for HiveSimpleUDF

2023-01-31 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682893#comment-17682893 ] BingKun Pan commented on SPARK-42052: - OK, Let me try. > Codegen Support for HiveSimpleUDF >

[jira] [Updated] (SPARK-42273) Skip Spark Connect tests if dependencies are not installed

2023-01-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42273: - Description: {code}

[jira] [Created] (SPARK-42273) Skip Spark Connect tests if dependencies are not installed

2023-01-31 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42273: Summary: Skip Spark Connect tests if dependencies are not installed Key: SPARK-42273 URL: https://issues.apache.org/jira/browse/SPARK-42273 Project: Spark

[jira] [Commented] (SPARK-42052) Codegen Support for HiveSimpleUDF

2023-01-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682890#comment-17682890 ] Kent Yao commented on SPARK-42052: -- Yes [~panbingkun], you can give it a try > Codegen Support for

[jira] [Comment Edited] (SPARK-42052) Codegen Support for HiveSimpleUDF

2023-01-31 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682883#comment-17682883 ] BingKun Pan edited comment on SPARK-42052 at 2/1/23 7:02 AM: - [~yao], Can I

[jira] [Assigned] (SPARK-42255) Assign name to _LEGACY_ERROR_TEMP_2430

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42255: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_2430 >

[jira] [Assigned] (SPARK-42255) Assign name to _LEGACY_ERROR_TEMP_2430

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42255: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_2430 >

[jira] [Commented] (SPARK-42255) Assign name to _LEGACY_ERROR_TEMP_2430

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682885#comment-17682885 ] Apache Spark commented on SPARK-42255: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-42052) Codegen Support for HiveSimpleUDF

2023-01-31 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682883#comment-17682883 ] BingKun Pan commented on SPARK-42052: - Can i do it [~yao] ? > Codegen Support for HiveSimpleUDF >

[jira] [Commented] (SPARK-40885) Spark will filter out data field sorting when dynamic partitions and data fields are sorted at the same time

2023-01-31 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682878#comment-17682878 ] Enrico Minack commented on SPARK-40885: --- This has been fixed in 3.4.0 and 3.5.0:

[jira] [Resolved] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42125. -- Resolution: Resolved Resolved by https://github.com/apache/spark/pull/39753 > Pandas UDF in

[jira] [Assigned] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42125: Assignee: Xinrong Meng > Pandas UDF in Spark Connect > --- > >

[jira] [Commented] (SPARK-42272) Use available ephemeral port for Spark Connect server in testing

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682863#comment-17682863 ] Apache Spark commented on SPARK-42272: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-42272) Use available ephemeral port for Spark Connect server in testing

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42272: Assignee: Apache Spark > Use available ephemeral port for Spark Connect server in

[jira] [Assigned] (SPARK-42272) Use available ephemeral port for Spark Connect server in testing

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42272: Assignee: (was: Apache Spark) > Use available ephemeral port for Spark Connect

[jira] [Created] (SPARK-42272) Use available ephemeral port for Spark Connect server in testing

2023-01-31 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42272: Summary: Use available ephemeral port for Spark Connect server in testing Key: SPARK-42272 URL: https://issues.apache.org/jira/browse/SPARK-42272 Project: Spark

[jira] [Resolved] (SPARK-42242) Upgrade snappy-java to 1.1.9.1

2023-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42242. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39811

[jira] [Assigned] (SPARK-42242) Upgrade snappy-java to 1.1.9.1

2023-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42242: - Assignee: Dongjoon Hyun > Upgrade snappy-java to 1.1.9.1 >

[jira] [Assigned] (SPARK-42270) Sort merge join may oom when right match rows are very large

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42270: Assignee: Apache Spark > Sort merge join may oom when right match rows are very large >

[jira] [Commented] (SPARK-42254) Assign name to _LEGACY_ERROR_TEMP_1117

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682854#comment-17682854 ] Apache Spark commented on SPARK-42254: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-42271) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682852#comment-17682852 ] Apache Spark commented on SPARK-42271: -- User 'xinrong-meng' has created a pull request for this

[jira] [Assigned] (SPARK-42271) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42271: Assignee: (was: Apache Spark) > Reuse UDF test cases under `pyspark.sql.tests` >

[jira] [Assigned] (SPARK-42271) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42271: Assignee: Apache Spark > Reuse UDF test cases under `pyspark.sql.tests` >

[jira] [Assigned] (SPARK-42254) Assign name to _LEGACY_ERROR_TEMP_1117

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42254: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_1117 >

[jira] [Assigned] (SPARK-42254) Assign name to _LEGACY_ERROR_TEMP_1117

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42254: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_1117 >

[jira] [Assigned] (SPARK-42270) Sort merge join may oom when right match rows are very large

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42270: Assignee: (was: Apache Spark) > Sort merge join may oom when right match rows are

[jira] [Commented] (SPARK-42270) Sort merge join may oom when right match rows are very large

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682853#comment-17682853 ] Apache Spark commented on SPARK-42270: -- User 'wankunde' has created a pull request for this issue:

[jira] [Commented] (SPARK-42271) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682851#comment-17682851 ] Apache Spark commented on SPARK-42271: -- User 'xinrong-meng' has created a pull request for this

[jira] [Created] (SPARK-42271) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42271: Summary: Reuse UDF test cases under `pyspark.sql.tests` Key: SPARK-42271 URL: https://issues.apache.org/jira/browse/SPARK-42271 Project: Spark Issue Type:

[jira] [Created] (SPARK-42270) Sort merge join may oom when right match rows are very large

2023-01-31 Thread Wan Kun (Jira)
Wan Kun created SPARK-42270: --- Summary: Sort merge join may oom when right match rows are very large Key: SPARK-42270 URL: https://issues.apache.org/jira/browse/SPARK-42270 Project: Spark Issue

[jira] [Updated] (SPARK-42269) Support user-specified return type as a collection DataType in DDL strings

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42269: - Summary: Support user-specified return type as a collection DataType in DDL strings (was:

[jira] [Created] (SPARK-42269) Support user-specified return type as a collection DataType

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42269: Summary: Support user-specified return type as a collection DataType Key: SPARK-42269 URL: https://issues.apache.org/jira/browse/SPARK-42269 Project: Spark

[jira] [Updated] (SPARK-42050) Hive UDF Codegen Support

2023-01-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-42050: - Affects Version/s: 3.5.0 (was: 3.4.0) > Hive UDF Codegen Support >

[jira] [Commented] (SPARK-41931) Improve UNSUPPORTED_DATA_TYPE message for complex types

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682848#comment-17682848 ] Apache Spark commented on SPARK-41931: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42268) Add UserDefinedType in protos

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42268: Assignee: (was: Apache Spark) > Add UserDefinedType in protos >

[jira] [Assigned] (SPARK-42268) Add UserDefinedType in protos

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42268: Assignee: Apache Spark > Add UserDefinedType in protos > - >

[jira] [Commented] (SPARK-42268) Add UserDefinedType in protos

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682846#comment-17682846 ] Apache Spark commented on SPARK-42268: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-42268) Add UserDefinedType in protos

2023-01-31 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-42268: - Summary: Add UserDefinedType in protos Key: SPARK-42268 URL: https://issues.apache.org/jira/browse/SPARK-42268 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-01-31 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682843#comment-17682843 ] Ritika Maheshwari commented on SPARK-41391: --- Modifying the agg method in

[jira] [Comment Edited] (SPARK-42261) K8s will not allocate more execs if there are any pending execs until next snapshot

2023-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682835#comment-17682835 ] Dongjoon Hyun edited comment on SPARK-42261 at 2/1/23 3:44 AM: --- I'm -1

[jira] [Commented] (SPARK-42261) K8s will not allocate more execs if there are any pending execs until next snapshot

2023-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682835#comment-17682835 ] Dongjoon Hyun commented on SPARK-42261: --- I'm -1 because this is a feature to prevent pending

[jira] [Updated] (SPARK-42242) Upgrade snappy-java to 1.1.9.1

2023-01-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42242: -- Summary: Upgrade snappy-java to 1.1.9.1 (was: Upgrade snappy-java to 1.1.9.0) > Upgrade

[jira] [Created] (SPARK-42267) Support left_outer join

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42267: Summary: Support left_outer join Key: SPARK-42267 URL: https://issues.apache.org/jira/browse/SPARK-42267 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-42266) Local mode should work with IPython

2023-01-31 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682829#comment-17682829 ] Ruifeng Zheng commented on SPARK-42266: --- cc [~gurwls223][~XinrongM] > Local mode should work with

[jira] [Created] (SPARK-42266) Local mode should work with IPython

2023-01-31 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-42266: - Summary: Local mode should work with IPython Key: SPARK-42266 URL: https://issues.apache.org/jira/browse/SPARK-42266 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41488) Assign name to _LEGACY_ERROR_TEMP_1176

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682822#comment-17682822 ] Apache Spark commented on SPARK-41488: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-42229) Migrate SparkCoreErrors into error class

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682820#comment-17682820 ] Apache Spark commented on SPARK-42229: -- User 'itholic' has created a pull request for this issue:

[jira] [Created] (SPARK-42265) DataFrame.createTempView - SparkConnectGrpcException: requirement failed

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42265: Summary: DataFrame.createTempView - SparkConnectGrpcException: requirement failed Key: SPARK-42265 URL: https://issues.apache.org/jira/browse/SPARK-42265 Project:

[jira] [Commented] (SPARK-42239) Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682819#comment-17682819 ] Apache Spark commented on SPARK-42239: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-41490) Assign name to _LEGACY_ERROR_TEMP_2441

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682818#comment-17682818 ] Apache Spark commented on SPARK-41490: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42208: Assignee: Xinrong Meng > Reuse UDF test cases under `pyspark.sql.tests` >

[jira] [Commented] (SPARK-41489) Assign name to _LEGACY_ERROR_TEMP_2415

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682817#comment-17682817 ] Apache Spark commented on SPARK-41489: -- User 'itholic' has created a pull request for this issue:

[jira] [Resolved] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42208. -- Resolution: Duplicate Duplicated by https://issues.apache.org/jira/browse/SPARK-42264 >

[jira] [Created] (SPARK-42264) Test Parity: pyspark.sql.tests.test_udf and pyspark.sql.tests.pandas.test_pandas_udf

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42264: Summary: Test Parity: pyspark.sql.tests.test_udf and pyspark.sql.tests.pandas.test_pandas_udf Key: SPARK-42264 URL: https://issues.apache.org/jira/browse/SPARK-42264

[jira] [Commented] (SPARK-42158) Assign name to _LEGACY_ERROR_TEMP_1003

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682815#comment-17682815 ] Apache Spark commented on SPARK-42158: -- User 'itholic' has created a pull request for this issue:

[jira] [Updated] (SPARK-42210) Standardize registered pickled Python UDFs

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42210: - Description: Implement spark.udf. > Standardize registered pickled Python UDFs >

[jira] [Updated] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42263: - Summary: Implement `spark.catalog.registerFunction` (was: Implement `spark.udf`) > Implement

[jira] [Updated] (SPARK-42263) Implement `spark.udf`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42263: - Summary: Implement `spark.udf` (was: Implement `spark.catalog.registerFunction`) > Implement

[jira] [Created] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42263: Summary: Implement `spark.catalog.registerFunction` Key: SPARK-42263 URL: https://issues.apache.org/jira/browse/SPARK-42263 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42051) Codegen Support for HiveGenericUDF

2023-01-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682806#comment-17682806 ] Kent Yao commented on SPARK-42051: -- resolvde by https://github.com/apache/spark/pull/39555 > Codegen

[jira] [Resolved] (SPARK-42051) Codegen Support for HiveGenericUDF

2023-01-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-42051. -- Fix Version/s: 3.5.0 Assignee: Kent Yao Resolution: Fixed > Codegen Support for

[jira] [Assigned] (SPARK-42191) Support udf luhn_check

2023-01-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42191: --- Assignee: Vinod KC > Support udf luhn_check > --- > >

[jira] [Resolved] (SPARK-42191) Support udf luhn_check

2023-01-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42191. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39747

[jira] [Resolved] (SPARK-42253) Add test for detecting duplicated error class

2023-01-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42253. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39821

[jira] [Assigned] (SPARK-42253) Add test for detecting duplicated error class

2023-01-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42253: Assignee: Haejoon Lee > Add test for detecting duplicated error class >

[jira] [Commented] (SPARK-36180) Support TimestampNTZ type in Hive

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682792#comment-17682792 ] Apache Spark commented on SPARK-36180: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36180) Support TimestampNTZ type in Hive

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682791#comment-17682791 ] Apache Spark commented on SPARK-36180: -- User 'gengliangwang' has created a pull request for this

[jira] [Resolved] (SPARK-42257) Remove unused variable in ExternalSorter

2023-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-42257. -- Fix Version/s: 3.5.0 Assignee: Khalid Mammadov Resolution: Fixed Resolved by

[jira] [Comment Edited] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682768#comment-17682768 ] Attila Zsolt Piros edited comment on SPARK-42068 at 1/31/23 9:42 PM: -

[jira] [Comment Edited] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682768#comment-17682768 ] Attila Zsolt Piros edited comment on SPARK-42068 at 1/31/23 9:40 PM: -

[jira] [Comment Edited] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682768#comment-17682768 ] Attila Zsolt Piros edited comment on SPARK-42068 at 1/31/23 9:39 PM: -

[jira] [Resolved] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-42068. Resolution: Workaround Resolving this issue as a workaround exits by using a

[jira] [Closed] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros closed SPARK-42068. -- Assignee: Attila Zsolt Piros > Implicit conversion is not working with

[jira] [Comment Edited] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682756#comment-17682756 ] Attila Zsolt Piros edited comment on SPARK-42068 at 1/31/23 9:10 PM: -

[jira] [Comment Edited] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682756#comment-17682756 ] Attila Zsolt Piros edited comment on SPARK-42068 at 1/31/23 9:09 PM: -

[jira] [Comment Edited] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682756#comment-17682756 ] Attila Zsolt Piros edited comment on SPARK-42068 at 1/31/23 9:08 PM: -

[jira] [Commented] (SPARK-42068) Implicit conversion is not working with parallelization in scala with java 11 and spark3

2023-01-31 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682756#comment-17682756 ] Attila Zsolt Piros commented on SPARK-42068: This issue is because of JDK bugfix

[jira] [Assigned] (SPARK-42262) Table schema changes via V2SessionCatalog with HiveExternalCatalog

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42262: Assignee: Apache Spark > Table schema changes via V2SessionCatalog with

[jira] [Commented] (SPARK-42262) Table schema changes via V2SessionCatalog with HiveExternalCatalog

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682733#comment-17682733 ] Apache Spark commented on SPARK-42262: -- User 'planga82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42262) Table schema changes via V2SessionCatalog with HiveExternalCatalog

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42262: Assignee: (was: Apache Spark) > Table schema changes via V2SessionCatalog with

[jira] [Commented] (SPARK-42262) Table schema changes via V2SessionCatalog with HiveExternalCatalog

2023-01-31 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682727#comment-17682727 ] Pablo Langa Blanco commented on SPARK-42262: I'm working on this > Table schema changes via

[jira] [Created] (SPARK-42262) Table schema changes via V2SessionCatalog with HiveExternalCatalog

2023-01-31 Thread Pablo Langa Blanco (Jira)
Pablo Langa Blanco created SPARK-42262: -- Summary: Table schema changes via V2SessionCatalog with HiveExternalCatalog Key: SPARK-42262 URL: https://issues.apache.org/jira/browse/SPARK-42262

[jira] [Assigned] (SPARK-42260) Log when the K8s Exec Pods Allocator Stalls

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42260: Assignee: Apache Spark (was: Holden Karau) > Log when the K8s Exec Pods Allocator

[jira] [Commented] (SPARK-42260) Log when the K8s Exec Pods Allocator Stalls

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682700#comment-17682700 ] Apache Spark commented on SPARK-42260: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42260) Log when the K8s Exec Pods Allocator Stalls

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42260: Assignee: Holden Karau (was: Apache Spark) > Log when the K8s Exec Pods Allocator

[jira] [Commented] (SPARK-42261) K8s will not allocate more execs if there are any pending execs until next snapshot

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682699#comment-17682699 ] Apache Spark commented on SPARK-42261: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42261) K8s will not allocate more execs if there are any pending execs until next snapshot

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42261: Assignee: Apache Spark > K8s will not allocate more execs if there are any pending execs

[jira] [Assigned] (SPARK-42261) K8s will not allocate more execs if there are any pending execs until next snapshot

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42261: Assignee: (was: Apache Spark) > K8s will not allocate more execs if there are any

[jira] [Created] (SPARK-42261) K8s will not allocate more execs if there are any pending execs until next snapshot

2023-01-31 Thread Holden Karau (Jira)
Holden Karau created SPARK-42261: Summary: K8s will not allocate more execs if there are any pending execs until next snapshot Key: SPARK-42261 URL: https://issues.apache.org/jira/browse/SPARK-42261

[jira] [Created] (SPARK-42260) Log when the K8s Exec Pods Allocator Stalls

2023-01-31 Thread Holden Karau (Jira)
Holden Karau created SPARK-42260: Summary: Log when the K8s Exec Pods Allocator Stalls Key: SPARK-42260 URL: https://issues.apache.org/jira/browse/SPARK-42260 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42259) ResolveGroupingAnalytics should take care of Python UDAF

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682676#comment-17682676 ] Apache Spark commented on SPARK-42259: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42259) ResolveGroupingAnalytics should take care of Python UDAF

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42259: Assignee: (was: Apache Spark) > ResolveGroupingAnalytics should take care of Python

[jira] [Assigned] (SPARK-42259) ResolveGroupingAnalytics should take care of Python UDAF

2023-01-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42259: Assignee: Apache Spark > ResolveGroupingAnalytics should take care of Python UDAF >

[jira] [Created] (SPARK-42259) ResolveGroupingAnalytics should take care of Python UDAF

2023-01-31 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-42259: --- Summary: ResolveGroupingAnalytics should take care of Python UDAF Key: SPARK-42259 URL: https://issues.apache.org/jira/browse/SPARK-42259 Project: Spark Issue

[jira] [Resolved] (SPARK-42236) Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT`

2023-01-31 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42236. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39804

[jira] [Assigned] (SPARK-42236) Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT`

2023-01-31 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42236: Assignee: Haejoon Lee > Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT` >

[jira] [Assigned] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-42245: Assignee: BingKun Pan > Upgrade scalafmt from 3.6.1 to 3.7.1 >

[jira] [Resolved] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-42245. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39816

[jira] [Updated] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-42245: - Priority: Trivial (was: Minor) > Upgrade scalafmt from 3.6.1 to 3.7.1 >

  1   2   >