[jira] [Created] (SPARK-44005) Improve the error messages when a UDTF returns a non-tuple value

2023-06-07 Thread Allison Wang (Jira)
Allison Wang created SPARK-44005: Summary: Improve the error messages when a UDTF returns a non-tuple value Key: SPARK-44005 URL: https://issues.apache.org/jira/browse/SPARK-44005 Project: Spark

[jira] [Assigned] (SPARK-43935) Add xpath_* functions to Scala and Python

2023-06-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43935: - Assignee: Ruifeng Zheng > Add xpath_* functions to Scala and Python >

[jira] [Resolved] (SPARK-43935) Add xpath_* functions to Scala and Python

2023-06-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43935. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41470

[jira] [Updated] (SPARK-42618) Support pandas 2.0.0.

2023-06-07 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42618: Summary: Support pandas 2.0.0. (was: Support pandas 2) > Support pandas 2.0.0. >

[jira] [Updated] (SPARK-42618) Support pandas 2

2023-06-07 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42618: Summary: Support pandas 2 (was: Support pandas 2.0.0) > Support pandas 2 > > >

[jira] [Commented] (SPARK-43273) Support writing Parquet files with lz4raw compression

2023-06-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730383#comment-17730383 ] Yuming Wang commented on SPARK-43273: - https://github.com/apache/spark/pull/41507 > Support writing

[jira] (SPARK-43273) Support writing Parquet files with lz4raw compression

2023-06-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43273 ] Yuming Wang deleted comment on SPARK-43273: - was (Author: q79969786): https://github.com/apache/spark/pull/41507 > Support writing Parquet files with lz4raw compression >

[jira] [Updated] (SPARK-43273) Support writing Parquet files with lz4raw compression

2023-06-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43273: Summary: Support writing Parquet files with lz4raw compression (was: Spark can't read parquet

[jira] [Commented] (SPARK-43273) Support writing Parquet files with lz4raw compression

2023-06-07 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730382#comment-17730382 ] Hudson commented on SPARK-43273: User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730377#comment-17730377 ] Ritika Maheshwari commented on SPARK-43514: --- Please provide the code to recreate the issue.

[jira] [Commented] (SPARK-43988) Add maven testing GA task for connect client module

2023-06-07 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730373#comment-17730373 ] Snoot.io commented on SPARK-43988: -- User 'LuciferYang' has created a pull request for this issue:

[jira] [Commented] (SPARK-43938) Add to_* functions to Scala and Python

2023-06-07 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730367#comment-17730367 ] Snoot.io commented on SPARK-43938: -- User 'panbingkun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-44000) Add join hint to disable broadcasting and replicating one side of a join

2023-06-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44000: - Assignee: Anton Okolnychyi > Add join hint to disable broadcasting and replicating one

[jira] [Resolved] (SPARK-44000) Add join hint to disable broadcasting and replicating one side of a join

2023-06-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44000. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41499

[jira] [Updated] (SPARK-43840) Switch `scala-213` GitHub Action Job to `scala-212`

2023-06-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43840: -- Fix Version/s: (was: 3.5.0) > Switch `scala-213` GitHub Action Job to `scala-212` >

[jira] [Reopened] (SPARK-43840) Switch `scala-213` GitHub Action Job to `scala-212`

2023-06-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-43840: --- Assignee: (was: Dongjoon Hyun) > Switch `scala-213` GitHub Action Job to `scala-212`

[jira] [Commented] (SPARK-43840) Switch `scala-213` GitHub Action Job to `scala-212`

2023-06-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730359#comment-17730359 ] Dongjoon Hyun commented on SPARK-43840: --- This is reverted via

[jira] [Commented] (SPARK-44003) DynamicPartitionDataSingleWriter is being starved by Parquet MemoryManager

2023-06-07 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730358#comment-17730358 ] Rafal Wojdyla commented on SPARK-44003: --- I believe {{MemoryManager}} has too strict

[jira] [Updated] (SPARK-44003) DynamicPartitionDataSingleWriter is being starved by Parquet MemoryManager

2023-06-07 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-44003: -- Description: We have a pyspark job that writes to a partitioned parquet dataset via:

[jira] [Updated] (SPARK-44003) DynamicPartitionDataSingleWriter is being starved by Parquet MemoryManager

2023-06-07 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-44003: -- Description: We have a pyspark job that writes to a partitioned parquet dataset via:

[jira] [Commented] (SPARK-44003) DynamicPartitionDataSingleWriter is being starved by Parquet MemoryManager

2023-06-07 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730353#comment-17730353 ] Rafal Wojdyla commented on SPARK-44003: --- And here's the [lock

[jira] [Created] (SPARK-44004) Assign name & improve error message for frequent LEGACY errors.

2023-06-07 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-44004: --- Summary: Assign name & improve error message for frequent LEGACY errors. Key: SPARK-44004 URL: https://issues.apache.org/jira/browse/SPARK-44004 Project: Spark

[jira] [Created] (SPARK-44003) DynamicPartitionDataSingleWriter is being starved by Parquet MemoryManager

2023-06-07 Thread Rafal Wojdyla (Jira)
Rafal Wojdyla created SPARK-44003: - Summary: DynamicPartitionDataSingleWriter is being starved by Parquet MemoryManager Key: SPARK-44003 URL: https://issues.apache.org/jira/browse/SPARK-44003

[jira] [Resolved] (SPARK-43920) Create sql/api module

2023-06-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-43920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-43920. --- Fix Version/s: 3.5.0 Resolution: Fixed > Create sql/api module >

[jira] [Resolved] (SPARK-43897) Reorg ps.Series unit tests

2023-06-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43897. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41401

[jira] [Assigned] (SPARK-43897) Reorg ps.Series unit tests

2023-06-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43897: - Assignee: Ruifeng Zheng > Reorg ps.Series unit tests > -- > >

[jira] [Resolved] (SPARK-44002) SparkConnectArtifactStatusesHandler freezes on second request

2023-06-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44002. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41500

[jira] [Comment Edited] (SPARK-43179) Add option for applications to control saving of metadata in the External Shuffle Service LevelDB

2023-06-07 Thread Chandni Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730307#comment-17730307 ] Chandni Singh edited comment on SPARK-43179 at 6/7/23 11:30 PM: Found a

[jira] [Reopened] (SPARK-43179) Add option for applications to control saving of metadata in the External Shuffle Service LevelDB

2023-06-07 Thread Chandni Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh reopened SPARK-43179: --- Found a bug so re-opening the issue.  > Add option for applications to control saving of

[jira] [Assigned] (SPARK-43540) Add working directory into classpath on the driver in K8S cluster mode

2023-06-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43540: - Assignee: Fei Wang > Add working directory into classpath on the driver in K8S cluster

[jira] [Resolved] (SPARK-43540) Add working directory into classpath on the driver in K8S cluster mode

2023-06-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43540. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41201

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Svyatoslav Semenyuk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730299#comment-17730299 ] Svyatoslav Semenyuk commented on SPARK-43514: - We were also having weird issue with data

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730287#comment-17730287 ] Ritika Maheshwari commented on SPARK-43514: --- I had deleted the comment because it was

[jira] [Created] (SPARK-44002) SparkConnectArtifactStatusesHandler freezes on second request

2023-06-07 Thread Max Gekk (Jira)
Max Gekk created SPARK-44002: Summary: SparkConnectArtifactStatusesHandler freezes on second request Key: SPARK-44002 URL: https://issues.apache.org/jira/browse/SPARK-44002 Project: Spark Issue

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Svyatoslav Semenyuk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730277#comment-17730277 ] Svyatoslav Semenyuk commented on SPARK-43514: - [~ritikam] is there a way to specify this

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730261#comment-17730261 ] Ritika Maheshwari commented on SPARK-43514: --- Hello please add this to your

[jira] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514 ] Ritika Maheshwari deleted comment on SPARK-43514: --- was (Author: ritikam): On further investigating the issue. It seems that the plan generated for 3.3.0 and 3.4.0 is same. Please see

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730239#comment-17730239 ] Ritika Maheshwari commented on SPARK-43514: --- On further investigating the issue. It seems that

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

2023-06-07 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: Plan.png > Unexpected NullPointerException or IllegalArgumentException

[jira] [Commented] (SPARK-43987) Separate finalizeShuffleMerge Processing to Dedicated Thread Pools

2023-06-07 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730206#comment-17730206 ] Hudson commented on SPARK-43987: User 'shuwang21' has created a pull request for this issue:

[jira] [Created] (SPARK-44001) Improve parsing of well known wrapper types

2023-06-07 Thread Parth Upadhyay (Jira)
Parth Upadhyay created SPARK-44001: -- Summary: Improve parsing of well known wrapper types Key: SPARK-44001 URL: https://issues.apache.org/jira/browse/SPARK-44001 Project: Spark Issue Type:

[jira] [Created] (SPARK-44000) Add join hint to disable broadcasting and replicating one side of a join

2023-06-07 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-44000: Summary: Add join hint to disable broadcasting and replicating one side of a join Key: SPARK-44000 URL: https://issues.apache.org/jira/browse/SPARK-44000

[jira] [Resolved] (SPARK-43933) Add linear regression aggregate functions to Scala and Python

2023-06-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43933. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41474

[jira] [Assigned] (SPARK-43933) Add linear regression aggregate functions to Scala and Python

2023-06-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43933: - Assignee: jiaan.geng > Add linear regression aggregate functions to Scala and Python >

[jira] [Commented] (SPARK-43999) Data is still fetched even though result was returned

2023-06-07 Thread Marek Byszewski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730143#comment-17730143 ] Marek Byszewski commented on SPARK-43999: - I'm experiencing the same issue. Outstanding queries

[jira] [Updated] (SPARK-43999) Data is still fetched even though result was returned

2023-06-07 Thread Kamil Kliczbor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Kliczbor updated SPARK-43999: --- Description: h2. Short problem description: I have two tables: * tab1 is empty * tab6

[jira] [Updated] (SPARK-43999) Data is still fetched even though result was returned

2023-06-07 Thread Kamil Kliczbor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Kliczbor updated SPARK-43999: --- Attachment: Profiler.PNG > Data is still fetched even though result was returned >

[jira] [Updated] (SPARK-43999) Data is still fetched even though result was returned

2023-06-07 Thread Kamil Kliczbor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Kliczbor updated SPARK-43999: --- Description: h2. Short problem description: I have two tables: * tab1 is empty * tab6

[jira] [Created] (SPARK-43999) Data is still fetched even though result was returned

2023-06-07 Thread Kamil Kliczbor (Jira)
Kamil Kliczbor created SPARK-43999: -- Summary: Data is still fetched even though result was returned Key: SPARK-43999 URL: https://issues.apache.org/jira/browse/SPARK-43999 Project: Spark

[jira] [Commented] (SPARK-43937) Add not,if,ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

2023-06-07 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730136#comment-17730136 ] BingKun Pan commented on SPARK-43937: - I work on it. > Add

[jira] [Updated] (SPARK-43991) Use the compression codec set by the spark config file when write compact log

2023-06-07 Thread shuyouZZ (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shuyouZZ updated SPARK-43991: - Description: Currently, if enable rolling log in SHS, only {{originalFilePath}} is used to determine

[jira] [Updated] (SPARK-43991) Use the value of spark.eventLog.compression.codec set by user when write compact file

2023-06-07 Thread shuyouZZ (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shuyouZZ updated SPARK-43991: - Summary: Use the value of spark.eventLog.compression.codec set by user when write compact file (was:

[jira] [Created] (SPARK-43998) Add support for UDAF

2023-06-07 Thread Venkata Sai Akhil Gudesa (Jira)
Venkata Sai Akhil Gudesa created SPARK-43998: Summary: Add support for UDAF Key: SPARK-43998 URL: https://issues.apache.org/jira/browse/SPARK-43998 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-43995) Implement UDFRegistration

2023-06-07 Thread Venkata Sai Akhil Gudesa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Sai Akhil Gudesa updated SPARK-43995: - Description: Reference file -

[jira] [Updated] (SPARK-43996) Implement call_udf function

2023-06-07 Thread Venkata Sai Akhil Gudesa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Sai Akhil Gudesa updated SPARK-43996: - Description:

[jira] [Updated] (SPARK-43997) Add support for Java UDFs

2023-06-07 Thread Venkata Sai Akhil Gudesa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Sai Akhil Gudesa updated SPARK-43997: - Description: API:  {code:java} def udf(f: UDF0[_], returnType:

[jira] [Created] (SPARK-43997) Add support for Java UDFs

2023-06-07 Thread Venkata Sai Akhil Gudesa (Jira)
Venkata Sai Akhil Gudesa created SPARK-43997: Summary: Add support for Java UDFs Key: SPARK-43997 URL: https://issues.apache.org/jira/browse/SPARK-43997 Project: Spark Issue

[jira] [Assigned] (SPARK-43984) Change to use foreach when map doesn't produce results

2023-06-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43984: Assignee: Yang Jie > Change to use foreach when map doesn't produce results >

[jira] [Resolved] (SPARK-43984) Change to use foreach when map doesn't produce results

2023-06-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43984. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41482

[jira] [Resolved] (SPARK-43993) Add tests for cache artifacts

2023-06-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43993. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41493

[jira] [Resolved] (SPARK-43790) Add API `copyLocalFileToHadoopFS`

2023-06-07 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-43790. Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41357

[jira] [Assigned] (SPARK-43921) Generate Protobuf descriptor files at build time

2023-06-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43921: Assignee: Raghu Angadi > Generate Protobuf descriptor files at build time >

[jira] [Resolved] (SPARK-43921) Generate Protobuf descriptor files at build time

2023-06-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43921. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41377

[jira] [Created] (SPARK-43996) Implement call_udf function

2023-06-07 Thread Venkata Sai Akhil Gudesa (Jira)
Venkata Sai Akhil Gudesa created SPARK-43996: Summary: Implement call_udf function Key: SPARK-43996 URL: https://issues.apache.org/jira/browse/SPARK-43996 Project: Spark Issue

[jira] [Created] (SPARK-43995) Implement UDFRegistration

2023-06-07 Thread Venkata Sai Akhil Gudesa (Jira)
Venkata Sai Akhil Gudesa created SPARK-43995: Summary: Implement UDFRegistration Key: SPARK-43995 URL: https://issues.apache.org/jira/browse/SPARK-43995 Project: Spark Issue

[jira] [Commented] (SPARK-43994) Upgrade zstd-jni to 1.5.5-4

2023-06-07 Thread Nikita Awasthi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730085#comment-17730085 ] Nikita Awasthi commented on SPARK-43994: User 'panbingkun' has created a pull request for this

[jira] [Updated] (SPARK-43994) Upgrade zstd-jni to 1.5.5-4

2023-06-07 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43994: Description: New version includes a bug fixed [Fix bug in ZstdBufferDecompressingStream when

[jira] [Created] (SPARK-43994) Upgrade zstd-jni to 1.5.5-4

2023-06-07 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43994: --- Summary: Upgrade zstd-jni to 1.5.5-4 Key: SPARK-43994 URL: https://issues.apache.org/jira/browse/SPARK-43994 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43993) Add tests for cache artifacts

2023-06-07 Thread Max Gekk (Jira)
Max Gekk created SPARK-43993: Summary: Add tests for cache artifacts Key: SPARK-43993 URL: https://issues.apache.org/jira/browse/SPARK-43993 Project: Spark Issue Type: Test Components:

[jira] [Updated] (SPARK-43992) Add optional pattern for Catalog.listFunctions

2023-06-07 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43992: --- Description: Currently, the syntax {code:java} SHOW FUNCTIONS LIKE pattern {code} supports a

[jira] [Updated] (SPARK-41289) Feature parity: Catalog API

2023-06-07 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-41289: --- Component/s: PySpark python SQL > Feature parity: Catalog API >

[jira] [Created] (SPARK-43992) Add optional pattern for Catalog.listFunctions

2023-06-07 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-43992: -- Summary: Add optional pattern for Catalog.listFunctions Key: SPARK-43992 URL: https://issues.apache.org/jira/browse/SPARK-43992 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43961) Add optional pattern for Catalog.listTables

2023-06-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43961. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41461

[jira] [Updated] (SPARK-43991) Use the compression codec set by the spark config file when write compact log

2023-06-07 Thread shuyouZZ (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shuyouZZ updated SPARK-43991: - Summary: Use the compression codec set by the spark config file when write compact log (was: Use the

[jira] [Created] (SPARK-43991) Use the compression codec set by the spark config file

2023-06-07 Thread shuyouZZ (Jira)
shuyouZZ created SPARK-43991: Summary: Use the compression codec set by the spark config file Key: SPARK-43991 URL: https://issues.apache.org/jira/browse/SPARK-43991 Project: Spark Issue Type:

[jira] [Created] (SPARK-43990) Upgrade kubernetes-client to 6.7.1

2023-06-07 Thread Jira
Bjørn Jørgensen created SPARK-43990: --- Summary: Upgrade kubernetes-client to 6.7.1 Key: SPARK-43990 URL: https://issues.apache.org/jira/browse/SPARK-43990 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43979) CollectedMetrics should be treated as the same one for self-join

2023-06-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43979. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41475

[jira] [Resolved] (SPARK-43717) Scala Client Dataset#reduce failed to handle null partitions for scala primitive types

2023-06-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43717. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41264

[jira] [Assigned] (SPARK-43717) Scala Client Dataset#reduce failed to handle null partitions for scala primitive types

2023-06-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43717: Assignee: Zhen Li > Scala Client Dataset#reduce failed to handle null partitions for scala >

[jira] [Created] (SPARK-43989) Add maven testing GA task for connect server module

2023-06-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-43989: Summary: Add maven testing GA task for connect server module Key: SPARK-43989 URL: https://issues.apache.org/jira/browse/SPARK-43989 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43988) Add maven testing GA task for connect client module

2023-06-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43988: - Summary: Add maven testing GA task for connect client module (was: Add independent maven testing GA

[jira] [Created] (SPARK-43988) Add independent maven testing GA task for connect client module

2023-06-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-43988: Summary: Add independent maven testing GA task for connect client module Key: SPARK-43988 URL: https://issues.apache.org/jira/browse/SPARK-43988 Project: Spark