[jira] [Resolved] (SPARK-44490) Remove TaskPagedTable in StagePage

2023-07-31 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-44490. Fix Version/s: 4.0.0 Assignee: dzcxzl Resolution: Fixed Issue resolved in

[jira] [Resolved] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44598. - Resolution: Not A Problem > spark 3.2+ can not read hive table with hbase serde when hbase Store

[jira] [Reopened] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-44598: - > spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize > is 0 > --

[jira] [Comment Edited] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749445#comment-17749445 ] ming95 edited comment on SPARK-44598 at 8/1/23 4:23 AM:

[jira] [Resolved] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ming95 resolved SPARK-44598. Resolution: Fixed > spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize

[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749445#comment-17749445 ] ming95 commented on SPARK-44598: it seem is a hbase bug. and hbase fix this bug

[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749444#comment-17749444 ] ming95 commented on SPARK-44598: [~ulysses] yes , it fix my case , thanks~ > sp

[jira] [Commented] (SPARK-44591) Add jobTags to SparkListenerSQLExecutionStart

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749442#comment-17749442 ] Snoot.io commented on SPARK-44591: -- User 'jasonli-db' has created a pull request for th

[jira] [Commented] (SPARK-44422) Fine grained interrupt in Spark Connect

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749435#comment-17749435 ] Snoot.io commented on SPARK-44422: -- User 'juliuszsompolski' has created a pull request

[jira] [Updated] (SPARK-44564) Refine the documents with LLM

2023-07-31 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44564: -- Description: Let's first focus on the Documents of *PySpark DataFrame APIs*. *1*, Chose a sub

[jira] [Commented] (SPARK-44424) Reattach to existing execute in Spark Connect (python client)

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749434#comment-17749434 ] Snoot.io commented on SPARK-44424: -- User 'HyukjinKwon' has created a pull request for t

[jira] [Resolved] (SPARK-44423) Reattach to existing execute in Spark Connect (scala client)

2023-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44423. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-44480) Add option for thread pool to perform maintenance for RocksDB/HDFS State Store Providers

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749432#comment-17749432 ] Snoot.io commented on SPARK-44480: -- User 'ericm-db' has created a pull request for this

[jira] [Commented] (SPARK-44480) Add option for thread pool to perform maintenance for RocksDB/HDFS State Store Providers

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749433#comment-17749433 ] Snoot.io commented on SPARK-44480: -- User 'ericm-db' has created a pull request for this

[jira] [Resolved] (SPARK-44421) Reattach to existing execute in Spark Connect (server mechanism)

2023-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44421. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-44421) Reattach to existing execute in Spark Connect (server mechanism)

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749430#comment-17749430 ] Snoot.io commented on SPARK-44421: -- User 'juliuszsompolski' has created a pull request

[jira] [Assigned] (SPARK-44423) Reattach to existing execute in Spark Connect (scala client)

2023-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44423: Assignee: Juliusz Sompolski > Reattach to existing execute in Spark Connect (scala client

[jira] [Assigned] (SPARK-44421) Reattach to existing execute in Spark Connect (server mechanism)

2023-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44421: Assignee: Juliusz Sompolski > Reattach to existing execute in Spark Connect (server mecha

[jira] [Commented] (SPARK-42941) Add support for streaming listener in Python

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749427#comment-17749427 ] Snoot.io commented on SPARK-42941: -- User 'bogao007' has created a pull request for this

[jira] [Commented] (SPARK-44218) Customize diff log in assertDataFrameEqual error message format

2023-07-31 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749426#comment-17749426 ] Snoot.io commented on SPARK-44218: -- User 'asl3' has created a pull request for this iss

[jira] [Resolved] (SPARK-44615) Rename spark connect client suites to avoid conflict

2023-07-31 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-44615. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-44583) `spark.*.io.connectionCreationTimeout` parameter documentation

2023-07-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44583. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-44583) `spark.*.io.connectionCreationTimeout` parameter documentation

2023-07-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44583: Assignee: dzcxzl > `spark.*.io.connectionCreationTimeout` parameter documentation > -

[jira] [Updated] (SPARK-44619) Free up disk space for pyspark container jobs

2023-07-31 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44619: -- Summary: Free up disk space for pyspark container jobs (was: Free up disk space for container

[jira] [Created] (SPARK-44619) Free up disk space for container jobs

2023-07-31 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-44619: - Summary: Free up disk space for container jobs Key: SPARK-44619 URL: https://issues.apache.org/jira/browse/SPARK-44619 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749421#comment-17749421 ] XiDuo You commented on SPARK-44598: --- please try `--conf spark.hadoopRDD.ignoreEmptySpl

[jira] [Resolved] (SPARK-44586) TorchDistributor should install cpu-only Torch for testing

2023-07-31 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44586. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42210 [https://

[jira] [Assigned] (SPARK-44586) TorchDistributor should install cpu-only Torch for testing

2023-07-31 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44586: - Assignee: Ruifeng Zheng > TorchDistributor should install cpu-only Torch for testing >

[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749418#comment-17749418 ] Yuming Wang commented on SPARK-44598: - How to reproduce this issue? > spark 3.2+ ca

[jira] [Assigned] (SPARK-44579) Support Interrupt On Cancel in SQLExecution

2023-07-31 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You reassigned SPARK-44579: - Assignee: Kent Yao > Support Interrupt On Cancel in SQLExecution >

[jira] [Resolved] (SPARK-44579) Support Interrupt On Cancel in SQLExecution

2023-07-31 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-44579. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42199 [https://github.c

[jira] [Resolved] (SPARK-44611) Remove exclusion for scala-xml for Spark Connect Scala Client

2023-07-31 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44611. --- Fix Version/s: 3.5.0 Resolution: Fixed > Remove exclusion for scala-xml for S

[jira] [Resolved] (SPARK-44599) Python client for reattaching to existing execute in Spark Connect (server mechanism)

2023-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44599. -- Resolution: Duplicate > Python client for reattaching to existing execute in Spark Connect (se

[jira] [Commented] (SPARK-44599) Python client for reattaching to existing execute in Spark Connect (server mechanism)

2023-07-31 Thread Juliusz Sompolski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749370#comment-17749370 ] Juliusz Sompolski commented on SPARK-44599: --- Duplicate of https://issues.apach

[jira] [Assigned] (SPARK-44617) Support comparison between lists of Rows

2023-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44617: Assignee: Amanda Liu > Support comparison between lists of Rows > ---

[jira] [Resolved] (SPARK-44617) Support comparison between lists of Rows

2023-07-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44617. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-44618) Free up disk space for non-container jobs

2023-07-31 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-44618: - Summary: Free up disk space for non-container jobs Key: SPARK-44618 URL: https://issues.apache.org/jira/browse/SPARK-44618 Project: Spark Issue Type: Sub-t

[jira] [Created] (SPARK-44617) Support comparison between list of Rows

2023-07-31 Thread Amanda Liu (Jira)
Amanda Liu created SPARK-44617: -- Summary: Support comparison between list of Rows Key: SPARK-44617 URL: https://issues.apache.org/jira/browse/SPARK-44617 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44617) Support comparison between lists of Rows

2023-07-31 Thread Amanda Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amanda Liu updated SPARK-44617: --- Summary: Support comparison between lists of Rows (was: Support comparison between list of Rows) >

[jira] [Created] (SPARK-44616) Hive Generic UDF support no longer supports short-circuiting of argument evaluation

2023-07-31 Thread Andy Grove (Jira)
Andy Grove created SPARK-44616: -- Summary: Hive Generic UDF support no longer supports short-circuiting of argument evaluation Key: SPARK-44616 URL: https://issues.apache.org/jira/browse/SPARK-44616 Proje

[jira] [Comment Edited] (SPARK-39441) Speed up DeduplicateRelations

2023-07-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh edited comment on SPARK-39441 at 7/31/23 9:03 PM: - After

[jira] [Comment Edited] (SPARK-39441) Speed up DeduplicateRelations

2023-07-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh edited comment on SPARK-39441 at 7/31/23 7:40 PM: - After

[jira] [Commented] (SPARK-39441) Speed up DeduplicateRelations

2023-07-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh commented on SPARK-39441: After applying this fix to 3.3.2, I still see some slow

[jira] [Updated] (SPARK-44578) Support pushing down BoundFunction in DSv2

2023-07-31 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-44578: - Description: See [https://github.com/apache/iceberg/pull/7886#discussion_r1257537662]  (was: We

[jira] [Resolved] (SPARK-44561) Fix AssertionError when converting UDTF output to a complex type

2023-07-31 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44561. --- Fix Version/s: 4.0.0 Assignee: Allison Wang Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-44615) Rename spark connect client suites to avoid conflict

2023-07-31 Thread Zhen Li (Jira)
Zhen Li created SPARK-44615: --- Summary: Rename spark connect client suites to avoid conflict Key: SPARK-44615 URL: https://issues.apache.org/jira/browse/SPARK-44615 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-44614) Add missing packages in setup.py

2023-07-31 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-44614: - Summary: Add missing packages in setup.py Key: SPARK-44614 URL: https://issues.apache.org/jira/browse/SPARK-44614 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-44613) Add Encoders.scala to Spark Connect Scala Client

2023-07-31 Thread Jira
Herman van Hövell created SPARK-44613: - Summary: Add Encoders.scala to Spark Connect Scala Client Key: SPARK-44613 URL: https://issues.apache.org/jira/browse/SPARK-44613 Project: Spark Is

[jira] [Resolved] (SPARK-44610) DeduplicateRelations should retain Alias metadata when creating a new instance

2023-07-31 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-44610. Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 42242 [https:

[jira] [Assigned] (SPARK-44610) DeduplicateRelations should retain Alias metadata when creating a new instance

2023-07-31 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-44610: -- Assignee: Wenchen Fan > DeduplicateRelations should retain Alias metadata when creati

[jira] [Updated] (SPARK-44508) Add user guide for Python UDTFs

2023-07-31 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-44508: - Summary: Add user guide for Python UDTFs (was: Add user guide and documentation for Python UDTF

[jira] [Resolved] (SPARK-44603) Add pyspark.testing to setup.py

2023-07-31 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44603. --- Fix Version/s: 3.5.0 Assignee: Amanda Liu Resolution: Fixed Issue resolved b

[jira] [Commented] (SPARK-44116) Utilize Hadoop vectorized APIs

2023-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749297#comment-17749297 ] Dongjoon Hyun commented on SPARK-44116: --- Thank you, [~ste...@apache.org]. > Utili

[jira] [Updated] (SPARK-44597) Migrate test_sql assert_eq to use assertDataFrameEqual

2023-07-31 Thread Amanda Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amanda Liu updated SPARK-44597: --- Description: The Jira ticket [[SPARK-44042] SPIP: PySpark Test Framework |https://issues.apache.org

[jira] [Updated] (SPARK-44597) Migrate test_sql assert_eq to use assertDataFrameEqual

2023-07-31 Thread Amanda Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amanda Liu updated SPARK-44597: --- Description: The Jira ticket [SPARK-44042] SPIP: PySpark Test Framework introduces a new PySpark t

[jira] [Resolved] (SPARK-43997) Add support for Java UDFs

2023-07-31 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-43997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-43997. --- Fix Version/s: 3.5.0 Resolution: Fixed > Add support for Java UDFs >

[jira] [Updated] (SPARK-44612) Use jobTags in SparkListenerSQLExecutionStart to get SQL Execution ID for Spark UI Connect page

2023-07-31 Thread Jason Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Li updated SPARK-44612: - Description: Follow up to https://issues.apache.org/jira/browse/SPARK-44394 and https://issues.apache.or

[jira] [Assigned] (SPARK-43997) Add support for Java UDFs

2023-07-31 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-43997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-43997: - Assignee: Venkata Sai Akhil Gudesa > Add support for Java UDFs > --

[jira] [Updated] (SPARK-44612) Use jobTags in SparkListenerSQLExecutionStart to get SQL Execution ID for Spark UI Connect page

2023-07-31 Thread Jason Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Li updated SPARK-44612: - Description: Follow up to https://issues.apache.org/jira/browse/SPARK-44394 and https://issues.apache.or

[jira] [Created] (SPARK-44612) Use jobTags in SparkListenerSQLExecutionStart to get SQL Execution ID for Spark UI Connect page

2023-07-31 Thread Jason Li (Jira)
Jason Li created SPARK-44612: Summary: Use jobTags in SparkListenerSQLExecutionStart to get SQL Execution ID for Spark UI Connect page Key: SPARK-44612 URL: https://issues.apache.org/jira/browse/SPARK-44612

[jira] [Commented] (SPARK-44116) Utilize Hadoop vectorized APIs

2023-07-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749266#comment-17749266 ] Steve Loughran commented on SPARK-44116: If this gets into the libraries, you do

[jira] [Commented] (SPARK-44124) Upgrade AWS SDK to v2

2023-07-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749264#comment-17749264 ] Steve Loughran commented on SPARK-44124: we are soon to move hadoop trunk up to

[jira] [Updated] (SPARK-44513) Upgrade snappy-java to 1.1.10.3

2023-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44513: -- Issue Type: Bug (was: Improvement) > Upgrade snappy-java to 1.1.10.3 > --

[jira] [Created] (SPARK-44611) Remove exclusion for scala-xml for Spark Connect Scala Client

2023-07-31 Thread Jira
Herman van Hövell created SPARK-44611: - Summary: Remove exclusion for scala-xml for Spark Connect Scala Client Key: SPARK-44611 URL: https://issues.apache.org/jira/browse/SPARK-44611 Project: Spar

[jira] [Created] (SPARK-44610) DeduplicateRelations should retain Alias metadata when creating a new instance

2023-07-31 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-44610: --- Summary: DeduplicateRelations should retain Alias metadata when creating a new instance Key: SPARK-44610 URL: https://issues.apache.org/jira/browse/SPARK-44610 Project:

[jira] [Created] (SPARK-44609) ExecutorPodsAllocator doesn't create new executors if no pod snapshot captured pod creation

2023-07-31 Thread Alibi Yeslambek (Jira)
Alibi Yeslambek created SPARK-44609: --- Summary: ExecutorPodsAllocator doesn't create new executors if no pod snapshot captured pod creation Key: SPARK-44609 URL: https://issues.apache.org/jira/browse/SPARK-44609

[jira] [Resolved] (SPARK-44605) refine internal ShuffleWriteProcessor API

2023-07-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44605. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42234 [https://gith

[jira] [Assigned] (SPARK-44605) refine internal ShuffleWriteProcessor API

2023-07-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44605: --- Assignee: Wenchen Fan > refine internal ShuffleWriteProcessor API > ---

[jira] [Created] (SPARK-44608) Remove unused definitions from `DataTypeExpression`

2023-07-31 Thread Yang Jie (Jira)
Yang Jie created SPARK-44608: Summary: Remove unused definitions from `DataTypeExpression` Key: SPARK-44608 URL: https://issues.apache.org/jira/browse/SPARK-44608 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-44607) Remove unused function `containsNestedColumn` from `Filter`

2023-07-31 Thread Yang Jie (Jira)
Yang Jie created SPARK-44607: Summary: Remove unused function `containsNestedColumn` from `Filter` Key: SPARK-44607 URL: https://issues.apache.org/jira/browse/SPARK-44607 Project: Spark Issue Ty

[jira] [Created] (SPARK-44606) Generate Java PB files and replace the package names in the files when testing

2023-07-31 Thread Yang Jie (Jira)
Yang Jie created SPARK-44606: Summary: Generate Java PB files and replace the package names in the files when testing Key: SPARK-44606 URL: https://issues.apache.org/jira/browse/SPARK-44606 Project: Spark

[jira] [Commented] (SPARK-43646) Make `connect` module daily test pass

2023-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749097#comment-17749097 ] ASF GitHub Bot commented on SPARK-43646: User 'LuciferYang' has created a pull r

[jira] [Commented] (SPARK-43646) Make `connect` module daily test pass

2023-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749098#comment-17749098 ] ASF GitHub Bot commented on SPARK-43646: User 'LuciferYang' has created a pull r

[jira] [Commented] (SPARK-44577) INSERT BY NAME returns non-sensical error message

2023-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749089#comment-17749089 ] ASF GitHub Bot commented on SPARK-44577: User 'Hisoka-X' has created a pull requ

[jira] [Created] (SPARK-44605) refine internal ShuffleWriteProcessor API

2023-07-31 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-44605: --- Summary: refine internal ShuffleWriteProcessor API Key: SPARK-44605 URL: https://issues.apache.org/jira/browse/SPARK-44605 Project: Spark Issue Type: Improveme