[jira] [Assigned] (SPARK-44367) Show error message on UI for each query

[ https://issues.apache.org/jira/browse/SPARK-44367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44367: Assignee: Kent Yao > Show error message on UI for each query >

[jira] [Resolved] (SPARK-44367) Show error message on UI for each query

[ https://issues.apache.org/jira/browse/SPARK-44367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44367. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-44491) Add `branch-3.5` to `publish_snapshot` GitHub Action job

BingKun Pan created SPARK-44491: --- Summary: Add `branch-3.5` to `publish_snapshot` GitHub Action job Key: SPARK-44491 URL: https://issues.apache.org/jira/browse/SPARK-44491 Project: Spark Issue

[jira] [Resolved] (SPARK-43778) RewriteCorrelatedScalarSubquery should handle duplicate attributes

[ https://issues.apache.org/jira/browse/SPARK-43778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Fan resolved SPARK-43778. - Fix Version/s: 4.0.0 Resolution: Fixed > RewriteCorrelatedScalarSubquery should handle duplicate

[jira] [Comment Edited] (SPARK-43778) RewriteCorrelatedScalarSubquery should handle duplicate attributes

[ https://issues.apache.org/jira/browse/SPARK-43778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744875#comment-17744875 ] Jia Fan edited comment on SPARK-43778 at 7/20/23 5:07 AM: -- This ticket already

[jira] [Commented] (SPARK-43778) RewriteCorrelatedScalarSubquery should handle duplicate attributes

[ https://issues.apache.org/jira/browse/SPARK-43778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744875#comment-17744875 ] Jia Fan commented on SPARK-43778: - This ticket already fixed by

[jira] [Updated] (SPARK-44490) Remove TaskPagedTable in StagePage

[ https://issues.apache.org/jira/browse/SPARK-44490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-44490: --- Description: In [SPARK-21809|https://issues.apache.org/jira/browse/SPARK-21809], we introduced

[jira] [Created] (SPARK-44490) Remove TaskPagedTable in StagePage

dzcxzl created SPARK-44490: -- Summary: Remove TaskPagedTable in StagePage Key: SPARK-44490 URL: https://issues.apache.org/jira/browse/SPARK-44490 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-44489) Make InsertIntoDataSourceDirCommand extends DataWritingCommand

[ https://issues.apache.org/jira/browse/SPARK-44489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan resolved SPARK-44489. --- Resolution: Not A Problem > Make InsertIntoDataSourceDirCommand extends DataWritingCommand >

[jira] [Commented] (SPARK-43966) Support non-deterministic Python UDTFs

[ https://issues.apache.org/jira/browse/SPARK-43966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744865#comment-17744865 ] Snoot.io commented on SPARK-43966: -- User 'allisonwang-db' has created a pull request for this issue:

[jira] [Commented] (SPARK-44380) Support for UDTF to analyze in Python

[ https://issues.apache.org/jira/browse/SPARK-44380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744864#comment-17744864 ] Snoot.io commented on SPARK-44380: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-44292) Assign names to the error class _LEGACY_ERROR_TEMP_[2315-2319]

[ https://issues.apache.org/jira/browse/SPARK-44292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744862#comment-17744862 ] Snoot.io commented on SPARK-44292: -- User 'beliefer' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44380) Support for UDTF to analyze in Python

[ https://issues.apache.org/jira/browse/SPARK-44380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44380. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41948

[jira] [Assigned] (SPARK-44380) Support for UDTF to analyze in Python

[ https://issues.apache.org/jira/browse/SPARK-44380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44380: Assignee: Takuya Ueshin > Support for UDTF to analyze in Python >

[jira] [Created] (SPARK-44489) Make InsertIntoDataSourceDirCommand extends DataWritingCommand

Cheng Pan created SPARK-44489: - Summary: Make InsertIntoDataSourceDirCommand extends DataWritingCommand Key: SPARK-44489 URL: https://issues.apache.org/jira/browse/SPARK-44489 Project: Spark

[jira] [Assigned] (SPARK-44431) Wrong semantics for null IN (empty list)

[ https://issues.apache.org/jira/browse/SPARK-44431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44431: --- Assignee: Jack Chen > Wrong semantics for null IN (empty list) >

[jira] [Resolved] (SPARK-44431) Wrong semantics for null IN (empty list)

[ https://issues.apache.org/jira/browse/SPARK-44431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44431. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 42007

[jira] [Created] (SPARK-44488) Support deserializing long fields into `Metadata` object

Richard Chen created SPARK-44488: Summary: Support deserializing long fields into `Metadata` object Key: SPARK-44488 URL: https://issues.apache.org/jira/browse/SPARK-44488 Project: Spark

[jira] [Resolved] (SPARK-43838) Subquery on single table with having clause can't be optimized

[ https://issues.apache.org/jira/browse/SPARK-43838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43838. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41347

[jira] [Assigned] (SPARK-43838) Subquery on single table with having clause can't be optimized

[ https://issues.apache.org/jira/browse/SPARK-43838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43838: --- Assignee: Jia Fan > Subquery on single table with having clause can't be optimized >

[jira] [Created] (SPARK-44487) KubernetesSuite report NPE when not set spark.kubernetes.test.unpackSparkDir

Jia Fan created SPARK-44487: --- Summary: KubernetesSuite report NPE when not set spark.kubernetes.test.unpackSparkDir Key: SPARK-44487 URL: https://issues.apache.org/jira/browse/SPARK-44487 Project: Spark

[jira] [Updated] (SPARK-44486) Implement PyArrow `self_destruct` feature for `toPandas`

[ https://issues.apache.org/jira/browse/SPARK-44486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-44486: - Description: Implement PyArrow `self_destruct` feature for `toPandas` To make the Spark

[jira] [Updated] (SPARK-44486) Implement PyArrow `self_destruct` feature for `toPandas`

[ https://issues.apache.org/jira/browse/SPARK-44486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-44486: - Description: Implement PyArrow `self_destruct` feature for `toPandas`   Now the Spark

[jira] [Created] (SPARK-44486) Implement PyArrow `self_destruct` feature for `toPandas`

Xinrong Meng created SPARK-44486: Summary: Implement PyArrow `self_destruct` feature for `toPandas` Key: SPARK-44486 URL: https://issues.apache.org/jira/browse/SPARK-44486 Project: Spark

[jira] [Assigned] (SPARK-44481) Make pyspark.sql.is_remote an API

[ https://issues.apache.org/jira/browse/SPARK-44481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44481: Assignee: Hyukjin Kwon > Make pyspark.sql.is_remote an API >

[jira] [Resolved] (SPARK-44481) Make pyspark.sql.is_remote an API

[ https://issues.apache.org/jira/browse/SPARK-44481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44481. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42072

[jira] [Resolved] (SPARK-44278) Implement a GRPC server interceptor that cleans up thread local properties

[ https://issues.apache.org/jira/browse/SPARK-44278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44278. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44278) Implement a GRPC server interceptor that cleans up thread local properties

[ https://issues.apache.org/jira/browse/SPARK-44278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44278: Assignee: Yihong He > Implement a GRPC server interceptor that cleans up thread local

[jira] [Created] (SPARK-44485) optimize generateTreeString code path

Ziqi Liu created SPARK-44485: Summary: optimize generateTreeString code path Key: SPARK-44485 URL: https://issues.apache.org/jira/browse/SPARK-44485 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-44484) Add missing json field batchDuration to StreamingQueryProgress

Wei Liu created SPARK-44484: --- Summary: Add missing json field batchDuration to StreamingQueryProgress Key: SPARK-44484 URL: https://issues.apache.org/jira/browse/SPARK-44484 Project: Spark Issue

[jira] [Updated] (SPARK-44265) Built-in XML data source support

[ https://issues.apache.org/jira/browse/SPARK-44265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandip Agarwala updated SPARK-44265: Description: XML is a widely used data format. An external spark-xml package

[jira] [Resolved] (SPARK-44396) Add direct Arrow deserialization

[ https://issues.apache.org/jira/browse/SPARK-44396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44396. --- Fix Version/s: 3.5.0 Resolution: Fixed > Add direct Arrow deserialization >

[jira] [Created] (SPARK-44483) When using Spark to read the hive table, the number of file partitions cannot be set using Spark's configuration settings

hao created SPARK-44483: --- Summary: When using Spark to read the hive table, the number of file partitions cannot be set using Spark's configuration settings Key: SPARK-44483 URL:

[jira] [Updated] (SPARK-44482) Connect server should can specify the bind address

[ https://issues.apache.org/jira/browse/SPARK-44482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44482: Summary: Connect server should can specify the bind address (was: Connect server can specify the

[jira] [Created] (SPARK-44482) Connect server can specify the bind address

BingKun Pan created SPARK-44482: --- Summary: Connect server can specify the bind address Key: SPARK-44482 URL: https://issues.apache.org/jira/browse/SPARK-44482 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-44265) Built-in XML data source support

[ https://issues.apache.org/jira/browse/SPARK-44265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744535#comment-17744535 ] Sandeep Katta commented on SPARK-44265: --- [~sandip.agarwala] could you update the SPIP link here

[jira] [Created] (SPARK-44481) Make pyspark.sql.is_remote an API

Hyukjin Kwon created SPARK-44481: Summary: Make pyspark.sql.is_remote an API Key: SPARK-44481 URL: https://issues.apache.org/jira/browse/SPARK-44481 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-44470) Setting version to 4.0.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-44470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44470. -- Fix Version/s: (was: 3.5.0) Resolution: Duplicate > Setting version to 4.0.0-SNAPSHOT >

[jira] (SPARK-44209) Expose amount of shuffle data available on the node

[ https://issues.apache.org/jira/browse/SPARK-44209 ] Deependra Patel deleted comment on SPARK-44209: - was (Author: deependra): Created PR - [https://github.com/apache/spark/pull/42071] > Expose amount of shuffle data available on the node

[jira] [Commented] (SPARK-44209) Expose amount of shuffle data available on the node

[ https://issues.apache.org/jira/browse/SPARK-44209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744503#comment-17744503 ] ASF GitHub Bot commented on SPARK-44209: User 'Deependra-Patel' has created a pull request for

[jira] [Commented] (SPARK-44209) Expose amount of shuffle data available on the node

[ https://issues.apache.org/jira/browse/SPARK-44209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744502#comment-17744502 ] ASF GitHub Bot commented on SPARK-44209: User 'Deependra-Patel' has created a pull request for

[jira] [Commented] (SPARK-44209) Expose amount of shuffle data available on the node

[ https://issues.apache.org/jira/browse/SPARK-44209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744500#comment-17744500 ] Deependra Patel commented on SPARK-44209: - Created PR -

[jira] [Resolved] (SPARK-41231) Built-in SQL Function Improvement

[ https://issues.apache.org/jira/browse/SPARK-41231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41231. --- Resolution: Resolved > Built-in SQL Function Improvement >

[jira] [Assigned] (SPARK-41231) Built-in SQL Function Improvement

[ https://issues.apache.org/jira/browse/SPARK-41231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41231: - Assignee: Ruifeng Zheng > Built-in SQL Function Improvement >

[jira] [Assigned] (SPARK-43907) Add SQL functions into Scala, Python and R API

[ https://issues.apache.org/jira/browse/SPARK-43907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43907: - Assignee: Ruifeng Zheng > Add SQL functions into Scala, Python and R API >

[jira] [Updated] (SPARK-44272) Path Inconsistency when Operating statCache within Yarn Client

[ https://issues.apache.org/jira/browse/SPARK-44272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-44272: Affects Version/s: (was: 0.9.1) (was: 2.3.0) >

[jira] [Resolved] (SPARK-44272) Path Inconsistency when Operating statCache within Yarn Client

[ https://issues.apache.org/jira/browse/SPARK-44272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-44272. - Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-44272) Path Inconsistency when Operating statCache within Yarn Client

[ https://issues.apache.org/jira/browse/SPARK-44272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-44272: --- Assignee: SHU WANG > Path Inconsistency when Operating statCache within