[jira] [Commented] (SPARK-37829) An outer-join using joinWith on DataFrames returns Rows with null fields instead of null values

[ https://issues.apache.org/jira/browse/SPARK-37829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707502#comment-17707502 ] Clément de Groc commented on SPARK-37829: - I'm not planning to resume. I don't know that part of

[jira] [Commented] (SPARK-41628) Support async query execution

[ https://issues.apache.org/jira/browse/SPARK-41628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707501#comment-17707501 ] Jia Fan commented on SPARK-41628: - I'm working on it > Support async query execution >

[jira] [Commented] (SPARK-42993) Make Torch Distributor compatible with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-42993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707493#comment-17707493 ] Snoot.io commented on SPARK-42993: -- User 'zhengruifeng' has created a pull request for this issue:

[jira] [Updated] (SPARK-42993) Make Torch Distributor compatible with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-42993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-42993: -- Summary: Make Torch Distributor compatible with Spark Connect (was: Make Torch Distributor

[jira] [Assigned] (SPARK-42998) Fix DataFrame.collect with null struct.

[ https://issues.apache.org/jira/browse/SPARK-42998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-42998: - Assignee: Takuya Ueshin > Fix DataFrame.collect with null struct. >

[jira] [Resolved] (SPARK-42998) Fix DataFrame.collect with null struct.

[ https://issues.apache.org/jira/browse/SPARK-42998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-42998. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40627

[jira] [Comment Edited] (SPARK-42860) Add analysed logical mode in org.apache.spark.sql.execution.ExplainMode

[ https://issues.apache.org/jira/browse/SPARK-42860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707402#comment-17707402 ] xiaochen zhou edited comment on SPARK-42860 at 4/1/23 1:20 AM: ---

[jira] [Comment Edited] (SPARK-42860) Add analysed logical mode in org.apache.spark.sql.execution.ExplainMode

[ https://issues.apache.org/jira/browse/SPARK-42860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707402#comment-17707402 ] xiaochen zhou edited comment on SPARK-42860 at 4/1/23 1:19 AM: --- [hi，I

[jira] [Comment Edited] (SPARK-42860) Add analysed logical mode in org.apache.spark.sql.execution.ExplainMode

[ https://issues.apache.org/jira/browse/SPARK-42860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707402#comment-17707402 ] xiaochen zhou edited comment on SPARK-42860 at 4/1/23 1:16 AM: ---

[jira] [Created] (SPARK-42999) Impl Dataset#foreach, foreachPartitions

Zhen Li created SPARK-42999: --- Summary: Impl Dataset#foreach, foreachPartitions Key: SPARK-42999 URL: https://issues.apache.org/jira/browse/SPARK-42999 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-42998) Fix DataFrame.collect with null struct.

Takuya Ueshin created SPARK-42998: - Summary: Fix DataFrame.collect with null struct. Key: SPARK-42998 URL: https://issues.apache.org/jira/browse/SPARK-42998 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42860) Add analysed logical mode in org.apache.spark.sql.execution.ExplainMode

[ https://issues.apache.org/jira/browse/SPARK-42860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707402#comment-17707402 ] xiaochen zhou commented on SPARK-42860: --- https://github.com/apache/spark/pull/40626 > Add

[jira] [Created] (SPARK-42997) TableOutputResolver must use correct column paths in error messages for arrays and maps

Anton Okolnychyi created SPARK-42997: Summary: TableOutputResolver must use correct column paths in error messages for arrays and maps Key: SPARK-42997 URL: https://issues.apache.org/jira/browse/SPARK-42997

[jira] [Resolved] (SPARK-42973) Upgrade buf to v1.16.0

[ https://issues.apache.org/jira/browse/SPARK-42973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42973. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40596

[jira] [Updated] (SPARK-42973) Upgrade buf to v1.16.0

[ https://issues.apache.org/jira/browse/SPARK-42973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42973: -- Affects Version/s: 3.5.0 (was: 3.4.1) > Upgrade buf to v1.16.0 >

[jira] [Assigned] (SPARK-42973) Upgrade buf to v1.16.0

[ https://issues.apache.org/jira/browse/SPARK-42973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42973: - Assignee: BingKun Pan > Upgrade buf to v1.16.0 > -- > >

[jira] [Commented] (SPARK-42663) Fix `default_session` to work properly

[ https://issues.apache.org/jira/browse/SPARK-42663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707279#comment-17707279 ] Wencong Liu commented on SPARK-42663: - Hello [~itholic] , I am a beginner in Spark and I am very

[jira] [Commented] (SPARK-42977) spark sql Disable vectorized faild

[ https://issues.apache.org/jira/browse/SPARK-42977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707256#comment-17707256 ] Jacek Laskowski commented on SPARK-42977: - Unless you can reproduce it without Iceberg, it's

[jira] [Assigned] (SPARK-42918) Generalize handling of metadata attributes in FileSourceStrategy

[ https://issues.apache.org/jira/browse/SPARK-42918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42918: --- Assignee: Johan Lasperas > Generalize handling of metadata attributes in

[jira] [Resolved] (SPARK-42918) Generalize handling of metadata attributes in FileSourceStrategy

[ https://issues.apache.org/jira/browse/SPARK-42918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42918. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40545

[jira] [Created] (SPARK-42996) Adding reason for test failure on Spark Connect parity tests.

Haejoon Lee created SPARK-42996: --- Summary: Adding reason for test failure on Spark Connect parity tests. Key: SPARK-42996 URL: https://issues.apache.org/jira/browse/SPARK-42996 Project: Spark

[jira] [Updated] (SPARK-42935) Optimze shuffle for union spark plan

[ https://issues.apache.org/jira/browse/SPARK-42935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Min updated SPARK-42935: - Description: Union plan does not take full advantage of children plan output partitionings when output

[jira] [Created] (SPARK-42995) Migrate Spark Connect DataFrame errors into error class

Haejoon Lee created SPARK-42995: --- Summary: Migrate Spark Connect DataFrame errors into error class Key: SPARK-42995 URL: https://issues.apache.org/jira/browse/SPARK-42995 Project: Spark Issue

[jira] [Resolved] (SPARK-42978) Derby: RENAME cannot qualify a new-table-Name with a schema-Name.

[ https://issues.apache.org/jira/browse/SPARK-42978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-42978. -- Fix Version/s: 3.5.0 Assignee: Kent Yao Resolution: Fixed issue resolved by

[jira] [Created] (SPARK-42994) Support sc.resources in Connect

Ruifeng Zheng created SPARK-42994: - Summary: Support sc.resources in Connect Key: SPARK-42994 URL: https://issues.apache.org/jira/browse/SPARK-42994 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42993) Make Torch Distributor support Spark Connect

Ruifeng Zheng created SPARK-42993: - Summary: Make Torch Distributor support Spark Connect Key: SPARK-42993 URL: https://issues.apache.org/jira/browse/SPARK-42993 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42972) ExecutorAllocationManager cannot allocate new instances when all executors down.

[ https://issues.apache.org/jira/browse/SPARK-42972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707177#comment-17707177 ] liang yu commented on SPARK-42972: -- I created a PR

[jira] [Comment Edited] (SPARK-42972) ExecutorAllocationManager cannot allocate new instances when all executors down.

[ https://issues.apache.org/jira/browse/SPARK-42972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707143#comment-17707143 ] liang yu edited comment on SPARK-42972 at 3/31/23 7:16 AM: --- Using structured

[jira] [Comment Edited] (SPARK-42972) ExecutorAllocationManager cannot allocate new instances when all executors down.

[ https://issues.apache.org/jira/browse/SPARK-42972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707143#comment-17707143 ] liang yu edited comment on SPARK-42972 at 3/31/23 7:13 AM: --- Using structured

[jira] [Updated] (SPARK-42991) Disable string type +/- interval in ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-42991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42991: Summary: Disable string type +/- interval in ANSI mode (was: Disable string +/- interval in ANSI

[jira] [Commented] (SPARK-42991) Disable string +/- interval in ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-42991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707158#comment-17707158 ] Yuming Wang commented on SPARK-42991: - https://github.com/apache/spark/pull/40616 > Disable string

[jira] [Created] (SPARK-42992) Introduce PySparkRuntimeError

Haejoon Lee created SPARK-42992: --- Summary: Introduce PySparkRuntimeError Key: SPARK-42992 URL: https://issues.apache.org/jira/browse/SPARK-42992 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42991) Disable string +/- interval in ANSI mode

Yuming Wang created SPARK-42991: --- Summary: Disable string +/- interval in ANSI mode Key: SPARK-42991 URL: https://issues.apache.org/jira/browse/SPARK-42991 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42972) ExecutorAllocationManager cannot allocate new instances when all executors down.

[ https://issues.apache.org/jira/browse/SPARK-42972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707143#comment-17707143 ] liang yu commented on SPARK-42972: -- When the ratio is lower than the scalingDownRatio, the client will