[jira] [Commented] (SPARK-44132) nesting full outer joins confuses code generator

[ https://issues.apache.org/jira/browse/SPARK-44132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736659#comment-17736659 ] Snoot.io commented on SPARK-44132: -- User 'steven-aerts' has created a pull request for

[jira] [Created] (SPARK-44164) Extract toAttribute method from StructField to Util class

Rui Wang created SPARK-44164: Summary: Extract toAttribute method from StructField to Util class Key: SPARK-44164 URL: https://issues.apache.org/jira/browse/SPARK-44164 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43974) Upgrade buf to v1.22.0

[ https://issues.apache.org/jira/browse/SPARK-43974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43974: Summary: Upgrade buf to v1.22.0 (was: Upgrade buf to v1.21.0) > Upgrade buf to v1.22.0 >

[jira] [Resolved] (SPARK-44163) Handle ModuleNotFoundError like ImportError

[ https://issues.apache.org/jira/browse/SPARK-44163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44163. --- Resolution: Invalid > Handle ModuleNotFoundError like ImportError >

[jira] [Closed] (SPARK-44163) Handle ModuleNotFoundError like ImportError

[ https://issues.apache.org/jira/browse/SPARK-44163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44163. - > Handle ModuleNotFoundError like ImportError > --- > >

[jira] [Created] (SPARK-44163) Handle ModuleNotFoundError like ImportError

Dongjoon Hyun created SPARK-44163: - Summary: Handle ModuleNotFoundError like ImportError Key: SPARK-44163 URL: https://issues.apache.org/jira/browse/SPARK-44163 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-44162) Support G1GC in `spark.eventLog.gcMetrics.*` without warning

Dongjoon Hyun created SPARK-44162: - Summary: Support G1GC in `spark.eventLog.gcMetrics.*` without warning Key: SPARK-44162 URL: https://issues.apache.org/jira/browse/SPARK-44162 Project: Spark

[jira] [Created] (SPARK-44161) Row as UDF inputs causes encoder errors

Zhen Li created SPARK-44161: --- Summary: Row as UDF inputs causes encoder errors Key: SPARK-44161 URL: https://issues.apache.org/jira/browse/SPARK-44161 Project: Spark Issue Type: Bug Compo

[jira] [Updated] (SPARK-44160) Extract shared code from StructType

[ https://issues.apache.org/jira/browse/SPARK-44160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-44160: - Description: The StructType has some methods that require CatalystParser and Catalyst expression. We are

[jira] [Created] (SPARK-44160) Extract shared code from StructType

Rui Wang created SPARK-44160: Summary: Extract shared code from StructType Key: SPARK-44160 URL: https://issues.apache.org/jira/browse/SPARK-44160 Project: Spark Issue Type: Sub-task Co

[jira] [Assigned] (SPARK-44158) Remove unused `spark.kubernetes.executor.lostCheck.maxAttempts`

[ https://issues.apache.org/jira/browse/SPARK-44158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44158: - Assignee: Dongjoon Hyun > Remove unused `spark.kubernetes.executor.lostCheck.maxAttempt

[jira] [Resolved] (SPARK-44158) Remove unused `spark.kubernetes.executor.lostCheck.maxAttempts`

[ https://issues.apache.org/jira/browse/SPARK-44158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44158. --- Fix Version/s: 3.3.3 3.5.0 3.4.2 Resolution: Fix

[jira] [Created] (SPARK-44159) Commands for writting (InsertIntoHadoopFsRelationCommand and InsertIntoHiveTable) should log what they are doing

Navin Kumar created SPARK-44159: --- Summary: Commands for writting (InsertIntoHadoopFsRelationCommand and InsertIntoHiveTable) should log what they are doing Key: SPARK-44159 URL: https://issues.apache.org/jira/browse

[jira] [Updated] (SPARK-44158) Remove unused `spark.kubernetes.executor.lostCheck.maxAttempts`

[ https://issues.apache.org/jira/browse/SPARK-44158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44158: -- Affects Version/s: 3.2.4 3.1.3 3.0.3

[jira] [Updated] (SPARK-44158) Remove unused `spark.kubernetes.executor.lostCheck.maxAttempts`

[ https://issues.apache.org/jira/browse/SPARK-44158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44158: -- Affects Version/s: 3.4.1 > Remove unused `spark.kubernetes.executor.lostCheck.maxAttempts` > -

[jira] [Created] (SPARK-44158) Remove unused `spark.kubernetes.executor.lostCheck.maxAttempts`

Dongjoon Hyun created SPARK-44158: - Summary: Remove unused `spark.kubernetes.executor.lostCheck.maxAttempts` Key: SPARK-44158 URL: https://issues.apache.org/jira/browse/SPARK-44158 Project: Spark

[jira] [Assigned] (SPARK-44151) Upgrade commons-codec from 1.15 to 1.16.0

[ https://issues.apache.org/jira/browse/SPARK-44151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44151: - Assignee: BingKun Pan > Upgrade commons-codec from 1.15 to 1.16.0 > ---

[jira] [Resolved] (SPARK-44151) Upgrade commons-codec from 1.15 to 1.16.0

[ https://issues.apache.org/jira/browse/SPARK-44151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44151. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41707 [https://

[jira] [Updated] (SPARK-44156) SortAggregation slows down dropDuplicates()

[ https://issues.apache.org/jira/browse/SPARK-44156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emanuel Velzi updated SPARK-44156: -- Description: TL;DR: SortAggregate makes dropDuplicates slower than HashAggregate. How to make

[jira] [Updated] (SPARK-44156) SortAggregation slows down dropDuplicates()

[ https://issues.apache.org/jira/browse/SPARK-44156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emanuel Velzi updated SPARK-44156: -- Description: TL;DR: SortAggregate makes dropDuplicates slower than HashAggregate. How to make

[jira] [Updated] (SPARK-44156) SortAggregation slows down dropDuplicates()

[ https://issues.apache.org/jira/browse/SPARK-44156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emanuel Velzi updated SPARK-44156: -- Summary: SortAggregation slows down dropDuplicates() (was: SortAggregation slows down dropDup

[jira] [Updated] (SPARK-44156) SortAggregation slows down dropDuplicates().

[ https://issues.apache.org/jira/browse/SPARK-44156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emanuel Velzi updated SPARK-44156: -- Summary: SortAggregation slows down dropDuplicates(). (was: Should HashAggregation improve dr

[jira] [Created] (SPARK-44157) Outdated JARs in PySpark package

Adrian Gonzalez-Martin created SPARK-44157: -- Summary: Outdated JARs in PySpark package Key: SPARK-44157 URL: https://issues.apache.org/jira/browse/SPARK-44157 Project: Spark Issue Ty

[jira] [Updated] (SPARK-44156) Should HashAggregation improve dropDuplicates()?

[ https://issues.apache.org/jira/browse/SPARK-44156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emanuel Velzi updated SPARK-44156: -- Description: TL;DR: SortAggregate makes dropDuplicates slower than HashAggregate. How to make

[jira] [Created] (SPARK-44156) Should HashAggregation improve dropDuplicates()?

Emanuel Velzi created SPARK-44156: - Summary: Should HashAggregation improve dropDuplicates()? Key: SPARK-44156 URL: https://issues.apache.org/jira/browse/SPARK-44156 Project: Spark Issue Type

[jira] [Updated] (SPARK-44134) Can't set resources (GPU/FPGA) to 0 when they are set to positive value in spark-defaults.conf

[ https://issues.apache.org/jira/browse/SPARK-44134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-44134: -- Fix Version/s: 3.4.2 (was: 3.4.1) > Can't set resources (GPU/FPGA) to 0

[jira] [Commented] (SPARK-44132) nesting full outer joins confuses code generator

[ https://issues.apache.org/jira/browse/SPARK-44132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736490#comment-17736490 ] Steven Aerts commented on SPARK-44132: -- [https://github.com/apache/spark/pull/41712

[jira] [Comment Edited] (SPARK-43438) Fix mismatched column list error on INSERT

[ https://issues.apache.org/jira/browse/SPARK-43438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736485#comment-17736485 ] BingKun Pan edited comment on SPARK-43438 at 6/23/23 12:48 PM: ---

[jira] [Comment Edited] (SPARK-43438) Fix mismatched column list error on INSERT

[ https://issues.apache.org/jira/browse/SPARK-43438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736485#comment-17736485 ] BingKun Pan edited comment on SPARK-43438 at 6/23/23 12:44 PM: ---

[jira] [Comment Edited] (SPARK-43438) Fix mismatched column list error on INSERT

[ https://issues.apache.org/jira/browse/SPARK-43438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736485#comment-17736485 ] BingKun Pan edited comment on SPARK-43438 at 6/23/23 12:42 PM: ---

[jira] [Comment Edited] (SPARK-43438) Fix mismatched column list error on INSERT

[ https://issues.apache.org/jira/browse/SPARK-43438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736485#comment-17736485 ] BingKun Pan edited comment on SPARK-43438 at 6/23/23 12:40 PM: ---

[jira] [Comment Edited] (SPARK-43438) Fix mismatched column list error on INSERT

[ https://issues.apache.org/jira/browse/SPARK-43438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736485#comment-17736485 ] BingKun Pan edited comment on SPARK-43438 at 6/23/23 12:39 PM: ---

[jira] [Commented] (SPARK-43438) Fix mismatched column list error on INSERT

[ https://issues.apache.org/jira/browse/SPARK-43438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736485#comment-17736485 ] BingKun Pan commented on SPARK-43438: - I checked and found that after `[https://gith

[jira] [Commented] (SPARK-44155) Adding a dev utility to improve error messages based on LLM

[ https://issues.apache.org/jira/browse/SPARK-44155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736424#comment-17736424 ] ASF GitHub Bot commented on SPARK-44155: User 'itholic' has created a pull reque

[jira] [Commented] (SPARK-44151) Upgrade commons-codec from 1.15 to 1.16.0

[ https://issues.apache.org/jira/browse/SPARK-44151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736421#comment-17736421 ] ASF GitHub Bot commented on SPARK-44151: User 'panbingkun' has created a pull re

[jira] [Commented] (SPARK-44153) Support `Heap Histogram` column in Executor tab

[ https://issues.apache.org/jira/browse/SPARK-44153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736420#comment-17736420 ] ASF GitHub Bot commented on SPARK-44153: User 'dongjoon-hyun' has created a pull

[jira] [Created] (SPARK-44155) Adding a dev utility to improve error messages based on LLM

Haejoon Lee created SPARK-44155: --- Summary: Adding a dev utility to improve error messages based on LLM Key: SPARK-44155 URL: https://issues.apache.org/jira/browse/SPARK-44155 Project: Spark I

[jira] [Commented] (SPARK-44152) Upgrade to spark 3.4.0 from 3.3.2 gives Exception in thread "main" java.nio.file.NoSuchFileException: , although jar is present in the location

[ https://issues.apache.org/jira/browse/SPARK-44152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736404#comment-17736404 ] Ramakrishna commented on SPARK-44152: - [~gurwls223]  Is this an issue spark 3.4.0 ?

[jira] [Commented] (SPARK-42877) Implement DataFrame.foreach

[ https://issues.apache.org/jira/browse/SPARK-42877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736399#comment-17736399 ] Gurpreet Singh commented on SPARK-42877: [~XinrongM] I am interested in working

[jira] [Resolved] (SPARK-44153) Support `Heap Histogram` column in Executor tab

[ https://issues.apache.org/jira/browse/SPARK-44153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44153. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41709 [https://

[jira] [Assigned] (SPARK-44153) Support `Heap Histogram` column in Executor tab

[ https://issues.apache.org/jira/browse/SPARK-44153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44153: - Assignee: Dongjoon Hyun > Support `Heap Histogram` column in Executor tab > ---