[jira] [Created] (SPARK-40783) Enable Spark on K8s integration test for official dockerfiles

2022-10-12 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-40783: --- Summary: Enable Spark on K8s integration test for official dockerfiles Key: SPARK-40783 URL: https://issues.apache.org/jira/browse/SPARK-40783 Project: Spark

[jira] [Assigned] (SPARK-40601) Improve error when cogrouping groups with mismatching key sizes

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40601: Assignee: Enrico Minack > Improve error when cogrouping groups with mismatching key sizes

[jira] [Resolved] (SPARK-40601) Improve error when cogrouping groups with mismatching key sizes

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40601. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38036 [https://gi

[jira] [Assigned] (SPARK-40782) Upgrade Jackson-databind to 2.13.4.1

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40782: Assignee: (was: Apache Spark) > Upgrade Jackson-databind to 2.13.4.1 > --

[jira] [Commented] (SPARK-40782) Upgrade Jackson-databind to 2.13.4.1

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616787#comment-17616787 ] Apache Spark commented on SPARK-40782: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-40782) Upgrade Jackson-databind to 2.13.4.1

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40782: Assignee: Apache Spark > Upgrade Jackson-databind to 2.13.4.1 > -

[jira] [Updated] (SPARK-40782) Upgrade Jackson-databind to 2.13.4.1

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40782: - Summary: Upgrade Jackson-databind to 2.13.4.1 (was: Upgrade Jackson to 2.13.4.1) > Upgrade Jackson-dat

[jira] [Created] (SPARK-40782) Upgrade Jackson to 2.13.4.1

2022-10-12 Thread Yang Jie (Jira)
Yang Jie created SPARK-40782: Summary: Upgrade Jackson to 2.13.4.1 Key: SPARK-40782 URL: https://issues.apache.org/jira/browse/SPARK-40782 Project: Spark Issue Type: Improvement Compone

[jira] [Commented] (SPARK-40761) Migrate type check failures of percentile expressions onto error classes

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616776#comment-17616776 ] Apache Spark commented on SPARK-40761: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-40781) Explain exit code 137 as killed due to OOM

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40781: Assignee: (was: Apache Spark) > Explain exit code 137 as killed due to OOM >

[jira] [Commented] (SPARK-40781) Explain exit code 137 as killed due to OOM

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616772#comment-17616772 ] Apache Spark commented on SPARK-40781: -- User 'warrenzhu25' has created a pull reque

[jira] [Assigned] (SPARK-40781) Explain exit code 137 as killed due to OOM

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40781: Assignee: Apache Spark > Explain exit code 137 as killed due to OOM > ---

[jira] [Commented] (SPARK-40780) Add WHERE to Connect proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616771#comment-17616771 ] Apache Spark commented on SPARK-40780: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-40780) Add WHERE to Connect proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40780: Assignee: (was: Apache Spark) > Add WHERE to Connect proto and DSL >

[jira] [Commented] (SPARK-40780) Add WHERE to Connect proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616770#comment-17616770 ] Apache Spark commented on SPARK-40780: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-40780) Add WHERE to Connect proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40780: Assignee: Apache Spark > Add WHERE to Connect proto and DSL > ---

[jira] [Created] (SPARK-40781) Explain exit code 137 as killed due to OOM

2022-10-12 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-40781: Summary: Explain exit code 137 as killed due to OOM Key: SPARK-40781 URL: https://issues.apache.org/jira/browse/SPARK-40781 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-40780) Add WHERE to Connect proto and DSL

2022-10-12 Thread Rui Wang (Jira)
Rui Wang created SPARK-40780: Summary: Add WHERE to Connect proto and DSL Key: SPARK-40780 URL: https://issues.apache.org/jira/browse/SPARK-40780 Project: Spark Issue Type: Sub-task Com

[jira] [Comment Edited] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-12 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616768#comment-17616768 ] Haejoon Lee edited comment on SPARK-40779 at 10/13/22 4:07 AM: ---

[jira] [Commented] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-12 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616768#comment-17616768 ] Haejoon Lee commented on SPARK-40779: - I'm taking a look. Seems like it's different

[jira] [Commented] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616767#comment-17616767 ] Apache Spark commented on SPARK-40778: -- User 'warrenzhu25' has created a pull reque

[jira] [Assigned] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40778: Assignee: (was: Apache Spark) > Make HeartbeatReceiver as an IsolatedRpcEndpoint > --

[jira] [Commented] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616766#comment-17616766 ] Apache Spark commented on SPARK-40778: -- User 'warrenzhu25' has created a pull reque

[jira] [Assigned] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40778: Assignee: Apache Spark > Make HeartbeatReceiver as an IsolatedRpcEndpoint > -

[jira] [Created] (SPARK-40779) Fix `corrwith` to work properly with different anchor.

2022-10-12 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-40779: --- Summary: Fix `corrwith` to work properly with different anchor. Key: SPARK-40779 URL: https://issues.apache.org/jira/browse/SPARK-40779 Project: Spark Issue Ty

[jira] [Created] (SPARK-40778) Make HeartbeatReceiver as an IsolatedRpcEndpoint

2022-10-12 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-40778: Summary: Make HeartbeatReceiver as an IsolatedRpcEndpoint Key: SPARK-40778 URL: https://issues.apache.org/jira/browse/SPARK-40778 Project: Spark Issue Type:

[jira] [Commented] (SPARK-40616) Loss of precision using SparkSQL shell on high-precision DECIMAL types

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616754#comment-17616754 ] Apache Spark commented on SPARK-40616: -- User 'xsystem2022' has created a pull reque

[jira] [Resolved] (SPARK-40765) Optimize redundant fs operations in `CommandUtils#calculateSingleLocationSize#getPathSize` method

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40765. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38214 [https://gi

[jira] [Assigned] (SPARK-40765) Optimize redundant fs operations in `CommandUtils#calculateSingleLocationSize#getPathSize` method

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40765: Assignee: Yang Jie > Optimize redundant fs operations in > `CommandUtils#calculateSingle

[jira] [Commented] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616750#comment-17616750 ] Apache Spark commented on SPARK-40739: -- User 'philwalk' has created a pull request

[jira] [Commented] (SPARK-40738) spark-shell fails with "bad array subscript" in cygwin or msys bash session

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616751#comment-17616751 ] Apache Spark commented on SPARK-40738: -- User 'philwalk' has created a pull request

[jira] [Assigned] (SPARK-40766) Upgrade the guava defined in `plugins.sbt` to `31.0.1-jre`

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40766: Assignee: Yang Jie > Upgrade the guava defined in `plugins.sbt` to `31.0.1-jre` > ---

[jira] [Resolved] (SPARK-40766) Upgrade the guava defined in `plugins.sbt` to `31.0.1-jre`

2022-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40766. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38217 [https://gi

[jira] [Assigned] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40775: Assignee: (was: Apache Spark) > V2 file scans have duplicative descriptions > ---

[jira] [Assigned] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40775: Assignee: Apache Spark > V2 file scans have duplicative descriptions > --

[jira] [Commented] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616738#comment-17616738 ] Apache Spark commented on SPARK-40775: -- User 'Kimahriman' has created a pull reques

[jira] [Created] (SPARK-40777) Use error classes for Protobuf exceptions

2022-10-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-40777: Summary: Use error classes for Protobuf exceptions Key: SPARK-40777 URL: https://issues.apache.org/jira/browse/SPARK-40777 Project: Spark Issue Type: Improve

[jira] [Created] (SPARK-40776) Add documentation (similar Avro functions).

2022-10-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-40776: Summary: Add documentation (similar Avro functions). Key: SPARK-40776 URL: https://issues.apache.org/jira/browse/SPARK-40776 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-40657) Add support for compiled classes (Java classes)

2022-10-12 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616711#comment-17616711 ] Raghu Angadi commented on SPARK-40657: -- I will add this.  cc: [~sanysand...@gmail.

[jira] [Assigned] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40772: - Assignee: Dongjoon Hyun > Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to

[jira] [Resolved] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40772. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38225 [https://

[jira] [Updated] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40739: External issue URL: https://github.com/apache/spark/pull/38228 (was: https://github.com/apache/sp

[jira] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739 ] Phil Walker deleted comment on SPARK-40739: - was (Author: JIRAUSER296755): The actual spark PR is now 38228, not sure why I wasn't able to modify the links above. > "sbt packageBin" fails i

[jira] [Updated] (SPARK-40738) spark-shell fails with "bad array subscript" in cygwin or msys bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40738: External issue URL: https://github.com/apache/spark/pull/38228 (was: https://github.com/apache/sp

[jira] [Updated] (SPARK-40738) spark-shell fails with "bad array subscript" in cygwin or msys bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40738: External issue ID: 38228 (was: 38167) > spark-shell fails with "bad array subscript" in cygwin or

[jira] [Commented] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616693#comment-17616693 ] Phil Walker commented on SPARK-40739: - The actual spark PR is now 38228, not sure wh

[jira] [Updated] (SPARK-40739) "sbt packageBin" fails in cygwin or other windows bash session

2022-10-12 Thread Phil Walker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Walker updated SPARK-40739: External issue ID: 38228 (was: 38167) Description:  In a Windows _*SHELL*_ environment,

[jira] [Created] (SPARK-40775) V2 file scans have duplicative descriptions

2022-10-12 Thread Adam Binford (Jira)
Adam Binford created SPARK-40775: Summary: V2 file scans have duplicative descriptions Key: SPARK-40775 URL: https://issues.apache.org/jira/browse/SPARK-40775 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616683#comment-17616683 ] Apache Spark commented on SPARK-40774: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40774: Assignee: Apache Spark > Add Sample to proto and DSL > --- > >

[jira] [Commented] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616682#comment-17616682 ] Apache Spark commented on SPARK-40774: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40774: Assignee: (was: Apache Spark) > Add Sample to proto and DSL > ---

[jira] [Created] (SPARK-40774) Add Sample to proto and DSL

2022-10-12 Thread Rui Wang (Jira)
Rui Wang created SPARK-40774: Summary: Add Sample to proto and DSL Key: SPARK-40774 URL: https://issues.apache.org/jira/browse/SPARK-40774 Project: Spark Issue Type: Sub-task Components

[jira] [Updated] (SPARK-40760) Migrate type check failures of interval expressions onto error classes

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-40760: - Description: Replace TypeCheckFailure by DataTypeMismatch in type checks in the interval expressions: 1

[jira] [Commented] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616661#comment-17616661 ] Apache Spark commented on SPARK-40773: -- User 'allisonwang-db' has created a pull re

[jira] [Assigned] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40773: Assignee: (was: Apache Spark) > Refactor checkCorrelationsInSubquery > --

[jira] [Assigned] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40773: Assignee: Apache Spark > Refactor checkCorrelationsInSubquery > -

[jira] [Created] (SPARK-40773) Refactor checkCorrelationsInSubquery

2022-10-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-40773: Summary: Refactor checkCorrelationsInSubquery Key: SPARK-40773 URL: https://issues.apache.org/jira/browse/SPARK-40773 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-40706) IllegalStateException when querying array values inside a nested struct

2022-10-12 Thread Rohan Barman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616654#comment-17616654 ] Rohan Barman commented on SPARK-40706: -- thanks [~bersprockets]. Those 2 spark setti

[jira] [Commented] (SPARK-40511) Upgrade slf4j to 2.x

2022-10-12 Thread Garret Wilson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616642#comment-17616642 ] Garret Wilson commented on SPARK-40511: --- Thank you for your work on this! It appea

[jira] [Resolved] (SPARK-40489) Spark 3.3.0 breaks with SFL4J 2.

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40489. -- Resolution: Duplicate Spark is on slf4j 2.x now, so I think SPARK-40511 supersedes this > Spa

[jira] [Resolved] (SPARK-40422) Upgrade hive to 4.0.0

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40422. -- Resolution: Won't Fix > Upgrade hive to 4.0.0 > - > > Key:

[jira] [Commented] (SPARK-40422) Upgrade hive to 4.0.0

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616623#comment-17616623 ] Sean R. Owen commented on SPARK-40422: -- This would be a massively breaking change -

[jira] [Resolved] (SPARK-40499) Spark 3.2.1 percentlie_approx query much slower than Spark 2.4.0

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40499. -- Resolution: Invalid It's hard to action this; this sounds like a question for the mailing list

[jira] [Resolved] (SPARK-40526) Upgrade Scala to 2.13.9

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40526. -- Resolution: Won't Fix > Upgrade Scala to 2.13.9 > --- > >

[jira] [Resolved] (SPARK-40650) Infer date type for Json schema inference

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40650. -- Fix Version/s: (was: 3.4.0) Resolution: Duplicate > Infer date type for Json schema

[jira] [Updated] (SPARK-40685) Update kryo transitive dependency to 5.2.0 or later

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40685: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) Does kryo 5 work with Sp

[jira] [Updated] (SPARK-40681) Update gson transitive dependency to 2.8.9 or later

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40681: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) Go ahead and try updatin

[jira] [Commented] (SPARK-40758) Upgrade Apache zookeeper to get rid of CVE-2020-10663

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616612#comment-17616612 ] Sean R. Owen commented on SPARK-40758: -- Does the CVE affect Spark? Can we update ZK

[jira] [Resolved] (SPARK-40721) Use `||` instead of `|` in TimSort

2022-10-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40721. -- Resolution: Not A Problem > Use `||` instead of `|` in TimSort > -

[jira] [Commented] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616552#comment-17616552 ] Apache Spark commented on SPARK-40772: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616553#comment-17616553 ] Apache Spark commented on SPARK-40772: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40772: Assignee: Apache Spark > Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to sup

[jira] [Assigned] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40772: Assignee: (was: Apache Spark) > Improve spark.sql.adaptive.skewJoin.skewedPartitionFa

[jira] [Created] (SPARK-40772) Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values

2022-10-12 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-40772: - Summary: Improve spark.sql.adaptive.skewJoin.skewedPartitionFactor to support float values Key: SPARK-40772 URL: https://issues.apache.org/jira/browse/SPARK-40772 P

[jira] [Commented] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616496#comment-17616496 ] Apache Spark commented on SPARK-40771: -- User 'eejbyfeldt' has created a pull reques

[jira] [Commented] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616495#comment-17616495 ] Apache Spark commented on SPARK-40771: -- User 'eejbyfeldt' has created a pull reques

[jira] [Assigned] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40771: Assignee: (was: Apache Spark) > Estimated size in log message can overflow Int >

[jira] [Assigned] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40771: Assignee: Apache Spark > Estimated size in log message can overflow Int > ---

[jira] [Created] (SPARK-40771) Estimated size in log message can overflow Int

2022-10-12 Thread Emil Ejbyfeldt (Jira)
Emil Ejbyfeldt created SPARK-40771: -- Summary: Estimated size in log message can overflow Int Key: SPARK-40771 URL: https://issues.apache.org/jira/browse/SPARK-40771 Project: Spark Issue Type

[jira] [Commented] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616483#comment-17616483 ] Apache Spark commented on SPARK-40770: -- User 'EnricoMi' has created a pull request

[jira] [Assigned] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40770: Assignee: Apache Spark > Improved error messages for applyInPandas for schema mismatch >

[jira] [Assigned] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40770: Assignee: (was: Apache Spark) > Improved error messages for applyInPandas for schema

[jira] [Commented] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616478#comment-17616478 ] Apache Spark commented on SPARK-40770: -- User 'EnricoMi' has created a pull request

[jira] [Created] (SPARK-40770) Improved error messages for applyInPandas for schema mismatch

2022-10-12 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-40770: - Summary: Improved error messages for applyInPandas for schema mismatch Key: SPARK-40770 URL: https://issues.apache.org/jira/browse/SPARK-40770 Project: Spark

[jira] [Commented] (SPARK-35084) [k8s] On Spark 3, jars listed in spark.jars and spark.jars.packages are not added to sparkContext

2022-10-12 Thread Gabor Roczei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616444#comment-17616444 ] Gabor Roczei commented on SPARK-35084: -- Hi [~ocworld], [Your pull request|https://

[jira] [Resolved] (SPARK-40762) Check error classes in ErrorParserSuite

2022-10-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40762. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38204 [https://github.com

[jira] [Commented] (SPARK-40736) Spark 3.3.0 doesn't works with Hive 3.1.2

2022-10-12 Thread Pratik Malani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616399#comment-17616399 ] Pratik Malani commented on SPARK-40736: --- Hi All, Removed hive-service jar from th

[jira] [Updated] (SPARK-40736) Spark 3.3.0 doesn't works with Hive 3.1.2

2022-10-12 Thread Pratik Malani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratik Malani updated SPARK-40736: -- Attachment: image-2022-10-12-18-19-24-455.png > Spark 3.3.0 doesn't works with Hive 3.1.2 > --

[jira] [Commented] (SPARK-40497) Upgrade Scala to 2.13.11

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616398#comment-17616398 ] Yang Jie commented on SPARK-40497: -- Change te Jira title due to [https://github.com/sca

[jira] [Updated] (SPARK-40497) Upgrade Scala to 2.13.11

2022-10-12 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40497: - Summary: Upgrade Scala to 2.13.11 (was: Upgrade Scala to 2.13.9) > Upgrade Scala to 2.13.11 > -

[jira] [Assigned] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40764: Assignee: (was: Apache Spark) > Extract partitioning through all children output expr

[jira] [Assigned] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40764: Assignee: Apache Spark > Extract partitioning through all children output expressions > -

[jira] [Commented] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616347#comment-17616347 ] Apache Spark commented on SPARK-40764: -- User 'wangyum' has created a pull request f

[jira] [Commented] (SPARK-40724) Simplify `corr` with method `inline`

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616333#comment-17616333 ] Apache Spark commented on SPARK-40724: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-40724) Simplify `corr` with method `inline`

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616332#comment-17616332 ] Apache Spark commented on SPARK-40724: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616303#comment-17616303 ] Yuming Wang commented on SPARK-40764: - Another case if pull out join condition: {co

[jira] [Commented] (SPARK-40764) Extract partitioning through all children output expressions

2022-10-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616301#comment-17616301 ] Yuming Wang commented on SPARK-40764: - Another case: {code:scala} withSQLConf(SQ

[jira] [Assigned] (SPARK-40464) Support automatic data format conversion for shuffle state db

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40464: Assignee: Apache Spark > Support automatic data format conversion for shuffle state db >

[jira] [Commented] (SPARK-40464) Support automatic data format conversion for shuffle state db

2022-10-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616278#comment-17616278 ] Apache Spark commented on SPARK-40464: -- User 'panbingkun' has created a pull reques

  1   2   >