[jira] [Resolved] (SPARK-32295) Add not null and size > 0 filters before inner explode to benefit from predicate pushdown

[ https://issues.apache.org/jira/browse/SPARK-32295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis resolved SPARK-32295. Resolution: Won't Do > Add not null and size > 0 filters before inner explode to benefit from > p

[jira] [Commented] (SPARK-32601) Issue in converting an RDD of Arrow RecordBatches in v3.0.0

[ https://issues.apache.org/jira/browse/SPARK-32601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194941#comment-17194941 ] Linar Savion commented on SPARK-32601: -- Hi [~tahmad], this can be easily done using

[jira] [Created] (SPARK-32868) Mark more aggregate functions as order irrelevant

Tanel Kiis created SPARK-32868: -- Summary: Mark more aggregate functions as order irrelevant Key: SPARK-32868 URL: https://issues.apache.org/jira/browse/SPARK-32868 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-32868) Mark more aggregate functions as order irrelevant

[ https://issues.apache.org/jira/browse/SPARK-32868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194966#comment-17194966 ] Apache Spark commented on SPARK-32868: -- User 'tanelk' has created a pull request fo

[jira] [Assigned] (SPARK-32868) Mark more aggregate functions as order irrelevant

[ https://issues.apache.org/jira/browse/SPARK-32868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32868: Assignee: Apache Spark > Mark more aggregate functions as order irrelevant >

[jira] [Assigned] (SPARK-32868) Mark more aggregate functions as order irrelevant

[ https://issues.apache.org/jira/browse/SPARK-32868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32868: Assignee: (was: Apache Spark) > Mark more aggregate functions as order irrelevant > -

[jira] [Commented] (SPARK-32868) Mark more aggregate functions as order irrelevant

[ https://issues.apache.org/jira/browse/SPARK-32868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194967#comment-17194967 ] Apache Spark commented on SPARK-32868: -- User 'tanelk' has created a pull request fo

[jira] [Created] (SPARK-32869) Ignore deprecation warnings for sbt

Kousuke Saruta created SPARK-32869: -- Summary: Ignore deprecation warnings for sbt Key: SPARK-32869 URL: https://issues.apache.org/jira/browse/SPARK-32869 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-32869) Ignore deprecation warnings for sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32869: --- Description: If we build Spark with scala-2.13 and sbt, some deprecation warnings are treate

[jira] [Updated] (SPARK-32869) Ignore deprecation warnings for sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32869: --- Parent: SPARK-25075 Issue Type: Sub-task (was: Bug) > Ignore deprecation warnings f

[jira] [Assigned] (SPARK-32869) Ignore deprecation warnings for sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32869: Assignee: Apache Spark (was: Kousuke Saruta) > Ignore deprecation warnings for sbt > ---

[jira] [Assigned] (SPARK-32869) Ignore deprecation warnings for sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32869: Assignee: Kousuke Saruta (was: Apache Spark) > Ignore deprecation warnings for sbt > ---

[jira] [Commented] (SPARK-32869) Ignore deprecation warnings for sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195101#comment-17195101 ] Apache Spark commented on SPARK-32869: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-32869) Ignore deprecation warnings for sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195102#comment-17195102 ] Apache Spark commented on SPARK-32869: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-32870) Make sure that all expressions have their ExpressionDescription properly filled

Tanel Kiis created SPARK-32870: -- Summary: Make sure that all expressions have their ExpressionDescription properly filled Key: SPARK-32870 URL: https://issues.apache.org/jira/browse/SPARK-32870 Project:

[jira] [Updated] (SPARK-32869) Ignore deprecation warnings for build with Scala-2.13 and sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32869: --- Summary: Ignore deprecation warnings for build with Scala-2.13 and sbt (was: Ignore depreca

[jira] [Created] (SPARK-32871) Append toMap to Map#filterKeys if the result of filter is concatinated with another Map

Kousuke Saruta created SPARK-32871: -- Summary: Append toMap to Map#filterKeys if the result of filter is concatinated with another Map Key: SPARK-32871 URL: https://issues.apache.org/jira/browse/SPARK-32871

[jira] [Updated] (SPARK-32871) Append toMap to Map#filterKeys if the result of filter is concatinated with another Map

[ https://issues.apache.org/jira/browse/SPARK-32871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32871: --- Parent: SPARK-25075 Issue Type: Sub-task (was: Bug) > Append toMap to Map#filterKey

[jira] [Updated] (SPARK-32871) Append toMap to Map#filterKeys if the result of filter is concatenated with another Map

[ https://issues.apache.org/jira/browse/SPARK-32871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32871: --- Summary: Append toMap to Map#filterKeys if the result of filter is concatenated with another

[jira] [Commented] (SPARK-32871) Append toMap to Map#filterKeys if the result of filter is concatenated with another Map

[ https://issues.apache.org/jira/browse/SPARK-32871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195110#comment-17195110 ] Apache Spark commented on SPARK-32871: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-32871) Append toMap to Map#filterKeys if the result of filter is concatenated with another Map

[ https://issues.apache.org/jira/browse/SPARK-32871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32871: Assignee: Kousuke Saruta (was: Apache Spark) > Append toMap to Map#filterKeys if the res

[jira] [Assigned] (SPARK-32871) Append toMap to Map#filterKeys if the result of filter is concatenated with another Map

[ https://issues.apache.org/jira/browse/SPARK-32871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32871: Assignee: Apache Spark (was: Kousuke Saruta) > Append toMap to Map#filterKeys if the res

[jira] [Commented] (SPARK-32454) Fix HistoryServerSuite in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195116#comment-17195116 ] Kousuke Saruta commented on SPARK-32454: [~dongjoon] Is this issue still reprodu

[jira] [Assigned] (SPARK-32870) Make sure that all expressions have their ExpressionDescription properly filled

[ https://issues.apache.org/jira/browse/SPARK-32870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32870: Assignee: Apache Spark > Make sure that all expressions have their ExpressionDescription

[jira] [Assigned] (SPARK-32870) Make sure that all expressions have their ExpressionDescription properly filled

[ https://issues.apache.org/jira/browse/SPARK-32870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32870: Assignee: (was: Apache Spark) > Make sure that all expressions have their ExpressionD

[jira] [Commented] (SPARK-32870) Make sure that all expressions have their ExpressionDescription properly filled

[ https://issues.apache.org/jira/browse/SPARK-32870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195122#comment-17195122 ] Apache Spark commented on SPARK-32870: -- User 'tanelk' has created a pull request fo

[jira] [Created] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

Ankur Dave created SPARK-32872: -- Summary: BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY Key: SPARK-32872 URL: https://issues.apache.org/jira/browse/SPARK-32872 Project: Spark

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Description: When BytesToBytesMap is close to {{MAX_CAPACITY}} and needs to grow, {{numKeys >= grow

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Description: When BytesToBytesMap is close to {{MAX_CAPACITY}} and needs to grow, {{numKeys >= grow

[jira] [Commented] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195130#comment-17195130 ] Apache Spark commented on SPARK-32872: -- User 'ankurdave' has created a pull request

[jira] [Assigned] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32872: Assignee: Ankur Dave (was: Apache Spark) > BytesToBytesMap exceeds growth threshold when

[jira] [Commented] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195131#comment-17195131 ] Apache Spark commented on SPARK-32872: -- User 'ankurdave' has created a pull request

[jira] [Assigned] (SPARK-32872) BytesToBytesMap exceeds growth threshold when close to MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32872: Assignee: Apache Spark (was: Ankur Dave) > BytesToBytesMap exceeds growth threshold when

[jira] [Updated] (SPARK-32871) Append toMap to Map#filterKeys if the result of filter is concatenated with another Map for Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32871: --- Summary: Append toMap to Map#filterKeys if the result of filter is concatenated with another

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold at MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Summary: BytesToBytesMap exceeds growth threshold at MAX_CAPACITY (was: BytesToBytesMap exceeds gro

[jira] [Updated] (SPARK-32872) BytesToBytesMap exceeds growth threshold at MAX_CAPACITY

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Description: When BytesToBytesMap is at {{MAX_CAPACITY}} and reaches the growth threshold,  {{numKey

[jira] [Updated] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave updated SPARK-32872: --- Summary: BytesToBytesMap at MAX_CAPACITY exceeds growth threshold (was: BytesToBytesMap exceeds gro

[jira] [Resolved] (SPARK-32869) Ignore deprecation warnings for build with Scala-2.13 and sbt

[ https://issues.apache.org/jira/browse/SPARK-32869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32869. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29741 [https://gi

[jira] [Assigned] (SPARK-31847) DAGSchedulerSuite: Rewrite the test framework to cover most of the existing major features of the Spark Scheduler, mock the necessary part wisely, and make the test fra

[ https://issues.apache.org/jira/browse/SPARK-31847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31847: Assignee: jiaan.geng > DAGSchedulerSuite: Rewrite the test framework to cover most of the

[jira] [Resolved] (SPARK-31847) DAGSchedulerSuite: Rewrite the test framework to cover most of the existing major features of the Spark Scheduler, mock the necessary part wisely, and make the test fra

[ https://issues.apache.org/jira/browse/SPARK-31847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31847. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29228 [https://gi

[jira] [Updated] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32873: --- Parent: SPARK-25075 Issue Type: Sub-task (was: Bug) > Fix code which causes error w

[jira] [Created] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

Kousuke Saruta created SPARK-32873: -- Summary: Fix code which causes error when build with sbt and Scala 2.13 Key: SPARK-32873 URL: https://issues.apache.org/jira/browse/SPARK-32873 Project: Spark

[jira] [Updated] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-32873: --- Description: When we try to build Spark with sbt and Spark 2.13, some piece of code causes

[jira] [Assigned] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32873: Assignee: Apache Spark (was: Kousuke Saruta) > Fix code which causes error when build wi

[jira] [Commented] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195172#comment-17195172 ] Apache Spark commented on SPARK-32873: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32873: Assignee: Kousuke Saruta (was: Apache Spark) > Fix code which causes error when build wi

[jira] [Commented] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195171#comment-17195171 ] Apache Spark commented on SPARK-32873: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-32839) Make Spark scripts working with the spaces in paths on Windows

[ https://issues.apache.org/jira/browse/SPARK-32839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32839: Assignee: Hyukjin Kwon > Make Spark scripts working with the spaces in paths on Windows >

[jira] [Resolved] (SPARK-32839) Make Spark scripts working with the spaces in paths on Windows

[ https://issues.apache.org/jira/browse/SPARK-32839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32839. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29706 [https://gi

[jira] [Resolved] (SPARK-32765) EliminateJoinToEmptyRelation should respect exchange behavior when canChangeNumPartitions == false

[ https://issues.apache.org/jira/browse/SPARK-32765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin resolved SPARK-32765. - Resolution: Not A Bug > EliminateJoinToEmptyRelation should respect exchange behavior when > ca

[jira] [Updated] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32872: -- Affects Version/s: 2.0.2 2.1.3 2.2.3

[jira] [Commented] (SPARK-32872) BytesToBytesMap at MAX_CAPACITY exceeds growth threshold

[ https://issues.apache.org/jira/browse/SPARK-32872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195204#comment-17195204 ] Dongjoon Hyun commented on SPARK-32872: --- Hi, [~ankurd]. According to the patch, I

[jira] [Created] (SPARK-32874) Enhance result set meta data check for execute statement operation for thrift server

Kent Yao created SPARK-32874: Summary: Enhance result set meta data check for execute statement operation for thrift server Key: SPARK-32874 URL: https://issues.apache.org/jira/browse/SPARK-32874 Project:

[jira] [Resolved] (SPARK-32873) Fix code which causes error when build with sbt and Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-32873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32873. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29745 [https://gi

[jira] [Assigned] (SPARK-32874) Enhance result set meta data check for execute statement operation for thrift server

[ https://issues.apache.org/jira/browse/SPARK-32874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32874: Assignee: Apache Spark > Enhance result set meta data check for execute statement operati

[jira] [Commented] (SPARK-32874) Enhance result set meta data check for execute statement operation for thrift server

[ https://issues.apache.org/jira/browse/SPARK-32874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195224#comment-17195224 ] Apache Spark commented on SPARK-32874: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-32874) Enhance result set meta data check for execute statement operation for thrift server

[ https://issues.apache.org/jira/browse/SPARK-32874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32874: Assignee: (was: Apache Spark) > Enhance result set meta data check for execute statem

[jira] [Updated] (SPARK-32846) Support createDataFrame from an RDD of pd.DataFrames

[ https://issues.apache.org/jira/browse/SPARK-32846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linar Savion updated SPARK-32846: - Flags: Patch Target Version/s: 3.1.0 > Support createDataFrame from an RDD of pd.

[jira] [Commented] (SPARK-32874) Enhance result set meta data check for execute statement operation for thrift server

[ https://issues.apache.org/jira/browse/SPARK-32874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195227#comment-17195227 ] Apache Spark commented on SPARK-32874: -- User 'yaooqinn' has created a pull request

[jira] [Commented] (SPARK-32846) Support createDataFrame from an RDD of pd.DataFrames

[ https://issues.apache.org/jira/browse/SPARK-32846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195228#comment-17195228 ] Linar Savion commented on SPARK-32846: -- PR is waiting for review: https://github.co