[jira] [Created] (SPARK-33706) Require fully specified partition identifier in partitionExists()

2020-12-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33706: -- Summary: Require fully specified partition identifier in partitionExists() Key: SPARK-33706 URL: https://issues.apache.org/jira/browse/SPARK-33706 Project: Spark

[jira] [Commented] (SPARK-33441) Add "unused-import" compile arg to scalac and remove all unused imports in Scala code

2020-12-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245723#comment-17245723 ] Maxim Gekk commented on SPARK-33441: Would it be possible to check unused imports in

[jira] [Resolved] (SPARK-29021) NoSuchElementException: key not found: hostPath.spark-local-dir-5.options.path

2020-12-07 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-29021. -- Resolution: Not A Problem > NoSuchElementException: key not found: hostPath.spark-local-dir-5.options.

[jira] [Commented] (SPARK-33142) SQL temp view should store SQL text as well

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245721#comment-17245721 ] Apache Spark commented on SPARK-33142: -- User 'linhongliu-db' has created a pull req

[jira] [Commented] (SPARK-33647) cache table not working for persisted view

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245720#comment-17245720 ] Apache Spark commented on SPARK-33647: -- User 'linhongliu-db' has created a pull req

[jira] [Commented] (SPARK-33142) SQL temp view should store SQL text as well

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245719#comment-17245719 ] Apache Spark commented on SPARK-33142: -- User 'linhongliu-db' has created a pull req

[jira] [Assigned] (SPARK-33704) Support latest version of initialize() in HiveGenericUDTF

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33704: Assignee: Apache Spark > Support latest version of initialize() in HiveGenericUDTF >

[jira] [Commented] (SPARK-33704) Support latest version of initialize() in HiveGenericUDTF

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245710#comment-17245710 ] Apache Spark commented on SPARK-33704: -- User 'southernriver' has created a pull req

[jira] [Assigned] (SPARK-33704) Support latest version of initialize() in HiveGenericUDTF

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33704: Assignee: (was: Apache Spark) > Support latest version of initialize() in HiveGeneric

[jira] [Assigned] (SPARK-33679) Enable spark.sql.adaptive.enabled true by default

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33679: - Assignee: Dongjoon Hyun > Enable spark.sql.adaptive.enabled true by default > -

[jira] [Resolved] (SPARK-33679) Enable spark.sql.adaptive.enabled true by default

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33679. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30628 [https://

[jira] [Updated] (SPARK-33705) Three cases always fail in hive-thriftserver module

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33705: - Environment: (was: Seems the following tests always failed both in Jenkins and Github Action: * org

[jira] [Created] (SPARK-33705) Three cases always fail in hive-thriftserver module

2020-12-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-33705: Summary: Three cases always fail in hive-thriftserver module Key: SPARK-33705 URL: https://issues.apache.org/jira/browse/SPARK-33705 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33705) Three cases always fail in hive-thriftserver module

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33705: - Description: Seems the following tests always failed both in Jenkins and Github Action: * org.apache.sp

[jira] [Commented] (SPARK-33705) Three cases always fail in hive-thriftserver module

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245697#comment-17245697 ] Yang Jie commented on SPARK-33705: -- cc [~dongjoon] > Three cases always fail in hive-t

[jira] [Assigned] (SPARK-33703) Migrate MSCK REPAIR TABLE to the new resolution framework

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33703: Assignee: Apache Spark > Migrate MSCK REPAIR TABLE to the new resolution framework >

[jira] [Assigned] (SPARK-33703) Migrate MSCK REPAIR TABLE to the new resolution framework

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33703: Assignee: (was: Apache Spark) > Migrate MSCK REPAIR TABLE to the new resolution frame

[jira] [Created] (SPARK-33704) Support latest version of initialize() in HiveGenericUDTF

2020-12-07 Thread chenliang (Jira)
chenliang created SPARK-33704: - Summary: Support latest version of initialize() in HiveGenericUDTF Key: SPARK-33704 URL: https://issues.apache.org/jira/browse/SPARK-33704 Project: Spark Issue Typ

[jira] [Commented] (SPARK-33703) Migrate MSCK REPAIR TABLE to the new resolution framework

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245689#comment-17245689 ] Apache Spark commented on SPARK-33703: -- User 'imback82' has created a pull request

[jira] [Created] (SPARK-33703) Migrate MSCK REPAIR TABLE to the new resolution framework

2020-12-07 Thread Terry Kim (Jira)
Terry Kim created SPARK-33703: - Summary: Migrate MSCK REPAIR TABLE to the new resolution framework Key: SPARK-33703 URL: https://issues.apache.org/jira/browse/SPARK-33703 Project: Spark Issue Typ

[jira] [Commented] (SPARK-33285) Too many "Auto-application to `()` is deprecated." related compilation warnings

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245682#comment-17245682 ] Yang Jie commented on SPARK-33285: -- thx [~smarter] > Too many "Auto-application to `()

[jira] [Updated] (SPARK-33700) Try to push down filters for parquet and orc should add extra `filters.nonEmpty` condition

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33700: - Summary: Try to push down filters for parquet and orc should add extra `filters.nonEmpty` condition (wa

[jira] [Commented] (SPARK-33702) Result of add "unused-locals" compile arg to scalac

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245681#comment-17245681 ] Yang Jie commented on SPARK-33702: -- ping [~hyukjin.kwon] > Result of add "unused-local

[jira] [Updated] (SPARK-33702) Result of add "unused-locals" compile arg to scalac

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33702: - Attachment: unused-locals.log > Result of add "unused-locals" compile arg to scalac > --

[jira] [Created] (SPARK-33702) Result of add "unused-locals" compile arg to scalac

2020-12-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-33702: Summary: Result of add "unused-locals" compile arg to scalac Key: SPARK-33702 URL: https://issues.apache.org/jira/browse/SPARK-33702 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-33701) Adaptive shuffle merge finalization for push-based shuffle

2020-12-07 Thread Min Shen (Jira)
Min Shen created SPARK-33701: Summary: Adaptive shuffle merge finalization for push-based shuffle Key: SPARK-33701 URL: https://issues.apache.org/jira/browse/SPARK-33701 Project: Spark Issue Type

[jira] [Commented] (SPARK-33700) Try to push down filters for parquet and orc should add add `filters.nonEmpty` condition

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245661#comment-17245661 ] Apache Spark commented on SPARK-33700: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-33700) Try to push down filters for parquet and orc should add add `filters.nonEmpty` condition

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33700: Assignee: Apache Spark > Try to push down filters for parquet and orc should add add > `

[jira] [Assigned] (SPARK-33700) Try to push down filters for parquet and orc should add add `filters.nonEmpty` condition

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33700: Assignee: (was: Apache Spark) > Try to push down filters for parquet and orc should a

[jira] [Commented] (SPARK-33700) Try to push down filters for parquet and orc should add add `filters.nonEmpty` condition

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245660#comment-17245660 ] Apache Spark commented on SPARK-33700: -- User 'LuciferYang' has created a pull reque

[jira] [Updated] (SPARK-33700) Try to push down filters for parquet and orc should add add `filters.nonEmpty` condition

2020-12-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33700: - Description:   {code:java} lazy val footerFileMetaData = ParquetFileReader.readFooter(conf, filePath,

[jira] [Created] (SPARK-33700) Try to push down filters for parquet and orc should add add `filters.nonEmpty` condition

2020-12-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-33700: Summary: Try to push down filters for parquet and orc should add add `filters.nonEmpty` condition Key: SPARK-33700 URL: https://issues.apache.org/jira/browse/SPARK-33700 Proj

[jira] [Assigned] (SPARK-33664) Migrate ALTER TABLE ... RENAME TO to new resolution framework

2020-12-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33664: --- Assignee: Terry Kim > Migrate ALTER TABLE ... RENAME TO to new resolution framework > -

[jira] [Resolved] (SPARK-33664) Migrate ALTER TABLE ... RENAME TO to new resolution framework

2020-12-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33664. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30610 [https://gith

[jira] [Commented] (SPARK-33686) Spark SQL unrecognized `db.tbl`

2020-12-07 Thread xiepengjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245651#comment-17245651 ] xiepengjie commented on SPARK-33686: [~srowen]  : thx for your report, it's really n

[jira] [Commented] (SPARK-33692) Permanent view shouldn't use current catalog and namespace to lookup function

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245649#comment-17245649 ] Apache Spark commented on SPARK-33692: -- User 'linhongliu-db' has created a pull req

[jira] [Commented] (SPARK-33692) Permanent view shouldn't use current catalog and namespace to lookup function

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245648#comment-17245648 ] Apache Spark commented on SPARK-33692: -- User 'linhongliu-db' has created a pull req

[jira] [Assigned] (SPARK-33692) Permanent view shouldn't use current catalog and namespace to lookup function

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33692: Assignee: (was: Apache Spark) > Permanent view shouldn't use current catalog and name

[jira] [Assigned] (SPARK-33692) Permanent view shouldn't use current catalog and namespace to lookup function

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33692: Assignee: Apache Spark > Permanent view shouldn't use current catalog and namespace to lo

[jira] [Updated] (SPARK-32680) CTAS with V2 catalog wrongly accessed unresolved query

2020-12-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-32680: Fix Version/s: 3.0.2 > CTAS with V2 catalog wrongly accessed unresolved query > --

[jira] [Updated] (SPARK-33692) Permanent view shouldn't use current catalog and namespace to lookup function

2020-12-07 Thread Linhong Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linhong Liu updated SPARK-33692: Summary: Permanent view shouldn't use current catalog and namespace to lookup function (was: Perm

[jira] [Resolved] (SPARK-32861) GenerateExec should require output column ordering

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32861. --- Fix Version/s: 3.1.0 Assignee: Allison Wang Resolution: Fixed This is resolv

[jira] [Updated] (SPARK-33698) Fix the compilation error of OracleIntegrationSuite for Scala 2.13

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33698: -- Component/s: Tests > Fix the compilation error of OracleIntegrationSuite for Scala 2.13 >

[jira] [Resolved] (SPARK-33698) Fix the compilation error of OracleIntegrationSuite for Scala 2.13

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33698. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30660 [https://

[jira] [Resolved] (SPARK-33609) word2vec reduce broadcast size

2020-12-07 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-33609. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30548 [https://gi

[jira] [Assigned] (SPARK-33609) word2vec reduce broadcast size

2020-12-07 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-33609: Assignee: zhengruifeng > word2vec reduce broadcast size > --

[jira] [Created] (SPARK-33699) Spark web ui kill application but the thread still exists background

2020-12-07 Thread Xiaoming W (Jira)
Xiaoming W created SPARK-33699: -- Summary: Spark web ui kill application but the thread still exists background Key: SPARK-33699 URL: https://issues.apache.org/jira/browse/SPARK-33699 Project: Spark

[jira] [Assigned] (SPARK-33698) Fix the compilation error of OracleIntegrationSuite for Scala 2.13

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33698: Assignee: Kousuke Saruta (was: Apache Spark) > Fix the compilation error of OracleIntegr

[jira] [Commented] (SPARK-33698) Fix the compilation error of OracleIntegrationSuite for Scala 2.13

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245629#comment-17245629 ] Apache Spark commented on SPARK-33698: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-33698) Fix the compilation error of OracleIntegrationSuite for Scala 2.13

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33698: Assignee: Apache Spark (was: Kousuke Saruta) > Fix the compilation error of OracleIntegr

[jira] [Created] (SPARK-33698) Fix the compilation error of OracleIntegrationSuite for Scala 2.13

2020-12-07 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-33698: -- Summary: Fix the compilation error of OracleIntegrationSuite for Scala 2.13 Key: SPARK-33698 URL: https://issues.apache.org/jira/browse/SPARK-33698 Project: Spark

[jira] [Assigned] (SPARK-33697) UnionExec should require column ordering in RemoveRedundantProjects

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33697: Assignee: Apache Spark > UnionExec should require column ordering in RemoveRedundantProje

[jira] [Assigned] (SPARK-33697) UnionExec should require column ordering in RemoveRedundantProjects

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33697: Assignee: (was: Apache Spark) > UnionExec should require column ordering in RemoveRed

[jira] [Assigned] (SPARK-33697) UnionExec should require column ordering in RemoveRedundantProjects

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33697: Assignee: Apache Spark > UnionExec should require column ordering in RemoveRedundantProje

[jira] [Commented] (SPARK-33697) UnionExec should require column ordering in RemoveRedundantProjects

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245609#comment-17245609 ] Apache Spark commented on SPARK-33697: -- User 'allisonwang-db' has created a pull re

[jira] [Resolved] (SPARK-32320) Remove mutable default arguments

2020-12-07 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-32320. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 29122 [https://gi

[jira] [Assigned] (SPARK-32320) Remove mutable default arguments

2020-12-07 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-32320: Assignee: Fokko Driesprong > Remove mutable default arguments > -

[jira] [Commented] (SPARK-22769) When driver stopping, there is errors: "Could not find CoarseGrainedScheduler" and "RpcEnv already stopped"

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245606#comment-17245606 ] Apache Spark commented on SPARK-22769: -- User 'sqlwindspeaker' has created a pull re

[jira] [Resolved] (SPARK-32943) Add improvement to filter pushdown and project combination

2020-12-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-32943. - Resolution: Duplicate > Add improvement to filter pushdown and project combination > ---

[jira] [Resolved] (SPARK-32945) Avoid project combination when it will hurst performance

2020-12-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-32945. - Resolution: Duplicate > Avoid project combination when it will hurst performance > -

[jira] [Resolved] (SPARK-32944) Avoid push down Filter through Project when it will hurts performance

2020-12-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-32944. - Resolution: Duplicate > Avoid push down Filter through Project when it will hurts performance >

[jira] [Commented] (SPARK-32943) Add improvement to filter pushdown and project combination

2020-12-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245604#comment-17245604 ] L. C. Hsieh commented on SPARK-32943: - This was superseded by my other work in gener

[jira] [Updated] (SPARK-33697) UnionExec should require column ordering in RemoveRedundantProjects

2020-12-07 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-33697: - Summary: UnionExec should require column ordering in RemoveRedundantProjects (was: UnionExec sh

[jira] [Created] (SPARK-33697) UnionExec should always require column ordering in RemoveRedundantProjects

2020-12-07 Thread Allison Wang (Jira)
Allison Wang created SPARK-33697: Summary: UnionExec should always require column ordering in RemoveRedundantProjects Key: SPARK-33697 URL: https://issues.apache.org/jira/browse/SPARK-33697 Project: S

[jira] [Assigned] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33696: Assignee: Apache Spark > Upgrade built-in Hive to 2.3.8 > --

[jira] [Assigned] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33696: Assignee: (was: Apache Spark) > Upgrade built-in Hive to 2.3.8 >

[jira] [Commented] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245590#comment-17245590 ] Apache Spark commented on SPARK-33696: -- User 'wangyum' has created a pull request f

[jira] [Updated] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33696: Description: Hive 2.3.8 changes: HIVE-19662: Upgrade Avro to 1.8.2 HIVE-24324: Remove deprecated A

[jira] [Created] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2020-12-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33696: --- Summary: Upgrade built-in Hive to 2.3.8 Key: SPARK-33696 URL: https://issues.apache.org/jira/browse/SPARK-33696 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-12-07 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245582#comment-17245582 ] Shane Knapp commented on SPARK-33044: - ok, i'm ready to get this sorted, but am wond

[jira] [Updated] (SPARK-33695) Bump Jackson to 2.10.5 and databind to 2.10.5.1

2020-12-07 Thread Nicholas Marion (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Marion updated SPARK-33695: Description: Jackson reported a vulnerability under CVE-2020-25649. The version pulled in

[jira] [Assigned] (SPARK-33695) Bump Jackson to 2.10.5 and databind to 2.10.5.1

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33695: Assignee: (was: Apache Spark) > Bump Jackson to 2.10.5 and databind to 2.10.5.1 > ---

[jira] [Assigned] (SPARK-33695) Bump Jackson to 2.10.5 and databind to 2.10.5.1

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33695: Assignee: Apache Spark > Bump Jackson to 2.10.5 and databind to 2.10.5.1 > --

[jira] [Assigned] (SPARK-33695) Bump Jackson to 2.10.5 and databind to 2.10.5.1

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33695: Assignee: Apache Spark > Bump Jackson to 2.10.5 and databind to 2.10.5.1 > --

[jira] [Commented] (SPARK-33695) Bump Jackson to 2.10.5 and databind to 2.10.5.1

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245575#comment-17245575 ] Apache Spark commented on SPARK-33695: -- User 'n-marion' has created a pull request

[jira] [Assigned] (SPARK-33621) Add a way to inject data source rewrite rules

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33621: - Assignee: Anton Okolnychyi > Add a way to inject data source rewrite rules > --

[jira] [Resolved] (SPARK-33621) Add a way to inject data source rewrite rules

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33621. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30577 [https://

[jira] [Created] (SPARK-33695) Bump Jackson to 2.10.5 and databind to 2.10.5.1

2020-12-07 Thread Nicholas Marion (Jira)
Nicholas Marion created SPARK-33695: --- Summary: Bump Jackson to 2.10.5 and databind to 2.10.5.1 Key: SPARK-33695 URL: https://issues.apache.org/jira/browse/SPARK-33695 Project: Spark Issue T

[jira] [Commented] (SPARK-33680) Fix PrunePartitionSuiteBase/BucketedReadWithHiveSupportSuite not to depend on the default conf

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245539#comment-17245539 ] Apache Spark commented on SPARK-33680: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-33680) Fix PrunePartitionSuiteBase/BucketedReadWithHiveSupportSuite not to depend on the default conf

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245540#comment-17245540 ] Apache Spark commented on SPARK-33680: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-33694) Auditing the API changes and behavior changes in Spark 3.1

2020-12-07 Thread Xiao Li (Jira)
Xiao Li created SPARK-33694: --- Summary: Auditing the API changes and behavior changes in Spark 3.1 Key: SPARK-33694 URL: https://issues.apache.org/jira/browse/SPARK-33694 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33694) Auditing the API changes and behavior changes in Spark 3.1

2020-12-07 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-33694: Priority: Blocker (was: Major) > Auditing the API changes and behavior changes in Spark 3.1 > ---

[jira] [Commented] (SPARK-33539) Standardize exception messages in Spark

2020-12-07 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245459#comment-17245459 ] Allison Wang commented on SPARK-33539: -- Hi [~srowen], this is to breakdown more tha

[jira] [Resolved] (SPARK-33693) deprecate spark.sql.hive.convertCTAS

2020-12-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33693. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30651 [https://

[jira] [Commented] (SPARK-33641) Invalidate new char-like type in public APIs that result incorrect results

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245440#comment-17245440 ] Apache Spark commented on SPARK-33641: -- User 'yaooqinn' has created a pull request

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-12-07 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245426#comment-17245426 ] Shane Knapp commented on SPARK-33044: - i'm waiting for someone to merge [https://gi

[jira] [Commented] (SPARK-32797) Install mypy on the Jenkins CI workers

2020-12-07 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245395#comment-17245395 ] Shane Knapp commented on SPARK-32797: - should this be installed for both pyton2.7 an

[jira] [Commented] (SPARK-33242) Install numpydoc in Jenkins machines

2020-12-07 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245394#comment-17245394 ] Shane Knapp commented on SPARK-33242: - should this be installed for both pyton2.7 an

[jira] [Commented] (SPARK-33044) Add a Jenkins build and test job for Scala 2.13

2020-12-07 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245377#comment-17245377 ] Shane Knapp commented on SPARK-33044: - 1) logins are temporarily disabled due to new

[jira] [Commented] (SPARK-33285) Too many "Auto-application to `()` is deprecated." related compilation warnings

2020-12-07 Thread Guillaume Martres (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245370#comment-17245370 ] Guillaume Martres commented on SPARK-33285: --- > But it seems that we can't supp

[jira] [Commented] (SPARK-33678) Numerical product aggregation

2020-12-07 Thread Richard Penney (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245338#comment-17245338 ] Richard Penney commented on SPARK-33678: Thanks [~srowen]  - I agree that there

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-07 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245329#comment-17245329 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:41 PM: ---

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-07 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245329#comment-17245329 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:33 PM: ---

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-07 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245329#comment-17245329 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:32 PM: ---

[jira] [Commented] (SPARK-33635) Performance regression in Kafka read

2020-12-07 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245329#comment-17245329 ] David Wyles commented on SPARK-33635: - The diffs on that library between 2.4.5 and 3

[jira] [Commented] (SPARK-33480) support char/varchar type

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245324#comment-17245324 ] Apache Spark commented on SPARK-33480: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-33480) support char/varchar type

2020-12-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245322#comment-17245322 ] Apache Spark commented on SPARK-33480: -- User 'cloud-fan' has created a pull request

[jira] [Comment Edited] (SPARK-33635) Performance regression in Kafka read

2020-12-07 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245304#comment-17245304 ] David Wyles edited comment on SPARK-33635 at 12/7/20, 4:20 PM: ---

[jira] [Issue Comment Deleted] (SPARK-33635) Performance regression in Kafka read

2020-12-07 Thread David Wyles (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wyles updated SPARK-33635: Comment: was deleted (was: Also note, in my example - the only thing that changes is the spark ve

[jira] [Updated] (SPARK-25914) Separate projection from grouping and aggregate in logical Aggregate

2020-12-07 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-25914: - Target Version/s: (was: 3.0.0) > Separate projection from grouping and aggregate in logical Ag

  1   2   >