[jira] [Resolved] (SPARK-35883) Migrate ALTER TABLE rename column command to the new resolution framework

2021-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35883. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33066

[jira] [Assigned] (SPARK-35883) Migrate ALTER TABLE rename column command to the new resolution framework

2021-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35883: --- Assignee: Terry Kim > Migrate ALTER TABLE rename column command to the new resolution

[jira] [Assigned] (SPARK-34892) Introduce MergingSortWithSessionWindowStateIterator sorting input rows and rows in state efficiently

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34892: Assignee: Apache Spark > Introduce MergingSortWithSessionWindowStateIterator sorting

[jira] [Assigned] (SPARK-34892) Introduce MergingSortWithSessionWindowStateIterator sorting input rows and rows in state efficiently

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34892: Assignee: (was: Apache Spark) > Introduce MergingSortWithSessionWindowStateIterator

[jira] [Commented] (SPARK-34892) Introduce MergingSortWithSessionWindowStateIterator sorting input rows and rows in state efficiently

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369247#comment-17369247 ] Apache Spark commented on SPARK-34892: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Resolved] (SPARK-35067) Test case for function COALESCE() does not cover the CodeGen support

2021-06-24 Thread zengrui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengrui resolved SPARK-35067. - Resolution: Won't Fix > Test case for function COALESCE() does not cover the CodeGen support >

[jira] [Resolved] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-06-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35471. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33073

[jira] [Assigned] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-06-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35471: Assignee: Takuya Ueshin > Enable disallow_untyped_defs mypy check for

[jira] [Commented] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-06-24 Thread Myeongju Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369241#comment-17369241 ] Myeongju Kim commented on SPARK-34927: -- [~dongjoon] Hello Dongjoon, Yes I'm still doing my Github

[jira] [Updated] (SPARK-35888) Add dataSize field in CoalescedPartitionSpec

2021-06-24 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35888: -- Description: Currently, all test suite about `CoalescedPartitionSpec` do not check the data size due

[jira] [Resolved] (SPARK-35817) Queries against wide Avro tables can be slow

2021-06-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35817. Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-35889) Support adding TimestampWithoutTZ with Interval types

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369228#comment-17369228 ] Apache Spark commented on SPARK-35889: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-35889) Support adding TimestampWithoutTZ with Interval types

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35889: Assignee: Gengliang Wang (was: Apache Spark) > Support adding TimestampWithoutTZ with

[jira] [Assigned] (SPARK-35889) Support adding TimestampWithoutTZ with Interval types

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35889: Assignee: Apache Spark (was: Gengliang Wang) > Support adding TimestampWithoutTZ with

[jira] [Assigned] (SPARK-35887) Find and set JAVA_HOME from javac location

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35887: - Assignee: William Hyun > Find and set JAVA_HOME from javac location >

[jira] [Updated] (SPARK-35889) Support adding TimestampWithoutTZ with Interval types

2021-06-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-35889: --- Summary: Support adding TimestampWithoutTZ with Interval types (was: Support ADDing

[jira] [Resolved] (SPARK-35887) Find and set JAVA_HOME from javac location

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35887. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33075

[jira] [Created] (SPARK-35889) Support ADDing TimestampWithoutTZ and Interval types

2021-06-24 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35889: -- Summary: Support ADDing TimestampWithoutTZ and Interval types Key: SPARK-35889 URL: https://issues.apache.org/jira/browse/SPARK-35889 Project: Spark

[jira] [Created] (SPARK-35888) Add dataSize field in CoalescedPartitionSpec

2021-06-24 Thread XiDuo You (Jira)
XiDuo You created SPARK-35888: - Summary: Add dataSize field in CoalescedPartitionSpec Key: SPARK-35888 URL: https://issues.apache.org/jira/browse/SPARK-35888 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-35887) Find and set JAVA_HOME from javac location

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35887: Assignee: (was: Apache Spark) > Find and set JAVA_HOME from javac location >

[jira] [Commented] (SPARK-35887) Find and set JAVA_HOME from javac location

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369224#comment-17369224 ] Apache Spark commented on SPARK-35887: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-35887) Find and set JAVA_HOME from javac location

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35887: Assignee: Apache Spark > Find and set JAVA_HOME from javac location >

[jira] [Created] (SPARK-35887) Find and set JAVA_HOME from javac location

2021-06-24 Thread William Hyun (Jira)
William Hyun created SPARK-35887: Summary: Find and set JAVA_HOME from javac location Key: SPARK-35887 URL: https://issues.apache.org/jira/browse/SPARK-35887 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-35886) Codegen issue for decimal type

2021-06-24 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369208#comment-17369208 ] angerszhu commented on SPARK-35886: --- Working on this > Codegen issue for decimal type >

[jira] [Commented] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369202#comment-17369202 ] Apache Spark commented on SPARK-34320: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369203#comment-17369203 ] Apache Spark commented on SPARK-34320: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35885: - Assignee: Dongjoon Hyun > Use keyserver.ubuntu.com as a keyserver for CRAN >

[jira] [Resolved] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35885. --- Fix Version/s: 3.1.3 3.2.0 3.0.4 Resolution:

[jira] [Created] (SPARK-35886) Codegen issue for decimal type

2021-06-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-35886: --- Summary: Codegen issue for decimal type Key: SPARK-35886 URL: https://issues.apache.org/jira/browse/SPARK-35886 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35471: Assignee: (was: Apache Spark) > Enable disallow_untyped_defs mypy check for

[jira] [Assigned] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35471: Assignee: Apache Spark > Enable disallow_untyped_defs mypy check for

[jira] [Commented] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369179#comment-17369179 ] Apache Spark commented on SPARK-35471: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-35817) Queries against wide Avro tables can be slow

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369172#comment-17369172 ] Apache Spark commented on SPARK-35817: -- User 'bersprockets' has created a pull request for this

[jira] [Updated] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35885: -- Description: This issue aims to use `keyserver.ubuntu.com` as a keyserver for CRAN. K8s

[jira] [Updated] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35885: -- Target Version/s: 3.2.0 > Use keyserver.ubuntu.com as a keyserver for CRAN >

[jira] [Updated] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35885: -- Affects Version/s: 3.0.3 3.1.2 > Use keyserver.ubuntu.com as a

[jira] [Assigned] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35885: Assignee: (was: Apache Spark) > Use keyserver.ubuntu.com as a keyserver for CRAN >

[jira] [Assigned] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35885: Assignee: Apache Spark > Use keyserver.ubuntu.com as a keyserver for CRAN >

[jira] [Commented] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369171#comment-17369171 ] Apache Spark commented on SPARK-35885: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-35873) Cleanup the version logic from the pandas API on Spark

2021-06-24 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35873: Summary: Cleanup the version logic from the pandas API on Spark (was: Remove the version check

[jira] [Updated] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35885: -- Description: ### What changes were proposed in this pull request? This PR aims to use

[jira] [Updated] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35885: -- Description: ### What changes were proposed in this pull request? This PR aims to use

[jira] [Updated] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35885: -- Priority: Blocker (was: Major) > Use keyserver.ubuntu.com as a keyserver for CRAN >

[jira] [Created] (SPARK-35885) Use keyserver.ubuntu.com as a keyserver for CRAN

2021-06-24 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35885: - Summary: Use keyserver.ubuntu.com as a keyserver for CRAN Key: SPARK-35885 URL: https://issues.apache.org/jira/browse/SPARK-35885 Project: Spark Issue

[jira] [Resolved] (SPARK-33298) FileCommitProtocol V2

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33298. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33012

[jira] [Assigned] (SPARK-33298) FileCommitProtocol V2

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33298: - Assignee: Cheng Su > FileCommitProtocol V2 > - > >

[jira] [Updated] (SPARK-33298) Introduce new API to FileCommitProtocol allow flexible file naming

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33298: -- Summary: Introduce new API to FileCommitProtocol allow flexible file naming (was:

[jira] [Assigned] (SPARK-35551) Handle the COUNT bug when rewriting lateral subqueries

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35551: Assignee: (was: Apache Spark) > Handle the COUNT bug when rewriting lateral

[jira] [Commented] (SPARK-35551) Handle the COUNT bug when rewriting lateral subqueries

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369162#comment-17369162 ] Apache Spark commented on SPARK-35551: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-35551) Handle the COUNT bug when rewriting lateral subqueries

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35551: Assignee: Apache Spark > Handle the COUNT bug when rewriting lateral subqueries >

[jira] [Updated] (SPARK-35801) SPIP: Row-level operations in Data Source V2

2021-06-24 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-35801: - Summary: SPIP: Row-level operations in Data Source V2 (was: SPIP: Support MERGE in

[jira] [Updated] (SPARK-35801) SPIP: Support MERGE in Data Source V2

2021-06-24 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Okolnychyi updated SPARK-35801: - Shepherd: L. C. Hsieh Description: Row-level operations such as UPDATE, DELETE,

[jira] [Resolved] (SPARK-35872) automatic some work for Spark releases

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35872. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33055

[jira] [Commented] (SPARK-35884) EXPLAIN FORMATTED for AQE

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369063#comment-17369063 ] Apache Spark commented on SPARK-35884: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35884) EXPLAIN FORMATTED for AQE

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35884: Assignee: Apache Spark > EXPLAIN FORMATTED for AQE > - > >

[jira] [Assigned] (SPARK-35884) EXPLAIN FORMATTED for AQE

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35884: Assignee: (was: Apache Spark) > EXPLAIN FORMATTED for AQE >

[jira] [Commented] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369061#comment-17369061 ] Dongjoon Hyun commented on SPARK-35868: --- Thank you for the comment, [~ste...@apache.org]. It's a

[jira] [Updated] (SPARK-35884) EXPLAIN FORMATTED for AQE

2021-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-35884: Issue Type: Bug (was: New Feature) > EXPLAIN FORMATTED for AQE > - > >

[jira] [Created] (SPARK-35884) EXPLAIN FORMATTED for AQE

2021-06-24 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-35884: --- Summary: EXPLAIN FORMATTED for AQE Key: SPARK-35884 URL: https://issues.apache.org/jira/browse/SPARK-35884 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-30283) V2 Command logical plan should use UnresolvedV2Relation for a table

2021-06-24 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim resolved SPARK-30283. --- Resolution: Duplicate Already resolved as separate subtasks. > V2 Command logical plan should use

[jira] [Resolved] (SPARK-30535) Migrate ALTER TABLE commands to the new resolution framework

2021-06-24 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim resolved SPARK-30535. --- Resolution: Duplicate Will be done as a separate subtask for each command. > Migrate ALTER TABLE

[jira] [Resolved] (SPARK-35403) Migrate ALTER TABLE commands that alter columns to new framework

2021-06-24 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim resolved SPARK-35403. --- Resolution: Duplicate Will be done as a separate subtask for each command. > Migrate ALTER TABLE

[jira] [Commented] (SPARK-35883) Migrate ALTER TABLE rename column command to the new resolution framework

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369038#comment-17369038 ] Apache Spark commented on SPARK-35883: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35883) Migrate ALTER TABLE rename column command to the new resolution framework

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35883: Assignee: Apache Spark > Migrate ALTER TABLE rename column command to the new resolution

[jira] [Assigned] (SPARK-35883) Migrate ALTER TABLE rename column command to the new resolution framework

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35883: Assignee: (was: Apache Spark) > Migrate ALTER TABLE rename column command to the new

[jira] [Created] (SPARK-35883) Migrate ALTER TABLE rename column command to the new resolution framework

2021-06-24 Thread Terry Kim (Jira)
Terry Kim created SPARK-35883: - Summary: Migrate ALTER TABLE rename column command to the new resolution framework Key: SPARK-35883 URL: https://issues.apache.org/jira/browse/SPARK-35883 Project: Spark

[jira] [Resolved] (SPARK-35870) Upgrade Jetty to 9.4.42

2021-06-24 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35870. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved

[jira] [Commented] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-06-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369030#comment-17369030 ] Dongjoon Hyun commented on SPARK-34927: --- Is there any update for this JIRA, [~Myeong]? > Support

[jira] [Updated] (SPARK-35882) Query performance degradation additional predicate and UDF call for explode

2021-06-24 Thread John Bateman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bateman updated SPARK-35882: - Description: This issue cannot be seen in 3.0.1, but has been introduced since and observed in

[jira] [Created] (SPARK-35882) Query performance degradation additional predicate and UDF call for explode

2021-06-24 Thread John Bateman (Jira)
John Bateman created SPARK-35882: Summary: Query performance degradation additional predicate and UDF call for explode Key: SPARK-35882 URL: https://issues.apache.org/jira/browse/SPARK-35882 Project:

[jira] [Commented] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

2021-06-24 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368970#comment-17368970 ] Takuya Ueshin commented on SPARK-35471: --- I'm working on this. > Enable disallow_untyped_defs mypy

[jira] [Commented] (SPARK-35880) [SS] Track the number of duplicates dropped in streaming dedupe operator

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368966#comment-17368966 ] Apache Spark commented on SPARK-35880: -- User 'vkorukanti' has created a pull request for this

[jira] [Assigned] (SPARK-35880) [SS] Track the number of duplicates dropped in streaming dedupe operator

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35880: Assignee: (was: Apache Spark) > [SS] Track the number of duplicates dropped in

[jira] [Commented] (SPARK-35880) [SS] Track the number of duplicates dropped in streaming dedupe operator

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368965#comment-17368965 ] Apache Spark commented on SPARK-35880: -- User 'vkorukanti' has created a pull request for this

[jira] [Assigned] (SPARK-35880) [SS] Track the number of duplicates dropped in streaming dedupe operator

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35880: Assignee: Apache Spark > [SS] Track the number of duplicates dropped in streaming dedupe

[jira] [Assigned] (SPARK-35290) unionByName with null filling fails for some nested structs

2021-06-24 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-35290: --- Assignee: Adam Binford > unionByName with null filling fails for some nested structs >

[jira] [Resolved] (SPARK-35290) unionByName with null filling fails for some nested structs

2021-06-24 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35290. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33040

[jira] [Assigned] (SPARK-35878) add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35878: Assignee: Apache Spark > add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null

[jira] [Assigned] (SPARK-35878) add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35878: Assignee: (was: Apache Spark) > add fs.s3a.endpoint if unset and

[jira] [Commented] (SPARK-35878) add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368943#comment-17368943 ] Apache Spark commented on SPARK-35878: -- User 'steveloughran' has created a pull request for this

[jira] [Updated] (SPARK-35881) [SQL] AQE does not support columnar execution for the final query stage

2021-06-24 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-35881: --- Description: In AdaptiveSparkPlanExec, a query is broken down into stages and these stages are

[jira] [Updated] (SPARK-35881) [SQL] AQE does not support columnar execution for the final query stage

2021-06-24 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-35881: --- Description: In AdaptiveSparkPlanExec, a query is broken down into stages and these stages are

[jira] [Created] (SPARK-35881) [SQL] AQE does not support columnar execution for the final query stage

2021-06-24 Thread Andy Grove (Jira)
Andy Grove created SPARK-35881: -- Summary: [SQL] AQE does not support columnar execution for the final query stage Key: SPARK-35881 URL: https://issues.apache.org/jira/browse/SPARK-35881 Project: Spark

[jira] [Created] (SPARK-35880) [SS] Track the number of duplicates dropped in streaming dedupe operator

2021-06-24 Thread Venki Korukanti (Jira)
Venki Korukanti created SPARK-35880: --- Summary: [SS] Track the number of duplicates dropped in streaming dedupe operator Key: SPARK-35880 URL: https://issues.apache.org/jira/browse/SPARK-35880

[jira] [Resolved] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34320. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32854

[jira] [Assigned] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34320: --- Assignee: Terry Kim > Migrate ALTER TABLE drop columns command to the new resolution

[jira] [Commented] (SPARK-34723) Correct parameter type for subexpression elimination under whole-stage

2021-06-24 Thread Andrew Olson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368894#comment-17368894 ] Andrew Olson commented on SPARK-34723: -- In case it might be helpful to anyone, the compilation

[jira] [Assigned] (SPARK-35871) Literal.create(value, dataType) should support fields

2021-06-24 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35871: Assignee: angerszhu > Literal.create(value, dataType) should support fields >

[jira] [Resolved] (SPARK-35871) Literal.create(value, dataType) should support fields

2021-06-24 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35871. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33056

[jira] [Commented] (SPARK-35879) Fix performance regression caused by collectFetchRequests

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368856#comment-17368856 ] Apache Spark commented on SPARK-35879: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-35879) Fix performance regression caused by collectFetchRequests

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368855#comment-17368855 ] Apache Spark commented on SPARK-35879: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35879) Fix performance regression caused by collectFetchRequests

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35879: Assignee: Apache Spark > Fix performance regression caused by collectFetchRequests >

[jira] [Assigned] (SPARK-35879) Fix performance regression caused by collectFetchRequests

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35879: Assignee: (was: Apache Spark) > Fix performance regression caused by

[jira] [Created] (SPARK-35879) Fix performance regression caused by collectFetchRequests

2021-06-24 Thread Kent Yao (Jira)
Kent Yao created SPARK-35879: Summary: Fix performance regression caused by collectFetchRequests Key: SPARK-35879 URL: https://issues.apache.org/jira/browse/SPARK-35879 Project: Spark Issue

[jira] [Commented] (SPARK-35877) Spark Protobuf jar has CVE issue CVE-2015-5237

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368828#comment-17368828 ] Apache Spark commented on SPARK-35877: -- User 'PavithraRamachandran' has created a pull request for

[jira] [Assigned] (SPARK-35877) Spark Protobuf jar has CVE issue CVE-2015-5237

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35877: Assignee: (was: Apache Spark) > Spark Protobuf jar has CVE issue CVE-2015-5237 >

[jira] [Commented] (SPARK-35877) Spark Protobuf jar has CVE issue CVE-2015-5237

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368829#comment-17368829 ] Apache Spark commented on SPARK-35877: -- User 'PavithraRamachandran' has created a pull request for

[jira] [Assigned] (SPARK-35877) Spark Protobuf jar has CVE issue CVE-2015-5237

2021-06-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35877: Assignee: Apache Spark > Spark Protobuf jar has CVE issue CVE-2015-5237 >

[jira] [Created] (SPARK-35878) add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null

2021-06-24 Thread Steve Loughran (Jira)
Steve Loughran created SPARK-35878: -- Summary: add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null Key: SPARK-35878 URL: https://issues.apache.org/jira/browse/SPARK-35878 Project: Spark

[jira] [Commented] (SPARK-35868) Add fs.s3a.downgrade.syncable.exceptions if not set

2021-06-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368822#comment-17368822 ] Steve Loughran commented on SPARK-35868: Which bits of code was this failing on? You got a stack

  1   2   >