[jira] [Commented] (SPARK-37545) V2 CreateTableAsSelect command should qualify location

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453306#comment-17453306 ] Apache Spark commented on SPARK-37545: -- User 'imback82' has created a pull request for this issue:

[jira] [Commented] (SPARK-37545) V2 CreateTableAsSelect command should qualify location

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453305#comment-17453305 ] Apache Spark commented on SPARK-37545: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37545) V2 CreateTableAsSelect command should qualify location

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37545: Assignee: (was: Apache Spark) > V2 CreateTableAsSelect command should qualify

[jira] [Assigned] (SPARK-37545) V2 CreateTableAsSelect command should qualify location

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37545: Assignee: Apache Spark > V2 CreateTableAsSelect command should qualify location >

[jira] [Updated] (SPARK-37545) V2 CreateTableAsSelect command should qualify location

2021-12-03 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-37545: -- Description: V2 CreateTableAsSelect command should qualify location. Currently,    {code:java}

[jira] [Assigned] (SPARK-37542) Optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to improve performance

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37542: Assignee: (was: Apache Spark) > Optimize the dynamic partitioning prune rules to

[jira] [Created] (SPARK-37545) V2 CreateTableAsSelect command should qualify location

2021-12-03 Thread Terry Kim (Jira)
Terry Kim created SPARK-37545: - Summary: V2 CreateTableAsSelect command should qualify location Key: SPARK-37545 URL: https://issues.apache.org/jira/browse/SPARK-37545 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-37542) Optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to improve performance

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37542: Assignee: Apache Spark > Optimize the dynamic partitioning prune rules to avoid

[jira] [Commented] (SPARK-37542) Optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to improve performance

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453301#comment-17453301 ] Apache Spark commented on SPARK-37542: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Updated] (SPARK-37542) Optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to improve performance

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Description: Currently, the dynamic partition pruning rule will insert a predicate on the filterable

[jira] [Updated] (SPARK-37542) Optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to improve performance

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Description: Currently, the dynamic partition pruning rule will insert a predicate on the filterable

[jira] [Updated] (SPARK-37542) optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to improve performance

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Summary: optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to

[jira] [Updated] (SPARK-37542) Optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to improve performance

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Summary: Optimize the dynamic partitioning prune rules to avoid inserting unnecessary predicates to

[jira] [Commented] (SPARK-37543) Document Java 17 support

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453296#comment-17453296 ] Apache Spark commented on SPARK-37543: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Closed] (SPARK-37110) Add Java 17 support for spark pull request builds

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37110. - > Add Java 17 support for spark pull request builds >

[jira] [Created] (SPARK-37544) sequence over dates with month interval is producing incorrect results

2021-12-03 Thread Vsevolod Ostapenko (Jira)
Vsevolod Ostapenko created SPARK-37544: -- Summary: sequence over dates with month interval is producing incorrect results Key: SPARK-37544 URL: https://issues.apache.org/jira/browse/SPARK-37544

[jira] [Assigned] (SPARK-37543) Document Java 17 support

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37543: - Assignee: Dongjoon Hyun > Document Java 17 support > > >

[jira] [Resolved] (SPARK-37543) Document Java 17 support

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37543. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34804

[jira] [Updated] (SPARK-37542) Improve the Dynamic partition pruning

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Description: Currently, the dynamic partition pruning rule will insert a predicate on the filterable

[jira] [Commented] (SPARK-37500) Incorrect scope when using named_windows in CTEs

2021-12-03 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453294#comment-17453294 ] Rui Wang commented on SPARK-37500: -- I am thinking this is no longer valid? Running on 3.2 spark-cli:

[jira] [Updated] (SPARK-37542) Improve the Dynamic partition pruning

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Description: Currently, the dynamic partition pruning rule will insert a predicate on the filterable

[jira] [Updated] (SPARK-37542) Improve the Dynamic partition pruning

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Description: Currently, the dynamic partition pruning rule will insert a predicate on the filterable

[jira] [Updated] (SPARK-37542) Improve the Dynamic partition pruning

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Description: Currently, the dynamic partition pruning rule will insert a predicate on the filterable

[jira] [Updated] (SPARK-37543) Document Java 17 support

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37543: -- Summary: Document Java 17 support (was: Add Java 17 support to the document) > Document

[jira] [Created] (SPARK-37543) Add JDK11 support to the document

2021-12-03 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-37543: - Summary: Add JDK11 support to the document Key: SPARK-37543 URL: https://issues.apache.org/jira/browse/SPARK-37543 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-37543) Add Java 17 support to the document

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37543: -- Summary: Add Java 17 support to the document (was: Add JDK17 support to the document) > Add

[jira] [Updated] (SPARK-37543) Add JDK17 support to the document

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37543: -- Summary: Add JDK17 support to the document (was: Add JDK11 support to the document) > Add

[jira] [Updated] (SPARK-37542) Improve the Dynamic partition pruning

2021-12-03 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37542: - Description: Currently, the dynamic partition pruning rule will insert a predicate on the filterable

[jira] [Created] (SPARK-37542) Improve the Dynamic partition pruning

2021-12-03 Thread weixiuli (Jira)
weixiuli created SPARK-37542: Summary: Improve the Dynamic partition pruning Key: SPARK-37542 URL: https://issues.apache.org/jira/browse/SPARK-37542 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-37330) Migrate ReplaceTableStatement to v2 command

2021-12-03 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-37330. Fix Version/s: 3.3.0 Assignee: dch nguyen Resolution: Fixed > Migrate

[jira] [Resolved] (SPARK-37510) Support TimedeltaIndex in pandas API on Spark

2021-12-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37510. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34787

[jira] [Assigned] (SPARK-37510) Support TimedeltaIndex in pandas API on Spark

2021-12-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37510: Assignee: Xinrong Meng > Support TimedeltaIndex in pandas API on Spark >

[jira] [Resolved] (SPARK-37495) Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

2021-12-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37495. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34750

[jira] [Assigned] (SPARK-37495) Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

2021-12-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37495: Assignee: dch nguyen > Skip identical index checking of Series.compare when config >

[jira] [Resolved] (SPARK-37541) Remove protobuf-java dependency from distribution

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37541. --- Resolution: Incomplete > Remove protobuf-java dependency from distribution >

[jira] [Assigned] (SPARK-37541) Remove protobuf-java dependency from distribution

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37541: Assignee: Apache Spark > Remove protobuf-java dependency from distribution >

[jira] [Commented] (SPARK-37541) Remove protobuf-java dependency from distribution

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453264#comment-17453264 ] Apache Spark commented on SPARK-37541: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-37541) Remove protobuf-java dependency from distribution

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453263#comment-17453263 ] Apache Spark commented on SPARK-37541: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-37541) Remove protobuf-java dependency from distribution

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37541: Assignee: (was: Apache Spark) > Remove protobuf-java dependency from distribution >

[jira] [Created] (SPARK-37541) Remove protobuf-java dependency

2021-12-03 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-37541: - Summary: Remove protobuf-java dependency Key: SPARK-37541 URL: https://issues.apache.org/jira/browse/SPARK-37541 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37541) Remove protobuf-java dependency from distribution

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37541: -- Summary: Remove protobuf-java dependency from distribution (was: Remove protobuf-java

[jira] [Commented] (SPARK-37500) Incorrect scope when using named_windows in CTEs

2021-12-03 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453215#comment-17453215 ] Rui Wang commented on SPARK-37500: -- I can take a look on this bug. > Incorrect scope when using

[jira] [Updated] (SPARK-34287) Spark Aggregate improvement

2021-12-03 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34287: - Summary: Spark Aggregate improvement (was: Aggregation improvement) > Spark Aggregate improvement >

[jira] [Updated] (SPARK-34287) Spark aggregate improvement

2021-12-03 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34287: - Summary: Spark aggregate improvement (was: Spark Aggregate improvement) > Spark aggregate improvement

[jira] [Updated] (SPARK-34287) Aggregation improvement

2021-12-03 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34287: - Description: Creating this umbrella Jira to track overall progress for Spark aggregate improvement. See

[jira] [Updated] (SPARK-34287) Aggregation improvement

2021-12-03 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34287: - Summary: Aggregation improvement (was: Object hash and sort aggregation improvement) > Aggregation

[jira] [Updated] (SPARK-36529) Decouple CPU with IO work in vectorized Parquet reader

2021-12-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36529: - Attachment: (was: image.png) > Decouple CPU with IO work in vectorized Parquet reader >

[jira] [Updated] (SPARK-36529) Decouple CPU with IO work in vectorized Parquet reader

2021-12-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36529: - Attachment: image.png > Decouple CPU with IO work in vectorized Parquet reader >

[jira] [Updated] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37534: -- Affects Version/s: (was: 3.2.1) > Bump dev.ludovic.netlib to 2.2.1 >

[jira] [Updated] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37534: -- Affects Version/s: (was: 3.3.0) > Bump dev.ludovic.netlib to 2.2.1 >

[jira] [Updated] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37534: -- Fix Version/s: 3.2.1 > Bump dev.ludovic.netlib to 2.2.1 > >

[jira] [Resolved] (SPARK-37471) spark-sql support nested bracketed comment

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37471. - Fix Version/s: 3.3.0 3.2.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-37471) spark-sql support nested bracketed comment

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37471: --- Assignee: angerszhu > spark-sql support nested bracketed comment >

[jira] [Assigned] (SPARK-37540) Detect more unsupported time travel

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37540: Assignee: Apache Spark > Detect more unsupported time travel >

[jira] [Commented] (SPARK-37540) Detect more unsupported time travel

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453120#comment-17453120 ] Apache Spark commented on SPARK-37540: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37540) Detect more unsupported time travel

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37540: Assignee: (was: Apache Spark) > Detect more unsupported time travel >

[jira] [Created] (SPARK-37540) Detect more unsupported time travel

2021-12-03 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-37540: --- Summary: Detect more unsupported time travel Key: SPARK-37540 URL: https://issues.apache.org/jira/browse/SPARK-37540 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37286) Move compileAggregates from JDBCRDD to JdbcDialect

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453080#comment-17453080 ] Apache Spark commented on SPARK-37286: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-37538) Replace single projection Expand with Project

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453075#comment-17453075 ] Apache Spark commented on SPARK-37538: -- User 'tanelk' has created a pull request for this issue:

[jira] [Commented] (SPARK-37538) Replace single projection Expand with Project

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453073#comment-17453073 ] Apache Spark commented on SPARK-37538: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37538) Replace single projection Expand with Project

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37538: Assignee: Apache Spark > Replace single projection Expand with Project >

[jira] [Assigned] (SPARK-37538) Replace single projection Expand with Project

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37538: Assignee: (was: Apache Spark) > Replace single projection Expand with Project >

[jira] [Created] (SPARK-37539) Create Spark interface for creating UDFs

2021-12-03 Thread Almaz Murzabekov (Jira)
Almaz Murzabekov created SPARK-37539: Summary: Create Spark interface for creating UDFs Key: SPARK-37539 URL: https://issues.apache.org/jira/browse/SPARK-37539 Project: Spark Issue Type:

[jira] [Created] (SPARK-37538) Replace single projection Expand with Project

2021-12-03 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-37538: -- Summary: Replace single projection Expand with Project Key: SPARK-37538 URL: https://issues.apache.org/jira/browse/SPARK-37538 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37392) Catalyst optimizer very time-consuming and memory-intensive with some "explode(array)"

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453034#comment-17453034 ] Wenchen Fan commented on SPARK-37392: - Great investigation, [~joshrosen] ! I think for this

[jira] [Created] (SPARK-37537) Spark 3.2.0 driver pod does not mount checkpoint filesystem from Kubernetes PVC

2021-12-03 Thread Petri (Jira)
Petri created SPARK-37537: - Summary: Spark 3.2.0 driver pod does not mount checkpoint filesystem from Kubernetes PVC Key: SPARK-37537 URL: https://issues.apache.org/jira/browse/SPARK-37537 Project: Spark

[jira] [Resolved] (SPARK-37455) Replace hash with sort aggregate if child is already sorted

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37455. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34702

[jira] [Assigned] (SPARK-37455) Replace hash with sort aggregate if child is already sorted

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37455: --- Assignee: Cheng Su (was: Apache Spark) > Replace hash with sort aggregate if child is

[jira] [Resolved] (SPARK-37286) Move compileAggregates from JDBCRDD to JdbcDialect

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37286. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34554

[jira] [Assigned] (SPARK-37286) Move compileAggregates from JDBCRDD to JdbcDialect

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37286: --- Assignee: jiaan.geng > Move compileAggregates from JDBCRDD to JdbcDialect >

[jira] [Resolved] (SPARK-36902) Migrate CreateTableAsSelectStatement to v2 command

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36902. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34667

[jira] [Assigned] (SPARK-36902) Migrate CreateTableAsSelectStatement to v2 command

2021-12-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36902: --- Assignee: dgd_contributor > Migrate CreateTableAsSelectStatement to v2 command >

[jira] [Commented] (SPARK-37527) Translate more standard aggregate functions for pushdown

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452926#comment-17452926 ] Apache Spark commented on SPARK-37527: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37527) Translate more standard aggregate functions for pushdown

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37527: Assignee: (was: Apache Spark) > Translate more standard aggregate functions for

[jira] [Commented] (SPARK-37527) Translate more standard aggregate functions for pushdown

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452923#comment-17452923 ] Apache Spark commented on SPARK-37527: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37527) Translate more standard aggregate functions for pushdown

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37527: Assignee: Apache Spark > Translate more standard aggregate functions for pushdown >

[jira] (SPARK-37527) Translate more standard aggregate functions for pushdown

2021-12-03 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37527 ] jiaan.geng deleted comment on SPARK-37527: was (Author: beliefer): I'm working on. > Translate more standard aggregate functions for pushdown >

[jira] [Created] (SPARK-37536) Allow for API user to disable Shuffle Operations while running locally

2021-12-03 Thread Rodrigo Boavida (Jira)
Rodrigo Boavida created SPARK-37536: --- Summary: Allow for API user to disable Shuffle Operations while running locally Key: SPARK-37536 URL: https://issues.apache.org/jira/browse/SPARK-37536

[jira] [Assigned] (SPARK-37535) Change default spark.io.compression.codec to zstd

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37535: Assignee: (was: Apache Spark) > Change default spark.io.compression.codec to zstd >

[jira] [Commented] (SPARK-37535) Change default spark.io.compression.codec to zstd

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452873#comment-17452873 ] Apache Spark commented on SPARK-37535: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37535) Change default spark.io.compression.codec to zstd

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37535: Assignee: Apache Spark > Change default spark.io.compression.codec to zstd >

[jira] [Created] (SPARK-37535) Change default spark.io.compression.codec to zstd

2021-12-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37535: --- Summary: Change default spark.io.compression.codec to zstd Key: SPARK-37535 URL: https://issues.apache.org/jira/browse/SPARK-37535 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452866#comment-17452866 ] Apache Spark commented on SPARK-37534: -- User 'luhenry' has created a pull request for this issue:

[jira] [Resolved] (SPARK-37530) Spark reads many paths very slow though newAPIHadoopFile

2021-12-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-37530. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34792

[jira] [Assigned] (SPARK-37530) Spark reads many paths very slow though newAPIHadoopFile

2021-12-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-37530: Assignee: Kent Yao > Spark reads many paths very slow though newAPIHadoopFile >

[jira] [Updated] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37534: -- Issue Type: Bug (was: Improvement) > Bump dev.ludovic.netlib to 2.2.1 >

[jira] [Resolved] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37534. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34783

[jira] [Assigned] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37534: - Assignee: Ludovic Henry > Bump dev.ludovic.netlib to 2.2.1 >

[jira] [Resolved] (SPARK-37531) Use PyArrow 6.0.0 in Python 3.9 tests at GitHub Action job

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37531. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34793

[jira] [Assigned] (SPARK-37531) Use PyArrow 6.0.0 in Python 3.9 tests at GitHub Action job

2021-12-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37531: - Assignee: Dongjoon Hyun > Use PyArrow 6.0.0 in Python 3.9 tests at GitHub Action job >

[jira] [Commented] (SPARK-37533) New SQL function: try_element_at

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452850#comment-17452850 ] Apache Spark commented on SPARK-37533: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-37533) New SQL function: try_element_at

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37533: Assignee: Apache Spark (was: Gengliang Wang) > New SQL function: try_element_at >

[jira] [Assigned] (SPARK-37533) New SQL function: try_element_at

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37533: Assignee: Gengliang Wang (was: Apache Spark) > New SQL function: try_element_at >

[jira] [Commented] (SPARK-37533) New SQL function: try_element_at

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452849#comment-17452849 ] Apache Spark commented on SPARK-37533: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452848#comment-17452848 ] Apache Spark commented on SPARK-37534: -- User 'luhenry' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37534: Assignee: (was: Apache Spark) > Bump dev.ludovic.netlib to 2.2.1 >

[jira] [Assigned] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37534: Assignee: Apache Spark > Bump dev.ludovic.netlib to 2.2.1 >

[jira] [Commented] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452847#comment-17452847 ] Apache Spark commented on SPARK-37534: -- User 'luhenry' has created a pull request for this issue:

[jira] [Created] (SPARK-37534) Bump dev.ludovic.netlib to 2.2.1

2021-12-03 Thread Ludovic Henry (Jira)
Ludovic Henry created SPARK-37534: - Summary: Bump dev.ludovic.netlib to 2.2.1 Key: SPARK-37534 URL: https://issues.apache.org/jira/browse/SPARK-37534 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-37533) New SQL function: try_element_at

2021-12-03 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-37533: -- Summary: New SQL function: try_element_at Key: SPARK-37533 URL: https://issues.apache.org/jira/browse/SPARK-37533 Project: Spark Issue Type: Sub-task

  1   2   >