[jira] [Commented] (SPARK-20384) supporting value classes over primitives in DataSets

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379651#comment-17379651 ] Apache Spark commented on SPARK-20384: -- User 'eejbyfeldt' has created a pull reques

[jira] [Commented] (SPARK-20384) supporting value classes over primitives in DataSets

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379650#comment-17379650 ] Apache Spark commented on SPARK-20384: -- User 'eejbyfeldt' has created a pull reques

[jira] [Commented] (SPARK-36118) [SQL] add bitmap functions for Spark SQL

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379621#comment-17379621 ] Apache Spark commented on SPARK-36118: -- User 'ReachInfi' has created a pull request

[jira] [Assigned] (SPARK-36118) [SQL] add bitmap functions for Spark SQL

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36118: Assignee: Apache Spark > [SQL] add bitmap functions for Spark SQL > -

[jira] [Assigned] (SPARK-36118) [SQL] add bitmap functions for Spark SQL

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36118: Assignee: (was: Apache Spark) > [SQL] add bitmap functions for Spark SQL > --

[jira] [Assigned] (SPARK-36118) [SQL] add bitmap functions for Spark SQL

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36118: Assignee: Apache Spark > [SQL] add bitmap functions for Spark SQL > -

[jira] [Created] (SPARK-36118) [SQL] add bitmap functions for Spark SQL

2021-07-12 Thread master_chief (Jira)
master_chief created SPARK-36118: Summary: [SQL] add bitmap functions for Spark SQL Key: SPARK-36118 URL: https://issues.apache.org/jira/browse/SPARK-36118 Project: Spark Issue Type: New Feat

[jira] [Updated] (SPARK-35553) Improve correlated subquery

2021-07-12 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-35553: - Description: This umbrella ticket is used to keep track of correlated subquery improvements and

[jira] [Issue Comment Deleted] (SPARK-36111) Support casting of timestamp without time zone to numeric type

2021-07-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36111: --- Comment: was deleted (was: I'm working on this.) > Support casting of timestamp without time zone t

[jira] [Resolved] (SPARK-36111) Support casting of timestamp without time zone to numeric type

2021-07-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng resolved SPARK-36111. Resolution: Won't Fix > Support casting of timestamp without time zone to numeric type > -

[jira] [Created] (SPARK-36117) Join can become unresolved after PullupCorrelatedPredicates

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36117: Summary: Join can become unresolved after PullupCorrelatedPredicates Key: SPARK-36117 URL: https://issues.apache.org/jira/browse/SPARK-36117 Project: Spark

[jira] [Created] (SPARK-36116) ResolveSubquery should skip unary node UnresolvedHaving

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36116: Summary: ResolveSubquery should skip unary node UnresolvedHaving Key: SPARK-36116 URL: https://issues.apache.org/jira/browse/SPARK-36116 Project: Spark Issue

[jira] [Created] (SPARK-36115) Handle the COUNT bug for correlated IN/EXISTS subquery

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36115: Summary: Handle the COUNT bug for correlated IN/EXISTS subquery Key: SPARK-36115 URL: https://issues.apache.org/jira/browse/SPARK-36115 Project: Spark Issue

[jira] [Created] (SPARK-36114) Support subqueries with correlated non-equality predicates

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36114: Summary: Support subqueries with correlated non-equality predicates Key: SPARK-36114 URL: https://issues.apache.org/jira/browse/SPARK-36114 Project: Spark Is

[jira] [Created] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36113: Summary: Unify the logic to handle COUNT bug for scalar and lateral subqueries Key: SPARK-36113 URL: https://issues.apache.org/jira/browse/SPARK-36113 Project: Spark

[jira] [Created] (SPARK-36112) Enable DecorrelateInnerQuery for IN/EXISTS subqueries

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36112: Summary: Enable DecorrelateInnerQuery for IN/EXISTS subqueries Key: SPARK-36112 URL: https://issues.apache.org/jira/browse/SPARK-36112 Project: Spark Issue T

[jira] [Resolved] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36110. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33312 [https://gi

[jira] [Commented] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379568#comment-17379568 ] Apache Spark commented on SPARK-36104: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-36111) Support casting of timestamp without time zone to numeric type

2021-07-12 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-36111: -- Summary: Support casting of timestamp without time zone to numeric type Key: SPARK-36111 URL: https://issues.apache.org/jira/browse/SPARK-36111 Project: Spark I

[jira] [Commented] (SPARK-36111) Support casting of timestamp without time zone to numeric type

2021-07-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379565#comment-17379565 ] jiaan.geng commented on SPARK-36111: I'm working on this. > Support casting of time

[jira] [Updated] (SPARK-35253) Upgrade Janino from 3.0.16 to 3.1.4

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35253: - Fix Version/s: (was: 3.2.0) > Upgrade Janino from 3.0.16 to 3.1.4 >

[jira] [Commented] (SPARK-35398) Simplify the way to get classes from ClassBodyEvaluator in CodeGenerator.updateAndGetCompilationStats method

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379550#comment-17379550 ] Hyukjin Kwon commented on SPARK-35398: -- Reverted at https://github.com/apache/spark

[jira] [Reopened] (SPARK-35253) Upgrade Janino from 3.0.16 to 3.1.4

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-35253: -- Assignee: (was: Takeshi Yamamuro) > Upgrade Janino from 3.0.16 to 3.1.4 > --

[jira] [Commented] (SPARK-35253) Upgrade Janino from 3.0.16 to 3.1.4

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379551#comment-17379551 ] Hyukjin Kwon commented on SPARK-35253: -- Reverted at https://github.com/apache/spark

[jira] [Reopened] (SPARK-35398) Simplify the way to get classes from ClassBodyEvaluator in CodeGenerator.updateAndGetCompilationStats method

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-35398: -- Assignee: (was: Yang Jie) > Simplify the way to get classes from ClassBodyEvaluator in

[jira] [Updated] (SPARK-35398) Simplify the way to get classes from ClassBodyEvaluator in CodeGenerator.updateAndGetCompilationStats method

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35398: - Fix Version/s: (was: 3.2.0) > Simplify the way to get classes from ClassBodyEvaluator in >

[jira] [Resolved] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36104. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33307 [https://gi

[jira] [Assigned] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36104: Assignee: Xinrong Meng > Manage InternalField in DataTypeOps.neg/abs > --

[jira] [Assigned] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36110: Assignee: Apache Spark (was: Kousuke Saruta) > Upgrade SBT to 1.5.5 > --

[jira] [Assigned] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36110: Assignee: Kousuke Saruta (was: Apache Spark) > Upgrade SBT to 1.5.5 > --

[jira] [Assigned] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36110: Assignee: Apache Spark (was: Kousuke Saruta) > Upgrade SBT to 1.5.5 > --

[jira] [Commented] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379512#comment-17379512 ] Apache Spark commented on SPARK-36110: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-36095) Group exception messages in core/rdd

2021-07-12 Thread dgd_contributor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379511#comment-17379511 ] dgd_contributor commented on SPARK-36095: - [~allisonwang-db] I would like to wor

[jira] [Updated] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36110: --- Description: SBT 1.5.5 was released, which includes 16 imrovements/bug fixes. https://github

[jira] [Updated] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36110: --- Issue Type: Improvement (was: Bug) > Upgrade SBT to 1.5.5 > > >

[jira] [Created] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36110: -- Summary: Upgrade SBT to 1.5.5 Key: SPARK-36110 URL: https://issues.apache.org/jira/browse/SPARK-36110 Project: Spark Issue Type: Bug Components

[jira] [Updated] (SPARK-36109) Fix flaky KafkaSourceStressSuite

2021-07-12 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-36109: Affects Version/s: 3.0.3 > Fix flaky KafkaSourceStressSuite > > >

[jira] [Updated] (SPARK-36109) Fix flaky KafkaSourceStressSuite

2021-07-12 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-36109: Affects Version/s: 3.1.2 > Fix flaky KafkaSourceStressSuite > > >

[jira] [Commented] (SPARK-36109) Fix flaky KafkaSourceStressSuite

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379495#comment-17379495 ] Apache Spark commented on SPARK-36109: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-36109) Fix flaky KafkaSourceStressSuite

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36109: Assignee: L. C. Hsieh (was: Apache Spark) > Fix flaky KafkaSourceStressSuite > -

[jira] [Assigned] (SPARK-36109) Fix flaky KafkaSourceStressSuite

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36109: Assignee: Apache Spark (was: L. C. Hsieh) > Fix flaky KafkaSourceStressSuite > -

[jira] [Created] (SPARK-36109) Fix flaky KafkaSourceStressSuite

2021-07-12 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-36109: --- Summary: Fix flaky KafkaSourceStressSuite Key: SPARK-36109 URL: https://issues.apache.org/jira/browse/SPARK-36109 Project: Spark Issue Type: Test Com

[jira] [Assigned] (SPARK-36105) OptimizeLocalShuffleReader support reading data of multiple mappers in one task

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36105: Assignee: Apache Spark > OptimizeLocalShuffleReader support reading data of multiple mapp

[jira] [Commented] (SPARK-36105) OptimizeLocalShuffleReader support reading data of multiple mappers in one task

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379480#comment-17379480 ] Apache Spark commented on SPARK-36105: -- User 'michaelzhang-db' has created a pull r

[jira] [Assigned] (SPARK-36105) OptimizeLocalShuffleReader support reading data of multiple mappers in one task

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36105: Assignee: (was: Apache Spark) > OptimizeLocalShuffleReader support reading data of mu

[jira] [Resolved] (SPARK-36103) Manage InternalField in DataTypeOps.invert

2021-07-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36103. -- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Fixed in https:

[jira] [Assigned] (SPARK-36106) Add error classes to QueryCompilationErrors

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36106: Assignee: (was: Apache Spark) > Add error classes to QueryCompilationErrors > ---

[jira] [Assigned] (SPARK-36106) Add error classes to QueryCompilationErrors

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36106: Assignee: Apache Spark > Add error classes to QueryCompilationErrors > --

[jira] [Commented] (SPARK-36106) Add error classes to QueryCompilationErrors

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379465#comment-17379465 ] Apache Spark commented on SPARK-36106: -- User 'karenfeng' has created a pull request

[jira] [Updated] (SPARK-36106) Add error classes to QueryCompilationErrors

2021-07-12 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36106: --- Description: Add error classes to [QueryCompilationErrors|https://github.com/apache/spark/blob/mast

[jira] [Commented] (SPARK-36106) Add error classes to QueryCompilationErrors

2021-07-12 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379463#comment-17379463 ] Karen Feng commented on SPARK-36106: I am working on the first 10 exceptions as a sa

[jira] [Created] (SPARK-36108) Add error classes to QueryParsingErrors

2021-07-12 Thread Karen Feng (Jira)
Karen Feng created SPARK-36108: -- Summary: Add error classes to QueryParsingErrors Key: SPARK-36108 URL: https://issues.apache.org/jira/browse/SPARK-36108 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36107) Add error classes to QueryExecutionErrors

2021-07-12 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36107: --- Description: Add error classes to [QueryExecutionErrors|https://github.com/apache/spark/blob/master/

[jira] [Updated] (SPARK-36106) Add error classes to QueryCompilationErrors

2021-07-12 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36106: --- Description: Add error classes to [QueryCompilationErrors|https://github.com/apache/spark/blob/maste

[jira] [Created] (SPARK-36107) Add error classes to QueryExecutionErrors

2021-07-12 Thread Karen Feng (Jira)
Karen Feng created SPARK-36107: -- Summary: Add error classes to QueryExecutionErrors Key: SPARK-36107 URL: https://issues.apache.org/jira/browse/SPARK-36107 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36106) Add error classes to QueryCompilationErrors

2021-07-12 Thread Karen Feng (Jira)
Karen Feng created SPARK-36106: -- Summary: Add error classes to QueryCompilationErrors Key: SPARK-36106 URL: https://issues.apache.org/jira/browse/SPARK-36106 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-35918) Consolidate logic between AvroSerializer/AvroDeserializer for schema mismatch handling and error messages

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379456#comment-17379456 ] Apache Spark commented on SPARK-35918: -- User 'xkrogen' has created a pull request f

[jira] [Assigned] (SPARK-35918) Consolidate logic between AvroSerializer/AvroDeserializer for schema mismatch handling and error messages

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35918: Assignee: Apache Spark > Consolidate logic between AvroSerializer/AvroDeserializer for sc

[jira] [Assigned] (SPARK-35918) Consolidate logic between AvroSerializer/AvroDeserializer for schema mismatch handling and error messages

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35918: Assignee: (was: Apache Spark) > Consolidate logic between AvroSerializer/AvroDeserial

[jira] [Commented] (SPARK-35918) Consolidate logic between AvroSerializer/AvroDeserializer for schema mismatch handling and error messages

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379455#comment-17379455 ] Apache Spark commented on SPARK-35918: -- User 'xkrogen' has created a pull request f

[jira] [Updated] (SPARK-36081) Update the document about the behavior change of trimming characters for cast

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36081: --- Description: sql-migration-guide.md mentions about the behavior of cast like as follows. {co

[jira] [Updated] (SPARK-36081) Update the document about the behavior change of trimming characters for cast

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36081: --- Summary: Update the document about the behavior change of trimming characters for cast (was

[jira] [Commented] (SPARK-36105) OptimizeLocalShuffleReader support reading data of multiple mappers in one task

2021-07-12 Thread Michael Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379452#comment-17379452 ] Michael Zhang commented on SPARK-36105: --- @[~cloud_fan] I'm working on this issue,

[jira] [Created] (SPARK-36105) OptimizeLocalShuffleReader support reading data of multiple mappers in one task

2021-07-12 Thread Michael Zhang (Jira)
Michael Zhang created SPARK-36105: - Summary: OptimizeLocalShuffleReader support reading data of multiple mappers in one task Key: SPARK-36105 URL: https://issues.apache.org/jira/browse/SPARK-36105 Pro

[jira] [Commented] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379437#comment-17379437 ] Apache Spark commented on SPARK-36104: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36104: Assignee: Apache Spark > Manage InternalField in DataTypeOps.neg/abs > --

[jira] [Assigned] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36104: Assignee: (was: Apache Spark) > Manage InternalField in DataTypeOps.neg/abs > ---

[jira] [Updated] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36104: - Description: The spark data type and nullability must be the same as the original when  {{DataTy

[jira] [Created] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36104: Summary: Manage InternalField in DataTypeOps.neg/abs Key: SPARK-36104 URL: https://issues.apache.org/jira/browse/SPARK-36104 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-35398) Simplify the way to get classes from ClassBodyEvaluator in CodeGenerator.updateAndGetCompilationStats method

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379396#comment-17379396 ] Apache Spark commented on SPARK-35398: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-36103) Manage InternalField in DataTypeOps.invert

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36103: Assignee: (was: Apache Spark) > Manage InternalField in DataTypeOps.invert >

[jira] [Commented] (SPARK-36103) Manage InternalField in DataTypeOps.invert

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379353#comment-17379353 ] Apache Spark commented on SPARK-36103: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-36103) Manage InternalField in DataTypeOps.invert

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36103: Assignee: Apache Spark > Manage InternalField in DataTypeOps.invert > ---

[jira] [Resolved] (SPARK-36046) Support new functions make_timestamp_ntz and make_timestamp_ltz

2021-07-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36046. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33299 [https://github.com

[jira] [Assigned] (SPARK-36046) Support new functions make_timestamp_ntz and make_timestamp_ltz

2021-07-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36046: Assignee: Gengliang Wang > Support new functions make_timestamp_ntz and make_timestamp_ltz >

[jira] [Created] (SPARK-36103) Manage InternalField in DataTypeOps.invert

2021-07-12 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36103: - Summary: Manage InternalField in DataTypeOps.invert Key: SPARK-36103 URL: https://issues.apache.org/jira/browse/SPARK-36103 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-35829) Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379332#comment-17379332 ] Apache Spark commented on SPARK-35829: -- User 'viirya' has created a pull request fo

[jira] [Resolved] (SPARK-35337) pandas API on Spark: Separate basic operations into data type based structures

2021-07-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35337. -- Resolution: Done > pandas API on Spark: Separate basic operations into data type based structu

[jira] [Resolved] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

2021-07-12 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36002. --- Fix Version/s: 3.2.0 Assignee: Yikun Jiang Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36102) Group exception messages in core/deploy

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36102: Summary: Group exception messages in core/deploy Key: SPARK-36102 URL: https://issues.apache.org/jira/browse/SPARK-36102 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36101) Group exception messages in core/api

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36101: Summary: Group exception messages in core/api Key: SPARK-36101 URL: https://issues.apache.org/jira/browse/SPARK-36101 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36100) Group exception messages in core/status

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36100: Summary: Group exception messages in core/status Key: SPARK-36100 URL: https://issues.apache.org/jira/browse/SPARK-36100 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36099) Group exception messages in core/util

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36099: Summary: Group exception messages in core/util Key: SPARK-36099 URL: https://issues.apache.org/jira/browse/SPARK-36099 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36098) Group exception messages in core/storage

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36098: Summary: Group exception messages in core/storage Key: SPARK-36098 URL: https://issues.apache.org/jira/browse/SPARK-36098 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36097) Group exception messages in core/scheduler

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36097: Summary: Group exception messages in core/scheduler Key: SPARK-36097 URL: https://issues.apache.org/jira/browse/SPARK-36097 Project: Spark Issue Type: Sub-ta

[jira] [Created] (SPARK-36096) Group exception messages in core/resource

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36096: Summary: Group exception messages in core/resource Key: SPARK-36096 URL: https://issues.apache.org/jira/browse/SPARK-36096 Project: Spark Issue Type: Sub-tas

[jira] [Created] (SPARK-36095) Group exception messages in core/rdd

2021-07-12 Thread Allison Wang (Jira)
Allison Wang created SPARK-36095: Summary: Group exception messages in core/rdd Key: SPARK-36095 URL: https://issues.apache.org/jira/browse/SPARK-36095 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36034) Incorrect datetime filter when reading Parquet files written in legacy mode

2021-07-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379312#comment-17379312 ] Max Gekk commented on SPARK-36034: -- I have reproduced the issue on master. Still troubl

[jira] [Commented] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379305#comment-17379305 ] Apache Spark commented on SPARK-32920: -- User 'venkata91' has created a pull request

[jira] [Commented] (SPARK-32920) Add support in Spark driver to coordinate the finalization of the push/merge phase in push-based shuffle for a given shuffle and the initiation of the reduce stage

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379303#comment-17379303 ] Apache Spark commented on SPARK-32920: -- User 'venkata91' has created a pull request

[jira] [Updated] (SPARK-36094) Group error messages in JSON file

2021-07-12 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36094: --- Component/s: SQL > Group error messages in JSON file > - > >

[jira] [Created] (SPARK-36094) Group error messages in JSON file

2021-07-12 Thread Karen Feng (Jira)
Karen Feng created SPARK-36094: -- Summary: Group error messages in JSON file Key: SPARK-36094 URL: https://issues.apache.org/jira/browse/SPARK-36094 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35866) Improve error message quality

2021-07-12 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-35866: --- Description: In the SPIP: Standardize Exception Messages in Spark, there are three major improvemen

[jira] [Resolved] (SPARK-33603) Group exception messages in execution/command

2021-07-12 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-33603. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32951 [https:

[jira] [Assigned] (SPARK-33603) Group exception messages in execution/command

2021-07-12 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-33603: -- Assignee: dgd_contributor > Group exception messages in execution/command > -

[jira] [Comment Edited] (SPARK-35957) Cannot convert Avro schema to catalyst type because schema at path is not compatible

2021-07-12 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379278#comment-17379278 ] Erik Krogen edited comment on SPARK-35957 at 7/12/21, 5:10 PM: ---

[jira] [Comment Edited] (SPARK-35957) Cannot convert Avro schema to catalyst type because schema at path is not compatible

2021-07-12 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379278#comment-17379278 ] Erik Krogen edited comment on SPARK-35957 at 7/12/21, 5:10 PM: ---

[jira] [Commented] (SPARK-35957) Cannot convert Avro schema to catalyst type because schema at path is not compatible

2021-07-12 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379278#comment-17379278 ] Erik Krogen commented on SPARK-35957: - Based on the discussion in [the linked Hudi

[jira] [Commented] (SPARK-35253) Upgrade Janino from 3.0.16 to 3.1.4

2021-07-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379267#comment-17379267 ] Apache Spark commented on SPARK-35253: -- User 'cloud-fan' has created a pull request

[jira] [Resolved] (SPARK-35861) Introduce "prefix match scan" feature on state store

2021-07-12 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-35861. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33038 [https://gith

  1   2   >