[jira] [Assigned] (SPARK-38345) Introduce SQL function ARRAY_SIZE

[ https://issues.apache.org/jira/browse/SPARK-38345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38345: Assignee: (was: Apache Spark) > Introduce SQL function ARRAY_SIZE >

[jira] [Commented] (SPARK-38345) Introduce SQL function ARRAY_SIZE

[ https://issues.apache.org/jira/browse/SPARK-38345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498741#comment-17498741 ] Apache Spark commented on SPARK-38345: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-38345) Introduce SQL function ARRAY_SIZE

[ https://issues.apache.org/jira/browse/SPARK-38345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38345: Assignee: Apache Spark > Introduce SQL function ARRAY_SIZE >

[jira] [Updated] (SPARK-38345) Introduce SQL function ARRAY_SIZE

[ https://issues.apache.org/jira/browse/SPARK-38345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38345: - Description: Counting elements within an array is a common use case. ARRAY_SIZE ensures the

[jira] [Created] (SPARK-38345) Introduce SQL function ARRAY_SIZE

Xinrong Meng created SPARK-38345: Summary: Introduce SQL function ARRAY_SIZE Key: SPARK-38345 URL: https://issues.apache.org/jira/browse/SPARK-38345 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-38345) Introduce SQL function ARRAY_SIZE

[ https://issues.apache.org/jira/browse/SPARK-38345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498729#comment-17498729 ] Xinrong Meng commented on SPARK-38345: -- I am working on that. > Introduce SQL function ARRAY_SIZE

[jira] [Commented] (SPARK-38344) Avoid to submit task when there are no requests to push up in push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-38344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498727#comment-17498727 ] Apache Spark commented on SPARK-38344: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Commented] (SPARK-38344) Avoid to submit task when there are no requests to push up in push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-38344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498726#comment-17498726 ] Apache Spark commented on SPARK-38344: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38344) Avoid to submit task when there are no requests to push up in push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-38344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38344: Assignee: (was: Apache Spark) > Avoid to submit task when there are no requests to

[jira] [Assigned] (SPARK-38344) Avoid to submit task when there are no requests to push up in push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-38344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38344: Assignee: Apache Spark > Avoid to submit task when there are no requests to push up in

[jira] [Assigned] (SPARK-38343) Fix SQLQuerySuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38343: Assignee: Apache Spark (was: Gengliang Wang) > Fix SQLQuerySuite under ANSI mode >

[jira] [Assigned] (SPARK-38343) Fix SQLQuerySuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38343: Assignee: Gengliang Wang (was: Apache Spark) > Fix SQLQuerySuite under ANSI mode >

[jira] [Commented] (SPARK-38343) Fix SQLQuerySuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498724#comment-17498724 ] Apache Spark commented on SPARK-38343: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38321) Fix BooleanSimplificationSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-38321: -- Assignee: Xinyi Yu > Fix BooleanSimplificationSuite under ANSI mode >

[jira] [Resolved] (SPARK-38321) Fix BooleanSimplificationSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38321. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35654

[jira] [Created] (SPARK-38344) Avoid to submit task when there are no requests to push up in push-based shuffle

weixiuli created SPARK-38344: Summary: Avoid to submit task when there are no requests to push up in push-based shuffle Key: SPARK-38344 URL: https://issues.apache.org/jira/browse/SPARK-38344 Project:

[jira] [Created] (SPARK-38343) Fix SQLQuerySuite under ANSI mode

Gengliang Wang created SPARK-38343: -- Summary: Fix SQLQuerySuite under ANSI mode Key: SPARK-38343 URL: https://issues.apache.org/jira/browse/SPARK-38343 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498712#comment-17498712 ] Yang Jie edited comment on SPARK-38341 at 2/28/22, 5:48 AM: I don't think

[jira] [Commented] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498712#comment-17498712 ] Yang Jie commented on SPARK-38341: -- I don't think it's a Spark 3.2.1 bug.   last_day('2020-06-30') is

[jira] [Assigned] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

[ https://issues.apache.org/jira/browse/SPARK-38204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38204: Assignee: (was: Apache Spark) > All state operators are at a risk of inconsistency

[jira] [Assigned] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

[ https://issues.apache.org/jira/browse/SPARK-38204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38204: Assignee: Apache Spark > All state operators are at a risk of inconsistency between

[jira] [Assigned] (SPARK-38342) Clean up deprecated api usage of Ivy

[ https://issues.apache.org/jira/browse/SPARK-38342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38342: Assignee: (was: Apache Spark) > Clean up deprecated api usage of Ivy >

[jira] [Commented] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

[ https://issues.apache.org/jira/browse/SPARK-38204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498706#comment-17498706 ] Apache Spark commented on SPARK-38204: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-38342) Clean up deprecated api usage of Ivy

[ https://issues.apache.org/jira/browse/SPARK-38342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498705#comment-17498705 ] Apache Spark commented on SPARK-38342: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-38342) Clean up deprecated api usage of Ivy

[ https://issues.apache.org/jira/browse/SPARK-38342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38342: Assignee: Apache Spark > Clean up deprecated api usage of Ivy >

[jira] [Created] (SPARK-38342) Clean up deprecated api usage of Ivy

Yang Jie created SPARK-38342: Summary: Clean up deprecated api usage of Ivy Key: SPARK-38342 URL: https://issues.apache.org/jira/browse/SPARK-38342 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-38326) aditya

[ https://issues.apache.org/jira/browse/SPARK-38326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vallepu Durga Aditya closed SPARK-38326. final > aditya > -- > > Key: SPARK-38326 > URL:

[jira] [Updated] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] davon.cao updated SPARK-38341: -- Component/s: SQL (was: Spark Submit) > Spark sql: 3.2.1 - Function of add_

[jira] [Updated] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] davon.cao updated SPARK-38341: -- Description: Step to reproduce: Version of spark sql: 3.2.1(latest  version in maven repository)

[jira] [Updated] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] davon.cao updated SPARK-38341: -- Summary: Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date (was: Spark sql -

[jira] [Updated] (SPARK-38341) Spark sql - Function of add_ Months returns an incorrect date

[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] davon.cao updated SPARK-38341: -- Description: Step to reproduce: Version of spark sql: 3.2.1(latest  version in maven repository)

[jira] [Updated] (SPARK-38331) csv parser exception when quote and escape are both double-quote and a value is just "," and column pruning enabled

[ https://issues.apache.org/jira/browse/SPARK-38331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38331: - Component/s: SQL (was: Input/Output) > csv parser exception when quote and

[jira] [Commented] (SPARK-38329) High I/O wait when Spark Structured Streaming checkpoint changed to EFS

[ https://issues.apache.org/jira/browse/SPARK-38329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498678#comment-17498678 ] Hyukjin Kwon commented on SPARK-38329: -- Spark 2.4.X is EOL. Can you test and see if the issue

[jira] [Commented] (SPARK-38327) JDBC Source with MariaDB connection returns column names as values

[ https://issues.apache.org/jira/browse/SPARK-38327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498677#comment-17498677 ] Hyukjin Kwon commented on SPARK-38327: -- I think it needs a MariaDB dialect that implements

[jira] [Resolved] (SPARK-38326) aditya

[ https://issues.apache.org/jira/browse/SPARK-38326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38326. -- Resolution: Invalid > aditya > -- > > Key: SPARK-38326 >

[jira] [Created] (SPARK-38341) Spark sql - Function of add_ Months returns an incorrect date

davon.cao created SPARK-38341: - Summary: Spark sql - Function of add_ Months returns an incorrect date Key: SPARK-38341 URL: https://issues.apache.org/jira/browse/SPARK-38341 Project: Spark

[jira] [Resolved] (SPARK-38337) Replace `toIterator` with `iterator` for `IterableLike`/`IterableOnce` to cleanup deprecated api usage

[ https://issues.apache.org/jira/browse/SPARK-38337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38337. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35665

[jira] [Assigned] (SPARK-38337) Replace `toIterator` with `iterator` for `IterableLike`/`IterableOnce` to cleanup deprecated api usage

[ https://issues.apache.org/jira/browse/SPARK-38337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38337: Assignee: Yang Jie > Replace `toIterator` with `iterator` for

[jira] [Resolved] (SPARK-38338) Remove test dependency of hamcrest

[ https://issues.apache.org/jira/browse/SPARK-38338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38338. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35666

[jira] [Assigned] (SPARK-38338) Remove test dependency of hamcrest

[ https://issues.apache.org/jira/browse/SPARK-38338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38338: Assignee: Yang Jie > Remove test dependency of hamcrest >

[jira] [Resolved] (SPARK-38339) Upgrade RoaringBitmap to 0.9.25

[ https://issues.apache.org/jira/browse/SPARK-38339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38339. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35668

[jira] [Assigned] (SPARK-38339) Upgrade RoaringBitmap to 0.9.25

[ https://issues.apache.org/jira/browse/SPARK-38339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38339: Assignee: Yang Jie > Upgrade RoaringBitmap to 0.9.25 > --- >

[jira] [Commented] (SPARK-38288) Aggregate push down doesnt work using Spark SQL jdbc datasource with postgresql

[ https://issues.apache.org/jira/browse/SPARK-38288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498669#comment-17498669 ] Daniel Fernández commented on SPARK-38288: -- [~andrewfmurphy] however in

[jira] [Commented] (SPARK-38288) Aggregate push down doesnt work using Spark SQL jdbc datasource with postgresql

[ https://issues.apache.org/jira/browse/SPARK-38288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498668#comment-17498668 ] Andrew Murphy commented on SPARK-38288: --- Hi [~llozano] I believe this is because JDBC DataSource

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498640#comment-17498640 ] Sean R. Owen commented on SPARK-25075: -- Unknown, though I'd guess not this year. What depends on

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498639#comment-17498639 ] Ismael Juma commented on SPARK-25075: - Is there a very rough timeline for 4.0 or it completely

[jira] [Comment Edited] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498639#comment-17498639 ] Ismael Juma edited comment on SPARK-25075 at 2/27/22, 5:39 PM: --- Is there a

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498622#comment-17498622 ] Sean R. Owen commented on SPARK-25075: -- I don't think that's the plan. Certainly not to remove 2.12

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498615#comment-17498615 ] Yang Jie commented on SPARK-25075: -- Do we plan to switch Scala 2.13 to the default Scala version in

[jira] [Commented] (SPARK-38112) Use error classes in the execution errors of date/timestamp handling

[ https://issues.apache.org/jira/browse/SPARK-38112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498612#comment-17498612 ] Apache Spark commented on SPARK-38112: -- User 'ivoson' has created a pull request for this issue:

[jira] [Commented] (SPARK-38112) Use error classes in the execution errors of date/timestamp handling

[ https://issues.apache.org/jira/browse/SPARK-38112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498611#comment-17498611 ] Apache Spark commented on SPARK-38112: -- User 'ivoson' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38112) Use error classes in the execution errors of date/timestamp handling

[ https://issues.apache.org/jira/browse/SPARK-38112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38112: Assignee: (was: Apache Spark) > Use error classes in the execution errors of

[jira] [Assigned] (SPARK-38112) Use error classes in the execution errors of date/timestamp handling

[ https://issues.apache.org/jira/browse/SPARK-38112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38112: Assignee: Apache Spark > Use error classes in the execution errors of date/timestamp

[jira] [Assigned] (SPARK-38041) DataFilter pushed down with PartitionFilter

[ https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38041: Assignee: (was: Apache Spark) > DataFilter pushed down with PartitionFilter >

[jira] [Commented] (SPARK-38041) DataFilter pushed down with PartitionFilter

[ https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498598#comment-17498598 ] Apache Spark commented on SPARK-38041: -- User 'stczwd' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38041) DataFilter pushed down with PartitionFilter

[ https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38041: Assignee: Apache Spark > DataFilter pushed down with PartitionFilter >

[jira] [Updated] (SPARK-38041) DataFilter pushed down with PartitionFilter

[ https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-38041: --- Description: At present, the Filter is divided into DataFilter and PartitionFilter when it is

[jira] [Updated] (SPARK-38340) Upgrade protobuf-java from 2.5.0 to 3.16.1

[ https://issues.apache.org/jira/browse/SPARK-38340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-38340: Description:

[jira] [Created] (SPARK-38340) Upgrade protobuf-java from 2.5.0 to 3.16.1

Bjørn Jørgensen created SPARK-38340: --- Summary: Upgrade protobuf-java from 2.5.0 to 3.16.1 Key: SPARK-38340 URL: https://issues.apache.org/jira/browse/SPARK-38340 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-38339) Upgrade RoaringBitmap to 0.9.25

[ https://issues.apache.org/jira/browse/SPARK-38339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38339: Assignee: (was: Apache Spark) > Upgrade RoaringBitmap to 0.9.25 >

[jira] [Assigned] (SPARK-38339) Upgrade RoaringBitmap to 0.9.25

[ https://issues.apache.org/jira/browse/SPARK-38339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38339: Assignee: Apache Spark > Upgrade RoaringBitmap to 0.9.25 >

[jira] [Commented] (SPARK-38339) Upgrade RoaringBitmap to 0.9.25

[ https://issues.apache.org/jira/browse/SPARK-38339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498577#comment-17498577 ] Apache Spark commented on SPARK-38339: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-38339) Upgrade RoaringBitmap to 0.9.25

Yang Jie created SPARK-38339: Summary: Upgrade RoaringBitmap to 0.9.25 Key: SPARK-38339 URL: https://issues.apache.org/jira/browse/SPARK-38339 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38337) Replace `toIterator` with `iterator` for `IterableLike`/`IterableOnce` to cleanup deprecated api usage

[ https://issues.apache.org/jira/browse/SPARK-38337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jj updated SPARK-38337: --- Attachment: Screenshot_20220227-050659.png > Replace `toIterator` with `iterator` for `IterableLike`/`IterableOnce`