[jira] [Assigned] (SPARK-39199) Implement pandas API missing parameters

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39199: Assignee: Xinrong Meng > Implement pandas API missing parameters >

[jira] [Assigned] (SPARK-39076) Standardize Statistical Functions of pandas API on Spark

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39076: Assignee: Xinrong Meng > Standardize Statistical Functions of pandas API on Spark >

[jira] [Updated] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39551: -- Issue Type: Bug (was: Improvement) > Add AQE invalid plan check > --

[jira] [Commented] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563568#comment-17563568 ] Dongjoon Hyun commented on SPARK-39551: --- This is backported for the bug fix patch,

[jira] [Updated] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39551: -- Fix Version/s: 3.2.2 > Add AQE invalid plan check > -- > >

[jira] [Updated] (SPARK-39705) Adding examples for 'Column' Documentation

2022-07-06 Thread William Zijie Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Zijie Zhang updated SPARK-39705: Description: # Figure out how rst files and sphinx build html docs # start going

[jira] [Created] (SPARK-39705) Adding examples for 'Column' Documentation

2022-07-06 Thread William Zijie Zhang (Jira)
William Zijie Zhang created SPARK-39705: --- Summary: Adding examples for 'Column' Documentation Key: SPARK-39705 URL: https://issues.apache.org/jira/browse/SPARK-39705 Project: Spark

[jira] [Assigned] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39704: Assignee: (was: Apache Spark) > Implement createIndex & dropIndex & IndexExists in

[jira] [Assigned] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39704: Assignee: Apache Spark > Implement createIndex & dropIndex & IndexExists in JDBC (H2

[jira] [Commented] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563542#comment-17563542 ] Apache Spark commented on SPARK-39704: -- User 'panbingkun' has created a pull request for this

[jira] [Updated] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-39704: Parent: SPARK-36525 Issue Type: Sub-task (was: Improvement) > Implement createIndex &

[jira] [Created] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-39704: --- Summary: Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect) Key: SPARK-39704 URL: https://issues.apache.org/jira/browse/SPARK-39704 Project: Spark

[jira] [Commented] (SPARK-39191) KafkaDataConsumer not support transional producer

2022-07-06 Thread SilkyAlex (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563537#comment-17563537 ] SilkyAlex commented on SPARK-39191: --- [~hyukjin.kwon] thanks for reply, but switch to 3+ is a big

[jira] [Updated] (SPARK-39622) ParquetIOSuite fails consistently on master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39622: - Priority: Major (was: Blocker) > ParquetIOSuite fails consistently on master branch >

[jira] [Commented] (SPARK-39622) ParquetIOSuite fails consistently on master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563532#comment-17563532 ] Hyukjin Kwon commented on SPARK-39622: -- Actually I think it's flaky. > ParquetIOSuite fails

[jira] [Commented] (SPARK-39622) ParquetIOSuite fails consistently on master branch

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563530#comment-17563530 ] Wenchen Fan commented on SPARK-39622: - > ParquetIOSuite fails consistently with master branch. I'm

[jira] [Commented] (SPARK-39503) Add session catalog name for v1 database table and function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563528#comment-17563528 ] Apache Spark commented on SPARK-39503: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-39503) Add session catalog name for v1 database table and function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563529#comment-17563529 ] Apache Spark commented on SPARK-39503: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39703: Assignee: (was: Apache Spark) > Mima complains with Scala 2.13 in the master branch

[jira] [Assigned] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39703: Assignee: Apache Spark > Mima complains with Scala 2.13 in the master branch >

[jira] [Commented] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563525#comment-17563525 ] Apache Spark commented on SPARK-39703: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-39702) Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563524#comment-17563524 ] Apache Spark commented on SPARK-39702: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-39703: Summary: Mima complains with Scala 2.13 in the master branch Key: SPARK-39703 URL: https://issues.apache.org/jira/browse/SPARK-39703 Project: Spark Issue

[jira] [Updated] (SPARK-39702) Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer

2022-07-06 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-39702: --- Component/s: YARN > Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel >

[jira] [Created] (SPARK-39702) Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer

2022-07-06 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-39702: -- Summary: Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer Key: SPARK-39702 URL: https://issues.apache.org/jira/browse/SPARK-39702

[jira] [Resolved] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39697. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37103

[jira] [Assigned] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39697: Assignee: Yikun Jiang > Add REFRESH_DATE flag and use previous cache to build cache

[jira] [Assigned] (SPARK-39648) Fix type hints of `like`, `rlike`, `ilike` of Column

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39648: Assignee: Xinrong Meng > Fix type hints of `like`, `rlike`, `ilike` of Column >

[jira] [Resolved] (SPARK-39648) Fix type hints of `like`, `rlike`, `ilike` of Column

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39648. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37038

[jira] [Commented] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563498#comment-17563498 ] Apache Spark commented on SPARK-39551: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39701. --- Fix Version/s: 3.4.0 Assignee: Dongjoon Hyun Resolution: Fixed This is

[jira] [Updated] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39701: -- Affects Version/s: (was: 3.3.0) > Move withSecretFile to SparkFunSuite to reuse >

[jira] [Updated] (SPARK-38953) Document PySpark common exceptions / errors

2022-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38953: - Description: There are common exceptions/errors in PySpark SQL, pandas API on Spark, and Py4J.

[jira] [Updated] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39701: -- Affects Version/s: 3.3.0 > Move withSecretFile to SparkFunSuite to reuse >

[jira] [Commented] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563437#comment-17563437 ] Apache Spark commented on SPARK-39689: -- User 'Yaohua628' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39689: Assignee: (was: Apache Spark) > Support 2-chars lineSep in CSV datasource >

[jira] [Assigned] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39689: Assignee: Apache Spark > Support 2-chars lineSep in CSV datasource >

[jira] [Updated] (SPARK-37865) Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns

2022-07-06 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-37865: --- Labels: correctness (was: ) > Spark should not dedup the groupingExpressions when the first child

[jira] [Commented] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563387#comment-17563387 ] Apache Spark commented on SPARK-39701: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39701: Assignee: (was: Apache Spark) > Move withSecretFile to SparkFunSuite to reuse >

[jira] [Assigned] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39701: Assignee: Apache Spark > Move withSecretFile to SparkFunSuite to reuse >

[jira] [Commented] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563386#comment-17563386 ] Apache Spark commented on SPARK-39701: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563385#comment-17563385 ] Apache Spark commented on SPARK-39700: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39700: Assignee: Apache Spark > Deprecate API that has parameters (DBName,

[jira] [Assigned] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39700: Assignee: (was: Apache Spark) > Deprecate API that has parameters (DBName,

[jira] [Created] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-39701: - Summary: Move withSecretFile to SparkFunSuite to reuse Key: SPARK-39701 URL: https://issues.apache.org/jira/browse/SPARK-39701 Project: Spark Issue Type:

[jira] [Created] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Rui Wang (Jira)
Rui Wang created SPARK-39700: Summary: Deprecate API that has parameters (DBName, tableName/FunctionName) Key: SPARK-39700 URL: https://issues.apache.org/jira/browse/SPARK-39700 Project: Spark

[jira] [Resolved] (SPARK-39076) Standardize Statistical Functions of pandas API on Spark

2022-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-39076. -- Resolution: Done > Standardize Statistical Functions of pandas API on Spark >

[jira] [Updated] (SPARK-39227) Reach parity with pandas boolean cast

2022-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39227: - Parent: (was: SPARK-39076) Issue Type: Improvement (was: Sub-task) > Reach parity

[jira] [Commented] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563334#comment-17563334 ] Apache Spark commented on SPARK-37287: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-39663) Miss UT for MysqlDialect's listIndexes

2022-07-06 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-39663: Assignee: BingKun Pan > Miss UT for MysqlDialect's listIndexes >

[jira] [Resolved] (SPARK-39663) Miss UT for MysqlDialect's listIndexes

2022-07-06 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39663. -- Resolution: Fixed Issue resolved by pull request 37060

[jira] [Resolved] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39691. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37100

[jira] [Assigned] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39691: - Assignee: Yang Jie > Supplement `MapStatusesConvertBenchmark` result generated by Java

[jira] [Created] (SPARK-39699) Make CollapseProject smarter about collection creation expressions

2022-07-06 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-39699: --- Summary: Make CollapseProject smarter about collection creation expressions Key: SPARK-39699 URL: https://issues.apache.org/jira/browse/SPARK-39699 Project: Spark

[jira] [Commented] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563283#comment-17563283 ] Apache Spark commented on SPARK-39698: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39698: Assignee: (was: Apache Spark) > Use TakeOrderedAndProject if maxRows below the

[jira] [Assigned] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39698: Assignee: Apache Spark > Use TakeOrderedAndProject if maxRows below the

[jira] [Created] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39698: --- Summary: Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold Key: SPARK-39698 URL: https://issues.apache.org/jira/browse/SPARK-39698 Project:

[jira] [Commented] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563271#comment-17563271 ] Apache Spark commented on SPARK-39697: -- User 'Yikun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39697: Assignee: (was: Apache Spark) > Add REFRESH_DATE flag and use previous cache to

[jira] [Assigned] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39697: Assignee: Apache Spark > Add REFRESH_DATE flag and use previous cache to build cache

[jira] [Commented] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563270#comment-17563270 ] Apache Spark commented on SPARK-39697: -- User 'Yikun' has created a pull request for this issue:

[jira] [Created] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-39697: --- Summary: Add REFRESH_DATE flag and use previous cache to build cache image Key: SPARK-39697 URL: https://issues.apache.org/jira/browse/SPARK-39697 Project: Spark

[jira] [Updated] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

2022-07-06 Thread Stephen Mcmullan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Mcmullan updated SPARK-39696: - Summary: Uncaught exception in thread executor-heartbeater

[jira] [Created] (SPARK-39696) Uncaught exception in thread executor-heartbeaterjava.util.ConcurrentModificationException: mutation occurred during iteration

2022-07-06 Thread Stephen Mcmullan (Jira)
Stephen Mcmullan created SPARK-39696: Summary: Uncaught exception in thread executor-heartbeaterjava.util.ConcurrentModificationException: mutation occurred during iteration Key: SPARK-39696 URL:

[jira] [Commented] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563184#comment-17563184 ] Apache Spark commented on SPARK-39694: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563182#comment-17563182 ] Apache Spark commented on SPARK-39694: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39694: Assignee: (was: Apache Spark) > Update `${sbtProject}/test:runMain` to

[jira] [Assigned] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39694: Assignee: Apache Spark > Update `${sbtProject}/test:runMain` to

[jira] [Commented] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563178#comment-17563178 ] Apache Spark commented on SPARK-39695: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563179#comment-17563179 ] Apache Spark commented on SPARK-39695: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39695: Assignee: Apache Spark (was: Max Gekk) > Add the REGEXP_SUBSTR function >

[jira] [Assigned] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39695: Assignee: Max Gekk (was: Apache Spark) > Add the REGEXP_SUBSTR function >

[jira] [Updated] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39695: - Description: The function should search a string for a regular expression pattern and returns it or

[jira] [Created] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Max Gekk (Jira)
Max Gekk created SPARK-39695: Summary: Add the REGEXP_SUBSTR function Key: SPARK-39695 URL: https://issues.apache.org/jira/browse/SPARK-39695 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39694: - Summary: Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain` (was: Update

[jira] [Updated] (SPARK-39694) Update `${moduleName}/test:runMain` to `${moduleName}/Test/runMain`

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39694: - Summary: Update `${moduleName}/test:runMain` to `${moduleName}/Test/runMain` (was: Update scaladoc to

[jira] [Created] (SPARK-39694) Update scaladoc to run mirco-benchmark using the new sbt shell syntax

2022-07-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-39694: Summary: Update scaladoc to run mirco-benchmark using the new sbt shell syntax Key: SPARK-39694 URL: https://issues.apache.org/jira/browse/SPARK-39694 Project: Spark

[jira] [Resolved] (SPARK-39584) Fix TPCDSQueryBenchmark Measuring Performance of Wrong Query Results

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39584. - Fix Version/s: 3.4.0 Assignee: Kazuyuki Tanimura Resolution: Fixed > Fix

[jira] [Updated] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39693: - Description: I found that `{{{}Generate an input dataset for TPCDSQueryBenchmark with SF=1`

[jira] [Updated] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39693: - Attachment: image-2022-07-06-17-08-56-728.png > `tpcds-1g-gen` shouldn't execute If benchmark GA does

[jira] [Created] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-39693: Summary: `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark Key: SPARK-39693 URL: https://issues.apache.org/jira/browse/SPARK-39693

[jira] [Updated] (SPARK-39683) Did not find value which can be converted into java.lang.String

2022-07-06 Thread zehra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zehra updated SPARK-39683: -- Language: python spark (was: python) > Did not find value which can be converted into java.lang.String >

[jira] [Assigned] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39691: Assignee: (was: Apache Spark) > Supplement `MapStatusesConvertBenchmark` result

[jira] [Assigned] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39691: Assignee: Apache Spark > Supplement `MapStatusesConvertBenchmark` result generated by

[jira] [Commented] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563033#comment-17563033 ] Apache Spark commented on SPARK-39691: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17563032#comment-17563032 ] Apache Spark commented on SPARK-39691: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-39692) Avoid misleading vector example in ML doc

2022-07-06 Thread thierry accart (Jira)
thierry accart created SPARK-39692: -- Summary: Avoid misleading vector example in ML doc Key: SPARK-39692 URL: https://issues.apache.org/jira/browse/SPARK-39692 Project: Spark Issue Type:

[jira] [Created] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-39691: Summary: Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17 Key: SPARK-39691 URL: https://issues.apache.org/jira/browse/SPARK-39691 Project: Spark

[jira] [Commented] (SPARK-39690) Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562984#comment-17562984 ] Apache Spark commented on SPARK-39690: -- User 'mskapilks' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39690) Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39690: Assignee: (was: Apache Spark) > Reuse exchange across subqueries is broken with AQE

[jira] [Commented] (SPARK-39690) Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562983#comment-17562983 ] Apache Spark commented on SPARK-39690: -- User 'mskapilks' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39690) Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39690: Assignee: Apache Spark > Reuse exchange across subqueries is broken with AQE if subquery

[jira] [Updated] (SPARK-39690) Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first

2022-07-06 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-39690: Description: When trying to reuse Exchange of a subquery in main plan, if the Exchange inside