[jira] [Created] (SPARK-39706) Set missing column with defaultValue as constant in `ParquetColumnVector`

2022-07-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-39706: Summary: Set missing column with defaultValue as constant in `ParquetColumnVector` Key: SPARK-39706 URL: https://issues.apache.org/jira/browse/SPARK-39706 Project: Spark

[jira] [Resolved] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39689. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37107 [https://gi

[jira] [Assigned] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39689: Assignee: Yaohua Cui > Support 2-chars lineSep in CSV datasource > --

[jira] [Assigned] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39703: Assignee: Hyukjin Kwon > Mima complains with Scala 2.13 in the master branch > --

[jira] [Resolved] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39703. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37109 [https://gi

[jira] [Assigned] (SPARK-39679) TakeOrderedAndProjectExec should respect child output ordering

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39679: --- Assignee: XiDuo You > TakeOrderedAndProjectExec should respect child output ordering >

[jira] [Resolved] (SPARK-39679) TakeOrderedAndProjectExec should respect child output ordering

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39679. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37085 [https://gith

[jira] [Assigned] (SPARK-39199) Implement pandas API missing parameters

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39199: Assignee: Xinrong Meng > Implement pandas API missing parameters > --

[jira] [Assigned] (SPARK-39076) Standardize Statistical Functions of pandas API on Spark

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39076: Assignee: Xinrong Meng > Standardize Statistical Functions of pandas API on Spark > -

[jira] [Updated] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39551: -- Issue Type: Bug (was: Improvement) > Add AQE invalid plan check > --

[jira] [Commented] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563568#comment-17563568 ] Dongjoon Hyun commented on SPARK-39551: --- This is backported for the bug fix patch,

[jira] [Updated] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39551: -- Fix Version/s: 3.2.2 > Add AQE invalid plan check > -- > >

[jira] [Updated] (SPARK-39705) Adding examples for 'Column' Documentation

2022-07-06 Thread William Zijie Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Zijie Zhang updated SPARK-39705: Description: # Figure out how rst files and sphinx build html docs # start going

[jira] [Created] (SPARK-39705) Adding examples for 'Column' Documentation

2022-07-06 Thread William Zijie Zhang (Jira)
William Zijie Zhang created SPARK-39705: --- Summary: Adding examples for 'Column' Documentation Key: SPARK-39705 URL: https://issues.apache.org/jira/browse/SPARK-39705 Project: Spark Issu

[jira] [Assigned] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39704: Assignee: (was: Apache Spark) > Implement createIndex & dropIndex & IndexExists in JD

[jira] [Assigned] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39704: Assignee: Apache Spark > Implement createIndex & dropIndex & IndexExists in JDBC (H2 dial

[jira] [Commented] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563542#comment-17563542 ] Apache Spark commented on SPARK-39704: -- User 'panbingkun' has created a pull reques

[jira] [Updated] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-39704: Parent: SPARK-36525 Issue Type: Sub-task (was: Improvement) > Implement createIndex & dro

[jira] [Created] (SPARK-39704) Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect)

2022-07-06 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-39704: --- Summary: Implement createIndex & dropIndex & IndexExists in JDBC (H2 dialect) Key: SPARK-39704 URL: https://issues.apache.org/jira/browse/SPARK-39704 Project: Spark

[jira] [Commented] (SPARK-39191) KafkaDataConsumer not support transional producer

2022-07-06 Thread SilkyAlex (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563537#comment-17563537 ] SilkyAlex commented on SPARK-39191: --- [~hyukjin.kwon] thanks for reply, but switch to 3

[jira] [Updated] (SPARK-39622) ParquetIOSuite fails consistently on master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39622: - Priority: Major (was: Blocker) > ParquetIOSuite fails consistently on master branch > -

[jira] [Commented] (SPARK-39622) ParquetIOSuite fails consistently on master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563532#comment-17563532 ] Hyukjin Kwon commented on SPARK-39622: -- Actually I think it's flaky. > ParquetIOSu

[jira] [Commented] (SPARK-39622) ParquetIOSuite fails consistently on master branch

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563530#comment-17563530 ] Wenchen Fan commented on SPARK-39622: - > ParquetIOSuite fails consistently with mast

[jira] [Commented] (SPARK-39503) Add session catalog name for v1 database table and function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563528#comment-17563528 ] Apache Spark commented on SPARK-39503: -- User 'ulysses-you' has created a pull reque

[jira] [Commented] (SPARK-39503) Add session catalog name for v1 database table and function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563529#comment-17563529 ] Apache Spark commented on SPARK-39503: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39703: Assignee: (was: Apache Spark) > Mima complains with Scala 2.13 in the master branch >

[jira] [Assigned] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39703: Assignee: Apache Spark > Mima complains with Scala 2.13 in the master branch > --

[jira] [Commented] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563525#comment-17563525 ] Apache Spark commented on SPARK-39703: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-39702) Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563524#comment-17563524 ] Apache Spark commented on SPARK-39702: -- User 'JoshRosen' has created a pull request

[jira] [Created] (SPARK-39703) Mima complains with Scala 2.13 in the master branch

2022-07-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-39703: Summary: Mima complains with Scala 2.13 in the master branch Key: SPARK-39703 URL: https://issues.apache.org/jira/browse/SPARK-39703 Project: Spark Issue Typ

[jira] [Updated] (SPARK-39702) Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer

2022-07-06 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-39702: --- Component/s: YARN > Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel > b

[jira] [Created] (SPARK-39702) Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer

2022-07-06 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-39702: -- Summary: Reduce memory overhead of TransportCipher$EncryptedMessage's byteRawChannel buffer Key: SPARK-39702 URL: https://issues.apache.org/jira/browse/SPARK-39702 Projec

[jira] [Resolved] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39697. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37103 [https://gi

[jira] [Assigned] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39697: Assignee: Yikun Jiang > Add REFRESH_DATE flag and use previous cache to build cache image

[jira] [Assigned] (SPARK-39648) Fix type hints of `like`, `rlike`, `ilike` of Column

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39648: Assignee: Xinrong Meng > Fix type hints of `like`, `rlike`, `ilike` of Column > -

[jira] [Resolved] (SPARK-39648) Fix type hints of `like`, `rlike`, `ilike` of Column

2022-07-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39648. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37038 [https://gi

[jira] [Commented] (SPARK-39551) Add AQE invalid plan check

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563498#comment-17563498 ] Apache Spark commented on SPARK-39551: -- User 'dongjoon-hyun' has created a pull req

[jira] [Resolved] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39701. --- Fix Version/s: 3.4.0 Assignee: Dongjoon Hyun Resolution: Fixed This is resol

[jira] [Updated] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39701: -- Affects Version/s: (was: 3.3.0) > Move withSecretFile to SparkFunSuite to reuse >

[jira] [Updated] (SPARK-38953) Document PySpark common exceptions / errors

2022-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38953: - Description: There are common exceptions/errors in PySpark SQL, pandas API on Spark, and Py4J.

[jira] [Updated] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39701: -- Affects Version/s: 3.3.0 > Move withSecretFile to SparkFunSuite to reuse > ---

[jira] [Commented] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563437#comment-17563437 ] Apache Spark commented on SPARK-39689: -- User 'Yaohua628' has created a pull request

[jira] [Assigned] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39689: Assignee: (was: Apache Spark) > Support 2-chars lineSep in CSV datasource > -

[jira] [Assigned] (SPARK-39689) Support 2-chars lineSep in CSV datasource

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39689: Assignee: Apache Spark > Support 2-chars lineSep in CSV datasource >

[jira] [Updated] (SPARK-37865) Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns

2022-07-06 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-37865: --- Labels: correctness (was: ) > Spark should not dedup the groupingExpressions when the first child o

[jira] [Commented] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563387#comment-17563387 ] Apache Spark commented on SPARK-39701: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39701: Assignee: (was: Apache Spark) > Move withSecretFile to SparkFunSuite to reuse > -

[jira] [Assigned] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39701: Assignee: Apache Spark > Move withSecretFile to SparkFunSuite to reuse >

[jira] [Commented] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563386#comment-17563386 ] Apache Spark commented on SPARK-39701: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563385#comment-17563385 ] Apache Spark commented on SPARK-39700: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39700: Assignee: Apache Spark > Deprecate API that has parameters (DBName, tableName/FunctionNam

[jira] [Assigned] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39700: Assignee: (was: Apache Spark) > Deprecate API that has parameters (DBName, tableName/

[jira] [Created] (SPARK-39701) Move withSecretFile to SparkFunSuite to reuse

2022-07-06 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-39701: - Summary: Move withSecretFile to SparkFunSuite to reuse Key: SPARK-39701 URL: https://issues.apache.org/jira/browse/SPARK-39701 Project: Spark Issue Type: T

[jira] [Created] (SPARK-39700) Deprecate API that has parameters (DBName, tableName/FunctionName)

2022-07-06 Thread Rui Wang (Jira)
Rui Wang created SPARK-39700: Summary: Deprecate API that has parameters (DBName, tableName/FunctionName) Key: SPARK-39700 URL: https://issues.apache.org/jira/browse/SPARK-39700 Project: Spark I

[jira] [Resolved] (SPARK-39076) Standardize Statistical Functions of pandas API on Spark

2022-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-39076. -- Resolution: Done > Standardize Statistical Functions of pandas API on Spark >

[jira] [Updated] (SPARK-39227) Reach parity with pandas boolean cast

2022-07-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-39227: - Parent: (was: SPARK-39076) Issue Type: Improvement (was: Sub-task) > Reach parity w

[jira] [Commented] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563334#comment-17563334 ] Apache Spark commented on SPARK-37287: -- User 'allisonwang-db' has created a pull re

[jira] [Assigned] (SPARK-39663) Miss UT for MysqlDialect's listIndexes

2022-07-06 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-39663: Assignee: BingKun Pan > Miss UT for MysqlDialect's listIndexes >

[jira] [Resolved] (SPARK-39663) Miss UT for MysqlDialect's listIndexes

2022-07-06 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39663. -- Resolution: Fixed Issue resolved by pull request 37060 [https://github.com/apache/spark/pull/3

[jira] [Resolved] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39691. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37100 [https://

[jira] [Assigned] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39691: - Assignee: Yang Jie > Supplement `MapStatusesConvertBenchmark` result generated by Java

[jira] [Created] (SPARK-39699) Make CollapseProject smarter about collection creation expressions

2022-07-06 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-39699: --- Summary: Make CollapseProject smarter about collection creation expressions Key: SPARK-39699 URL: https://issues.apache.org/jira/browse/SPARK-39699 Project: Spark

[jira] [Commented] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563283#comment-17563283 ] Apache Spark commented on SPARK-39698: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39698: Assignee: (was: Apache Spark) > Use TakeOrderedAndProject if maxRows below the topKSo

[jira] [Assigned] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39698: Assignee: Apache Spark > Use TakeOrderedAndProject if maxRows below the topKSortFallbackT

[jira] [Created] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39698: --- Summary: Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold Key: SPARK-39698 URL: https://issues.apache.org/jira/browse/SPARK-39698 Project: Spark

[jira] [Commented] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563271#comment-17563271 ] Apache Spark commented on SPARK-39697: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39697: Assignee: (was: Apache Spark) > Add REFRESH_DATE flag and use previous cache to build

[jira] [Assigned] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39697: Assignee: Apache Spark > Add REFRESH_DATE flag and use previous cache to build cache imag

[jira] [Commented] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563270#comment-17563270 ] Apache Spark commented on SPARK-39697: -- User 'Yikun' has created a pull request for

[jira] [Created] (SPARK-39697) Add REFRESH_DATE flag and use previous cache to build cache image

2022-07-06 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-39697: --- Summary: Add REFRESH_DATE flag and use previous cache to build cache image Key: SPARK-39697 URL: https://issues.apache.org/jira/browse/SPARK-39697 Project: Spark

[jira] [Updated] (SPARK-39696) Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificationException: mutation occurred during iteration

2022-07-06 Thread Stephen Mcmullan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Mcmullan updated SPARK-39696: - Summary: Uncaught exception in thread executor-heartbeater java.util.ConcurrentModificat

[jira] [Created] (SPARK-39696) Uncaught exception in thread executor-heartbeaterjava.util.ConcurrentModificationException: mutation occurred during iteration

2022-07-06 Thread Stephen Mcmullan (Jira)
Stephen Mcmullan created SPARK-39696: Summary: Uncaught exception in thread executor-heartbeaterjava.util.ConcurrentModificationException: mutation occurred during iteration Key: SPARK-39696 URL: https://issu

[jira] [Commented] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563184#comment-17563184 ] Apache Spark commented on SPARK-39694: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563182#comment-17563182 ] Apache Spark commented on SPARK-39694: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39694: Assignee: (was: Apache Spark) > Update `${sbtProject}/test:runMain` to `${sbtProject}

[jira] [Assigned] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39694: Assignee: Apache Spark > Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMa

[jira] [Commented] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563179#comment-17563179 ] Apache Spark commented on SPARK-39695: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563178#comment-17563178 ] Apache Spark commented on SPARK-39695: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39695: Assignee: Apache Spark (was: Max Gekk) > Add the REGEXP_SUBSTR function > --

[jira] [Assigned] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39695: Assignee: Max Gekk (was: Apache Spark) > Add the REGEXP_SUBSTR function > --

[jira] [Updated] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39695: - Description: The function should search a string for a regular expression pattern and returns it or NUL

[jira] [Created] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-06 Thread Max Gekk (Jira)
Max Gekk created SPARK-39695: Summary: Add the REGEXP_SUBSTR function Key: SPARK-39695 URL: https://issues.apache.org/jira/browse/SPARK-39695 Project: Spark Issue Type: New Feature Comp

[jira] [Updated] (SPARK-39694) Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain`

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39694: - Summary: Update `${sbtProject}/test:runMain` to `${sbtProject}/Test/runMain` (was: Update `${moduleName

[jira] [Updated] (SPARK-39694) Update `${moduleName}/test:runMain` to `${moduleName}/Test/runMain`

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39694: - Summary: Update `${moduleName}/test:runMain` to `${moduleName}/Test/runMain` (was: Update scaladoc to r

[jira] [Created] (SPARK-39694) Update scaladoc to run mirco-benchmark using the new sbt shell syntax

2022-07-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-39694: Summary: Update scaladoc to run mirco-benchmark using the new sbt shell syntax Key: SPARK-39694 URL: https://issues.apache.org/jira/browse/SPARK-39694 Project: Spark

[jira] [Resolved] (SPARK-39584) Fix TPCDSQueryBenchmark Measuring Performance of Wrong Query Results

2022-07-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39584. - Fix Version/s: 3.4.0 Assignee: Kazuyuki Tanimura Resolution: Fixed > Fix TPCDSQu

[jira] [Updated] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39693: - Description: I found that `{{{}Generate an input dataset for TPCDSQueryBenchmark with SF=1` {{alway

[jira] [Updated] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-39693: - Attachment: image-2022-07-06-17-08-56-728.png > `tpcds-1g-gen` shouldn't execute If benchmark GA does no

[jira] [Created] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-39693: Summary: `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark Key: SPARK-39693 URL: https://issues.apache.org/jira/browse/SPARK-39693

[jira] [Updated] (SPARK-39683) Did not find value which can be converted into java.lang.String

2022-07-06 Thread zehra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zehra updated SPARK-39683: -- Language: python spark (was: python) > Did not find value which can be converted into java.lang.String >

[jira] [Assigned] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39691: Assignee: (was: Apache Spark) > Supplement `MapStatusesConvertBenchmark` result gener

[jira] [Assigned] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39691: Assignee: Apache Spark > Supplement `MapStatusesConvertBenchmark` result generated by Jav

[jira] [Commented] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563033#comment-17563033 ] Apache Spark commented on SPARK-39691: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563032#comment-17563032 ] Apache Spark commented on SPARK-39691: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-39692) Avoid misleading vector example in ML doc

2022-07-06 Thread thierry accart (Jira)
thierry accart created SPARK-39692: -- Summary: Avoid misleading vector example in ML doc Key: SPARK-39692 URL: https://issues.apache.org/jira/browse/SPARK-39692 Project: Spark Issue Type: Imp

[jira] [Created] (SPARK-39691) Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17

2022-07-06 Thread Yang Jie (Jira)
Yang Jie created SPARK-39691: Summary: Supplement `MapStatusesConvertBenchmark` result generated by Java 11 and 17 Key: SPARK-39691 URL: https://issues.apache.org/jira/browse/SPARK-39691 Project: Spark