[jira] [Commented] (SPARK-36667) Close resources properly in StateStoreSuite/RocksDBStateStoreSuite

2021-09-03 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409779#comment-17409779 ] Jungtaek Lim commented on SPARK-36667: -- Will submit a PR soon. > Close resources properly in

[jira] [Created] (SPARK-36667) Close resources properly in StateStoreSuite/RocksDBStateStoreSuite

2021-09-03 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-36667: Summary: Close resources properly in StateStoreSuite/RocksDBStateStoreSuite Key: SPARK-36667 URL: https://issues.apache.org/jira/browse/SPARK-36667 Project: Spark

[jira] [Updated] (SPARK-36666) [SQL] Regression in AQEShuffleReadExec

2021-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3: -- Priority: Blocker (was: Major) > [SQL] Regression in AQEShuffleReadExec >

[jira] [Assigned] (SPARK-36666) [SQL] Regression in AQEShuffleReadExec

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3: Assignee: Apache Spark > [SQL] Regression in AQEShuffleReadExec >

[jira] [Commented] (SPARK-36666) [SQL] Regression in AQEShuffleReadExec

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409761#comment-17409761 ] Apache Spark commented on SPARK-3: -- User 'andygrove' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36666) [SQL] Regression in AQEShuffleReadExec

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3: Assignee: (was: Apache Spark) > [SQL] Regression in AQEShuffleReadExec >

[jira] [Created] (SPARK-36666) [SQL] Regression in AQEShuffleReadExec

2021-09-03 Thread Andy Grove (Jira)
Andy Grove created SPARK-3: -- Summary: [SQL] Regression in AQEShuffleReadExec Key: SPARK-3 URL: https://issues.apache.org/jira/browse/SPARK-3 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36666) [SQL] Regression in AQEShuffleReadExec

2021-09-03 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-3: --- Description: I am currently testing the RAPIDS Accelerator for Apache Spark with the Spark 3.2

[jira] [Commented] (SPARK-36665) Add more Not operator optimizations

2021-09-03 Thread Kazuyuki Tanimura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409746#comment-17409746 ] Kazuyuki Tanimura commented on SPARK-36665: --- I am working on this > Add more Not operator

[jira] [Created] (SPARK-36665) Add more Not operator optimizations

2021-09-03 Thread Kazuyuki Tanimura (Jira)
Kazuyuki Tanimura created SPARK-36665: - Summary: Add more Not operator optimizations Key: SPARK-36665 URL: https://issues.apache.org/jira/browse/SPARK-36665 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36655) Add `versionadded` for API added in Spark 3.3.0

2021-09-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36655. --- Fix Version/s: 3.3.0 Assignee: Xinrong Meng Resolution: Fixed Issue

[jira] [Resolved] (SPARK-36401) Implement Series.cov

2021-09-03 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36401. --- Fix Version/s: 3.3.0 Assignee: dgd_contributor Resolution: Fixed Issue

[jira] [Commented] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409634#comment-17409634 ] Dongjoon Hyun commented on SPARK-36659: --- Although RC2 will fail, I set the fixed version with

[jira] [Updated] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36659: -- Fix Version/s: (was: 3.3.0) 3.2.1 > Promote

[jira] [Updated] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36659: -- Fix Version/s: (was: 3.2.0) 3.3.0 > Promote

[jira] [Updated] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36659: -- Fix Version/s: (was: 3.3.0) 3.2.0 > Promote

[jira] [Commented] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409620#comment-17409620 ] Apache Spark commented on SPARK-36639: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409619#comment-17409619 ] Apache Spark commented on SPARK-36639: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-36664) Log time spent waiting for cluster resources

2021-09-03 Thread Holden Karau (Jira)
Holden Karau created SPARK-36664: Summary: Log time spent waiting for cluster resources Key: SPARK-36664 URL: https://issues.apache.org/jira/browse/SPARK-36664 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-09-03 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409617#comment-17409617 ] pralabhkumar commented on SPARK-36622: -- [~thejdeep]  Its better to have _HOST , its been common

[jira] [Commented] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409539#comment-17409539 ] Kousuke Saruta commented on SPARK-36639: Issue resolved in

[jira] [Resolved] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36639. Assignee: Kousuke Saruta Resolution: Fixed > SQL sequence function with interval

[jira] [Updated] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36639: --- Fix Version/s: 3.1.3 3.2.0 > SQL sequence function with interval returns

[jira] [Commented] (SPARK-36663) When the existing field name is a number, an error will be reported when reading the orc file

2021-09-03 Thread mcdull_zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409492#comment-17409492 ] mcdull_zhang commented on SPARK-36663: -- cc  [~hyukjin.kwon]      [~cloud_fan] > When the existing

[jira] [Updated] (SPARK-36663) When the existing field name is a number, an error will be reported when reading the orc file

2021-09-03 Thread mcdull_zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mcdull_zhang updated SPARK-36663: - Description: You can use the following methods to reproduce the problem: {quote}val path =

[jira] [Updated] (SPARK-36663) When the existing field name is a number, an error will be reported when reading the orc file

2021-09-03 Thread mcdull_zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mcdull_zhang updated SPARK-36663: - Attachment: image-2021-09-03-20-56-28-846.png > When the existing field name is a number, an

[jira] [Created] (SPARK-36663) When the existing field name is a number, an error will be reported when reading the orc file

2021-09-03 Thread mcdull_zhang (Jira)
mcdull_zhang created SPARK-36663: Summary: When the existing field name is a number, an error will be reported when reading the orc file Key: SPARK-36663 URL: https://issues.apache.org/jira/browse/SPARK-36663

[jira] [Resolved] (SPARK-36609) Add `errors` argument for `ps.to_numeric`.

2021-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36609. -- Fix Version/s: 3.3.0 Assignee: Haejoon Lee Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-36659. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33904

[jira] [Assigned] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-03 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-36659: Assignee: Kent Yao > Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

[jira] [Assigned] (SPARK-36661) Support TimestampNTZ in Py4J

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36661: Assignee: (was: Apache Spark) > Support TimestampNTZ in Py4J >

[jira] [Commented] (SPARK-36661) Support TimestampNTZ in Py4J

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409447#comment-17409447 ] Apache Spark commented on SPARK-36661: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36661) Support TimestampNTZ in Py4J

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36661: Assignee: Apache Spark > Support TimestampNTZ in Py4J > > >

[jira] [Commented] (SPARK-26208) Empty dataframe does not roundtrip for csv with header

2021-09-03 Thread Ranga Reddy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409427#comment-17409427 ] Ranga Reddy commented on SPARK-26208: - cc [~hyukjin.kwon] > Empty dataframe does not roundtrip for

[jira] [Assigned] (SPARK-36662) special timestamps values support for path filters - modifiedBefore/modifiedAfter

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36662: Assignee: Apache Spark > special timestamps values support for path filters - >

[jira] [Commented] (SPARK-36662) special timestamps values support for path filters - modifiedBefore/modifiedAfter

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409356#comment-17409356 ] Apache Spark commented on SPARK-36662: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36662) special timestamps values support for path filters - modifiedBefore/modifiedAfter

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36662: Assignee: (was: Apache Spark) > special timestamps values support for path filters -

[jira] [Resolved] (SPARK-36644) Push down boolean column filter

2021-09-03 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-36644. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33898

[jira] [Assigned] (SPARK-36644) Push down boolean column filter

2021-09-03 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-36644: --- Assignee: Kazuyuki Tanimura > Push down boolean column filter > --- >

[jira] [Created] (SPARK-36662) special timestamps values support for path filters - modifiedBefore/modifiedAfter

2021-09-03 Thread Kent Yao (Jira)
Kent Yao created SPARK-36662: Summary: special timestamps values support for path filters - modifiedBefore/modifiedAfter Key: SPARK-36662 URL: https://issues.apache.org/jira/browse/SPARK-36662 Project:

[jira] [Assigned] (SPARK-36610) Add `thousands` argument to `ps.read_csv`.

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36610: Assignee: Apache Spark > Add `thousands` argument to `ps.read_csv`. >

[jira] [Assigned] (SPARK-36610) Add `thousands` argument to `ps.read_csv`.

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36610: Assignee: (was: Apache Spark) > Add `thousands` argument to `ps.read_csv`. >

[jira] [Commented] (SPARK-36610) Add `thousands` argument to `ps.read_csv`.

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409328#comment-17409328 ] Apache Spark commented on SPARK-36610: -- User 'itholic' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36633) DivideDTInterval should throw the same exception when divide by zero.

2021-09-03 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng resolved SPARK-36633. Resolution: Duplicate > DivideDTInterval should throw the same exception when divide by zero. >

[jira] [Commented] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409287#comment-17409287 ] Apache Spark commented on SPARK-36660: -- User 'yutoacts' has created a pull request for this issue:

[jira] [Updated] (SPARK-36632) DivideYMInterval and DivideDTInterval should throw the same exception when divide by zero.

2021-09-03 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36632: --- Summary: DivideYMInterval and DivideDTInterval should throw the same exception when divide by zero.

[jira] [Assigned] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36660: Assignee: (was: Apache Spark) > Cotangent is not supported by Dataframe >

[jira] [Assigned] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36660: Assignee: Apache Spark > Cotangent is not supported by Dataframe >

[jira] [Commented] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409288#comment-17409288 ] Apache Spark commented on SPARK-36660: -- User 'yutoacts' has created a pull request for this issue: