[jira] [Commented] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

2021-09-15 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415891#comment-17415891 ] Mridul Muralidharan commented on SPARK-36772: - Thanks [~Gengliang.Wang] ! And thanks for

[jira] [Commented] (SPARK-36587) Migrate CreateNamespaceStatement to v2 command framework

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415884#comment-17415884 ] Apache Spark commented on SPARK-36587: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36775) Add documentation for ANSI store assignment rules

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36775: Assignee: Apache Spark (was: Gengliang Wang) > Add documentation for ANSI store

[jira] [Commented] (SPARK-36775) Add documentation for ANSI store assignment rules

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415883#comment-17415883 ] Apache Spark commented on SPARK-36775: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-36775) Add documentation for ANSI store assignment rules

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36775: Assignee: Gengliang Wang (was: Apache Spark) > Add documentation for ANSI store

[jira] [Commented] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

2021-09-15 Thread Ye Zhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415879#comment-17415879 ] Ye Zhou commented on SPARK-36772: - I will work on this one and post PR ASAP. > FinalizeShuffleMerge

[jira] [Created] (SPARK-36775) Add documentation for ANSI store assignment rules

2021-09-15 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-36775: -- Summary: Add documentation for ANSI store assignment rules Key: SPARK-36775 URL: https://issues.apache.org/jira/browse/SPARK-36775 Project: Spark Issue

[jira] [Assigned] (SPARK-36745) ExtractEquiJoinKeys should return the original predicates on join keys

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36745: --- Assignee: Yannis Sismanis > ExtractEquiJoinKeys should return the original predicates on

[jira] [Resolved] (SPARK-36745) Cleanup pattern ExtractEquiJoinKeys

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36745. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33985

[jira] [Updated] (SPARK-36745) ExtractEquiJoinKeys should return the original predicates on join keys

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36745: Summary: ExtractEquiJoinKeys should return the original predicates on join keys (was: Cleanup

[jira] [Commented] (SPARK-36774) Use SparkSubmitTestUtils to core and use it in SparkSubmitSuite

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415867#comment-17415867 ] Apache Spark commented on SPARK-36774: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-36774) Use SparkSubmitTestUtils to core and use it in SparkSubmitSuite

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415869#comment-17415869 ] Apache Spark commented on SPARK-36774: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Updated] (SPARK-36755) array_overlap should handle Double.NaN and Float.NaN

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36755: Fix Version/s: 3.0.4 > array_overlap should handle Double.NaN and Float.NaN >

[jira] [Resolved] (SPARK-36702) array_union should not produce duplicated result

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36702. - Fix Version/s: 3.0.4 3.1.3 3.2.0 Assignee:

[jira] [Commented] (SPARK-36673) Incorrect Unions of struct with mismatched field name case

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415864#comment-17415864 ] Wenchen Fan commented on SPARK-36673: - This seems like a bug. [~viirya] what do you think? >

[jira] [Created] (SPARK-36774) Use SparkSubmitTestUtils to core and use it in SparkSubmitSuite

2021-09-15 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-36774: -- Summary: Use SparkSubmitTestUtils to core and use it in SparkSubmitSuite Key: SPARK-36774 URL: https://issues.apache.org/jira/browse/SPARK-36774 Project: Spark

[jira] [Commented] (SPARK-36678) Migrate SHOW TABLES to use V2 command by default

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415863#comment-17415863 ] Wenchen Fan commented on SPARK-36678: - Temp views don't belong to any catalog. I think the SHOW

[jira] [Commented] (SPARK-36773) Fix the uts to check the parquet compression

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415854#comment-17415854 ] Apache Spark commented on SPARK-36773: -- User 'BelodengKlaus' has created a pull request for this

[jira] [Commented] (SPARK-36773) Fix the uts to check the parquet compression

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415853#comment-17415853 ] Apache Spark commented on SPARK-36773: -- User 'BelodengKlaus' has created a pull request for this

[jira] [Assigned] (SPARK-36773) Fix the uts to check the parquet compression

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36773: Assignee: Apache Spark > Fix the uts to check the parquet compression >

[jira] [Assigned] (SPARK-36773) Fix the uts to check the parquet compression

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36773: Assignee: (was: Apache Spark) > Fix the uts to check the parquet compression >

[jira] [Updated] (SPARK-36773) Fix the uts to check the parquet compression

2021-09-15 Thread jingpan xiong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jingpan xiong updated SPARK-36773: -- Summary: Fix the uts to check the parquet compression (was: Fix the uts to check the parquet

[jira] [Created] (SPARK-36773) Fix the uts to check the parquet codec

2021-09-15 Thread jingpan xiong (Jira)
jingpan xiong created SPARK-36773: - Summary: Fix the uts to check the parquet codec Key: SPARK-36773 URL: https://issues.apache.org/jira/browse/SPARK-36773 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-36702) array_union should not produce duplicated result

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415841#comment-17415841 ] Apache Spark commented on SPARK-36702: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

2021-09-15 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415810#comment-17415810 ] Gengliang Wang commented on SPARK-36772: [~mridulm80]Thanks for the ping. I will cut RC3 after

[jira] [Comment Edited] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

2021-09-15 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415803#comment-17415803 ] Mridul Muralidharan edited comment on SPARK-36772 at 9/15/21, 11:55 PM:

[jira] [Commented] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

2021-09-15 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415803#comment-17415803 ] Mridul Muralidharan commented on SPARK-36772: - +CC [~Gengliang.Wang] > FinalizeShuffleMerge

[jira] [Created] (SPARK-36772) FinalizeShuffleMerge fails with an exception due to attempt id not matching

2021-09-15 Thread Mridul Muralidharan (Jira)
Mridul Muralidharan created SPARK-36772: --- Summary: FinalizeShuffleMerge fails with an exception due to attempt id not matching Key: SPARK-36772 URL: https://issues.apache.org/jira/browse/SPARK-36772

[jira] [Created] (SPARK-36771) Fix `pop` of Categorical Series

2021-09-15 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36771: Summary: Fix `pop` of Categorical Series Key: SPARK-36771 URL: https://issues.apache.org/jira/browse/SPARK-36771 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36771) Fix `pop` of Categorical Series

2021-09-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415796#comment-17415796 ] Xinrong Meng commented on SPARK-36771: -- I am working on that. > Fix `pop` of Categorical Series >

[jira] [Commented] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window ROWS frame

2021-09-15 Thread Guibin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415787#comment-17415787 ] Guibin Zhang commented on SPARK-36770: -- [https://github.com/apache/spark/pull/34010]   > Improve

[jira] [Issue Comment Deleted] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window ROWS frame

2021-09-15 Thread Guibin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guibin Zhang updated SPARK-36770: - Comment: was deleted (was: [https://github.com/apache/spark/pull/34010]  ) > Improve run-time

[jira] [Commented] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window ROWS frame

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415786#comment-17415786 ] Apache Spark commented on SPARK-36770: -- User 'guibin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window ROWS frame

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36770: Assignee: (was: Apache Spark) > Improve run-time performance for window function

[jira] [Assigned] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window ROWS frame

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36770: Assignee: Apache Spark > Improve run-time performance for window function first and last

[jira] [Commented] (SPARK-34378) Support extra optional Avro fields in AvroSerializer

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415783#comment-17415783 ] Apache Spark commented on SPARK-34378: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34378) Support extra optional Avro fields in AvroSerializer

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34378: Assignee: (was: Apache Spark) > Support extra optional Avro fields in AvroSerializer

[jira] [Assigned] (SPARK-34378) Support extra optional Avro fields in AvroSerializer

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34378: Assignee: Apache Spark > Support extra optional Avro fields in AvroSerializer >

[jira] [Resolved] (SPARK-36735) Adjust overhead of cached relation for DPP

2021-09-15 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36735. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33975

[jira] [Assigned] (SPARK-36735) Adjust overhead of cached relation for DPP

2021-09-15 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36735: --- Assignee: L. C. Hsieh > Adjust overhead of cached relation for DPP >

[jira] [Resolved] (SPARK-36759) Upgrade Scala to 2.12.15

2021-09-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36759. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33999

[jira] [Assigned] (SPARK-36759) Upgrade Scala to 2.12.15

2021-09-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36759: - Assignee: Dongjoon Hyun > Upgrade Scala to 2.12.15 > > >

[jira] [Comment Edited] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

2021-09-15 Thread Yongjun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415740#comment-17415740 ] Yongjun Zhang edited comment on SPARK-31646 at 9/15/21, 8:27 PM: - HI

[jira] [Comment Edited] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

2021-09-15 Thread Yongjun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415740#comment-17415740 ] Yongjun Zhang edited comment on SPARK-31646 at 9/15/21, 8:27 PM: - HI

[jira] [Commented] (SPARK-31646) Remove unused registeredConnections counter from ShuffleMetrics

2021-09-15 Thread Yongjun Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415740#comment-17415740 ] Yongjun Zhang commented on SPARK-31646: --- HI [~mauzhang], [~dongjoon], [~marek.simunek], Thanks

[jira] [Commented] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11 and 1.12

2021-09-15 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415725#comment-17415725 ] Takuya Ueshin commented on SPARK-34276: --- SPARK-36696 was fixed by upgrading parquet to 1.12.1.

[jira] [Resolved] (SPARK-36696) spark.read.parquet loads empty dataset

2021-09-15 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36696. --- Fix Version/s: 3.2.0 Resolution: Fixed I confirmed the example file can be read

[jira] [Comment Edited] (SPARK-36696) spark.read.parquet loads empty dataset

2021-09-15 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415703#comment-17415703 ] DB Tsai edited comment on SPARK-36696 at 9/15/21, 7:21 PM: --- This issue is

[jira] [Comment Edited] (SPARK-36696) spark.read.parquet loads empty dataset

2021-09-15 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415703#comment-17415703 ] DB Tsai edited comment on SPARK-36696 at 9/15/21, 7:20 PM: --- This issue is

[jira] [Commented] (SPARK-36696) spark.read.parquet loads empty dataset

2021-09-15 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415703#comment-17415703 ] DB Tsai commented on SPARK-36696: - This issue is addressed by

[jira] [Resolved] (SPARK-36726) Upgrade Parquet to 1.12.1

2021-09-15 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-36726. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33969

[jira] [Assigned] (SPARK-36726) Upgrade Parquet to 1.12.1

2021-09-15 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-36726: --- Assignee: Chao Sun > Upgrade Parquet to 1.12.1 > - > >

[jira] [Updated] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window ROWS frame

2021-09-15 Thread Guibin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guibin Zhang updated SPARK-36770: - Summary: Improve run-time performance for window function first and last against

[jira] [Updated] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window frame

2021-09-15 Thread Guibin Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guibin Zhang updated SPARK-36770: - Description: *Context* The *UnboundedFollowingWindowFunctionFrame* has the time complexity of

[jira] [Created] (SPARK-36770) Improve run-time performance for window function first and last against UnboundedFollowing window frame

2021-09-15 Thread Guibin Zhang (Jira)
Guibin Zhang created SPARK-36770: Summary: Improve run-time performance for window function first and last against UnboundedFollowing window frame Key: SPARK-36770 URL:

[jira] [Resolved] (SPARK-36722) Problems with update function in koalas - pyspark pandas.

2021-09-15 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36722. --- Fix Version/s: 3.2.0 Assignee: dgd_contributor Resolution: Fixed Issue

[jira] [Commented] (SPARK-36769) Improve `filter` of single-indexed DataFrame

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415673#comment-17415673 ] Apache Spark commented on SPARK-36769: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36769) Improve `filter` of single-indexed DataFrame

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36769: Assignee: (was: Apache Spark) > Improve `filter` of single-indexed DataFrame >

[jira] [Assigned] (SPARK-36769) Improve `filter` of single-indexed DataFrame

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36769: Assignee: Apache Spark > Improve `filter` of single-indexed DataFrame >

[jira] [Created] (SPARK-36769) Improve `filter` of single-indexed DataFrame

2021-09-15 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36769: Summary: Improve `filter` of single-indexed DataFrame Key: SPARK-36769 URL: https://issues.apache.org/jira/browse/SPARK-36769 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36768) Cannot resolve attribute with table reference

2021-09-15 Thread Willi Raschkowski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415631#comment-17415631 ] Willi Raschkowski commented on SPARK-36768: --- In case you wonder why we care or why we can't

[jira] [Commented] (SPARK-36768) Cannot resolve attribute with table reference

2021-09-15 Thread Willi Raschkowski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415630#comment-17415630 ] Willi Raschkowski commented on SPARK-36768: --- In the debugger I see [on this

[jira] [Commented] (SPARK-36768) Cannot resolve attribute with table reference

2021-09-15 Thread Willi Raschkowski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415621#comment-17415621 ] Willi Raschkowski commented on SPARK-36768: --- This also reproduces on master at time of

[jira] [Updated] (SPARK-36768) Cannot resolve attribute with table reference

2021-09-15 Thread Willi Raschkowski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Willi Raschkowski updated SPARK-36768: -- Description: Spark seems in some cases unable to resolve attributes that contain

[jira] [Updated] (SPARK-36768) Cannot resolve attribute with table reference

2021-09-15 Thread Willi Raschkowski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Willi Raschkowski updated SPARK-36768: -- Description: Spark seems in some cases unable to resolve attributes that contain

[jira] [Created] (SPARK-36768) Cannot resolve attribute with table reference

2021-09-15 Thread Willi Raschkowski (Jira)
Willi Raschkowski created SPARK-36768: - Summary: Cannot resolve attribute with table reference Key: SPARK-36768 URL: https://issues.apache.org/jira/browse/SPARK-36768 Project: Spark

[jira] [Commented] (SPARK-36740) collection operators should handle duplicated NaN

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415608#comment-17415608 ] Apache Spark commented on SPARK-36740: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36740) collection operators should handle duplicated NaN

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36740: Assignee: Apache Spark > collection operators should handle duplicated NaN >

[jira] [Assigned] (SPARK-36740) collection operators should handle duplicated NaN

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36740: Assignee: (was: Apache Spark) > collection operators should handle duplicated NaN >

[jira] [Commented] (SPARK-36740) collection operators should handle duplicated NaN

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415607#comment-17415607 ] Apache Spark commented on SPARK-36740: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-36767) ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415588#comment-17415588 ] Shockang commented on SPARK-36767: -- [~angerszhuuu] OK > ArrayMin/ArrayMax/SortArray/ArraySort add

[jira] [Commented] (SPARK-36767) ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT

2021-09-15 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415584#comment-17415584 ] angerszhu commented on SPARK-36767: --- [~Shockang] Sorry, I have write the pr. >

[jira] [Commented] (SPARK-36767) ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415583#comment-17415583 ] Shockang commented on SPARK-36767: -- working on this. > ArrayMin/ArrayMax/SortArray/ArraySort add

[jira] [Updated] (SPARK-36766) Spark SQL DDL does not recognize fs.s3.impl implied filesystem in LOCATION tag

2021-09-15 Thread Sungwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sungwon updated SPARK-36766: Description: CREATE TABLE with the LOCATION tag via SPARK SQL DDL does not seem to recognize the

[jira] [Created] (SPARK-36767) ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT

2021-09-15 Thread angerszhu (Jira)
angerszhu created SPARK-36767: - Summary: ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT Key: SPARK-36767 URL: https://issues.apache.org/jira/browse/SPARK-36767 Project: Spark Issue

[jira] [Issue Comment Deleted] (SPARK-30175) Eliminate warnings: part 5

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shockang updated SPARK-30175: - Comment: was deleted (was: I'm working on this.) > Eliminate warnings: part 5 >

[jira] [Created] (SPARK-36766) Spark SQL DDL does not recognize fs.s3.impl implied filesystem in LOCATION tag

2021-09-15 Thread Sungwon (Jira)
Sungwon created SPARK-36766: --- Summary: Spark SQL DDL does not recognize fs.s3.impl implied filesystem in LOCATION tag Key: SPARK-36766 URL: https://issues.apache.org/jira/browse/SPARK-36766 Project: Spark

[jira] [Updated] (SPARK-30175) Eliminate warnings: part 5

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shockang updated SPARK-30175: - Description:

[jira] [Commented] (SPARK-30177) Eliminate warnings: part 7

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415564#comment-17415564 ] Shockang commented on SPARK-30177: -- After careful inspection, I found that except for the last problem,

[jira] [Issue Comment Deleted] (SPARK-30177) Eliminate warnings: part 7

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shockang updated SPARK-30177: - Comment: was deleted (was: After careful inspection, I found that except for the last one, everything

[jira] [Issue Comment Deleted] (SPARK-30177) Eliminate warnings: part 7

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shockang updated SPARK-30177: - Comment: was deleted (was: I'm working on this.) > Eliminate warnings: part 7 >

[jira] [Resolved] (SPARK-36755) array_overlap should handle Double.NaN and Float.NaN

2021-09-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36755. - Fix Version/s: 3.1.3 3.2.0 Assignee: angerszhu Resolution:

[jira] [Updated] (SPARK-30175) Eliminate warnings: part 5

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shockang updated SPARK-30175: - Description:

[jira] [Updated] (SPARK-30177) Eliminate warnings: part 7

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shockang updated SPARK-30177: - Description: -/mllib/src/test/scala/org/apache/spark/ml/clustering/BisectingKMeansSuite.scala-

[jira] [Commented] (SPARK-30177) Eliminate warnings: part 7

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415551#comment-17415551 ] Shockang commented on SPARK-30177: -- I'm working on this. > Eliminate warnings: part 7 >

[jira] [Commented] (SPARK-30177) Eliminate warnings: part 7

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415550#comment-17415550 ] Shockang commented on SPARK-30177: -- After careful inspection, I found that except for the last one,

[jira] [Updated] (SPARK-36765) Spark Support for MS Sql JDBC connector with Kerberos/Keytab

2021-09-15 Thread Dilip Thallam Sridhar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Thallam Sridhar updated SPARK-36765: -- Description: Hi Team,   We are using the Spark-3.0.2 to connect to MS

[jira] [Updated] (SPARK-36765) Spark Support for MS Sql JDBC connector with Kerberos/Keytab

2021-09-15 Thread Dilip Thallam Sridhar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Thallam Sridhar updated SPARK-36765: -- Description: Hi Team,   We are using the Spark-3.0.2 to connect to MS

[jira] [Commented] (SPARK-30175) Eliminate warnings: part 5

2021-09-15 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415542#comment-17415542 ] Shockang commented on SPARK-30175: -- I'm working on this. > Eliminate warnings: part 5 >

[jira] [Updated] (SPARK-36765) Spark Support for MS Sql JDBC connector with Kerberos/Keytab

2021-09-15 Thread Dilip Thallam Sridhar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Thallam Sridhar updated SPARK-36765: -- Summary: Spark Support for MS Sql JDBC connector with Kerberos/Keytab (was:

[jira] [Updated] (SPARK-36765) Support MS Sql JDBC connector with Kerberos/Keytab

2021-09-15 Thread Dilip Thallam Sridhar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Thallam Sridhar updated SPARK-36765: -- Summary: Support MS Sql JDBC connector with Kerberos/Keytab (was: Support MS

[jira] [Updated] (SPARK-36765) Support MS Sql JDBC connectorJDBC with Kerberos/Keytab

2021-09-15 Thread Dilip Thallam Sridhar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Thallam Sridhar updated SPARK-36765: -- Summary: Support MS Sql JDBC connectorJDBC with Kerberos/Keytab (was:

[jira] [Created] (SPARK-36765) Support MS Sql JDBC connectorJDBC with Keytab

2021-09-15 Thread Dilip Thallam Sridhar (Jira)
Dilip Thallam Sridhar created SPARK-36765: - Summary: Support MS Sql JDBC connectorJDBC with Keytab Key: SPARK-36765 URL: https://issues.apache.org/jira/browse/SPARK-36765 Project: Spark

[jira] [Commented] (SPARK-36710) Support new syntax in function apply APIs

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415494#comment-17415494 ] Apache Spark commented on SPARK-36710: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-36710) Support new syntax in function apply APIs

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36710: Assignee: (was: Apache Spark) > Support new syntax in function apply APIs >

[jira] [Assigned] (SPARK-36710) Support new syntax in function apply APIs

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36710: Assignee: Apache Spark > Support new syntax in function apply APIs >

[jira] [Commented] (SPARK-36710) Support new syntax in function apply APIs

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415493#comment-17415493 ] Apache Spark commented on SPARK-36710: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-36755) array_overlap should handle Double.NaN and Float.NaN

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415459#comment-17415459 ] Apache Spark commented on SPARK-36755: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-36755) array_overlap should handle Double.NaN and Float.NaN

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415457#comment-17415457 ] Apache Spark commented on SPARK-36755: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36755) array_overlap should handle Double.NaN and Float.NaN

2021-09-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36755: Assignee: (was: Apache Spark) > array_overlap should handle Double.NaN and Float.NaN

  1   2   >