[jira] [Assigned] (SPARK-34399) Add file commit time to metrics and shown in SQL Tab UI

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34399: --- Assignee: angerszhu > Add file commit time to metrics and shown in SQL Tab UI >

[jira] [Resolved] (SPARK-34399) Add file commit time to metrics and shown in SQL Tab UI

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34399. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33542

[jira] [Assigned] (SPARK-36312) ParquetWritter should check inner field

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36312: --- Assignee: angerszhu > ParquetWritter should check inner field >

[jira] [Resolved] (SPARK-36312) ParquetWritter should check inner field

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36312. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33531

[jira] [Resolved] (SPARK-35639) Add metrics about coalesced partitions to CustomShuffleReader in AQE

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35639. - Resolution: Fixed Issue resolved by pull request 32776

[jira] [Resolved] (SPARK-36275) ResolveAggregateFunctions should work with nested fields

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36275. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33498

[jira] [Assigned] (SPARK-36275) ResolveAggregateFunctions should work with nested fields

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36275: --- Assignee: Allison Wang > ResolveAggregateFunctions should work with nested fields >

[jira] [Updated] (SPARK-36028) Allow Project to host outer references in scalar subqueries

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-36028: Fix Version/s: (was: 3.3.0) 3.2.0 > Allow Project to host outer references

[jira] [Assigned] (SPARK-36323) Support ANSI interval literals for TimeWindow

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36323: Assignee: Kousuke Saruta (was: Apache Spark) > Support ANSI interval literals for

[jira] [Assigned] (SPARK-36323) Support ANSI interval literals for TimeWindow

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36323: Assignee: Apache Spark (was: Kousuke Saruta) > Support ANSI interval literals for

[jira] [Resolved] (SPARK-36318) Update docs about mapping of ANSI interval types to Java/Scala/SQL types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36318. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved in

[jira] [Commented] (SPARK-36323) Support ANSI interval literals for TimeWindow

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388433#comment-17388433 ] Apache Spark commented on SPARK-36323: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-36323) Support ANSI interval literals for TimeWindow

2021-07-27 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36323: -- Summary: Support ANSI interval literals for TimeWindow Key: SPARK-36323 URL: https://issues.apache.org/jira/browse/SPARK-36323 Project: Spark Issue

[jira] [Updated] (SPARK-36322) Client cannot authenticate via:[TOKEN, KERBEROS]

2021-07-27 Thread MengYao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MengYao updated SPARK-36322: Description: When I run spark thriftserver in spark on k8s, the -- principal parameter and -- KeyTab

[jira] [Created] (SPARK-36322) Client cannot authenticate via:[TOKEN, KERBEROS]

2021-07-27 Thread MengYao (Jira)
MengYao created SPARK-36322: --- Summary: Client cannot authenticate via:[TOKEN, KERBEROS] Key: SPARK-36322 URL: https://issues.apache.org/jira/browse/SPARK-36322 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-36321) Do not fail application in kubernetes if name is too long

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36321: Assignee: (was: Apache Spark) > Do not fail application in kubernetes if name is too

[jira] [Assigned] (SPARK-36321) Do not fail application in kubernetes if name is too long

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36321: Assignee: Apache Spark > Do not fail application in kubernetes if name is too long >

[jira] [Commented] (SPARK-36321) Do not fail application in kubernetes if name is too long

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388420#comment-17388420 ] Apache Spark commented on SPARK-36321: -- User 'ulysses-you' has created a pull request for this

[jira] [Created] (SPARK-36321) Do not fail application in kubernetes if name is too long

2021-07-27 Thread XiDuo You (Jira)
XiDuo You created SPARK-36321: - Summary: Do not fail application in kubernetes if name is too long Key: SPARK-36321 URL: https://issues.apache.org/jira/browse/SPARK-36321 Project: Spark Issue

[jira] [Assigned] (SPARK-36314) Update Sessionization example to use native support of session window

2021-07-27 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36314: --- Assignee: Jungtaek Lim > Update Sessionization example to use native support of session

[jira] [Resolved] (SPARK-36314) Update Sessionization example to use native support of session window

2021-07-27 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36314. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33548

[jira] [Assigned] (SPARK-36320) Fix Series/Index.copy() to drop extra columns.

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36320: Assignee: Apache Spark > Fix Series/Index.copy() to drop extra columns. >

[jira] [Assigned] (SPARK-36320) Fix Series/Index.copy() to drop extra columns.

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36320: Assignee: (was: Apache Spark) > Fix Series/Index.copy() to drop extra columns. >

[jira] [Commented] (SPARK-36320) Fix Series/Index.copy() to drop extra columns.

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388392#comment-17388392 ] Apache Spark commented on SPARK-36320: -- User 'ueshin' has created a pull request for this issue:

[jira] [Created] (SPARK-36320) Fix Series/Index.copy() to drop extra columns.

2021-07-27 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36320: - Summary: Fix Series/Index.copy() to drop extra columns. Key: SPARK-36320 URL: https://issues.apache.org/jira/browse/SPARK-36320 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36099) Group exception messages in core/util

2021-07-27 Thread dgd_contributor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388387#comment-17388387 ] dgd_contributor commented on SPARK-36099: - Sorry I wasn't checking the comment recently, I've

[jira] [Assigned] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36310: Assignee: Xinrong Meng > Fix hasnan() window function in IndexOpsMixin >

[jira] [Resolved] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36310. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33547

[jira] [Updated] (SPARK-36094) Group SQL component error messages in Spark error class JSON file

2021-07-27 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36094: --- Description: To improve auditing, reduce duplication, and improve quality of error messages thrown

[jira] [Resolved] (SPARK-35997) Implement comparison operators for CategoricalDtype in pandas API on Spark

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35997. -- Resolution: Done > Implement comparison operators for CategoricalDtype in pandas API on Spark

[jira] [Commented] (SPARK-36314) Update Sessionization example to use native support of session window

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388345#comment-17388345 ] Apache Spark commented on SPARK-36314: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-36314) Update Sessionization example to use native support of session window

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388344#comment-17388344 ] Apache Spark commented on SPARK-36314: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-36314) Update Sessionization example to use native support of session window

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36314: Assignee: (was: Apache Spark) > Update Sessionization example to use native support

[jira] [Assigned] (SPARK-36314) Update Sessionization example to use native support of session window

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36314: Assignee: Apache Spark > Update Sessionization example to use native support of session

[jira] [Assigned] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36190: Assignee: (was: Apache Spark) > Improve the rest of DataTypeOps tests by avoiding

[jira] [Assigned] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36310: Assignee: (was: Apache Spark) > Fix hasnan() window function in IndexOpsMixin >

[jira] [Assigned] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36310: Assignee: Apache Spark > Fix hasnan() window function in IndexOpsMixin >

[jira] [Commented] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388312#comment-17388312 ] Apache Spark commented on SPARK-36310: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388309#comment-17388309 ] Apache Spark commented on SPARK-36190: -- User 'xinrong-databricks' has created a pull request for

[jira] [Commented] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388308#comment-17388308 ] Apache Spark commented on SPARK-36190: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36190: Assignee: Apache Spark > Improve the rest of DataTypeOps tests by avoiding joins >

[jira] [Updated] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36190: - Summary: Improve the rest of DataTypeOps tests by avoiding joins (was: Improve all DataTypeOps

[jira] [Updated] (SPARK-36190) Improve all DataTypeOps tests by avoiding joins

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36190: - Summary: Improve all DataTypeOps tests by avoiding joins (was: Improve the rest of DataTypeOps

[jira] [Updated] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36310: - Description:   {code:java} File "/__w/spark/spark/python/pyspark/pandas/groupby.py", line 1497,

[jira] [Updated] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36310: - Summary: Fix hasnan() window function in IndexOpsMixin (was: Fix hasnan(), any(), and all()

[jira] [Commented] (SPARK-36319) Have Observation return Map instead of Row

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388269#comment-17388269 ] Apache Spark commented on SPARK-36319: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36319) Have Observation return Map instead of Row

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36319: Assignee: (was: Apache Spark) > Have Observation return Map instead of Row >

[jira] [Assigned] (SPARK-36319) Have Observation return Map instead of Row

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36319: Assignee: Apache Spark > Have Observation return Map instead of Row >

[jira] [Commented] (SPARK-36319) Have Observation return Map instead of Row

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388268#comment-17388268 ] Apache Spark commented on SPARK-36319: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Commented] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388265#comment-17388265 ] Apache Spark commented on SPARK-34927: -- User 'MyeongKim' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34927: Assignee: (was: Apache Spark) > Support TPCDSQueryBenchmark in Benchmarks >

[jira] [Assigned] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34927: Assignee: Apache Spark > Support TPCDSQueryBenchmark in Benchmarks >

[jira] [Commented] (SPARK-34927) Support TPCDSQueryBenchmark in Benchmarks

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388266#comment-17388266 ] Apache Spark commented on SPARK-34927: -- User 'MyeongKim' has created a pull request for this issue:

[jira] [Updated] (SPARK-36094) Group SQL component error messages in Spark error class JSON file

2021-07-27 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-36094: --- Description: To improve auditing, reduce duplication, and improve quality of error messages thrown

[jira] [Created] (SPARK-36319) Have Observation return Map instead of Row

2021-07-27 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-36319: - Summary: Have Observation return Map instead of Row Key: SPARK-36319 URL: https://issues.apache.org/jira/browse/SPARK-36319 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-36318) Update docs about mapping of ANSI interval types to Java/Scala/SQL types

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36318: Assignee: Max Gekk (was: Apache Spark) > Update docs about mapping of ANSI interval

[jira] [Commented] (SPARK-36318) Update docs about mapping of ANSI interval types to Java/Scala/SQL types

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388236#comment-17388236 ] Apache Spark commented on SPARK-36318: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36318) Update docs about mapping of ANSI interval types to Java/Scala/SQL types

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36318: Assignee: Apache Spark (was: Max Gekk) > Update docs about mapping of ANSI interval

[jira] [Created] (SPARK-36318) Update docs about mapping of ANSI interval types to Java/Scala/SQL types

2021-07-27 Thread Max Gekk (Jira)
Max Gekk created SPARK-36318: Summary: Update docs about mapping of ANSI interval types to Java/Scala/SQL types Key: SPARK-36318 URL: https://issues.apache.org/jira/browse/SPARK-36318 Project: Spark

[jira] [Resolved] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36263. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33484

[jira] [Assigned] (SPARK-36263) Add Dataset.observe(Observation, Column, Column*) to PySpark

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36263: --- Assignee: Enrico Minack > Add Dataset.observe(Observation, Column, Column*) to PySpark >

[jira] [Resolved] (SPARK-36185) Implement functions in CategoricalAccessor/CategoricalIndex

2021-07-27 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36185. --- Fix Version/s: 3.2.0 Resolution: Done I'd close this since we've done the tasks we

[jira] [Commented] (SPARK-36317) PruneFileSourcePartitionsSuite tests are failing after the fix to SPARK-36136

2021-07-27 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388204#comment-17388204 ] Chao Sun commented on SPARK-36317: -- [~vsowrirajan]: the change is already reverted - are you still

[jira] [Updated] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36242: -- Affects Version/s: 3.2.0 3.0.3 3.1.2 > Ensure

[jira] [Updated] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36242: -- Fix Version/s: 3.0.4 > Ensure spill file closed before set success to true in >

[jira] [Updated] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36242: -- Issue Type: Bug (was: Improvement) > Ensure spill file closed before set success to true in

[jira] [Updated] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36242: -- Fix Version/s: 3.1.3 > Ensure spill file closed before set success to true in >

[jira] [Commented] (SPARK-34399) Add file commit time to metrics and shown in SQL Tab UI

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388195#comment-17388195 ] Apache Spark commented on SPARK-34399: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-36317) PruneFileSourcePartitionsSuite tests are failing after the fix to SPARK-36136

2021-07-27 Thread Venkata krishnan Sowrirajan (Jira)
Venkata krishnan Sowrirajan created SPARK-36317: --- Summary: PruneFileSourcePartitionsSuite tests are failing after the fix to SPARK-36136 Key: SPARK-36317 URL:

[jira] [Created] (SPARK-36316) NoClassDefFoundError for org.slf4j.impl.StaticLoggerBinder in org.apache.spark.Logging#isLog4j12 when using SLF4J/Logback 2.x

2021-07-27 Thread Ian Springer (Jira)
Ian Springer created SPARK-36316: Summary: NoClassDefFoundError for org.slf4j.impl.StaticLoggerBinder in org.apache.spark.Logging#isLog4j12 when using SLF4J/Logback 2.x Key: SPARK-36316 URL:

[jira] [Commented] (SPARK-36315) Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388171#comment-17388171 ] Apache Spark commented on SPARK-36315: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36315) Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36315: Assignee: Apache Spark > Only skip AQEShuffleReadRule in the final stage if it breaks

[jira] [Assigned] (SPARK-36315) Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36315: Assignee: (was: Apache Spark) > Only skip AQEShuffleReadRule in the final stage if

[jira] [Commented] (SPARK-36315) Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388169#comment-17388169 ] Apache Spark commented on SPARK-36315: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-36315) Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-27 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36315: --- Summary: Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement Key: SPARK-36315 URL: https://issues.apache.org/jira/browse/SPARK-36315

[jira] [Comment Edited] (SPARK-36086) The case of the delta table is inconsistent with parquet

2021-07-27 Thread Ruslan Krivoshein (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388156#comment-17388156 ] Ruslan Krivoshein edited comment on SPARK-36086 at 7/27/21, 4:03 PM: -

[jira] [Commented] (SPARK-36086) The case of the delta table is inconsistent with parquet

2021-07-27 Thread Ruslan Krivoshein (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388156#comment-17388156 ] Ruslan Krivoshein commented on SPARK-36086: --- Let me get on with it, please > The case of the

[jira] [Commented] (SPARK-36099) Group exception messages in core/util

2021-07-27 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388107#comment-17388107 ] Shockang commented on SPARK-36099: -- It's good for you to wait a day. I was going to submit the PR

[jira] [Commented] (SPARK-36099) Group exception messages in core/util

2021-07-27 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388101#comment-17388101 ] Shockang commented on SPARK-36099: -- I’m sorry for my gaffe.[~dc-heros] > Group exception messages in

[jira] [Commented] (SPARK-36099) Group exception messages in core/util

2021-07-27 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388095#comment-17388095 ] Shockang commented on SPARK-36099: -- I think you should ask my permission, or my time will be

[jira] [Commented] (SPARK-36099) Group exception messages in core/util

2021-07-27 Thread Shockang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388090#comment-17388090 ] Shockang commented on SPARK-36099: -- You should tell me in advance.My code has written more than 200

[jira] [Assigned] (SPARK-36102) Group exception messages in core/deploy

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36102: Assignee: Apache Spark > Group exception messages in core/deploy >

[jira] [Commented] (SPARK-36102) Group exception messages in core/deploy

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388089#comment-17388089 ] Apache Spark commented on SPARK-36102: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-36102) Group exception messages in core/deploy

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36102: Assignee: (was: Apache Spark) > Group exception messages in core/deploy >

[jira] [Assigned] (SPARK-36101) Group exception messages in core/api

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36101: Assignee: Apache Spark > Group exception messages in core/api >

[jira] [Commented] (SPARK-36101) Group exception messages in core/api

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388070#comment-17388070 ] Apache Spark commented on SPARK-36101: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-36101) Group exception messages in core/api

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36101: Assignee: (was: Apache Spark) > Group exception messages in core/api >

[jira] [Resolved] (SPARK-34249) Add documentation for ANSI implicit cast rules

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34249. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33516

[jira] [Assigned] (SPARK-34249) Add documentation for ANSI implicit cast rules

2021-07-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34249: --- Assignee: Gengliang Wang > Add documentation for ANSI implicit cast rules >

[jira] [Updated] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34619: --- Affects Version/s: 3.2.0 > Update the Spark SQL guide about day-time and year-month

[jira] [Updated] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34619: --- Affects Version/s: (was: 3.3.0) > Update the Spark SQL guide about day-time and

[jira] [Assigned] (SPARK-35398) Simplify the way to get classes from ClassBodyEvaluator in CodeGenerator.updateAndGetCompilationStats method

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35398: Assignee: Apache Spark > Simplify the way to get classes from ClassBodyEvaluator in >

[jira] [Assigned] (SPARK-35398) Simplify the way to get classes from ClassBodyEvaluator in CodeGenerator.updateAndGetCompilationStats method

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35398: Assignee: (was: Apache Spark) > Simplify the way to get classes from

[jira] [Assigned] (SPARK-35253) Upgrade Janino from 3.0.16 to 3.1.4

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35253: Assignee: Apache Spark > Upgrade Janino from 3.0.16 to 3.1.4 >

[jira] [Assigned] (SPARK-35253) Upgrade Janino from 3.0.16 to 3.1.4

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35253: Assignee: (was: Apache Spark) > Upgrade Janino from 3.0.16 to 3.1.4 >

[jira] [Commented] (SPARK-36295) Refactor sixth set of 20 query execution errors to use error classes

2021-07-27 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387979#comment-17387979 ] PengLei commented on SPARK-36295: - woking on this > Refactor sixth set of 20 query execution errors to

[jira] [Commented] (SPARK-36294) Refactor fifth set of 20 query execution errors to use error classes

2021-07-27 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387978#comment-17387978 ] PengLei commented on SPARK-36294: - working on this > Refactor fifth set of 20 query execution errors to

[jira] [Issue Comment Deleted] (SPARK-36291) Refactor second set of 20 query execution errors to use error classes

2021-07-27 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PengLei updated SPARK-36291: Comment: was deleted (was: working on this) > Refactor second set of 20 query execution errors to use

[jira] [Updated] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34619: --- Affects Version/s: (was: 3.2.0) 3.3.0 > Update the Spark SQL

[jira] [Commented] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387969#comment-17387969 ] Apache Spark commented on SPARK-34619: -- User 'sarutak' has created a pull request for this issue:

  1   2   >