[jira] [Created] (SPARK-39438) Add a threshold to not in line CTE

Yuming Wang created SPARK-39438: --- Summary: Add a threshold to not in line CTE Key: SPARK-39438 URL: https://issues.apache.org/jira/browse/SPARK-39438 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-39437) normalize plan id separately in PlanStabilitySuite

[ https://issues.apache.org/jira/browse/SPARK-39437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552558#comment-17552558 ] Apache Spark commented on SPARK-39437: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39437) normalize plan id separately in PlanStabilitySuite

[ https://issues.apache.org/jira/browse/SPARK-39437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39437: Assignee: (was: Apache Spark) > normalize plan id separately in PlanStabilitySuite >

[jira] [Assigned] (SPARK-39437) normalize plan id separately in PlanStabilitySuite

[ https://issues.apache.org/jira/browse/SPARK-39437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39437: Assignee: Apache Spark > normalize plan id separately in PlanStabilitySuite >

[jira] [Commented] (SPARK-39437) normalize plan id separately in PlanStabilitySuite

[ https://issues.apache.org/jira/browse/SPARK-39437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552557#comment-17552557 ] Apache Spark commented on SPARK-39437: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-39436) graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsException: -1

[ https://issues.apache.org/jira/browse/SPARK-39436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hanyingjun updated SPARK-39436: --- Description: val graph = Graph(vertices, edges).partitionBy(PartitionStrategy.RandomVertexCut)

[jira] [Created] (SPARK-39436) graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsException: -1

hanyingjun created SPARK-39436: -- Summary: graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsException: -1 Key: SPARK-39436 URL: https://issues.apache.org/jira/browse/SPARK-39436 Project:

[jira] [Created] (SPARK-39435) graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsException: -1

hanyingjun created SPARK-39435: -- Summary: graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsException: -1 Key: SPARK-39435 URL: https://issues.apache.org/jira/browse/SPARK-39435 Project:

[jira] [Commented] (SPARK-5480) GraphX pageRank: java.lang.ArrayIndexOutOfBoundsException:

[ https://issues.apache.org/jira/browse/SPARK-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552541#comment-17552541 ] hanyingjun commented on SPARK-5480: ---   This problem has not been solved in spark v2.4.3，   Does anyone

[jira] [Updated] (SPARK-39392) Refine ANSI error messages and remove 'To return NULL instead'

[ https://issues.apache.org/jira/browse/SPARK-39392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39392: - Fix Version/s: 3.3.0 (was: 3.3.1) > Refine ANSI error messages and remove 'To

[jira] [Updated] (SPARK-39286) Documentation for the decode function has an incorrect reference

[ https://issues.apache.org/jira/browse/SPARK-39286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39286: - Fix Version/s: 3.3.0 (was: 3.3.1) > Documentation for the decode function has an

[jira] [Updated] (SPARK-39394) Improve PySpark structured streaming page more readable

[ https://issues.apache.org/jira/browse/SPARK-39394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39394: - Fix Version/s: 3.3.0 (was: 3.3.1) > Improve PySpark structured streaming page

[jira] [Updated] (SPARK-39376) Do not output duplicated columns in star expansion of subquery alias of NATURAL/USING JOIN

[ https://issues.apache.org/jira/browse/SPARK-39376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39376: - Fix Version/s: 3.3.0 (was: 3.3.1) > Do not output duplicated columns in star

[jira] [Updated] (SPARK-39411) Release candidates do not have the correct version for PySpark

[ https://issues.apache.org/jira/browse/SPARK-39411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39411: - Fix Version/s: 3.3.0 (was: 3.3.1) > Release candidates do not have the correct

[jira] [Updated] (SPARK-39412) IllegalStateException from connector does not work well with error class framework

[ https://issues.apache.org/jira/browse/SPARK-39412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39412: - Fix Version/s: 3.3.0 (was: 3.3.1) > IllegalStateException from connector does

[jira] [Updated] (SPARK-39398) when doing iteration compute in graphx, checkpoint need support storagelevel

[ https://issues.apache.org/jira/browse/SPARK-39398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangwenli updated SPARK-39398: -- Affects Version/s: 2.4.4 (was: 3.2.1) > when doing iteration compute in

[jira] [Assigned] (SPARK-39434) Provide runtime error query context when array index is out of bound

[ https://issues.apache.org/jira/browse/SPARK-39434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39434: Assignee: Apache Spark (was: Gengliang Wang) > Provide runtime error query context when

[jira] [Commented] (SPARK-39434) Provide runtime error query context when array index is out of bound

[ https://issues.apache.org/jira/browse/SPARK-39434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552483#comment-17552483 ] Apache Spark commented on SPARK-39434: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-39434) Provide runtime error query context when array index is out of bound

[ https://issues.apache.org/jira/browse/SPARK-39434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39434: Assignee: Gengliang Wang (was: Apache Spark) > Provide runtime error query context when

[jira] [Created] (SPARK-39434) Provide runtime error query context when array index is out of bound

Gengliang Wang created SPARK-39434: -- Summary: Provide runtime error query context when array index is out of bound Key: SPARK-39434 URL: https://issues.apache.org/jira/browse/SPARK-39434 Project:

[jira] [Created] (SPARK-39433) to_date function returns a null for the first week of the year

CHARLES HOGG created SPARK-39433: Summary: to_date function returns a null for the first week of the year Key: SPARK-39433 URL: https://issues.apache.org/jira/browse/SPARK-39433 Project: Spark

[jira] [Created] (SPARK-39432) element_at(*, 0) does not return INVALID_ARRAY_INDEX_IN_ELEMENT_AT

Serge Rielau created SPARK-39432: Summary: element_at(*, 0) does not return INVALID_ARRAY_INDEX_IN_ELEMENT_AT Key: SPARK-39432 URL: https://issues.apache.org/jira/browse/SPARK-39432 Project: Spark

[jira] [Assigned] (SPARK-39431) Update PySpark dependency in Installation doc

[ https://issues.apache.org/jira/browse/SPARK-39431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39431: - Assignee: Dongjoon Hyun > Update PySpark dependency in Installation doc >

[jira] [Resolved] (SPARK-39431) Update PySpark dependency in Installation doc

[ https://issues.apache.org/jira/browse/SPARK-39431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39431. --- Fix Version/s: 3.3.1 Resolution: Fixed Issue resolved by pull request 36825

[jira] [Commented] (SPARK-39431) Update PySpark dependency in Installation doc

[ https://issues.apache.org/jira/browse/SPARK-39431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552416#comment-17552416 ] Apache Spark commented on SPARK-39431: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-39431) Update PySpark dependency in Installation doc

[ https://issues.apache.org/jira/browse/SPARK-39431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39431: Assignee: Apache Spark > Update PySpark dependency in Installation doc >

[jira] [Assigned] (SPARK-39431) Update PySpark dependency in Installation doc

[ https://issues.apache.org/jira/browse/SPARK-39431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39431: Assignee: (was: Apache Spark) > Update PySpark dependency in Installation doc >

[jira] [Created] (SPARK-39431) Update PySpark dependency in Installation doc

Dongjoon Hyun created SPARK-39431: - Summary: Update PySpark dependency in Installation doc Key: SPARK-39431 URL: https://issues.apache.org/jira/browse/SPARK-39431 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-39422) SHOW CREATE TABLE should suggest 'AS SERDE' for Hive tables with unsupported serde configurations

[ https://issues.apache.org/jira/browse/SPARK-39422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-39422. Fix Version/s: 3.3.1 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-39430) The inconsistent timezone in Spark History Server UI

[ https://issues.apache.org/jira/browse/SPARK-39430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surbhi updated SPARK-39430: --- Attachment: Screenshot 2022-06-10 at 12.59.50 AM.png > The inconsistent timezone in Spark History Server UI

[jira] [Updated] (SPARK-39430) The inconsistent timezone in Spark History Server UI

[ https://issues.apache.org/jira/browse/SPARK-39430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surbhi updated SPARK-39430: --- Attachment: Screenshot 2022-06-10 at 12.59.36 AM.png > The inconsistent timezone in Spark History Server UI

[jira] [Updated] (SPARK-39430) The inconsistent timezone in Spark History Server UI

[ https://issues.apache.org/jira/browse/SPARK-39430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surbhi updated SPARK-39430: --- Description: The spark history server is running in UTC timezone. But we are trying to view history server

[jira] [Created] (SPARK-39430) The inconsistent timezone in Spark History Server UI

Surbhi created SPARK-39430: -- Summary: The inconsistent timezone in Spark History Server UI Key: SPARK-39430 URL: https://issues.apache.org/jira/browse/SPARK-39430 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-39420) Support ANALYZE TABLE on v2 tables

[ https://issues.apache.org/jira/browse/SPARK-39420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552309#comment-17552309 ] Felipe commented on SPARK-39420: Hi [~hyukjin.kwon], are you aware of any work in progress or other

[jira] [Updated] (SPARK-39420) Support ANALYZE TABLE on v2 tables

[ https://issues.apache.org/jira/browse/SPARK-39420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe updated SPARK-39420: --- Description: According to https://github.com/delta-io/delta/pull/840 to implement ANALYZE TABLE in Delta,

[jira] [Resolved] (SPARK-39414) Upgrade Scala to 2.12.16

[ https://issues.apache.org/jira/browse/SPARK-39414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39414. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36807

[jira] [Assigned] (SPARK-39414) Upgrade Scala to 2.12.16

[ https://issues.apache.org/jira/browse/SPARK-39414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39414: - Assignee: Yang Jie > Upgrade Scala to 2.12.16 > > >

[jira] [Updated] (SPARK-39269) spark3.2.0 commit tmp file is not found when rename

[ https://issues.apache.org/jira/browse/SPARK-39269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cxb updated SPARK-39269: Summary: spark3.2.0 commit tmp file is not found when rename (was: spark3.2.0 offset tmp file is not found when

[jira] [Comment Edited] (SPARK-39269) spark3.2.0 offset tmp file is not found when rename

[ https://issues.apache.org/jira/browse/SPARK-39269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552211#comment-17552211 ] cxb edited comment on SPARK-39269 at 6/9/22 1:42 PM: - a batch will running long time

[jira] [Comment Edited] (SPARK-39269) spark3.2.0 offset tmp file is not found when rename

[ https://issues.apache.org/jira/browse/SPARK-39269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552211#comment-17552211 ] cxb edited comment on SPARK-39269 at 6/9/22 1:40 PM: - a batch will running long time

[jira] [Commented] (SPARK-39269) spark3.2.0 offset tmp file is not found when rename

[ https://issues.apache.org/jira/browse/SPARK-39269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552211#comment-17552211 ] cxb commented on SPARK-39269: - a batch will running long time when it happening the error and  the file of

[jira] [Resolved] (SPARK-39427) Disable ANSI intervals in the percentile functions

[ https://issues.apache.org/jira/browse/SPARK-39427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-39427. -- Fix Version/s: 3.3.0 3.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-39412) IllegalStateException from connector does not work well with error class framework

[ https://issues.apache.org/jira/browse/SPARK-39412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552172#comment-17552172 ] Apache Spark commented on SPARK-39412: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-39259) Timestamps returned by now() and equivalent functions are not consistent in subqueries

[ https://issues.apache.org/jira/browse/SPARK-39259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552130#comment-17552130 ] Apache Spark commented on SPARK-39259: -- User 'olaky' has created a pull request for this issue:

[jira] [Commented] (SPARK-39259) Timestamps returned by now() and equivalent functions are not consistent in subqueries

[ https://issues.apache.org/jira/browse/SPARK-39259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552129#comment-17552129 ] Apache Spark commented on SPARK-39259: -- User 'olaky' has created a pull request for this issue:

[jira] [Commented] (SPARK-39429) Convert Inner Join With Aggregate To Semi Join

[ https://issues.apache.org/jira/browse/SPARK-39429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552127#comment-17552127 ] Apache Spark commented on SPARK-39429: -- User 'lxian' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39429) Convert Inner Join With Aggregate To Semi Join

[ https://issues.apache.org/jira/browse/SPARK-39429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39429: Assignee: (was: Apache Spark) > Convert Inner Join With Aggregate To Semi Join >

[jira] [Assigned] (SPARK-39429) Convert Inner Join With Aggregate To Semi Join

[ https://issues.apache.org/jira/browse/SPARK-39429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39429: Assignee: Apache Spark > Convert Inner Join With Aggregate To Semi Join >

[jira] [Commented] (SPARK-39429) Convert Inner Join With Aggregate To Semi Join

[ https://issues.apache.org/jira/browse/SPARK-39429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552123#comment-17552123 ] Li Xian commented on SPARK-39429: - [https://github.com/apache/spark/pull/36823] I've submitted a PR for

[jira] [Created] (SPARK-39429) Convert Inner Join With Aggregate To Semi Join

Li Xian created SPARK-39429: --- Summary: Convert Inner Join With Aggregate To Semi Join Key: SPARK-39429 URL: https://issues.apache.org/jira/browse/SPARK-39429 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37670) Support predicate pushdown and column pruning for de-duped CTEs

[ https://issues.apache.org/jira/browse/SPARK-37670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552072#comment-17552072 ] Apache Spark commented on SPARK-37670: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39428: Assignee: Apache Spark > use code block for `Coalesce Hints for SQL Queries` >

[jira] [Commented] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552071#comment-17552071 ] Apache Spark commented on SPARK-39428: -- User 'dcoliversun' has created a pull request for this

[jira] [Assigned] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39428: Assignee: (was: Apache Spark) > use code block for `Coalesce Hints for SQL Queries`

[jira] [Commented] (SPARK-39226) Fix the precision of the return type of round-like functions

[ https://issues.apache.org/jira/browse/SPARK-39226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552070#comment-17552070 ] Apache Spark commented on SPARK-39226: -- User 'wangyum' has created a pull request for this issue:

[jira] (SPARK-39393) Parquet data source only supports push-down predicate filters for non-repeated primitive types

[ https://issues.apache.org/jira/browse/SPARK-39393 ] Dongjoon Hyun deleted comment on SPARK-39393: --- was (Author: apachespark): User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/36819 >

[jira] [Updated] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian updated SPARK-39428: - Attachment: Coalesce.png > use code block for `Coalesce Hints for SQL Queries` >

[jira] [Created] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

qian created SPARK-39428: Summary: use code block for `Coalesce Hints for SQL Queries` Key: SPARK-39428 URL: https://issues.apache.org/jira/browse/SPARK-39428 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian updated SPARK-39428: - Description: Use code block for `Coalesce Hints for SQL Queries`, now it is plain block. (was: Use code block

[jira] [Assigned] (SPARK-39427) Disable ANSI intervals in the percentile functions

[ https://issues.apache.org/jira/browse/SPARK-39427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39427: Assignee: Max Gekk (was: Apache Spark) > Disable ANSI intervals in the percentile

[jira] [Commented] (SPARK-39393) Parquet data source only supports push-down predicate filters for non-repeated primitive types

[ https://issues.apache.org/jira/browse/SPARK-39393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552034#comment-17552034 ] Apache Spark commented on SPARK-39393: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-39427) Disable ANSI intervals in the percentile functions

[ https://issues.apache.org/jira/browse/SPARK-39427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552032#comment-17552032 ] Apache Spark commented on SPARK-39427: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39427) Disable ANSI intervals in the percentile functions

[ https://issues.apache.org/jira/browse/SPARK-39427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39427: Assignee: Apache Spark (was: Max Gekk) > Disable ANSI intervals in the percentile

[jira] [Commented] (SPARK-39259) Timestamps returned by now() and equivalent functions are not consistent in subqueries

[ https://issues.apache.org/jira/browse/SPARK-39259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552033#comment-17552033 ] Apache Spark commented on SPARK-39259: -- User 'olaky' has created a pull request for this issue:

[jira] [Created] (SPARK-39427) Disable ANSI intervals in the percentile functions

Max Gekk created SPARK-39427: Summary: Disable ANSI intervals in the percentile functions Key: SPARK-39427 URL: https://issues.apache.org/jira/browse/SPARK-39427 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-39417) Handle Null partition values in PartitioningUtils

[ https://issues.apache.org/jira/browse/SPARK-39417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-39417. Fix Version/s: 3.3.0 3.4.0 Resolution: Fixed > Handle Null partition

[jira] [Assigned] (SPARK-39417) Handle Null partition values in PartitioningUtils

[ https://issues.apache.org/jira/browse/SPARK-39417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39417: --- Assignee: Prashant Singh > Handle Null partition values in PartitioningUtils >