[jira] [Created] (SPARK-46036) raise_error should only take the error class parameter for internal usage

2023-11-21 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46036: --- Summary: raise_error should only take the error class parameter for internal usage Key: SPARK-46036 URL: https://issues.apache.org/jira/browse/SPARK-46036 Project: Spar

[jira] [Commented] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-11-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788306#comment-17788306 ] Wenchen Fan commented on SPARK-45649: - This has a bug. I've uploaded the test data.

[jira] [Updated] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-11-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-45649: Attachment: test_table.parquet.zip > Unify the prepare framework for `OffsetWindowFunctionFrame` >

[jira] [Commented] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-11-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788307#comment-17788307 ] Wenchen Fan commented on SPARK-45649: - I'm reverting the commit. > Unify the prepar

[jira] [Reopened] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-11-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-45649: - > Unify the prepare framework for `OffsetWindowFunctionFrame` >

[jira] [Resolved] (SPARK-33393) Support SHOW TABLE EXTENDED in DSv2

2023-11-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33393. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 37588 [https://gith

[jira] [Resolved] (SPARK-45905) least common type between decimal types should retain integral digits first

2023-11-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45905. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43781 [https://gith

[jira] [Assigned] (SPARK-43393) Sequence expression can overflow

2023-11-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43393: --- Assignee: Deepayan Patra > Sequence expression can overflow > -

[jira] [Resolved] (SPARK-43393) Sequence expression can overflow

2023-11-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43393. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Created] (SPARK-45932) GROUP BY ordinal should not replicate the grouping expression

2023-11-14 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45932: --- Summary: GROUP BY ordinal should not replicate the grouping expression Key: SPARK-45932 URL: https://issues.apache.org/jira/browse/SPARK-45932 Project: Spark

[jira] [Resolved] (SPARK-45898) Rewrite catalog table APIs to use unresolved logical plan

2023-11-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45898. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43771 [https://gith

[jira] [Assigned] (SPARK-45898) Rewrite catalog table APIs to use unresolved logical plan

2023-11-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45898: --- Assignee: Yihong He > Rewrite catalog table APIs to use unresolved logical plan > -

[jira] [Created] (SPARK-45920) group by ordinal should be idempotent

2023-11-13 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45920: --- Summary: group by ordinal should be idempotent Key: SPARK-45920 URL: https://issues.apache.org/jira/browse/SPARK-45920 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-45892) Refactor the optimizer plan validation to decouple validateSchemaOutput and validateExprIdUniqueness

2023-11-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45892. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43761 [https://gith

[jira] [Assigned] (SPARK-45892) Refactor the optimizer plan validation to decouple validateSchemaOutput and validateExprIdUniqueness

2023-11-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45892: --- Assignee: Xi Liang > Refactor the optimizer plan validation to decouple validateSchemaOutpu

[jira] [Created] (SPARK-45905) least common type between decimal types should retain integral digits first

2023-11-13 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45905: --- Summary: least common type between decimal types should retain integral digits first Key: SPARK-45905 URL: https://issues.apache.org/jira/browse/SPARK-45905 Project: Sp

[jira] [Assigned] (SPARK-45815) Provide an interface for Streaming sources to add _metadata columns

2023-11-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45815: --- Assignee: Yaohua Zhao > Provide an interface for Streaming sources to add _metadata columns

[jira] [Resolved] (SPARK-45815) Provide an interface for Streaming sources to add _metadata columns

2023-11-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45815. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43692 [https://gith

[jira] [Resolved] (SPARK-44886) Introduce CLUSTER BY SQL clause to CREATE/REPLACE TABLE

2023-11-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44886. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42577 [https://gith

[jira] [Assigned] (SPARK-44886) Introduce CLUSTER BY SQL clause to CREATE/REPLACE TABLE

2023-11-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44886: --- Assignee: Terry Kim > Introduce CLUSTER BY SQL clause to CREATE/REPLACE TABLE > ---

[jira] [Resolved] (SPARK-45752) Unreferenced CTE should all be checked by CheckAnalysis0

2023-11-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45752. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43614 [https://gith

[jira] [Resolved] (SPARK-45760) Add With expression to avoid duplicating expressions

2023-11-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45760. - Fix Version/s: 4.0.0 Resolution: Fixed > Add With expression to avoid duplicating express

[jira] [Created] (SPARK-45760) Add With expression to avoid duplicating expressions

2023-11-01 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45760: --- Summary: Add With expression to avoid duplicating expressions Key: SPARK-45760 URL: https://issues.apache.org/jira/browse/SPARK-45760 Project: Spark Issue Type

[jira] [Resolved] (SPARK-45713) Support registering Python data sources

2023-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45713. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43566 [https://gith

[jira] [Assigned] (SPARK-45713) Support registering Python data sources

2023-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45713: --- Assignee: Allison Wang > Support registering Python data sources >

[jira] [Resolved] (SPARK-45592) AQE and InMemoryTableScanExec correctness bug

2023-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45592. - Fix Version/s: 4.0.0 3.5.1 Resolution: Fixed Issue resolved by pull re

[jira] [Created] (SPARK-45725) remove the non-default IN subquery runtime filter

2023-10-30 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45725: --- Summary: remove the non-default IN subquery runtime filter Key: SPARK-45725 URL: https://issues.apache.org/jira/browse/SPARK-45725 Project: Spark Issue Type: I

[jira] [Resolved] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-10-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45649. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43507 [https://gith

[jira] [Resolved] (SPARK-45430) FramelessOffsetWindowFunctionFrame fails when ignore nulls and offset > # of rows

2023-10-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45430. - Fix Version/s: 3.3.4 3.5.1 4.0.0 3.4.2

[jira] [Assigned] (SPARK-45430) FramelessOffsetWindowFunctionFrame fails when ignore nulls and offset > # of rows

2023-10-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45430: --- Assignee: Vitalii Li > FramelessOffsetWindowFunctionFrame fails when ignore nulls and offse

[jira] [Resolved] (SPARK-45616) Usages of ParVector are unsafe because it does not propagate ThreadLocals or SparkSession

2023-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45616. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Resolved] (SPARK-45595) Expose SQLSTATE in error message

2023-10-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45595. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43438 [https://gith

[jira] [Assigned] (SPARK-45595) Expose SQLSTATE in error message

2023-10-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45595: --- Assignee: Serge Rielau > Expose SQLSTATE in error message > ---

[jira] [Assigned] (SPARK-45584) Execution fails when there are subqueries in TakeOrderedAndProjectExec

2023-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45584: --- Assignee: Allison Wang > Execution fails when there are subqueries in TakeOrderedAndProject

[jira] [Resolved] (SPARK-45584) Execution fails when there are subqueries in TakeOrderedAndProjectExec

2023-10-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45584. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45507) Correctness bug in correlated scalar subqueries with COUNT aggregates

2023-10-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45507: --- Assignee: Andy Lam > Correctness bug in correlated scalar subqueries with COUNT aggregates

[jira] [Resolved] (SPARK-45507) Correctness bug in correlated scalar subqueries with COUNT aggregates

2023-10-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45507. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43341 [https://gith

[jira] [Assigned] (SPARK-45586) Reduce compilation time for large expression trees

2023-10-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45586: --- Assignee: Kelvin Jiang > Reduce compilation time for large expression trees > -

[jira] [Resolved] (SPARK-45586) Reduce compilation time for large expression trees

2023-10-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45586. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43420 [https://gith

[jira] [Assigned] (SPARK-44649) Runtime Filter supports passing equivalent creation side expressions

2023-10-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44649: --- Assignee: Jiaan Geng > Runtime Filter supports passing equivalent creation side expressions

[jira] [Resolved] (SPARK-44649) Runtime Filter supports passing equivalent creation side expressions

2023-10-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44649. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42317 [https://gith

[jira] [Created] (SPARK-45575) support time travel options for df read API

2023-10-17 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45575: --- Summary: support time travel options for df read API Key: SPARK-45575 URL: https://issues.apache.org/jira/browse/SPARK-45575 Project: Spark Issue Type: New Fea

[jira] [Resolved] (SPARK-45531) Add more comments and rename some variable name for InjectRuntimeFilter

2023-10-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45531. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43359 [https://gith

[jira] [Resolved] (SPARK-45491) Replace: _LEGACY_ERROR_TEMP_2196 with a better error class

2023-10-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45491. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43376 [https://gith

[jira] [Assigned] (SPARK-45009) Correlated EXISTS subqueries in join ON condition unsupported and fail with internal error

2023-10-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45009: --- Assignee: Jack Chen > Correlated EXISTS subqueries in join ON condition unsupported and fai

[jira] [Resolved] (SPARK-45009) Correlated EXISTS subqueries in join ON condition unsupported and fail with internal error

2023-10-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45009. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42725 [https://gith

[jira] [Resolved] (SPARK-45498) Followup: Ignore task completion from old stage after retrying indeterminate stages

2023-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45498. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45498) Followup: Ignore task completion from old stage after retrying indeterminate stages

2023-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45498: --- Assignee: Mayur Bhosale > Followup: Ignore task completion from old stage after retrying in

[jira] [Resolved] (SPARK-45418) Change CURRENT_SCHEMA() column alias to match function name

2023-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45418. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43235 [https://gith

[jira] [Assigned] (SPARK-45418) Change CURRENT_SCHEMA() column alias to match function name

2023-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45418: --- Assignee: Michael Zhang > Change CURRENT_SCHEMA() column alias to match function name > ---

[jira] [Assigned] (SPARK-45132) Fix IDENTIFIER clause for functions

2023-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45132: --- Assignee: Serge Rielau > Fix IDENTIFIER clause for functions >

[jira] [Resolved] (SPARK-45132) Fix IDENTIFIER clause for functions

2023-10-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45132. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Resolved] (SPARK-45451) Make the default storage level of dataset cache configurable

2023-10-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45451. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43259 [https://gith

[jira] [Assigned] (SPARK-45451) Make the default storage level of dataset cache configurable

2023-10-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45451: --- Assignee: XiDuo You > Make the default storage level of dataset cache configurable > --

[jira] [Assigned] (SPARK-45449) Cache Invalidation Issue with JDBC Table

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45449: --- Assignee: liangyongyuan > Cache Invalidation Issue with JDBC Table > --

[jira] [Resolved] (SPARK-45449) Cache Invalidation Issue with JDBC Table

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45449. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45205) Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by SPARK-35278

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45205: --- Assignee: Qiang Yang > Since version 3.2.0, Spark SQL has taken longer to execute "show >

[jira] [Resolved] (SPARK-45205) Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by SPARK-35278

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45205. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Commented] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773146#comment-17773146 ] Wenchen Fan commented on SPARK-38341: - It seems like a common behavior to preserve t

[jira] [Reopened] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-38341: - > Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date > ---

[jira] [Assigned] (SPARK-44219) Add extra per-rule validation for optimization rewrites.

2023-10-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44219: --- Assignee: Yannis Sismanis > Add extra per-rule validation for optimization rewrites. >

[jira] [Resolved] (SPARK-44219) Add extra per-rule validation for optimization rewrites.

2023-10-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44219. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41763 [https://gith

[jira] [Resolved] (SPARK-45421) Catch AnalysisException over InlineCTE

2023-10-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45421. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43227 [https://gith

[jira] [Assigned] (SPARK-36112) Enable DecorrelateInnerQuery for IN/EXISTS subqueries

2023-10-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36112: --- Assignee: Andrey Gubichev > Enable DecorrelateInnerQuery for IN/EXISTS subqueries > ---

[jira] [Resolved] (SPARK-36112) Enable DecorrelateInnerQuery for IN/EXISTS subqueries

2023-10-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36112. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43111 [https://gith

[jira] [Resolved] (SPARK-45346) Parquet schema inference should respect case sensitive flag when merging schema

2023-09-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45346. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45346) Parquet schema inference should respect case sensitive flag when merging schema

2023-09-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45346: --- Assignee: Wenchen Fan > Parquet schema inference should respect case sensitive flag when me

[jira] [Resolved] (SPARK-44780) Document SQL Session variables

2023-09-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44780. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42467 [https://gith

[jira] [Assigned] (SPARK-44780) Document SQL Session variables

2023-09-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44780: --- Assignee: Serge Rielau > Document SQL Session variables > -- >

[jira] [Created] (SPARK-45346) Parquet schema inference should respect case sensitive flag when merging schema

2023-09-26 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45346: --- Summary: Parquet schema inference should respect case sensitive flag when merging schema Key: SPARK-45346 URL: https://issues.apache.org/jira/browse/SPARK-45346 Project

[jira] [Assigned] (SPARK-45271) Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some unused method in QueryCompilationErrors

2023-09-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45271: --- Assignee: BingKun Pan > Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some un

[jira] [Resolved] (SPARK-45271) Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some unused method in QueryCompilationErrors

2023-09-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45271. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43044 [https://gith

[jira] [Resolved] (SPARK-45182) Ignore task completion from old stage after retrying indeterminate stages

2023-09-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45182. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45182) Ignore task completion from old stage after retrying indeterminate stages

2023-09-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45182: --- Assignee: Mayur Bhosale > Ignore task completion from old stage after retrying indeterminat

[jira] [Resolved] (SPARK-44550) Wrong semantics for null IN (empty list)

2023-09-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44550. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43068 [https://gith

[jira] [Assigned] (SPARK-45306) Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans

2023-09-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45306: --- Assignee: Yang Jie > Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans

[jira] [Resolved] (SPARK-45306) Make `InMemoryColumnarBenchmark` use AQE-aware utils to collect plans

2023-09-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45306. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Resolved] (SPARK-41086) Consolidate SecondArgumentXXX error to INVALID_PARAMETER_VALUE

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41086. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45163: --- Assignee: BingKun Pan > Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into > UNSUPPORTED

[jira] [Resolved] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45163. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42917 [https://gith

[jira] [Resolved] (SPARK-45108) Improve the InjectRuntimeFilter for check probably shuffle

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45108. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42861 [https://gith

[jira] [Assigned] (SPARK-45108) Improve the InjectRuntimeFilter for check probably shuffle

2023-09-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45108: --- Assignee: jiaan.geng > Improve the InjectRuntimeFilter for check probably shuffle > ---

[jira] [Assigned] (SPARK-45112) Use UnresolvedFunction based resolution in SQL Dataset functions

2023-09-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45112: --- Assignee: Peter Toth > Use UnresolvedFunction based resolution in SQL Dataset functions > -

[jira] [Resolved] (SPARK-45112) Use UnresolvedFunction based resolution in SQL Dataset functions

2023-09-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45112. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42864 [https://gith

[jira] [Created] (SPARK-45203) join tests in KeyGroupedPartitioningSuite should use merge join hint

2023-09-18 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45203: --- Summary: join tests in KeyGroupedPartitioningSuite should use merge join hint Key: SPARK-45203 URL: https://issues.apache.org/jira/browse/SPARK-45203 Project: Spark

[jira] [Resolved] (SPARK-45184) Remove orphaned error class documents

2023-09-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45184. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42952 [https://gith

[jira] [Assigned] (SPARK-45184) Remove orphaned error class documents

2023-09-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45184: --- Assignee: BingKun Pan > Remove orphaned error class documents > ---

[jira] [Resolved] (SPARK-36191) Support ORDER BY and LIMIT to be on the correlation path

2023-09-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36191. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42705 [https://gith

[jira] [Assigned] (SPARK-36191) Support ORDER BY and LIMIT to be on the correlation path

2023-09-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36191: --- Assignee: Allison Wang > Support ORDER BY and LIMIT to be on the correlation path > ---

[jira] [Resolved] (SPARK-45117) Implement missing otherCopyArgs for the MultiCommutativeOp expression

2023-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45117. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45117) Implement missing otherCopyArgs for the MultiCommutativeOp expression

2023-09-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45117: --- Assignee: Supun Nakandala > Implement missing otherCopyArgs for the MultiCommutativeOp expr

[jira] [Resolved] (SPARK-45069) SQL variable should always be resolved after outer reference

2023-09-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45069. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42803 [https://gith

[jira] [Assigned] (SPARK-45069) SQL variable should always be resolved after outer reference

2023-09-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45069: --- Assignee: Wenchen Fan > SQL variable should always be resolved after outer reference >

[jira] [Created] (SPARK-45069) SQL variable should always be resolved after outer reference

2023-09-04 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-45069: --- Summary: SQL variable should always be resolved after outer reference Key: SPARK-45069 URL: https://issues.apache.org/jira/browse/SPARK-45069 Project: Spark I

[jira] [Updated] (SPARK-44577) INSERT BY NAME returns non-sensical error message

2023-09-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-44577: Fix Version/s: (was: 4.0.0) > INSERT BY NAME returns non-sensical error message >

[jira] [Resolved] (SPARK-44577) INSERT BY NAME returns non-sensical error message

2023-09-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44577. - Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-44577) INSERT BY NAME returns non-sensical error message

2023-09-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44577: --- Assignee: Jia Fan > INSERT BY NAME returns non-sensical error message > ---

[jira] [Assigned] (SPARK-44743) Reflect function behavior different from Hive

2023-09-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44743: --- Assignee: Jia Fan > Reflect function behavior different from Hive > ---

[jira] [Assigned] (SPARK-44743) Reflect function behavior different from Hive

2023-09-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44743: --- Assignee: (was: Fan Jiang) > Reflect function behavior different from Hive > --

[jira] [Assigned] (SPARK-44743) Reflect function behavior different from Hive

2023-09-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44743: --- Assignee: Fan Jiang > Reflect function behavior different from Hive > -

<    3   4   5   6   7   8   9   10   11   12   >