[jira] [Commented] (SPARK-33275) ANSI mode: runtime errors instead of returning null

2020-12-16 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250261#comment-17250261 ] Leanken.Lin commented on SPARK-33275: - [~Chongguang] Yes, Please go ahead. > ANSI mode: runtime

[jira] [Updated] (SPARK-33619) GetMapValueUtil code generation error

2020-12-01 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33619: Description: ``` In -SPARK-33460--- there is a bug as follow. GetMapValueUtil generated code

[jira] [Updated] (SPARK-33619) GetMapValueUtil code generation error

2020-12-01 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33619: Description: ``` In -SPARK-33460--- there is a bug as follow. GetMapValueUtil generated code

[jira] [Created] (SPARK-33619) GetMapValueUtil code generation error

2020-12-01 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33619: --- Summary: GetMapValueUtil code generation error Key: SPARK-33619 URL: https://issues.apache.org/jira/browse/SPARK-33619 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33498) Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

2020-11-20 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33498: Summary: Datetime parsing should fail if the input string can't be parsed, or the pattern string

[jira] [Updated] (SPARK-33498) Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

2020-11-20 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33498: Description: Datetime parsing should fail if the input string can't be parsed, or the pattern

[jira] [Created] (SPARK-33498) Datetime parsing should fail if the input string can't be parsed, or he pattern string is invalid

2020-11-19 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33498: --- Summary: Datetime parsing should fail if the input string can't be parsed, or he pattern string is invalid Key: SPARK-33498 URL: https://issues.apache.org/jira/browse/SPARK-33498

[jira] [Updated] (SPARK-33386) Accessing array elements should failed if index is out of bound.

2020-11-15 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33386: Description: When ansi mode enabled, accessing array element with out of bound index should

[jira] [Updated] (SPARK-33460) Accessing map values should fail if key is not found.

2020-11-15 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33460: Description: When ansi mode enabled, accessing map values should failed with exception if key

[jira] [Created] (SPARK-33460) Accessing map values should fail if key is not found.

2020-11-15 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33460: --- Summary: Accessing map values should fail if key is not found. Key: SPARK-33460 URL: https://issues.apache.org/jira/browse/SPARK-33460 Project: Spark Issue

[jira] [Updated] (SPARK-33391) element_at with CreateArray not respect one based index

2020-11-08 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33391: Description: var df = spark.sql("select element_at(array(3, 2, 1), 0)") df.printSchema() df =

[jira] [Updated] (SPARK-33391) element_at with CreateArray not respect one based index

2020-11-08 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33391: Summary: element_at with CreateArray not respect one based index (was: element_at not respect

[jira] [Created] (SPARK-33391) element_at not respect one based index

2020-11-08 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33391: --- Summary: element_at not respect one based index Key: SPARK-33391 URL: https://issues.apache.org/jira/browse/SPARK-33391 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-33386) Accessing array elements should failed if index is out of bound.

2020-11-08 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33386: Description: When ansi mode enabled, accessing array element should failed with exception, but

[jira] [Created] (SPARK-33386) Accessing array elements should failed if index is out of bound.

2020-11-08 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33386: --- Summary: Accessing array elements should failed if index is out of bound. Key: SPARK-33386 URL: https://issues.apache.org/jira/browse/SPARK-33386 Project: Spark

[jira] [Updated] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-10-14 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33140: Summary: make Analyzer rules using SQLConf.get (was: make Analyzer and Optimizer rules using

[jira] [Updated] (SPARK-33139) protect setActiveSession and clearActiveSession

2020-10-14 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33139: Description: This PR is a sub-task of

[jira] [Commented] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-14 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213824#comment-17213824 ] Leanken.Lin commented on SPARK-33138: - Thanks for your consideration, but me and my colleague are

[jira] [Created] (SPARK-33142) SQL temp view should store SQL text as well

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33142: --- Summary: SQL temp view should store SQL text as well Key: SPARK-33142 URL: https://issues.apache.org/jira/browse/SPARK-33142 Project: Spark Issue Type:

[jira] [Created] (SPARK-33141) capture SQL configs when creating permanent views

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33141: --- Summary: capture SQL configs when creating permanent views Key: SPARK-33141 URL: https://issues.apache.org/jira/browse/SPARK-33141 Project: Spark Issue Type:

[jira] [Created] (SPARK-33140) make Analyzer and Optimizer rules using SQLConf.get

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33140: --- Summary: make Analyzer and Optimizer rules using SQLConf.get Key: SPARK-33140 URL: https://issues.apache.org/jira/browse/SPARK-33140 Project: Spark Issue

[jira] [Created] (SPARK-33139) protect setActiveSession and clearActiveSession

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33139: --- Summary: protect setActiveSession and clearActiveSession Key: SPARK-33139 URL: https://issues.apache.org/jira/browse/SPARK-33139 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33138: Description: Currently, temp view store mapping of temp view name and its logicalPlan, and

[jira] [Created] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33138: --- Summary: unify temp view and permanent view behaviors Key: SPARK-33138 URL: https://issues.apache.org/jira/browse/SPARK-33138 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33138: Description: Currently, temp view store mapping of temp view name and its logicalPlan, and

[jira] [Updated] (SPARK-33016) Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on.

2020-09-28 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33016: Affects Version/s: (was: 3.0.1) 3.0.0 > Potential SQLMetrics missed

[jira] [Created] (SPARK-33016) Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on.

2020-09-28 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33016: --- Summary: Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on. Key: SPARK-33016 URL: https://issues.apache.org/jira/browse/SPARK-33016

[jira] [Resolved] (SPARK-32765) EliminateJoinToEmptyRelation should respect exchange behavior when canChangeNumPartitions == false

2020-09-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin resolved SPARK-32765. - Resolution: Not A Bug > EliminateJoinToEmptyRelation should respect exchange behavior when >

[jira] [Updated] (SPARK-32765) EliminateJoinToEmptyRelation should respect exchange behavior when canChangeNumPartitions == false

2020-09-01 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32765: Environment: (was: Currently, EliminateJoinToEmptyRelation Rule will convert Join into

[jira] [Updated] (SPARK-32765) EliminateJoinToEmptyRelation should respect exchange behavior when canChangeNumPartitions == false

2020-09-01 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32765: Description: Currently, EliminateJoinToEmptyRelation Rule will convert Join into EmptyRelation

[jira] [Created] (SPARK-32765) EliminateJoinToEmptyRelation should respect exchange behavior when canChangeNumPartitions == false

2020-09-01 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32765: --- Summary: EliminateJoinToEmptyRelation should respect exchange behavior when canChangeNumPartitions == false Key: SPARK-32765 URL: https://issues.apache.org/jira/browse/SPARK-32765

[jira] [Created] (SPARK-32705) EmptyHashedRelation$; no valid constructor

2020-08-26 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32705: --- Summary: EmptyHashedRelation$; no valid constructor Key: SPARK-32705 URL: https://issues.apache.org/jira/browse/SPARK-32705 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-32678) Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ generated code

2020-08-21 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32678: --- Summary: Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ generated code Key: SPARK-32678 URL: https://issues.apache.org/jira/browse/SPARK-32678 Project:

[jira] [Commented] (SPARK-32649) Optimize BHJ/SHJ inner and semi join with empty hashed relation

2020-08-18 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179397#comment-17179397 ] Leanken.Lin commented on SPARK-32649: - Feel free to just send out PR for reviewing,^_^ > Optimize

[jira] [Comment Edited] (SPARK-32649) Optimize BHJ/SHJ inner and semi join with empty hashed relation

2020-08-18 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179397#comment-17179397 ] Leanken.Lin edited comment on SPARK-32649 at 8/18/20, 6:37 AM: --- Feel free

[jira] [Updated] (SPARK-32644) NAAJ support for ShuffleHashJoin when AQE is on

2020-08-17 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32644: Description: In SPARK-32290, we managed to optimize NAAJ scenario from BNLJ to BHJ, but skipped

[jira] [Created] (SPARK-32644) NAAJ support for ShuffleHashJoin when AQE is on

2020-08-17 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32644: --- Summary: NAAJ support for ShuffleHashJoin when AQE is on Key: SPARK-32644 URL: https://issues.apache.org/jira/browse/SPARK-32644 Project: Spark Issue Type:

[jira] [Updated] (SPARK-32615) Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32615: Description: {code:java} // Reproduce Step sql/test-only

[jira] [Updated] (SPARK-32615) Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32615: Description: {code:java} // Reproduce Step sql/test-only

[jira] [Updated] (SPARK-32615) Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32615: Description: {code:java} // Reproduce Step sql/test-only

[jira] [Updated] (SPARK-32615) Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32615: Summary: Fix AQE aggregateMetrics java.util.NoSuchElementException (was: AQE aggregateMetrics

[jira] [Created] (SPARK-32615) AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32615: --- Summary: AQE aggregateMetrics java.util.NoSuchElementException Key: SPARK-32615 URL: https://issues.apache.org/jira/browse/SPARK-32615 Project: Spark Issue

[jira] [Updated] (SPARK-32573) Anti Join Improvement with EmptyHashedRelation and EmptyHashedRelationWithAllNullKeys

2020-08-10 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32573: Description: In SPARK-32290, we introduced several new types of HashedRelation *

[jira] [Created] (SPARK-32573) Eliminate Anti Join when BuildSide is Empty

2020-08-08 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32573: --- Summary: Eliminate Anti Join when BuildSide is Empty Key: SPARK-32573 URL: https://issues.apache.org/jira/browse/SPARK-32573 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-32494) Null Aware Anti Join Optimize Support Multi-Column

2020-08-05 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin resolved SPARK-32494. - Resolution: Later > Null Aware Anti Join Optimize Support Multi-Column >

[jira] [Updated] (SPARK-32494) Null Aware Anti Join Optimize Support Multi-Column

2020-07-30 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32494: Description: In Issue SPARK-32290, we managed to optimize BroadcastNestedLoopJoin into 

[jira] [Created] (SPARK-32494) Null Aware Anti Join Optimize Support Multi-Column

2020-07-30 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32494: --- Summary: Null Aware Anti Join Optimize Support Multi-Column Key: SPARK-32494 URL: https://issues.apache.org/jira/browse/SPARK-32494 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-32474) NullAwareAntiJoin multi-column support

2020-07-30 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin resolved SPARK-32474. - Resolution: Duplicate > NullAwareAntiJoin multi-column support >

[jira] [Updated] (SPARK-32474) NullAwareAntiJoin multi-column support

2020-07-28 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32474: Description: This is a follow up improvement of Issue SPARK-32290. In SPARK-32290, we already

[jira] [Updated] (SPARK-32474) NullAwareAntiJoin multi-column support

2020-07-28 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32474: Description: This is a follow up improvement of Issue SPARK-32290. In SPARK-32290, we already

[jira] [Created] (SPARK-32474) NullAwareAntiJoin multi-column support

2020-07-28 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32474: --- Summary: NullAwareAntiJoin multi-column support Key: SPARK-32474 URL: https://issues.apache.org/jira/browse/SPARK-32474 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-32290) NotInSubquery SingleColumn Optimize

2020-07-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32290: Fix Version/s: 3.0.1 > NotInSubquery SingleColumn Optimize > ---

[jira] [Created] (SPARK-32290) NotInSubquery SingleColumn Optimize

2020-07-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32290: --- Summary: NotInSubquery SingleColumn Optimize Key: SPARK-32290 URL: https://issues.apache.org/jira/browse/SPARK-32290 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-28050) DataFrameWriter support insertInto a specific table partition

2019-06-14 Thread Leanken.Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-28050: Description: {code:java} // Some comments here val ptTableName = "mc_test_pt_table" sql(s"CREATE

[jira] [Updated] (SPARK-28050) DataFrameWriter support insertInto a specific table partition

2019-06-14 Thread Leanken.Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-28050: Description: ``` val ptTableName = "mc_test_pt_table" sql(s"CREATE TABLE ${ptTableName} (name

[jira] [Created] (SPARK-28050) DataFrameWriter support insertInto a specific table partition

2019-06-14 Thread Leanken.Lin (JIRA)
Leanken.Lin created SPARK-28050: --- Summary: DataFrameWriter support insertInto a specific table partition Key: SPARK-28050 URL: https://issues.apache.org/jira/browse/SPARK-28050 Project: Spark