[jira] [Updated] (SPARK-37035) Improve error message when use vectorize reader

2021-10-19 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37035: -- Parent: SPARK-37065 Issue Type: Sub-task (was: Improvement) > Improve error message when use

[jira] [Created] (SPARK-37065) improve File reader's error message for each data source

2021-10-19 Thread angerszhu (Jira)
angerszhu created SPARK-37065: - Summary: improve File reader's error message for each data source Key: SPARK-37065 URL: https://issues.apache.org/jira/browse/SPARK-37065 Project: Spark Issue

[jira] [Created] (SPARK-37058) Add spark-shell command line unit test

2021-10-19 Thread angerszhu (Jira)
angerszhu created SPARK-37058: - Summary: Add spark-shell command line unit test Key: SPARK-37058 URL: https://issues.apache.org/jira/browse/SPARK-37058 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-37056) Fix unused code in test about history server & MetricsSystem

2021-10-19 Thread angerszhu (Jira)
angerszhu created SPARK-37056: - Summary: Fix unused code in test about history server & MetricsSystem Key: SPARK-37056 URL: https://issues.apache.org/jira/browse/SPARK-37056 Project: Spark

[jira] [Created] (SPARK-37053) Add metrics system for History server

2021-10-18 Thread angerszhu (Jira)
angerszhu created SPARK-37053: - Summary: Add metrics system for History server Key: SPARK-37053 URL: https://issues.apache.org/jira/browse/SPARK-37053 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-37052) Fix spark-3.2 can use --verbose with spark-shell

2021-10-18 Thread angerszhu (Jira)
angerszhu created SPARK-37052: - Summary: Fix spark-3.2 can use --verbose with spark-shell Key: SPARK-37052 URL: https://issues.apache.org/jira/browse/SPARK-37052 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-37035) Improve error message when use vectorize reader

2021-10-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429809#comment-17429809 ] angerszhu commented on SPARK-37035: --- raise a pr soon > Improve error message when use vectorize

[jira] [Updated] (SPARK-37035) Improve error message when use vectorize reader

2021-10-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37035: -- Description: Vectorized reader won't show which file read failed.   None-vectorize parquet reader

[jira] [Updated] (SPARK-37035) Improve error message when use vectorize reader

2021-10-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37035: -- Description: Vectorized reader won't show which file read failed.   Ono-vectorize parquet reader

[jira] [Updated] (SPARK-37035) Improve error message when use vectorize reader

2021-10-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37035: -- Description: Vectorized reader won't show which file read failed.     was: Vectorized reader

[jira] [Created] (SPARK-37035) Improve error message when use vectorize reader

2021-10-17 Thread angerszhu (Jira)
angerszhu created SPARK-37035: - Summary: Improve error message when use vectorize reader Key: SPARK-37035 URL: https://issues.apache.org/jira/browse/SPARK-37035 Project: Spark Issue Type:

[jira] [Created] (SPARK-37032) Remove unuseable link in spark-3.2.0's doc

2021-10-17 Thread angerszhu (Jira)
angerszhu created SPARK-37032: - Summary: Remove unuseable link in spark-3.2.0's doc Key: SPARK-37032 URL: https://issues.apache.org/jira/browse/SPARK-37032 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-36987) Add Doc about FROM statement

2021-10-15 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36987. --- Resolution: Not A Problem > Add Doc about FROM statement > > >

[jira] [Created] (SPARK-37003) Merge INSERT related docs

2021-10-13 Thread angerszhu (Jira)
angerszhu created SPARK-37003: - Summary: Merge INSERT related docs Key: SPARK-37003 URL: https://issues.apache.org/jira/browse/SPARK-37003 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-36995) Add new SQLFileCommitProtocol

2021-10-13 Thread angerszhu (Jira)
angerszhu created SPARK-36995: - Summary: Add new SQLFileCommitProtocol Key: SPARK-36995 URL: https://issues.apache.org/jira/browse/SPARK-36995 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36987) Add Doc about FROM statement

2021-10-12 Thread angerszhu (Jira)
angerszhu created SPARK-36987: - Summary: Add Doc about FROM statement Key: SPARK-36987 URL: https://issues.apache.org/jira/browse/SPARK-36987 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-36980) Insert support query with CTE

2021-10-11 Thread angerszhu (Jira)
angerszhu created SPARK-36980: - Summary: Insert support query with CTE Key: SPARK-36980 URL: https://issues.apache.org/jira/browse/SPARK-36980 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-36975) Refactor HiveClientImpl collect hive client call logic

2021-10-11 Thread angerszhu (Jira)
angerszhu created SPARK-36975: - Summary: Refactor HiveClientImpl collect hive client call logic Key: SPARK-36975 URL: https://issues.apache.org/jira/browse/SPARK-36975 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36975) Refactor HiveClientImpl collect hive client call logic

2021-10-11 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427053#comment-17427053 ] angerszhu commented on SPARK-36975: --- Raise a pr soon > Refactor HiveClientImpl collect hive client

[jira] [Resolved] (SPARK-36974) Try to raise memory and parallelism again for GA

2021-10-11 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36974. --- Resolution: Duplicate > Try to raise memory and parallelism again for GA >

[jira] [Created] (SPARK-36974) Try to raise memory and parallelism again for GA

2021-10-11 Thread angerszhu (Jira)
angerszhu created SPARK-36974: - Summary: Try to raise memory and parallelism again for GA Key: SPARK-36974 URL: https://issues.apache.org/jira/browse/SPARK-36974 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-36238) Spark UI load event timeline too slow for huge stage

2021-10-08 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36238. --- Resolution: Duplicate > Spark UI load event timeline too slow for huge stage >

[jira] [Commented] (SPARK-36238) Spark UI load event timeline too slow for huge stage

2021-10-08 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426441#comment-17426441 ] angerszhu commented on SPARK-36238: --- [~thejdeep]Seems so > Spark UI load event timeline too slow for

[jira] [Created] (SPARK-36957) Add product function to SQL function

2021-10-08 Thread angerszhu (Jira)
angerszhu created SPARK-36957: - Summary: Add product function to SQL function Key: SPARK-36957 URL: https://issues.apache.org/jira/browse/SPARK-36957 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-36876) Support Dynamic Partition pruning for HiveTableScanExec

2021-09-28 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36876: -- Description: Support dynamic partition pruning for hive serde scan > Support Dynamic Partition

[jira] [Created] (SPARK-36876) Support Dynamic Partition pruning for HiveTableScanExec

2021-09-28 Thread angerszhu (Jira)
angerszhu created SPARK-36876: - Summary: Support Dynamic Partition pruning for HiveTableScanExec Key: SPARK-36876 URL: https://issues.apache.org/jira/browse/SPARK-36876 Project: Spark Issue

[jira] [Updated] (SPARK-36838) Improve InSet NaN check generated code performance

2021-09-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Description: Since Set can't check is NaN value is contained in current set. With codegen, only when

[jira] [Updated] (SPARK-36838) Improve InSet NaN check generated code performance

2021-09-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Summary: Improve InSet NaN check generated code performance (was: Improve InSet NaN check gen code

[jira] [Updated] (SPARK-36838) Improve InSet NaN check gen code performance

2021-09-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Summary: Improve InSet NaN check gen code performance (was: Keep In/InSet use same nullSafeEval and

[jira] [Updated] (SPARK-36838) Keep In/InSet use same nullSafeEval and refactor

2021-09-24 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Summary: Keep In/InSet use same nullSafeEval and refactor (was: Keep In/InSet use same nullSafeEval)

[jira] [Updated] (SPARK-36838) Keep In/InSet use same nullSafeEval

2021-09-24 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Summary: Keep In/InSet use same nullSafeEval (was: In/InSet should handle NULL correctly ) > Keep

[jira] [Reopened] (SPARK-36838) Keep In/InSet use same nullSafeEval

2021-09-24 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu reopened SPARK-36838: --- > Keep In/InSet use same nullSafeEval > --- > > Key:

[jira] [Resolved] (SPARK-36838) In/InSet should handle NULL correctly

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36838. --- Resolution: Not A Problem > In/InSet should handle NULL correctly >

[jira] [Commented] (SPARK-36838) In/InSet should handle NULL correctly

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419532#comment-17419532 ] angerszhu commented on SPARK-36838: --- raising a  pr   > In/InSet should handle NULL correctly >

[jira] [Created] (SPARK-36838) In/InSet should handle NULL correctly

2021-09-23 Thread angerszhu (Jira)
angerszhu created SPARK-36838: - Summary: In/InSet should handle NULL correctly Key: SPARK-36838 URL: https://issues.apache.org/jira/browse/SPARK-36838 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36829) Refactor collectionOperation related Null check related code

2021-09-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36829: -- Summary: Refactor collectionOperation related Null check related code (was: Refactor Null check

[jira] [Created] (SPARK-36829) Refactor Null check related code

2021-09-23 Thread angerszhu (Jira)
angerszhu created SPARK-36829: - Summary: Refactor Null check related code Key: SPARK-36829 URL: https://issues.apache.org/jira/browse/SPARK-36829 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36792) Inset should handle Double.NaN and Float.NaN

2021-09-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416743#comment-17416743 ] angerszhu commented on SPARK-36792: --- raise a pr soon > Inset should handle Double.NaN and Float.NaN >

[jira] [Created] (SPARK-36792) Inset should handle Double.NaN and Float.NaN

2021-09-17 Thread angerszhu (Jira)
angerszhu created SPARK-36792: - Summary: Inset should handle Double.NaN and Float.NaN Key: SPARK-36792 URL: https://issues.apache.org/jira/browse/SPARK-36792 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36767) ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT

2021-09-15 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415584#comment-17415584 ] angerszhu commented on SPARK-36767: --- [~Shockang] Sorry, I have write the pr. >

[jira] [Created] (SPARK-36767) ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT

2021-09-15 Thread angerszhu (Jira)
angerszhu created SPARK-36767: - Summary: ArrayMin/ArrayMax/SortArray/ArraySort add comment and UT Key: SPARK-36767 URL: https://issues.apache.org/jira/browse/SPARK-36767 Project: Spark Issue

[jira] [Commented] (SPARK-36755) array_overlap should handle Double.NaN and Float.NaN

2021-09-14 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414980#comment-17414980 ] angerszhu commented on SPARK-36755: --- Working on this > array_overlap should handle Double.NaN and

[jira] [Created] (SPARK-36755) array_overlap should handle Double.NaN and Float.NaN

2021-09-14 Thread angerszhu (Jira)
angerszhu created SPARK-36755: - Summary: array_overlap should handle Double.NaN and Float.NaN Key: SPARK-36755 URL: https://issues.apache.org/jira/browse/SPARK-36755 Project: Spark Issue Type:

[jira] [Created] (SPARK-36754) array_intersect should handle Double.NaN and Float.NaN

2021-09-14 Thread angerszhu (Jira)
angerszhu created SPARK-36754: - Summary: array_intersect should handle Double.NaN and Float.NaN Key: SPARK-36754 URL: https://issues.apache.org/jira/browse/SPARK-36754 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36753) array_except should handle Double.NaN and Float.NaN

2021-09-14 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36753: -- Summary: array_except should handle Double.NaN and Float.NaN (was: ArrayExcept should handle

[jira] [Created] (SPARK-36753) ArrayExcept should handle Double.NaN and Float.NaN

2021-09-14 Thread angerszhu (Jira)
angerszhu created SPARK-36753: - Summary: ArrayExcept should handle Double.NaN and Float.NaN Key: SPARK-36753 URL: https://issues.apache.org/jira/browse/SPARK-36753 Project: Spark Issue Type:

[jira] [Created] (SPARK-36741) array_distinct should not return duplicated NaN

2021-09-13 Thread angerszhu (Jira)
angerszhu created SPARK-36741: - Summary: array_distinct should not return duplicated NaN Key: SPARK-36741 URL: https://issues.apache.org/jira/browse/SPARK-36741 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36702) array_union should not produce duplicated result

2021-09-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36702: -- Parent: SPARK-36740 Issue Type: Sub-task (was: Bug) > array_union should not produce

[jira] [Created] (SPARK-36740) collection operators should handle duplicated NaN

2021-09-13 Thread angerszhu (Jira)
angerszhu created SPARK-36740: - Summary: collection operators should handle duplicated NaN Key: SPARK-36740 URL: https://issues.apache.org/jira/browse/SPARK-36740 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-36692) Improve Error statement when requesting thread dump while executor already stopped

2021-09-08 Thread angerszhu (Jira)
angerszhu created SPARK-36692: - Summary: Improve Error statement when requesting thread dump while executor already stopped Key: SPARK-36692 URL: https://issues.apache.org/jira/browse/SPARK-36692

[jira] [Updated] (SPARK-36691) PythonRunner failed should pass error message to ApplicationMaster too

2021-09-08 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36691: -- Description: In current pyspark, stderr and stdout are print together, if py-script failed, will

[jira] [Created] (SPARK-36691) PythonRunner failed should pass error message to ApplicationMaster too

2021-09-08 Thread angerszhu (Jira)
angerszhu created SPARK-36691: - Summary: PythonRunner failed should pass error message to ApplicationMaster too Key: SPARK-36691 URL: https://issues.apache.org/jira/browse/SPARK-36691 Project: Spark

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-09-06 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410492#comment-17410492 ] angerszhu commented on SPARK-35181: --- {code} 2021-08-28 22:38:19 INFO CodecPool:184 - Got brand-new

[jira] [Commented] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-09-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410319#comment-17410319 ] angerszhu commented on SPARK-36622: --- +1 on this > spark.history.kerberos.principal doesn't take value

[jira] [Created] (SPARK-36650) ApplicationMaster shutdown hook should catch timeout exception

2021-09-02 Thread angerszhu (Jira)
angerszhu created SPARK-36650: - Summary: ApplicationMaster shutdown hook should catch timeout exception Key: SPARK-36650 URL: https://issues.apache.org/jira/browse/SPARK-36650 Project: Spark

[jira] [Created] (SPARK-36624) When application killed, sc should not exit with code 0

2021-08-31 Thread angerszhu (Jira)
angerszhu created SPARK-36624: - Summary: When application killed, sc should not exit with code 0 Key: SPARK-36624 URL: https://issues.apache.org/jira/browse/SPARK-36624 Project: Spark Issue

[jira] [Updated] (SPARK-36615) SparkContext should register shutdown hook earlier

2021-08-31 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36615: -- Parent: SPARK-36623 Issue Type: Sub-task (was: Bug) > SparkContext should register shutdown

[jira] [Updated] (SPARK-36540) AM should not just finish with Success when dissconnected

2021-08-31 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36540: -- Parent: SPARK-36623 Issue Type: Sub-task (was: Bug) > AM should not just finish with Success

[jira] [Created] (SPARK-36623) Improve spark exit logic

2021-08-31 Thread angerszhu (Jira)
angerszhu created SPARK-36623: - Summary: Improve spark exit logic Key: SPARK-36623 URL: https://issues.apache.org/jira/browse/SPARK-36623 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-08-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407090#comment-17407090 ] angerszhu commented on SPARK-35181: --- [~dongjoon]Thanks for your information, I will try this. > Use

[jira] [Commented] (SPARK-35181) Use zstd for spark.io.compression.codec by default

2021-08-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407027#comment-17407027 ] angerszhu commented on SPARK-35181: --- Hi [~dongjoon] Can I know if I use it as default in spark-2.4,

[jira] [Created] (SPARK-36615) SparkContext should register shutdown hook earlier

2021-08-30 Thread angerszhu (Jira)
angerszhu created SPARK-36615: - Summary: SparkContext should register shutdown hook earlier Key: SPARK-36615 URL: https://issues.apache.org/jira/browse/SPARK-36615 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-36539) trimNonTopLevelAlias should not change StructType inner alias

2021-08-26 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36539. --- Resolution: Not A Problem > trimNonTopLevelAlias should not change StructType inner alias >

[jira] [Created] (SPARK-36579) Make spark stagingDir can be congfigurable

2021-08-24 Thread angerszhu (Jira)
angerszhu created SPARK-36579: - Summary: Make spark stagingDir can be congfigurable Key: SPARK-36579 URL: https://issues.apache.org/jira/browse/SPARK-36579 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36571) Optimized FileOutputCommitter with StagingDir

2021-08-23 Thread angerszhu (Jira)
angerszhu created SPARK-36571: - Summary: Optimized FileOutputCommitter with StagingDir Key: SPARK-36571 URL: https://issues.apache.org/jira/browse/SPARK-36571 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-32838) Connot insert overwite different partition with same table

2021-08-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32838: -- Parent: SPARK-36562 Issue Type: Sub-task (was: Bug) > Connot insert overwite different

[jira] [Created] (SPARK-36563) dynamicPartitionOverwrite can direct rename to targetPath instead of partition path one by one

2021-08-23 Thread angerszhu (Jira)
angerszhu created SPARK-36563: - Summary: dynamicPartitionOverwrite can direct rename to targetPath instead of partition path one by one Key: SPARK-36563 URL: https://issues.apache.org/jira/browse/SPARK-36563

[jira] [Created] (SPARK-36562) Improve InsertIntoHadoopFsRelation file commit logic

2021-08-23 Thread angerszhu (Jira)
angerszhu created SPARK-36562: - Summary: Improve InsertIntoHadoopFsRelation file commit logic Key: SPARK-36562 URL: https://issues.apache.org/jira/browse/SPARK-36562 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31936) Implement ScriptTransform in sql/core

2021-08-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-31936: -- Fix Version/s: 3.2.0 > Implement ScriptTransform in sql/core > -

[jira] [Commented] (SPARK-31936) Implement ScriptTransform in sql/core

2021-08-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403030#comment-17403030 ] angerszhu commented on SPARK-31936: --- [~Gengliang.Wang] The main work has been done, we can close this.

[jira] [Resolved] (SPARK-31936) Implement ScriptTransform in sql/core

2021-08-23 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-31936. --- Resolution: Implemented > Implement ScriptTransform in sql/core >

[jira] [Updated] (SPARK-36549) Add taskStatus support multiple value to doc

2021-08-19 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36549: -- Summary: Add taskStatus support multiple value to doc (was: Improve doc about [SPARK-34092][SQL]

[jira] [Created] (SPARK-36549) Improve doc about [SPARK-34092][SQL] Support Stage level restful api filter task details by task status

2021-08-19 Thread angerszhu (Jira)
angerszhu created SPARK-36549: - Summary: Improve doc about [SPARK-34092][SQL] Support Stage level restful api filter task details by task status Key: SPARK-36549 URL: https://issues.apache.org/jira/browse/SPARK-36549

[jira] [Updated] (SPARK-36540) AM should not just finish with Success when dissconnected

2021-08-18 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36540: -- Description: We meet a case AM lose connection {code} 21/08/18 02:14:15 ERROR

[jira] [Updated] (SPARK-36540) AM should not just finish with Success when dissconnected

2021-08-18 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36540: -- Description: We meet a case AM lose connection {code} 21/08/18 02:14:15 ERROR

[jira] [Created] (SPARK-36540) AM should not just finish with Success when dissconnected

2021-08-18 Thread angerszhu (Jira)
angerszhu created SPARK-36540: - Summary: AM should not just finish with Success when dissconnected Key: SPARK-36540 URL: https://issues.apache.org/jira/browse/SPARK-36540 Project: Spark Issue

[jira] [Created] (SPARK-36539) trimNonTopLevelAlias should not change StructType inner alias

2021-08-18 Thread angerszhu (Jira)
angerszhu created SPARK-36539: - Summary: trimNonTopLevelAlias should not change StructType inner alias Key: SPARK-36539 URL: https://issues.apache.org/jira/browse/SPARK-36539 Project: Spark

[jira] [Created] (SPARK-36486) Support type constructed string as dat time interval

2021-08-11 Thread angerszhu (Jira)
angerszhu created SPARK-36486: - Summary: Support type constructed string as dat time interval Key: SPARK-36486 URL: https://issues.apache.org/jira/browse/SPARK-36486 Project: Spark Issue Type:

[jira] [Created] (SPARK-36485) Support cast type constructed string to year month interval

2021-08-11 Thread angerszhu (Jira)
angerszhu created SPARK-36485: - Summary: Support cast type constructed string to year month interval Key: SPARK-36485 URL: https://issues.apache.org/jira/browse/SPARK-36485 Project: Spark Issue

[jira] [Updated] (SPARK-36473) Add non-unified memory metrics

2021-08-11 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36473: -- Description: Spark split memory to managed memory (storage + execution) and un-managed memory. And

[jira] [Created] (SPARK-36475) Add doc about spark.shuffle.service.fetch.rdd.enabled

2021-08-11 Thread angerszhu (Jira)
angerszhu created SPARK-36475: - Summary: Add doc about spark.shuffle.service.fetch.rdd.enabled Key: SPARK-36475 URL: https://issues.apache.org/jira/browse/SPARK-36475 Project: Spark Issue Type:

[jira] [Created] (SPARK-36473) Add non-unified memory metrics

2021-08-10 Thread angerszhu (Jira)
angerszhu created SPARK-36473: - Summary: Add non-unified memory metrics Key: SPARK-36473 URL: https://issues.apache.org/jira/browse/SPARK-36473 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36431) Support comparison of ANSI intervals with different fields

2021-08-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394235#comment-17394235 ] angerszhu commented on SPARK-36431: --- Working on this > Support comparison of ANSI intervals with

[jira] [Resolved] (SPARK-36355) NamedExpression add method `withName(newName: String)`

2021-08-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36355. --- Resolution: Not A Problem > NamedExpression add method `withName(newName: String)` >

[jira] [Resolved] (SPARK-36356) RemoveRedundantAlias should keep output schema

2021-08-03 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36356. --- Resolution: Fixed > RemoveRedundantAlias should keep output schema >

[jira] [Created] (SPARK-36391) When fetch chunk throw NPE, improve the error message

2021-08-03 Thread angerszhu (Jira)
angerszhu created SPARK-36391: - Summary: When fetch chunk throw NPE, improve the error message Key: SPARK-36391 URL: https://issues.apache.org/jira/browse/SPARK-36391 Project: Spark Issue Type:

[jira] [Created] (SPARK-36356) RemoveRedundantAlias should keep output schema

2021-07-30 Thread angerszhu (Jira)
angerszhu created SPARK-36356: - Summary: RemoveRedundantAlias should keep output schema Key: SPARK-36356 URL: https://issues.apache.org/jira/browse/SPARK-36356 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36353) RemoveNoopOperators should keep output schema

2021-07-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36353: -- Description: !image-2021-07-30-17-46-59-196.png|width=539,height=220!

[jira] [Commented] (SPARK-36353) RemoveNoopOperators should keep output schema

2021-07-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390458#comment-17390458 ] angerszhu commented on SPARK-36353: --- Raise a pr soon > RemoveNoopOperators should keep output schema

[jira] [Updated] (SPARK-36353) RemoveNoopOperators should keep output schema

2021-07-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36353: -- Attachment: image-2021-07-30-17-46-59-196.png > RemoveNoopOperators should keep output schema >

[jira] [Created] (SPARK-36355) NamedExpression add method `withName(newName: String)`

2021-07-30 Thread angerszhu (Jira)
angerszhu created SPARK-36355: - Summary: NamedExpression add method `withName(newName: String)` Key: SPARK-36355 URL: https://issues.apache.org/jira/browse/SPARK-36355 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36093) (RemoveRedundantAliases should keep output schema name) The result incorrect if the partition path case is inconsistent

2021-07-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36093: -- Summary: (RemoveRedundantAliases should keep output schema name) The result incorrect if the

[jira] [Created] (SPARK-36353) RemoveNoopOperators should keep output schema

2021-07-30 Thread angerszhu (Jira)
angerszhu created SPARK-36353: - Summary: RemoveNoopOperators should keep output schema Key: SPARK-36353 URL: https://issues.apache.org/jira/browse/SPARK-36353 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36352) Spark should check result plan's output schema name

2021-07-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390338#comment-17390338 ] angerszhu commented on SPARK-36352: --- RemoveNoopOperators CollapseProject > Spark should check result

[jira] [Updated] (SPARK-36093) The result incorrect if the partition path case is inconsistent

2021-07-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36093: -- Parent: SPARK-36352 Issue Type: Sub-task (was: Bug) > The result incorrect if the partition

[jira] [Updated] (SPARK-36086) The case of the delta table is inconsistent with parquet

2021-07-30 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36086: -- Parent: SPARK-36352 Issue Type: Sub-task (was: Bug) > The case of the delta table is

[jira] [Created] (SPARK-36352) Spark should check result plan's output schema name

2021-07-30 Thread angerszhu (Jira)
angerszhu created SPARK-36352: - Summary: Spark should check result plan's output schema name Key: SPARK-36352 URL: https://issues.apache.org/jira/browse/SPARK-36352 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36341) In stage page, 'Aggregated Metrics by Executor' the underline displayed when the mouse is moved to the link is blocked

2021-07-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389312#comment-17389312 ] angerszhu commented on SPARK-36341: --- Raise a pr soon > In stage page, 'Aggregated Metrics by

[jira] [Updated] (SPARK-36341) In stage page, 'Aggregated Metrics by Executor' the underline displayed when the mouse is moved to the link is blocked

2021-07-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36341: -- Description: !image-2021-07-29-15-15-50-593.png! (was: !image-2021-07-29-15-15-33-469.png!) > In

[jira] [Updated] (SPARK-36341) In stage page, 'Aggregated Metrics by Executor' the underline displayed when the mouse is moved to the link is blocked

2021-07-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36341: -- Attachment: image-2021-07-29-15-15-50-593.png > In stage page, 'Aggregated Metrics by Executor' the

<    1   2   3   4   5   6   7   8   9   10   >