[jira] [Updated] (SPARK-37651) Use existing active Spark session in all places of pandas API on Spark

2021-12-27 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37651: -- Parent: SPARK-37503 Issue Type: Sub-task (was: Improvement) > Use existing active Spark sessi

[jira] [Updated] (SPARK-37503) Improve SparkSession/PySpark SparkSession startup

2021-12-27 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37503: -- Summary: Improve SparkSession/PySpark SparkSession startup (was: Improve SparkSession startup issue)

[jira] [Updated] (SPARK-37638) Use existing active Spark session instead of SparkSession.getOrCreate in pandas API on Spark

2021-12-27 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37638: -- Parent: SPARK-37503 Issue Type: Sub-task (was: Improvement) > Use existing active Spark sessi

[jira] [Resolved] (SPARK-37503) Improve SparkSession/PySpark SparkSession startup

2021-12-27 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-37503. --- Fix Version/s: 3.3.0 Resolution: Fixed > Improve SparkSession/PySpark SparkSession startup >

[jira] [Commented] (SPARK-37503) Improve SparkSession/PySpark SparkSession startup

2021-12-27 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465987#comment-17465987 ] angerszhu commented on SPARK-37503: --- [~hyukjin.kwon] Move all related pr as subtask to

[jira] [Commented] (SPARK-37507) Add the TO_BINARY() function

2021-12-28 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465989#comment-17465989 ] angerszhu commented on SPARK-37507: --- Checked with [~beliefer] I will take this one. >

[jira] [Commented] (SPARK-37503) Improve SparkSession/PySpark SparkSession startup

2021-12-28 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465990#comment-17465990 ] angerszhu commented on SPARK-37503: --- Seem I didn't see the tag of Resolved, not have t

[jira] [Commented] (SPARK-37507) Add the TO_BINARY() function

2021-12-28 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465995#comment-17465995 ] angerszhu commented on SPARK-37507: --- Yea , ok. > Add the TO_BINARY() function > -

[jira] [Commented] (SPARK-37503) Improve SparkSession/PySpark SparkSession startup

2021-12-28 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466009#comment-17466009 ] angerszhu commented on SPARK-37503: --- Yea, thanks > Improve SparkSession/PySpark Spar

[jira] [Created] (SPARK-37780) QueryExecutionListener should also support SQLConf

2021-12-29 Thread angerszhu (Jira)
angerszhu created SPARK-37780: - Summary: QueryExecutionListener should also support SQLConf Key: SPARK-37780 URL: https://issues.apache.org/jira/browse/SPARK-37780 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-37786) CLONE - StreamingQueryListener should also support SQLConf

2021-12-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466713#comment-17466713 ] angerszhu commented on SPARK-37786: --- Raise a pr soon > CLONE - StreamingQueryListener

[jira] [Created] (SPARK-37786) CLONE - StreamingQueryListener should also support SQLConf

2021-12-29 Thread angerszhu (Jira)
angerszhu created SPARK-37786: - Summary: CLONE - StreamingQueryListener should also support SQLConf Key: SPARK-37786 URL: https://issues.apache.org/jira/browse/SPARK-37786 Project: Spark Issue Ty

[jira] [Updated] (SPARK-37786) StreamingQueryListener should also support SQLConf

2021-12-29 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37786: -- Summary: StreamingQueryListener should also support SQLConf (was: CLONE - StreamingQueryListener shou

[jira] [Created] (SPARK-37816) Create table like should change to current user

2022-01-05 Thread angerszhu (Jira)
angerszhu created SPARK-37816: - Summary: Create table like should change to current user Key: SPARK-37816 URL: https://issues.apache.org/jira/browse/SPARK-37816 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-37816) Create table like should change to current user

2022-01-05 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-37816. --- Resolution: Won't Fix > Create table like should change to current user > --

[jira] [Created] (SPARK-37906) spark-sql should not pass last simple comment to backend

2022-01-13 Thread angerszhu (Jira)
angerszhu created SPARK-37906: - Summary: spark-sql should not pass last simple comment to backend Key: SPARK-37906 URL: https://issues.apache.org/jira/browse/SPARK-37906 Project: Spark Issue Typ

[jira] [Created] (SPARK-37907) StaticInvoke should support ConstantFolding

2022-01-13 Thread angerszhu (Jira)
angerszhu created SPARK-37907: - Summary: StaticInvoke should support ConstantFolding Key: SPARK-37907 URL: https://issues.apache.org/jira/browse/SPARK-37907 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-37907) StaticInvoke should support ConstantFolding

2022-01-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475980#comment-17475980 ] angerszhu commented on SPARK-37907: --- Raise a pr soon > StaticInvoke should support Co

[jira] [Reopened] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2022-01-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu reopened SPARK-27442: --- > ParquetFileFormat fails to read column named with invalid characters > ---

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2022-01-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477089#comment-17477089 ] angerszhu commented on SPARK-27442: --- Yea, will try to fix this. > ParquetFileFormat f

[jira] [Created] (SPARK-37951) Refactor ImageFileFormatSuite

2022-01-17 Thread angerszhu (Jira)
angerszhu created SPARK-37951: - Summary: Refactor ImageFileFormatSuite Key: SPARK-37951 URL: https://issues.apache.org/jira/browse/SPARK-37951 Project: Spark Issue Type: Task Components

[jira] [Updated] (SPARK-37951) Refactor ImageFileFormatSuite

2022-01-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37951: -- Description: Not use standard API, sometimes failed, optimize it. > Refactor ImageFileFormatSuite > --

[jira] [Created] (SPARK-37965) Remove check field name when reading existing data in parquet

2022-01-19 Thread angerszhu (Jira)
angerszhu created SPARK-37965: - Summary: Remove check field name when reading existing data in parquet Key: SPARK-37965 URL: https://issues.apache.org/jira/browse/SPARK-37965 Project: Spark Issu

[jira] [Updated] (SPARK-37965) Remove check field name when reading/writing existing data in ORC

2022-01-19 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37965: -- Description: Remove check field name when reading existing data in Orc > Remove check field name when

[jira] [Updated] (SPARK-37965) Remove check field name when reading/writing existing data in ORC

2022-01-19 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37965: -- Summary: Remove check field name when reading/writing existing data in ORC (was: Remove check field n

[jira] [Created] (SPARK-37966) Static insert should write _SUCCESS under partition path

2022-01-19 Thread angerszhu (Jira)
angerszhu created SPARK-37966: - Summary: Static insert should write _SUCCESS under partition path Key: SPARK-37966 URL: https://issues.apache.org/jira/browse/SPARK-37966 Project: Spark Issue Type

[jira] [Created] (SPARK-37967) Literal support ObjectType

2022-01-19 Thread angerszhu (Jira)
angerszhu created SPARK-37967: - Summary: Literal support ObjectType Key: SPARK-37967 URL: https://issues.apache.org/jira/browse/SPARK-37967 Project: Spark Issue Type: Task Components: S

[jira] [Updated] (SPARK-37967) ConstantFolding/ Literal.create support ObjectType

2022-01-19 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37967: -- Summary: ConstantFolding/ Literal.create support ObjectType (was: Literal support ObjectType) > Cons

[jira] [Created] (SPARK-37969) Hive Serde insert should check schema before execution

2022-01-20 Thread angerszhu (Jira)
angerszhu created SPARK-37969: - Summary: Hive Serde insert should check schema before execution Key: SPARK-37969 URL: https://issues.apache.org/jira/browse/SPARK-37969 Project: Spark Issue Type:

[jira] [Created] (SPARK-37985) Fix flaky test SPARK-37578

2022-01-21 Thread angerszhu (Jira)
angerszhu created SPARK-37985: - Summary: Fix flaky test SPARK-37578 Key: SPARK-37985 URL: https://issues.apache.org/jira/browse/SPARK-37985 Project: Spark Issue Type: Task Components: S

[jira] [Created] (SPARK-38043) Refactor FileDataSourceBaseSuite

2022-01-27 Thread angerszhu (Jira)
angerszhu created SPARK-38043: - Summary: Refactor FileDataSourceBaseSuite Key: SPARK-38043 URL: https://issues.apache.org/jira/browse/SPARK-38043 Project: Spark Issue Type: Task Compone

[jira] [Commented] (SPARK-35531) Can not insert into hive bucket table if create table with upper case schema

2022-02-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488612#comment-17488612 ] angerszhu commented on SPARK-35531: --- Sure. > Can not insert into hive bucket table if

[jira] [Created] (SPARK-38150) Update comment of RelationConversions

2022-02-08 Thread angerszhu (Jira)
angerszhu created SPARK-38150: - Summary: Update comment of RelationConversions Key: SPARK-38150 URL: https://issues.apache.org/jira/browse/SPARK-38150 Project: Spark Issue Type: Task Co

[jira] [Created] (SPARK-38197) Improve error message of BlockManager.fetchRemoteManagedBuffer

2022-02-13 Thread angerszhu (Jira)
angerszhu created SPARK-38197: - Summary: Improve error message of BlockManager.fetchRemoteManagedBuffer Key: SPARK-38197 URL: https://issues.apache.org/jira/browse/SPARK-38197 Project: Spark Iss

[jira] [Created] (SPARK-38215) InsertIntoHiveDir support convert metadata

2022-02-15 Thread angerszhu (Jira)
angerszhu created SPARK-38215: - Summary: InsertIntoHiveDir support convert metadata Key: SPARK-38215 URL: https://issues.apache.org/jira/browse/SPARK-38215 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-46006) YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop

2023-11-20 Thread angerszhu (Jira)
angerszhu created SPARK-46006: - Summary: YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop Key: SPARK-46006 URL: https://issues.apache.org/jira/browse/SPARK-46006

[jira] [Updated] (SPARK-46006) YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop

2023-11-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46006: -- Description: We meet a case that user call sc.stop() after run all custom code, but stuck in some pla

[jira] [Updated] (SPARK-46006) YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop

2023-11-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46006: -- Description: We meet a case that user call sc.stop() after run all custom code, but stuck in some pla

[jira] [Updated] (SPARK-46006) YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop

2023-11-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46006: -- Description: We meet a case that user call sc.stop() after run all custom code, but stuck in some pla

[jira] [Updated] (SPARK-46006) YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop

2023-11-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46006: -- Attachment: image-2023-11-20-17-56-45-212.png > YarnAllocator miss clean targetNumExecutorsPerResource

[jira] [Updated] (SPARK-46006) YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop

2023-11-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46006: -- Attachment: image-2023-11-20-17-56-56-507.png > YarnAllocator miss clean targetNumExecutorsPerResource

[jira] [Updated] (SPARK-46006) YarnAllocator miss clean targetNumExecutorsPerResourceProfileId after YarnSchedulerBackend call stop

2023-11-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46006: -- Description: We meet a case that user call sc.stop() after run all custom code, but stuck in some pla

[jira] [Created] (SPARK-46034) SparkContext add file should also copy file to local root path

2023-11-21 Thread angerszhu (Jira)
angerszhu created SPARK-46034: - Summary: SparkContext add file should also copy file to local root path Key: SPARK-46034 URL: https://issues.apache.org/jira/browse/SPARK-46034 Project: Spark Iss

[jira] [Updated] (SPARK-46034) SparkContext add file should also copy file to local root path

2023-11-21 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46034: -- Description: For below case, it failed with FileNotFoundException {code:java} add jar hdfs://path/sear

[jira] [Created] (SPARK-46741) CacheTable AsSelect should inherit from CTEInChildren to make sure it can be matched

2024-01-16 Thread angerszhu (Jira)
angerszhu created SPARK-46741: - Summary: CacheTable AsSelect should inherit from CTEInChildren to make sure it can be matched Key: SPARK-46741 URL: https://issues.apache.org/jira/browse/SPARK-46741 Projec

[jira] [Updated] (SPARK-46741) CacheTable AsSelect should inherit from CTEInChildren to make sure it can be matched

2024-01-16 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46741: -- Attachment: image-2024-01-17-11-48-28-867.png > CacheTable AsSelect should inherit from CTEInChildren

[jira] [Updated] (SPARK-46741) CacheTable AsSelect should inherit from CTEInChildren to make sure it can be matched

2024-01-16 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-46741: -- Description: Current code since CaheTableAsSelelct not inherit CETInChildren,  still return forceInli

[jira] [Created] (SPARK-39527) V2Catalog rename not support newIdent with catalog

2022-06-19 Thread angerszhu (Jira)
angerszhu created SPARK-39527: - Summary: V2Catalog rename not support newIdent with catalog Key: SPARK-39527 URL: https://issues.apache.org/jira/browse/SPARK-39527 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-39776) Join‘ verbose string didn't contains JoinType

2022-07-14 Thread angerszhu (Jira)
angerszhu created SPARK-39776: - Summary: Join‘ verbose string didn't contains JoinType Key: SPARK-39776 URL: https://issues.apache.org/jira/browse/SPARK-39776 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-39776) Join‘ verbose string didn't contains JoinType

2022-07-14 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-39776: -- Description: Current verbose string don't have joinType {code:java} override def verboseStringWithOpe

[jira] [Updated] (SPARK-39776) Join‘ verbose string didn't contains JoinType

2022-07-14 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-39776: -- Description: Current verbose string don't have joinType {code:java} (5) BroadcastHashJoin [codegen

[jira] [Created] (SPARK-40019) Refactor comment of ArrayType and MapType

2022-08-09 Thread angerszhu (Jira)
angerszhu created SPARK-40019: - Summary: Refactor comment of ArrayType and MapType Key: SPARK-40019 URL: https://issues.apache.org/jira/browse/SPARK-40019 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-40019) Refactor comment of ArrayType

2022-08-09 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-40019: -- Summary: Refactor comment of ArrayType (was: Refactor comment of ArrayType and MapType) > Refactor c

[jira] [Created] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread angerszhu (Jira)
angerszhu created SPARK-42698: - Summary: Client mode submit task client should keep same exitcode with AM Key: SPARK-42698 URL: https://issues.apache.org/jira/browse/SPARK-42698 Project: Spark I

[jira] [Updated] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-42698: -- Description: ``` try { app.start(childArgs.toArray, sparkConf) } catch { case t: T

[jira] [Updated] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-42698: -- Parent: SPARK-36623 Issue Type: Sub-task (was: Bug) > Client mode submit task client should k

[jira] [Created] (SPARK-42699) SparkConnectServer should make client and AM same exit code

2023-03-07 Thread angerszhu (Jira)
angerszhu created SPARK-42699: - Summary: SparkConnectServer should make client and AM same exit code Key: SPARK-42699 URL: https://issues.apache.org/jira/browse/SPARK-42699 Project: Spark Issue

[jira] [Created] (SPARK-43064) Spark SQL CLI SQL tab should only show once statement once

2023-04-07 Thread angerszhu (Jira)
angerszhu created SPARK-43064: - Summary: Spark SQL CLI SQL tab should only show once statement once Key: SPARK-43064 URL: https://issues.apache.org/jira/browse/SPARK-43064 Project: Spark Issue Ty

[jira] [Updated] (SPARK-43064) Spark SQL CLI SQL tab should only show once statement once

2023-04-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-43064: -- Attachment: screenshot-1.png > Spark SQL CLI SQL tab should only show once statement once > --

[jira] [Updated] (SPARK-43064) Spark SQL CLI SQL tab should only show once statement once

2023-04-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-43064: -- Description: !screenshot-1.png! > Spark SQL CLI SQL tab should only show once statement once > -

[jira] [Updated] (SPARK-43064) Spark SQL CLI SQL tab should only show once statement once

2023-04-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-43064: -- Description: !screenshot-1.png|width=996,height=554! (was: !screenshot-1.png! ) > Spark SQL CLI SQL

<    5   6   7   8   9   10