[jira] [Updated] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-06 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengZhou updated SPARK-42694: - Description: We are currently using Spark version 3.1.1 in our production environment. We have noticed

[jira] [Created] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-06 Thread FengZhou (Jira)
FengZhou created SPARK-42694: Summary: Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1 Key: SPARK-42694 URL: https://issues.apache.org/jira/browse/SPARK-42694

[jira] [Created] (SPARK-42693) API Auditing

2023-03-06 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42693: Summary: API Auditing Key: SPARK-42693 URL: https://issues.apache.org/jira/browse/SPARK-42693 Project: Spark Issue Type: Sub-task Components: ML,

[jira] [Resolved] (SPARK-42665) `simple udf` test failed using Maven

2023-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42665. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40304

[jira] [Assigned] (SPARK-42665) `simple udf` test failed using Maven

2023-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42665: Assignee: Yang Jie > `simple udf` test failed using Maven >

[jira] [Commented] (SPARK-42559) Implement DataFrameNaFunctions

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697249#comment-17697249 ] Apache Spark commented on SPARK-42559: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-42376) Introduce watermark propagation among operators

2023-03-06 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42376: Assignee: Jungtaek Lim > Introduce watermark propagation among operators >

[jira] [Resolved] (SPARK-42376) Introduce watermark propagation among operators

2023-03-06 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42376. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39931

[jira] [Comment Edited] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697216#comment-17697216 ] Yang Jie edited comment on SPARK-42579 at 3/7/23 3:22 AM: -- [~hvanhovell]  In

[jira] [Updated] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42579: - Attachment: (was: image-2023-03-07-11-00-39-429.png) > Extend function.lit() to match

[jira] [Updated] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42579: - Attachment: (was: image-2023-03-07-11-00-49-379.png) > Extend function.lit() to match

[jira] [Comment Edited] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697216#comment-17697216 ] Yang Jie edited comment on SPARK-42579 at 3/7/23 3:09 AM: -- [~hvanhovell]  In

[jira] [Commented] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697216#comment-17697216 ] Yang Jie commented on SPARK-42579: -- [~hvanhovell]  In fact, if we compare

[jira] [Updated] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42579: - Attachment: image-2023-03-07-11-00-49-379.png > Extend function.lit() to match Literal.apply() >

[jira] [Updated] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42579: - Attachment: image-2023-03-07-11-00-39-429.png > Extend function.lit() to match Literal.apply() >

[jira] [Resolved] (SPARK-42680) Create the helper function withSQLConf for connect's test

2023-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42680. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40296

[jira] [Assigned] (SPARK-42680) Create the helper function withSQLConf for connect's test

2023-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42680: Assignee: jiaan.geng > Create the helper function withSQLConf for connect's test >

[jira] [Resolved] (SPARK-42688) Rename Connect proto Request client_id to session_id

2023-03-06 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42688. --- Fix Version/s: 3.4.1 Assignee: Rui Wang Resolution: Fixed > Rename

[jira] [Updated] (SPARK-42656) Spark Connect Scala Client Shell Script

2023-03-06 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-42656: -- Epic Link: SPARK-42554 > Spark Connect Scala Client Shell Script >

[jira] [Commented] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are

2023-03-06 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697201#comment-17697201 ] LiJie2023 commented on SPARK-42682: --- [~yumwang]  Why does this happen and how to avoid it > When I

[jira] [Commented] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697188#comment-17697188 ] Apache Spark commented on SPARK-42022: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42022: Assignee: (was: Apache Spark) > createDataFrame should autogenerate missing column

[jira] [Assigned] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42022: Assignee: Apache Spark > createDataFrame should autogenerate missing column names >

[jira] [Commented] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-06 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697174#comment-17697174 ] Herman van Hövell commented on SPARK-42579: --- [~LuciferYang] PR 40218 did not completely fix

[jira] [Assigned] (SPARK-42687) Better error message for unspported `Pivot` operator in Structure Streaming

2023-03-06 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42687: Assignee: Huanli Wang > Better error message for unspported `Pivot` operator in

[jira] [Resolved] (SPARK-42687) Better error message for unspported `Pivot` operator in Structure Streaming

2023-03-06 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42687. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40306

[jira] [Assigned] (SPARK-42688) Rename Connect proto Request client_id to session_id

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42688: Assignee: (was: Apache Spark) > Rename Connect proto Request client_id to session_id

[jira] [Commented] (SPARK-42688) Rename Connect proto Request client_id to session_id

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697163#comment-17697163 ] Apache Spark commented on SPARK-42688: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42688) Rename Connect proto Request client_id to session_id

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42688: Assignee: Apache Spark > Rename Connect proto Request client_id to session_id >

[jira] [Commented] (SPARK-42656) Spark Connect Scala Client Shell Script

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697162#comment-17697162 ] Apache Spark commented on SPARK-42656: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Updated] (SPARK-42478) Make a serializable jobTrackerId instead of a non-serializable JobID in FileWriterFactory

2023-03-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42478: -- Fix Version/s: 3.2.4 3.3.3 > Make a serializable jobTrackerId instead of a

[jira] [Assigned] (SPARK-42151) Align UPDATE assignments with table attributes

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42151: Assignee: (was: Apache Spark) > Align UPDATE assignments with table attributes >

[jira] [Assigned] (SPARK-42151) Align UPDATE assignments with table attributes

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42151: Assignee: Apache Spark > Align UPDATE assignments with table attributes >

[jira] [Commented] (SPARK-42151) Align UPDATE assignments with table attributes

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697153#comment-17697153 ] Apache Spark commented on SPARK-42151: -- User 'aokolnychyi' has created a pull request for this

[jira] [Commented] (SPARK-42689) Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697121#comment-17697121 ] Apache Spark commented on SPARK-42689: -- User 'mridulm' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42689) Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42689: Assignee: Apache Spark > Allow ShuffleDriverComponent to declare if shuffle data is

[jira] [Assigned] (SPARK-42689) Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42689: Assignee: (was: Apache Spark) > Allow ShuffleDriverComponent to declare if shuffle

[jira] [Commented] (SPARK-42689) Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697120#comment-17697120 ] Apache Spark commented on SPARK-42689: -- User 'mridulm' has created a pull request for this issue:

[jira] [Created] (SPARK-42692) Implement Dataset.toJson

2023-03-06 Thread Jira
Herman van Hövell created SPARK-42692: - Summary: Implement Dataset.toJson Key: SPARK-42692 URL: https://issues.apache.org/jira/browse/SPARK-42692 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-42691) Implement Dataset.semanticHash

2023-03-06 Thread Jira
Herman van Hövell created SPARK-42691: - Summary: Implement Dataset.semanticHash Key: SPARK-42691 URL: https://issues.apache.org/jira/browse/SPARK-42691 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-42689) Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

2023-03-06 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-42689: Description: Currently, if there is an executor node loss, we assume the shuffle

[jira] [Created] (SPARK-42690) Implement CSV/JSON parsing funcions

2023-03-06 Thread Jira
Herman van Hövell created SPARK-42690: - Summary: Implement CSV/JSON parsing funcions Key: SPARK-42690 URL: https://issues.apache.org/jira/browse/SPARK-42690 Project: Spark Issue Type:

[jira] [Created] (SPARK-42689) Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

2023-03-06 Thread Mridul Muralidharan (Jira)
Mridul Muralidharan created SPARK-42689: --- Summary: Allow ShuffleDriverComponent to declare if shuffle data is reliably stored Key: SPARK-42689 URL: https://issues.apache.org/jira/browse/SPARK-42689

[jira] [Assigned] (SPARK-42688) Rename Connect proto Request client_id to session_id

2023-03-06 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned SPARK-42688: Assignee: (was: Rui Wang) > Rename Connect proto Request client_id to session_id >

[jira] [Resolved] (SPARK-42559) Implement DataFrameNaFunctions

2023-03-06 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42559. --- Fix Version/s: 3.4.1 Resolution: Fixed > Implement DataFrameNaFunctions >

[jira] [Updated] (SPARK-42688) Rename Connect proto Request client_id to session_id

2023-03-06 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-42688: - Summary: Rename Connect proto Request client_id to session_id (was: Rename client_id to session_id) >

[jira] [Created] (SPARK-42688) Rename client_id to session_id

2023-03-06 Thread Rui Wang (Jira)
Rui Wang created SPARK-42688: Summary: Rename client_id to session_id Key: SPARK-42688 URL: https://issues.apache.org/jira/browse/SPARK-42688 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-42673) Make build/mvn build Spark only with the verified maven version

2023-03-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42673: -- Affects Version/s: 3.2.4 3.3.3 (was: 3.5.0)

[jira] [Updated] (SPARK-42673) Make build/mvn build Spark only with the verified maven version

2023-03-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42673: -- Issue Type: Bug (was: Improvement) > Make build/mvn build Spark only with the verified maven

[jira] [Resolved] (SPARK-42673) Make build/mvn build Spark only with the verified maven version

2023-03-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42673. --- Fix Version/s: 3.3.3 3.2.4 3.4.0 Resolution:

[jira] [Assigned] (SPARK-42673) Make build/mvn build Spark only with the verified maven version

2023-03-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42673: - Assignee: Yang Jie > Make build/mvn build Spark only with the verified maven version >

[jira] [Assigned] (SPARK-42687) Better error message for unspported `Pivot` operator in Structure Streaming

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42687: Assignee: (was: Apache Spark) > Better error message for unspported `Pivot` operator

[jira] [Commented] (SPARK-42687) Better error message for unspported `Pivot` operator in Structure Streaming

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697100#comment-17697100 ] Apache Spark commented on SPARK-42687: -- User 'huanliwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-42687) Better error message for unspported `Pivot` operator in Structure Streaming

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42687: Assignee: Apache Spark > Better error message for unspported `Pivot` operator in

[jira] [Created] (SPARK-42687) Better error message for unspported `Pivot` operator in Structure Streaming

2023-03-06 Thread Huanli Wang (Jira)
Huanli Wang created SPARK-42687: --- Summary: Better error message for unspported `Pivot` operator in Structure Streaming Key: SPARK-42687 URL: https://issues.apache.org/jira/browse/SPARK-42687 Project:

[jira] [Updated] (SPARK-42485) SPIP: Shutting down spark structured streaming when the streaming process completed current process

2023-03-06 Thread Mich Talebzadeh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mich Talebzadeh updated SPARK-42485: Attachment: sparkStructuredStreaming01.png > SPIP: Shutting down spark structured

[jira] [Commented] (SPARK-42665) `simple udf` test failed using Maven

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697043#comment-17697043 ] Apache Spark commented on SPARK-42665: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Commented] (SPARK-42665) `simple udf` test failed using Maven

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697042#comment-17697042 ] Apache Spark commented on SPARK-42665: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42665) `simple udf` test failed using Maven

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42665: Assignee: Apache Spark > `simple udf` test failed using Maven >

[jira] [Commented] (SPARK-42665) `simple udf` test failed using Maven

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697041#comment-17697041 ] Apache Spark commented on SPARK-42665: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42665) `simple udf` test failed using Maven

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42665: Assignee: (was: Apache Spark) > `simple udf` test failed using Maven >

[jira] [Commented] (SPARK-42685) optimize byteToString routines

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697036#comment-17697036 ] Apache Spark commented on SPARK-42685: -- User 'alkis' has created a pull request for this issue:

[jira] [Commented] (SPARK-42685) optimize byteToString routines

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697034#comment-17697034 ] Apache Spark commented on SPARK-42685: -- User 'alkis' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42685) optimize byteToString routines

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42685: Assignee: (was: Apache Spark) > optimize byteToString routines >

[jira] [Assigned] (SPARK-42685) optimize byteToString routines

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42685: Assignee: Apache Spark > optimize byteToString routines > --

[jira] [Commented] (SPARK-42656) Spark Connect Scala Client Shell Script

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697033#comment-17697033 ] Apache Spark commented on SPARK-42656: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42686) TaskMemoryManager debug logging is expensive

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42686: Assignee: Apache Spark > TaskMemoryManager debug logging is expensive >

[jira] [Assigned] (SPARK-42686) TaskMemoryManager debug logging is expensive

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42686: Assignee: (was: Apache Spark) > TaskMemoryManager debug logging is expensive >

[jira] [Commented] (SPARK-42686) TaskMemoryManager debug logging is expensive

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697032#comment-17697032 ] Apache Spark commented on SPARK-42686: -- User 'alkis' has created a pull request for this issue:

[jira] [Created] (SPARK-42686) TaskMemoryManager debug logging is expensive

2023-03-06 Thread Alkis Evlogimenos (Jira)
Alkis Evlogimenos created SPARK-42686: - Summary: TaskMemoryManager debug logging is expensive Key: SPARK-42686 URL: https://issues.apache.org/jira/browse/SPARK-42686 Project: Spark Issue

[jira] [Assigned] (SPARK-42683) Automatically rename metadata columns that conflict with data schema columns

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42683: Assignee: Apache Spark > Automatically rename metadata columns that conflict with data

[jira] [Assigned] (SPARK-42683) Automatically rename metadata columns that conflict with data schema columns

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42683: Assignee: (was: Apache Spark) > Automatically rename metadata columns that conflict

[jira] [Commented] (SPARK-42683) Automatically rename metadata columns that conflict with data schema columns

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697017#comment-17697017 ] Apache Spark commented on SPARK-42683: -- User 'ryan-johnson-databricks' has created a pull request

[jira] [Created] (SPARK-42685) optimize byteToString routines

2023-03-06 Thread Alkis Evlogimenos (Jira)
Alkis Evlogimenos created SPARK-42685: - Summary: optimize byteToString routines Key: SPARK-42685 URL: https://issues.apache.org/jira/browse/SPARK-42685 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-42684) v2 catalog should not allow column default value by default

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42684: Assignee: Apache Spark > v2 catalog should not allow column default value by default >

[jira] [Assigned] (SPARK-42684) v2 catalog should not allow column default value by default

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42684: Assignee: (was: Apache Spark) > v2 catalog should not allow column default value by

[jira] [Assigned] (SPARK-42595) Support query inserted partitions after insert data into table when hive.exec.dynamic.partition=true

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42595: Assignee: (was: Apache Spark) > Support query inserted partitions after insert data

[jira] [Commented] (SPARK-42595) Support query inserted partitions after insert data into table when hive.exec.dynamic.partition=true

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697007#comment-17697007 ] Apache Spark commented on SPARK-42595: -- User 'haoyanzhang' has created a pull request for this

[jira] [Assigned] (SPARK-42595) Support query inserted partitions after insert data into table when hive.exec.dynamic.partition=true

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42595: Assignee: Apache Spark > Support query inserted partitions after insert data into table

[jira] [Commented] (SPARK-42684) v2 catalog should not allow column default value by default

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697006#comment-17697006 ] Apache Spark commented on SPARK-42684: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-42684) v2 catalog should not allow column default value by default

2023-03-06 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-42684: --- Summary: v2 catalog should not allow column default value by default Key: SPARK-42684 URL: https://issues.apache.org/jira/browse/SPARK-42684 Project: Spark

[jira] [Created] (SPARK-42683) Automatically rename metadata columns that conflict with data schema columns

2023-03-06 Thread Ryan Johnson (Jira)
Ryan Johnson created SPARK-42683: Summary: Automatically rename metadata columns that conflict with data schema columns Key: SPARK-42683 URL: https://issues.apache.org/jira/browse/SPARK-42683

[jira] [Commented] (SPARK-42412) Initial prototype implementation for PySparkML

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696957#comment-17696957 ] Apache Spark commented on SPARK-42412: -- User 'WeichenXu123' has created a pull request for this

[jira] [Updated] (SPARK-42643) Register Java (aggregate) user-defined functions

2023-03-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42643: - Summary: Register Java (aggregate) user-defined functions (was: Implement

[jira] [Resolved] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are

2023-03-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42682. - Resolution: Invalid This is because the spark context is not fully started. > When I execute

[jira] (SPARK-42680) Create the helper function withSQLConf for connect's test

2023-03-06 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42680 ] jiaan.geng deleted comment on SPARK-42680: was (Author: beliefer): I'm working on. > Create the helper function withSQLConf for connect's test >

[jira] [Assigned] (SPARK-42668) Catch exception while trying to close compressed stream in HDFSStateStoreProvider abort

2023-03-06 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42668: Assignee: Anish Shrigondekar > Catch exception while trying to close compressed stream

[jira] [Resolved] (SPARK-42668) Catch exception while trying to close compressed stream in HDFSStateStoreProvider abort

2023-03-06 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42668. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40273

[jira] [Commented] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are

2023-03-06 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696841#comment-17696841 ] LiJie2023 commented on SPARK-42682: --- Have anyone ever encountered this kind of problem?  

[jira] [Updated] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are s

2023-03-06 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiJie2023 updated SPARK-42682: -- Attachment: image-2023-03-06-18-12-51-441.png > When I execute the spark-shell command, "WARN

[jira] [Updated] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are s

2023-03-06 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiJie2023 updated SPARK-42682: -- Description: !image-2023-03-06-18-12-51-441.png!   2023-03-06 17:11:52,464 WARN yarn.Client:

[jira] [Created] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are s

2023-03-06 Thread LiJie2023 (Jira)
LiJie2023 created SPARK-42682: - Summary: When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are starting up Spark, please wait a while

[jira] [Assigned] (SPARK-42680) Create the helper function withSQLConf for connect's test

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42680: Assignee: (was: Apache Spark) > Create the helper function withSQLConf for connect's

[jira] [Assigned] (SPARK-42680) Create the helper function withSQLConf for connect's test

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42680: Assignee: Apache Spark > Create the helper function withSQLConf for connect's test >

[jira] [Commented] (SPARK-42680) Create the helper function withSQLConf for connect's test

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696824#comment-17696824 ] Apache Spark commented on SPARK-42680: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-42650) link issue SPARK-42550

2023-03-06 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696810#comment-17696810 ] kevinshin commented on SPARK-42650: --- Thank you @[~yumwang]  > link issue SPARK-42550 >

[jira] [Assigned] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42681: Assignee: Apache Spark > Relax ordering constraint for ALTER TABLE ADD|REPLACE column

[jira] [Assigned] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42681: Assignee: (was: Apache Spark) > Relax ordering constraint for ALTER TABLE

[jira] [Commented] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696767#comment-17696767 ] Apache Spark commented on SPARK-42681: -- User 'vitaliili-db' has created a pull request for this

[jira] [Created] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-06 Thread Vitalii Li (Jira)
Vitalii Li created SPARK-42681: -- Summary: Relax ordering constraint for ALTER TABLE ADD|REPLACE column options Key: SPARK-42681 URL: https://issues.apache.org/jira/browse/SPARK-42681 Project: Spark

  1   2   >