[jira] [Updated] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengZhou updated SPARK-42694: - Description: We are currently using Spark version 3.1.1 in our production environment. We have noticed

[jira] [Updated] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengZhou updated SPARK-42694: - Attachment: image-2023-03-07-15-59-27-665.png > Data duplication and loss occur after executing 'insert

[jira] [Updated] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengZhou updated SPARK-42694: - Attachment: image-2023-03-07-15-59-08-818.png > Data duplication and loss occur after executing 'insert

[jira] [Updated] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengZhou updated SPARK-42694: - Priority: Major (was: Blocker) > Data duplication and loss occur after executing 'insert overwrite...'

[jira] [Updated] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengZhou updated SPARK-42694: - Priority: Critical (was: Major) > Data duplication and loss occur after executing 'insert overwrite...'

[jira] [Created] (SPARK-42695) Skew join handling in stream side of broadcast hash join

2023-03-07 Thread Xingchao, Zhang (Jira)
Xingchao, Zhang created SPARK-42695: --- Summary: Skew join handling in stream side of broadcast hash join Key: SPARK-42695 URL: https://issues.apache.org/jira/browse/SPARK-42695 Project: Spark

[jira] [Updated] (SPARK-42695) Skew join handling in stream side of broadcast hash join

2023-03-07 Thread Xingchao, Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingchao, Zhang updated SPARK-42695: Attachment: before-01.png > Skew join handling in stream side of broadcast hash join > ---

[jira] [Updated] (SPARK-42695) Skew join handling in stream side of broadcast hash join

2023-03-07 Thread Xingchao, Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingchao, Zhang updated SPARK-42695: Description: We can extended the current  OptimizeSkewedJoin if data skew detected in stre

[jira] [Created] (SPARK-42696) Speed up parquet reading with Java Vector API

2023-03-07 Thread jiangjiguang0719 (Jira)
jiangjiguang0719 created SPARK-42696: Summary: Speed up parquet reading with Java Vector API Key: SPARK-42696 URL: https://issues.apache.org/jira/browse/SPARK-42696 Project: Spark Issue T

[jira] [Assigned] (SPARK-42695) Skew join handling in stream side of broadcast hash join

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42695: Assignee: (was: Apache Spark) > Skew join handling in stream side of broadcast hash j

[jira] [Commented] (SPARK-42695) Skew join handling in stream side of broadcast hash join

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697306#comment-17697306 ] Apache Spark commented on SPARK-42695: -- User 'xingchaozh' has created a pull reques

[jira] [Assigned] (SPARK-42695) Skew join handling in stream side of broadcast hash join

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42695: Assignee: Apache Spark > Skew join handling in stream side of broadcast hash join > -

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697316#comment-17697316 ] Yuming Wang commented on SPARK-42694: - Could you upgrade to Spark 3.1.3 or Spark 3.3

[jira] [Created] (SPARK-42697) /api/v1/applications return 0 for duration

2023-03-07 Thread Kent Yao (Jira)
Kent Yao created SPARK-42697: Summary: /api/v1/applications return 0 for duration Key: SPARK-42697 URL: https://issues.apache.org/jira/browse/SPARK-42697 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-42679) createDataFrame doesn't work with non-nullable schema.

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42679: - Parent: SPARK-41281 Issue Type: Sub-task (was: Bug) > createDataFrame doesn't work with

[jira] [Commented] (SPARK-42650) link issue SPARK-42550

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697348#comment-17697348 ] Hyukjin Kwon commented on SPARK-42650: -- [~kevinshin] mind self-contained reproducer

[jira] [Updated] (SPARK-42606) Improvements to built in Protobuf functions

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42606: - Component/s: Protobuf (was: Spark Core) > Improvements to built in Protobuf

[jira] [Assigned] (SPARK-42697) /api/v1/applications return 0 for duration

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42697: Assignee: Apache Spark > /api/v1/applications return 0 for duration > ---

[jira] [Assigned] (SPARK-42697) /api/v1/applications return 0 for duration

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42697: Assignee: (was: Apache Spark) > /api/v1/applications return 0 for duration >

[jira] [Resolved] (SPARK-42517) Add documentation for Protobuf connector

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42517. -- Resolution: Duplicate > Add documentation for Protobuf connector > ---

[jira] [Commented] (SPARK-42697) /api/v1/applications return 0 for duration

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697356#comment-17697356 ] Apache Spark commented on SPARK-42697: -- User 'yaooqinn' has created a pull request

[jira] [Created] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread angerszhu (Jira)
angerszhu created SPARK-42698: - Summary: Client mode submit task client should keep same exitcode with AM Key: SPARK-42698 URL: https://issues.apache.org/jira/browse/SPARK-42698 Project: Spark I

[jira] [Updated] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-42698: -- Description: ``` try { app.start(childArgs.toArray, sparkConf) } catch { case t: T

[jira] [Updated] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-42698: -- Parent: SPARK-36623 Issue Type: Sub-task (was: Bug) > Client mode submit task client should k

[jira] [Commented] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697365#comment-17697365 ] Apache Spark commented on SPARK-42698: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42698: Assignee: Apache Spark > Client mode submit task client should keep same exitcode with AM

[jira] [Commented] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697364#comment-17697364 ] Apache Spark commented on SPARK-42698: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-42698) Client mode submit task client should keep same exitcode with AM

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42698: Assignee: (was: Apache Spark) > Client mode submit task client should keep same exitc

[jira] [Created] (SPARK-42699) SparkConnectServer should make client and AM same exit code

2023-03-07 Thread angerszhu (Jira)
angerszhu created SPARK-42699: - Summary: SparkConnectServer should make client and AM same exit code Key: SPARK-42699 URL: https://issues.apache.org/jira/browse/SPARK-42699 Project: Spark Issue

[jira] [Assigned] (SPARK-42699) SparkConnectServer should make client and AM same exit code

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42699: Assignee: (was: Apache Spark) > SparkConnectServer should make client and AM same exi

[jira] [Assigned] (SPARK-42699) SparkConnectServer should make client and AM same exit code

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42699: Assignee: Apache Spark > SparkConnectServer should make client and AM same exit code > --

[jira] [Commented] (SPARK-42699) SparkConnectServer should make client and AM same exit code

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697373#comment-17697373 ] Apache Spark commented on SPARK-42699: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: Yuming Wang > Speed up parquet reading with Java Vector API > ---

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: jiangjiguang0719 (was: Yuming Wang) > Speed up parquet reading with Java Vector

[jira] [Commented] (SPARK-42356) Cannot resolve orderby attributes in DISTINCT

2023-03-07 Thread Zhou Chong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697390#comment-17697390 ] Zhou Chong commented on SPARK-42356: Hi [~blackpig]  could I know more detail reason

[jira] [Created] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-42700: Summary: Add h2 as test dependency of connect-server module Key: SPARK-42700 URL: https://issues.apache.org/jira/browse/SPARK-42700 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-42679) createDataFrame doesn't work with non-nullable schema.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42679: Assignee: (was: Apache Spark) > createDataFrame doesn't work with non-nullable schema

[jira] [Commented] (SPARK-42679) createDataFrame doesn't work with non-nullable schema.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697412#comment-17697412 ] Apache Spark commented on SPARK-42679: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-42679) createDataFrame doesn't work with non-nullable schema.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42679: Assignee: Apache Spark > createDataFrame doesn't work with non-nullable schema. > ---

[jira] [Assigned] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42700: Assignee: Apache Spark > Add h2 as test dependency of connect-server module > ---

[jira] [Assigned] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42700: Assignee: (was: Apache Spark) > Add h2 as test dependency of connect-server module >

[jira] [Commented] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697417#comment-17697417 ] Apache Spark commented on SPARK-42700: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-42701) Add the try_aes_decrypt() function

2023-03-07 Thread Max Gekk (Jira)
Max Gekk created SPARK-42701: Summary: Add the try_aes_decrypt() function Key: SPARK-42701 URL: https://issues.apache.org/jira/browse/SPARK-42701 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-42656) Spark Connect Scala Client Shell Script

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697423#comment-17697423 ] Apache Spark commented on SPARK-42656: -- User 'LuciferYang' has created a pull reque

[jira] [Updated] (SPARK-42701) Add the try_aes_decrypt() function

2023-03-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-42701: - Labels: starter (was: ) > Add the try_aes_decrypt() function > -- > >

[jira] [Commented] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697426#comment-17697426 ] Yang Jie commented on SPARK-42579: -- [~hvanhovell] should we make this one as `Fixed`,  

[jira] [Commented] (SPARK-42701) Add the try_aes_decrypt() function

2023-03-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697429#comment-17697429 ] Max Gekk commented on SPARK-42701: -- [~panbingkun] [~ivoson] [~xleesf] [~YActs] [~lvshao

[jira] [Updated] (SPARK-42551) Support subexpression elimination in FilterExec and JoinExec

2023-03-07 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-42551: Summary: Support subexpression elimination in FilterExec and JoinExec (was: Support subexpression elimina

[jira] [Created] (SPARK-42702) Support parameterized CTE

2023-03-07 Thread Max Gekk (Jira)
Max Gekk created SPARK-42702: Summary: Support parameterized CTE Key: SPARK-42702 URL: https://issues.apache.org/jira/browse/SPARK-42702 Project: Spark Issue Type: New Feature Component

[jira] [Assigned] (SPARK-42702) Support parameterized CTE

2023-03-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42702: Assignee: Max Gekk > Support parameterized CTE > - > > Ke

[jira] [Assigned] (SPARK-42692) Implement Dataset.toJson

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42692: Assignee: (was: Apache Spark) > Implement Dataset.toJson > >

[jira] [Commented] (SPARK-42692) Implement Dataset.toJson

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697467#comment-17697467 ] Apache Spark commented on SPARK-42692: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-42692) Implement Dataset.toJson

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42692: Assignee: Apache Spark > Implement Dataset.toJson > > >

[jira] [Created] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
LiJie2023 created SPARK-42703: - Summary: How to use Fair Scheduler Pools Key: SPARK-42703 URL: https://issues.apache.org/jira/browse/SPARK-42703 Project: Spark Issue Type: Question Comp

[jira] [Updated] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiJie2023 updated SPARK-42703: -- Description: I have two questions to ask: #   I wrote a demo referring to the official website, but i

[jira] [Commented] (SPARK-42690) Implement CSV/JSON parsing funcions

2023-03-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697489#comment-17697489 ] Yang Jie commented on SPARK-42690: -- {code:java} message Parse { // (Required) Input r

[jira] [Updated] (SPARK-42683) Automatically rename metadata columns that conflict with data schema columns

2023-03-07 Thread Ryan Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Johnson updated SPARK-42683: - Target Version/s: 3.5.0 (was: 3.4.0) > Automatically rename metadata columns that conflict with

[jira] [Created] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Ryan Johnson (Jira)
Ryan Johnson created SPARK-42704: Summary: SubqueryAlias should propagate metadata columns its child already selects Key: SPARK-42704 URL: https://issues.apache.org/jira/browse/SPARK-42704 Project: S

[jira] [Updated] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Ryan Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Johnson updated SPARK-42704: - Description: The `AddMetadataColumns` analyzer rule intends to make resolve available metadata

[jira] [Assigned] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42704: Assignee: (was: Apache Spark) > SubqueryAlias should propagate metadata columns its c

[jira] [Commented] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697565#comment-17697565 ] Apache Spark commented on SPARK-42704: -- User 'ryan-johnson-databricks' has created

[jira] [Assigned] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42704: Assignee: Apache Spark > SubqueryAlias should propagate metadata columns its child alread

[jira] [Commented] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697612#comment-17697612 ] Bjørn Jørgensen commented on SPARK-42703: - Hi [~lijie1912] we have a mail list h

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697614#comment-17697614 ] Bjørn Jørgensen commented on SPARK-42694: - Spark 3.1 [is EOL|https://github.com

[jira] [Resolved] (SPARK-42591) Document SS guide doc for introducing watermark propagation among operators

2023-03-07 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42591. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40215 [https://gi

[jira] [Assigned] (SPARK-42591) Document SS guide doc for introducing watermark propagation among operators

2023-03-07 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42591: Assignee: Jungtaek Lim > Document SS guide doc for introducing watermark propagation amon

[jira] [Commented] (SPARK-42702) Support parameterized CTE

2023-03-07 Thread Entong Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697624#comment-17697624 ] Entong Shen commented on SPARK-42702: - And subqueries > Support parameterized CTE >

[jira] [Created] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-42705: - Summary: SparkSession.sql doesn't return values from commands. Key: SPARK-42705 URL: https://issues.apache.org/jira/browse/SPARK-42705 Project: Spark Issue

[jira] [Commented] (SPARK-41775) Implement training functions as input

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697658#comment-17697658 ] Apache Spark commented on SPARK-41775: -- User 'rithwik-db' has created a pull reques

[jira] [Assigned] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42705: Assignee: (was: Apache Spark) > SparkSession.sql doesn't return values from commands.

[jira] [Commented] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697660#comment-17697660 ] Apache Spark commented on SPARK-42705: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42705: Assignee: Apache Spark > SparkSession.sql doesn't return values from commands. >

[jira] [Resolved] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42022. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40310 [https://gi

[jira] [Assigned] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42022: Assignee: Hyukjin Kwon > createDataFrame should autogenerate missing column names > -

[jira] [Created] (SPARK-42706) List the error class to user-facing documentation.

2023-03-07 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42706: --- Summary: List the error class to user-facing documentation. Key: SPARK-42706 URL: https://issues.apache.org/jira/browse/SPARK-42706 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42706) List the error class to user-facing documentation.

2023-03-07 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697665#comment-17697665 ] Haejoon Lee commented on SPARK-42706: - I'm working on this > List the error class t

[jira] [Updated] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiJie2023 updated SPARK-42703: -- Attachment: image-2023-03-08-09-53-35-867.png > How to use Fair Scheduler Pools >

[jira] [Commented] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697679#comment-17697679 ] LiJie2023 commented on SPARK-42703: --- Sorry, I can't find the answer in StackOverflow a

[jira] [Comment Edited] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697679#comment-17697679 ] LiJie2023 edited comment on SPARK-42703 at 3/8/23 1:57 AM: --- So

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697683#comment-17697683 ] FengZhou commented on SPARK-42694: -- [~bjornjorgensen]  As the current upgrade would hav

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697682#comment-17697682 ] FengZhou commented on SPARK-42694: -- [~yumwang]  This version has been running in produc

[jira] [Comment Edited] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697679#comment-17697679 ] LiJie2023 edited comment on SPARK-42703 at 3/8/23 2:22 AM: --- [~

[jira] [Commented] (SPARK-42496) Introduction Spark Connect at main page.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697699#comment-17697699 ] Apache Spark commented on SPARK-42496: -- User 'allanf-db' has created a pull request

[jira] [Updated] (SPARK-42650) insert overwrite table will casue table location lost if java.lang.ArithmeticException is thrown

2023-03-07 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevinshin updated SPARK-42650: -- Summary: insert overwrite table will casue table location lost if java.lang.ArithmeticException is thr

[jira] [Resolved] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-42705. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40323 [https://

[jira] [Assigned] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-42705: - Assignee: Takuya Ueshin > SparkSession.sql doesn't return values from commands. > -

[jira] [Created] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42707: Summary: Remove experimental warning in developer documentation Key: SPARK-42707 URL: https://issues.apache.org/jira/browse/SPARK-42707 Project: Spark Issue

[jira] [Assigned] (SPARK-39399) proxy-user not working for Spark on k8s in cluster deploy mode

2023-03-07 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39399: Assignee: Shrikant Prasad > proxy-user not working for Spark on k8s in cluster deploy mode >

[jira] [Resolved] (SPARK-39399) proxy-user not working for Spark on k8s in cluster deploy mode

2023-03-07 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39399. -- Fix Version/s: 3.3.3 3.2.4 3.4.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42707: Assignee: Apache Spark > Remove experimental warning in developer documentation > ---

[jira] [Assigned] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42707: Assignee: (was: Apache Spark) > Remove experimental warning in developer documentatio

[jira] [Commented] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697717#comment-17697717 ] Apache Spark commented on SPARK-42707: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Jia Fan (Jira)
Jia Fan created SPARK-42708: --- Summary: The generated protobuf java file is too large Key: SPARK-42708 URL: https://issues.apache.org/jira/browse/SPARK-42708 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-42681. Resolution: Fixed Issue resolved by pull request 40295 [https://github.com/apache/spark/pu

[jira] [Assigned] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-42681: -- Assignee: Vitalii Li > Relax ordering constraint for ALTER TABLE ADD|REPLACE column o

[jira] [Assigned] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42708: Assignee: Apache Spark > The generated protobuf java file is too large >

[jira] [Commented] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697719#comment-17697719 ] Apache Spark commented on SPARK-42708: -- User 'Hisoka-X' has created a pull request

[jira] [Assigned] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42708: Assignee: (was: Apache Spark) > The generated protobuf java file is too large > -

[jira] [Commented] (SPARK-42623) parameter markers not blocked in DDL

2023-03-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697721#comment-17697721 ] Wenchen Fan commented on SPARK-42623: - I think the problem occurs when we store the

[jira] [Updated] (SPARK-42623) parameter markers not blocked in DDL

2023-03-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-42623: Fix Version/s: (was: 3.4.0) > parameter markers not blocked in DDL > -

  1   2   >