[jira] [Updated] (SPARK-40158) Remove useless configuration & extract common code for parquet read

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40158: Fix Version/s: (was: 3.4.0) > Remove useless configuration & extract common code for parquet r

[jira] [Updated] (SPARK-40355) Improve pushdown for orc & parquet when cast scenario

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40355: Fix Version/s: (was: 3.4.0) > Improve pushdown for orc & parquet when cast scenario >

[jira] [Resolved] (SPARK-42520) Spark Connect Scala Client: Window

2023-02-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42520. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40107 [https://gith

[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691973#comment-17691973 ] Apache Spark commented on SPARK-26365: -- User 'zwangsheng' has created a pull reques

[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691972#comment-17691972 ] Apache Spark commented on SPARK-26365: -- User 'zwangsheng' has created a pull reques

[jira] [Assigned] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26365: Assignee: (was: Apache Spark) > spark-submit for k8s cluster doesn't propagate exit c

[jira] [Assigned] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26365: Assignee: Apache Spark > spark-submit for k8s cluster doesn't propagate exit code > -

[jira] [Resolved] (SPARK-41283) Feature parity: Functions API in Spark Connect

2023-02-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41283. -- Resolution: Fixed Yup, thanks for pointing this out. > Feature parity: Functions API in Spark

[jira] [Resolved] (SPARK-41283) Feature parity: Functions API in Spark Connect

2023-02-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41283. -- Resolution: Done > Feature parity: Functions API in Spark Connect > --

[jira] [Reopened] (SPARK-41283) Feature parity: Functions API in Spark Connect

2023-02-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-41283: -- > Feature parity: Functions API in Spark Connect > --

[jira] [Commented] (SPARK-42427) Conv should return an error if the internal conversion overflows

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691954#comment-17691954 ] Apache Spark commented on SPARK-42427: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691930#comment-17691930 ] Apache Spark commented on SPARK-41391: -- User 'ritikam2' has created a pull request

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691929#comment-17691929 ] Apache Spark commented on SPARK-41391: -- User 'ritikam2' has created a pull request

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691928#comment-17691928 ] Ritika Maheshwari commented on SPARK-41391: --- I did send a PR > The output col

[jira] [Commented] (SPARK-41283) Feature parity: Functions API in Spark Connect

2023-02-21 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691922#comment-17691922 ] L. C. Hsieh commented on SPARK-41283: - Seems all sub-tasks are resolved? Can we reso

[jira] [Assigned] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42525: Assignee: Apache Spark > collapse two adjacent windows with the same partition/order in s

[jira] [Commented] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691904#comment-17691904 ] Apache Spark commented on SPARK-42525: -- User 'zml1206' has created a pull request f

[jira] [Assigned] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42525: Assignee: (was: Apache Spark) > collapse two adjacent windows with the same partition

[jira] [Commented] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691905#comment-17691905 ] Apache Spark commented on SPARK-42525: -- User 'zml1206' has created a pull request f

[jira] [Updated] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

2023-02-21 Thread zhuml (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuml updated SPARK-42525: -- Description: Extend the CollapseWindow rule to collapse Window nodes, when one window in subquery.   {code:j

[jira] [Assigned] (SPARK-42513) Push down topK through join

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42513: Assignee: (was: Apache Spark) > Push down topK through join > ---

[jira] [Assigned] (SPARK-42513) Push down topK through join

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42513: Assignee: Apache Spark > Push down topK through join > --- > >

[jira] [Commented] (SPARK-42513) Push down topK through join

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691896#comment-17691896 ] Apache Spark commented on SPARK-42513: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-42523) Apache Spark 3.4 release

2023-02-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42523: - Assignee: Xinrong Meng > Apache Spark 3.4 release > > >

[jira] [Created] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

2023-02-21 Thread zhuml (Jira)
zhuml created SPARK-42525: - Summary: collapse two adjacent windows with the same partition/order in subquery Key: SPARK-42525 URL: https://issues.apache.org/jira/browse/SPARK-42525 Project: Spark Is

[jira] [Assigned] (SPARK-42524) Upgrade numpy and pandas in the release Dockerfile

2023-02-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42524: Assignee: Xinrong Meng > Upgrade numpy and pandas in the release Dockerfile > ---

[jira] [Resolved] (SPARK-42524) Upgrade numpy and pandas in the release Dockerfile

2023-02-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42524. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40111 [https://gi

[jira] [Updated] (SPARK-42343) Ignore `IOException` in `handleBlockRemovalFailure` if SparkContext is stopped

2023-02-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42343: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Bug) > Ignore `IOException` in `handle

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691885#comment-17691885 ] Ruifeng Zheng commented on SPARK-41391: --- [~ritikam] thanks, would you mind sending

[jira] [Updated] (SPARK-42357) Log `exitCode` when `SparkContext.stop` starts

2023-02-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42357: -- Parent: SPARK-41550 Issue Type: Sub-task (was: Improvement) > Log `exitCode` when `Sp

[jira] [Commented] (SPARK-40278) Used databricks spark-sql-pref with Spark 3.3 to run 3TB tpcds test failed

2023-02-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691877#comment-17691877 ] Yang Jie commented on SPARK-40278: -- OK, I change the status to `RESOLVED` > Used datab

[jira] [Commented] (SPARK-42509) WindowGroupLimitExec supports codegen

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691876#comment-17691876 ] Apache Spark commented on SPARK-42509: -- User 'beliefer' has created a pull request

[jira] [Resolved] (SPARK-40278) Used databricks spark-sql-pref with Spark 3.3 to run 3TB tpcds test failed

2023-02-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-40278. -- Resolution: Duplicate > Used databricks spark-sql-pref with Spark 3.3 to run 3TB tpcds test failed > -

[jira] [Assigned] (SPARK-42509) WindowGroupLimitExec supports codegen

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42509: Assignee: (was: Apache Spark) > WindowGroupLimitExec supports codegen > -

[jira] [Commented] (SPARK-41933) Provide local mode that automatically starts the server

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691875#comment-17691875 ] Apache Spark commented on SPARK-41933: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-42509) WindowGroupLimitExec supports codegen

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42509: Assignee: Apache Spark > WindowGroupLimitExec supports codegen >

[jira] [Commented] (SPARK-42509) WindowGroupLimitExec supports codegen

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691873#comment-17691873 ] Apache Spark commented on SPARK-42509: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-41933) Provide local mode that automatically starts the server

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691871#comment-17691871 ] Apache Spark commented on SPARK-41933: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-42524) Upgrade numpy and pandas in the release Dockerfile

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42524: Assignee: (was: Apache Spark) > Upgrade numpy and pandas in the release Dockerfile >

[jira] [Assigned] (SPARK-42524) Upgrade numpy and pandas in the release Dockerfile

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42524: Assignee: Apache Spark > Upgrade numpy and pandas in the release Dockerfile > ---

[jira] [Commented] (SPARK-42524) Upgrade numpy and pandas in the release Dockerfile

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691867#comment-17691867 ] Apache Spark commented on SPARK-42524: -- User 'xinrong-meng' has created a pull requ

[jira] [Updated] (SPARK-42524) Upgrade numpy and pandas in the release Dockerfile

2023-02-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42524: - Component/s: (was: PySpark) > Upgrade numpy and pandas in the release Dockerfile > -

[jira] [Created] (SPARK-42524) Upgrade numpy and pandas in the release Dockerfile

2023-02-21 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42524: Summary: Upgrade numpy and pandas in the release Dockerfile Key: SPARK-42524 URL: https://issues.apache.org/jira/browse/SPARK-42524 Project: Spark Issue Type

[jira] [Updated] (SPARK-42184) Setting version to 3.5.0-SNAPSHOT

2023-02-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42184: - Parent: SPARK-42523 Issue Type: Sub-task (was: Task) > Setting version to 3.5.0-SNAPSHO

[jira] [Created] (SPARK-42523) Apache Spark 3.4 release

2023-02-21 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42523: Summary: Apache Spark 3.4 release Key: SPARK-42523 URL: https://issues.apache.org/jira/browse/SPARK-42523 Project: Spark Issue Type: Umbrella Compo

[jira] [Commented] (SPARK-40278) Used databricks spark-sql-pref with Spark 3.3 to run 3TB tpcds test failed

2023-02-21 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691861#comment-17691861 ] XiDuo You commented on SPARK-40278: --- It should work now(3.4). We figure out SQL execut

[jira] [Resolved] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-21 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-42406. Resolution: Fixed Resolved in https://github.com/apache/spark/pull/40080 > [PROTOBUF] Rec

[jira] [Updated] (SPARK-42513) Push down topK through join

2023-02-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42513: Summary: Push down topK through join (was: Push topK through join) > Push down topK through join

[jira] [Resolved] (SPARK-42514) Scala Client add partition transforms functions

2023-02-21 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42514. --- Fix Version/s: 3.4.0 Assignee: Yang Jie Resolution: Fixed > Scala Cl

[jira] [Commented] (SPARK-41775) Implement training functions as input

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691836#comment-17691836 ] Apache Spark commented on SPARK-41775: -- User 'rithwik-db' has created a pull reques

[jira] [Assigned] (SPARK-42522) Fix DataFrameWriterV2 to find the default source

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42522: Assignee: Apache Spark > Fix DataFrameWriterV2 to find the default source > -

[jira] [Commented] (SPARK-42522) Fix DataFrameWriterV2 to find the default source

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691801#comment-17691801 ] Apache Spark commented on SPARK-42522: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-42522) Fix DataFrameWriterV2 to find the default source

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42522: Assignee: (was: Apache Spark) > Fix DataFrameWriterV2 to find the default source > --

[jira] [Created] (SPARK-42522) Fix DataFrameWriterV2 to find the default source

2023-02-21 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-42522: - Summary: Fix DataFrameWriterV2 to find the default source Key: SPARK-42522 URL: https://issues.apache.org/jira/browse/SPARK-42522 Project: Spark Issue Type

[jira] [Commented] (SPARK-42485) SPIP: Shutting down spark structured streaming when the streaming process completed current process

2023-02-21 Thread Mich Talebzadeh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691790#comment-17691790 ] Mich Talebzadeh commented on SPARK-42485: - Hi Boyang,   Please find my respons

[jira] [Assigned] (SPARK-42521) Add NULL values for INSERT commands with user-specified lists of fewer columns than the target table

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42521: Assignee: (was: Apache Spark) > Add NULL values for INSERT commands with user-specifi

[jira] [Assigned] (SPARK-42521) Add NULL values for INSERT commands with user-specified lists of fewer columns than the target table

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42521: Assignee: Apache Spark > Add NULL values for INSERT commands with user-specified lists of

[jira] [Commented] (SPARK-42521) Add NULL values for INSERT commands with user-specified lists of fewer columns than the target table

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691786#comment-17691786 ] Apache Spark commented on SPARK-42521: -- User 'dtenedor' has created a pull request

[jira] [Commented] (SPARK-42521) Add NULL values for INSERT commands with user-specified lists of fewer columns than the target table

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691785#comment-17691785 ] Apache Spark commented on SPARK-42521: -- User 'dtenedor' has created a pull request

[jira] [Created] (SPARK-42521) Add NULL values for INSERT commands with user-specified lists of fewer columns than the target table

2023-02-21 Thread Daniel (Jira)
Daniel created SPARK-42521: -- Summary: Add NULL values for INSERT commands with user-specified lists of fewer columns than the target table Key: SPARK-42521 URL: https://issues.apache.org/jira/browse/SPARK-42521

[jira] [Commented] (SPARK-42520) Spark Connect Scala Client: Window

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691774#comment-17691774 ] Apache Spark commented on SPARK-42520: -- User 'amaliujia' has created a pull request

[jira] [Assigned] (SPARK-42520) Spark Connect Scala Client: Window

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42520: Assignee: Apache Spark (was: Rui Wang) > Spark Connect Scala Client: Window > --

[jira] [Assigned] (SPARK-42520) Spark Connect Scala Client: Window

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42520: Assignee: Rui Wang (was: Apache Spark) > Spark Connect Scala Client: Window > --

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 8:06 PM: --

[jira] [Created] (SPARK-42520) Spark Connect Scala Client: Window

2023-02-21 Thread Rui Wang (Jira)
Rui Wang created SPARK-42520: Summary: Spark Connect Scala Client: Window Key: SPARK-42520 URL: https://issues.apache.org/jira/browse/SPARK-42520 Project: Spark Issue Type: Task Compone

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:56 PM: --

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:35 PM: --

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:34 PM: --

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:33 PM: --

[jira] [Commented] (SPARK-42485) SPIP: Shutting down spark structured streaming when the streaming process completed current process

2023-02-21 Thread Boyang Jerry Peng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691762#comment-17691762 ] Boyang Jerry Peng commented on SPARK-42485: --- [~mich.talebza...@gmail.com] This

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691760#comment-17691760 ] Ritika Maheshwari commented on SPARK-41391: --- A better solution   In *Relatio

[jira] [Assigned] (SPARK-42518) Scala client Write API V2

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42518: Assignee: (was: Apache Spark) > Scala client Write API V2 > -

[jira] [Commented] (SPARK-42518) Scala client Write API V2

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691758#comment-17691758 ] Apache Spark commented on SPARK-42518: -- User 'zhenlineo' has created a pull request

[jira] [Assigned] (SPARK-42518) Scala client Write API V2

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42518: Assignee: Apache Spark > Scala client Write API V2 > - > >

[jira] [Commented] (SPARK-31754) Spark Structured Streaming: NullPointerException in Stream Stream join

2023-02-21 Thread Yoel Benharrous (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691757#comment-17691757 ] Yoel Benharrous commented on SPARK-31754: - Seems relate to this bug  [SPARK-3880

[jira] [Updated] (SPARK-42519) Add more WriteTo tests after Scala Client session config is supported

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li updated SPARK-42519: Description: Add more test cases following the examples in "SparkConnectProtoSuite("WriteTo")" tests and a

[jira] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2023-02-21 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391 ] Ritika Maheshwari deleted comment on SPARK-41391: --- was (Author: ritikam): Modifying the agg method in RelationalGroupedDataset.scala to @scala.annotation.varargs def agg(expr: Column,

[jira] [Updated] (SPARK-42519) Add more WriteTo tests after Scala Client session config is supported

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li updated SPARK-42519: Summary: Add more WriteTo tests after Scala Client session config is supported (was: Scala Client session

[jira] [Created] (SPARK-42519) Scala Client session config

2023-02-21 Thread Zhen Li (Jira)
Zhen Li created SPARK-42519: --- Summary: Scala Client session config Key: SPARK-42519 URL: https://issues.apache.org/jira/browse/SPARK-42519 Project: Spark Issue Type: Improvement Component

[jira] [Commented] (SPARK-42517) Add documentation for Protobuf connector

2023-02-21 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691754#comment-17691754 ] Gengliang Wang commented on SPARK-42517: cc [~rangadi]  > Add documentation for

[jira] [Created] (SPARK-42518) Scala client Write API V2

2023-02-21 Thread Zhen Li (Jira)
Zhen Li created SPARK-42518: --- Summary: Scala client Write API V2 Key: SPARK-42518 URL: https://issues.apache.org/jira/browse/SPARK-42518 Project: Spark Issue Type: Improvement Components:

[jira] [Created] (SPARK-42517) Add documentation for Protobuf connector

2023-02-21 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-42517: -- Summary: Add documentation for Protobuf connector Key: SPARK-42517 URL: https://issues.apache.org/jira/browse/SPARK-42517 Project: Spark Issue Type: Task

[jira] [Closed] (SPARK-42482) Scala client Write API V1

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-42482. --- > Scala client Write API V1 > - > > Key: SPARK-42482 > U

[jira] [Closed] (SPARK-42457) Scala Client Session Read API

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-42457. --- > Scala Client Session Read API > - > > Key: SPARK-42457 >

[jira] [Closed] (SPARK-42202) Scala Client E2E test stop the server gracefully

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-42202. --- > Scala Client E2E test stop the server gracefully > > >

[jira] [Closed] (SPARK-42429) IntelliJ Build issue: value getArgument is not a member of org.mockito.invocation.InvocationOnMock

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-42429. --- > IntelliJ Build issue: value getArgument is not a member of > org.mockito.invocation.InvocationOnMock > --

[jira] [Closed] (SPARK-42172) Compatibility check for Scala Client

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-42172. --- > Compatibility check for Scala Client > > > Key: SPARK-421

[jira] [Closed] (SPARK-42043) Basic Scala Client Result Implementation

2023-02-21 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li closed SPARK-42043. --- > Basic Scala Client Result Implementation > - > > Key:

[jira] [Commented] (SPARK-42002) Implement DataFrameWriterV2 (ReadwriterV2Tests)

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691721#comment-17691721 ] Apache Spark commented on SPARK-42002: -- User 'amaliujia' has created a pull request

[jira] [Commented] (SPARK-42002) Implement DataFrameWriterV2 (ReadwriterV2Tests)

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691720#comment-17691720 ] Apache Spark commented on SPARK-42002: -- User 'amaliujia' has created a pull request

[jira] [Updated] (SPARK-42516) Non-captured session time zone in view creation

2023-02-21 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-42516: - Description: The session time zone config is captured only when it is set explicitly but if it is not th

[jira] [Commented] (SPARK-42516) Non-captured session time zone in view creation

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691715#comment-17691715 ] Apache Spark commented on SPARK-42516: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-42516) Non-captured session time zone in view creation

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42516: Assignee: Max Gekk (was: Apache Spark) > Non-captured session time zone in view creation

[jira] [Assigned] (SPARK-42516) Non-captured session time zone in view creation

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42516: Assignee: Apache Spark (was: Max Gekk) > Non-captured session time zone in view creation

[jira] [Created] (SPARK-42516) Non-captured session time zone in view creation

2023-02-21 Thread Max Gekk (Jira)
Max Gekk created SPARK-42516: Summary: Non-captured session time zone in view creation Key: SPARK-42516 URL: https://issues.apache.org/jira/browse/SPARK-42516 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-42515) ClientE2ETestSuite local test failed

2023-02-21 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691702#comment-17691702 ] Yang Jie commented on SPARK-42515: -- will fix this later > ClientE2ETestSuite local tes

[jira] [Created] (SPARK-42515) ClientE2ETestSuite local test failed

2023-02-21 Thread Yang Jie (Jira)
Yang Jie created SPARK-42515: Summary: ClientE2ETestSuite local test failed Key: SPARK-42515 URL: https://issues.apache.org/jira/browse/SPARK-42515 Project: Spark Issue Type: Bug Compon

[jira] [Assigned] (SPARK-42514) Scala Client add partition transforms functions

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42514: Assignee: (was: Apache Spark) > Scala Client add partition transforms functions > ---

[jira] [Commented] (SPARK-42514) Scala Client add partition transforms functions

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691692#comment-17691692 ] Apache Spark commented on SPARK-42514: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-42514) Scala Client add partition transforms functions

2023-02-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42514: Assignee: Apache Spark > Scala Client add partition transforms functions > --

  1   2   >