[jira] [Assigned] (SPARK-42576) Add 2nd groupBy method to Dataset

2023-02-24 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned SPARK-42576: Assignee: Rui Wang > Add 2nd groupBy method to Dataset > - > >

[jira] [Commented] (SPARK-42576) Add 2nd groupBy method to Dataset

2023-02-24 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693406#comment-17693406 ] Herman van Hövell commented on SPARK-42576: --- cc [~ruiwcdl]  > Add 2nd groupBy method to

[jira] [Created] (SPARK-42576) Add 2nd groupBy method to Dataset

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42576: - Summary: Add 2nd groupBy method to Dataset Key: SPARK-42576 URL: https://issues.apache.org/jira/browse/SPARK-42576 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-42576) Add 2nd groupBy method to Dataset

2023-02-24 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-42576: -- Summary: Add 2nd groupBy method to Dataset (was: Add s2nd groupBy method to Dataset)

[jira] [Updated] (SPARK-42576) Add s2nd groupBy method to Dataset

2023-02-24 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-42576: -- Summary: Add s2nd groupBy method to Dataset (was: Add 2nd groupBy method to Dataset)

[jira] [Resolved] (SPARK-42541) Support Pivot with provided pivot column values

2023-02-24 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42541. --- Fix Version/s: 3.4.1 Resolution: Fixed > Support Pivot with provided pivot

[jira] [Assigned] (SPARK-42574) DataFrame.toPandas should handle duplicated column names

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42574: Assignee: Apache Spark > DataFrame.toPandas should handle duplicated column names >

[jira] [Commented] (SPARK-42574) DataFrame.toPandas should handle duplicated column names

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693405#comment-17693405 ] Apache Spark commented on SPARK-42574: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42574) DataFrame.toPandas should handle duplicated column names

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42574: Assignee: (was: Apache Spark) > DataFrame.toPandas should handle duplicated column

[jira] [Assigned] (SPARK-42575) Replace `AnyFunSuite` with `ConnectFunSuite` for scala client tests

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42575: Assignee: (was: Apache Spark) > Replace `AnyFunSuite` with `ConnectFunSuite` for

[jira] [Assigned] (SPARK-42575) Replace `AnyFunSuite` with `ConnectFunSuite` for scala client tests

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42575: Assignee: Apache Spark > Replace `AnyFunSuite` with `ConnectFunSuite` for scala client

[jira] [Commented] (SPARK-42575) Replace `AnyFunSuite` with `ConnectFunSuite` for scala client tests

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693404#comment-17693404 ] Apache Spark commented on SPARK-42575: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Commented] (SPARK-42564) Implement Dataset.version and Dataset.time

2023-02-24 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693401#comment-17693401 ] BingKun Pan commented on SPARK-42564: - I work on it. > Implement Dataset.version and Dataset.time >

[jira] [Created] (SPARK-42575) Replace `AnyFunSuite` with `ConnectFunSuite` for scala client tests

2023-02-24 Thread Zhen Li (Jira)
Zhen Li created SPARK-42575: --- Summary: Replace `AnyFunSuite` with `ConnectFunSuite` for scala client tests Key: SPARK-42575 URL: https://issues.apache.org/jira/browse/SPARK-42575 Project: Spark

[jira] [Created] (SPARK-42574) DataFrame.toPandas should handle duplicated column names

2023-02-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-42574: - Summary: DataFrame.toPandas should handle duplicated column names Key: SPARK-42574 URL: https://issues.apache.org/jira/browse/SPARK-42574 Project: Spark

[jira] [Assigned] (SPARK-42568) SparkConnectStreamHandler should manage configs properly while creating plans.

2023-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42568: Assignee: Takuya Ueshin > SparkConnectStreamHandler should manage configs properly while

[jira] [Resolved] (SPARK-42568) SparkConnectStreamHandler should manage configs properly while creating plans.

2023-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42568. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40165

[jira] [Updated] (SPARK-41834) Implement SparkSession.conf

2023-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41834: - Fix Version/s: 3.4.0 (was: 3.5.0) > Implement SparkSession.conf >

[jira] [Assigned] (SPARK-41834) Implement SparkSession.conf

2023-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41834: Assignee: Takuya Ueshin > Implement SparkSession.conf > --- > >

[jira] [Resolved] (SPARK-41834) Implement SparkSession.conf

2023-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41834. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40150

[jira] [Commented] (SPARK-42573) Enable binary compatibility tests for SparkSession/Dataset/Column/functions

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693397#comment-17693397 ] Apache Spark commented on SPARK-42573: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42573) Enable binary compatibility tests for SparkSession/Dataset/Column/functions

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42573: Assignee: Apache Spark > Enable binary compatibility tests for

[jira] [Assigned] (SPARK-42573) Enable binary compatibility tests for SparkSession/Dataset/Column/functions

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42573: Assignee: (was: Apache Spark) > Enable binary compatibility tests for

[jira] [Created] (SPARK-42573) Enable binary compatibility tests for SparkSession/Dataset/Column/functions

2023-02-24 Thread Zhen Li (Jira)
Zhen Li created SPARK-42573: --- Summary: Enable binary compatibility tests for SparkSession/Dataset/Column/functions Key: SPARK-42573 URL: https://issues.apache.org/jira/browse/SPARK-42573 Project: Spark

[jira] [Updated] (SPARK-42497) Support of pandas API on Spark for Spark Connect.

2023-02-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42497: - Epic Link: SPARK-39375 Issue Type: Umbrella (was: New Feature) Summary: Support of

[jira] [Updated] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

2023-02-24 Thread Wei Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu updated SPARK-42572: Description: SPARK-42484 Changed the logic for  > Logic error for StateStore.validateStateRowFormat >

[jira] [Updated] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

2023-02-24 Thread Wei Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu updated SPARK-42572: Description: SPARK-42484 Changed the logic of whether to check state store format in

[jira] [Created] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

2023-02-24 Thread Wei Liu (Jira)
Wei Liu created SPARK-42572: --- Summary: Logic error for StateStore.validateStateRowFormat Key: SPARK-42572 URL: https://issues.apache.org/jira/browse/SPARK-42572 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-42571) Provide a mode to replace Py4J for local communication

2023-02-24 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42571: Summary: Provide a mode to replace Py4J for local communication Key: SPARK-42571 URL: https://issues.apache.org/jira/browse/SPARK-42571 Project: Spark Issue

[jira] [Assigned] (SPARK-42561) Add TempView APIs to Dataset

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42561: Assignee: Apache Spark (was: Rui Wang) > Add TempView APIs to Dataset >

[jira] [Commented] (SPARK-42561) Add TempView APIs to Dataset

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693388#comment-17693388 ] Apache Spark commented on SPARK-42561: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42561) Add TempView APIs to Dataset

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42561: Assignee: Rui Wang (was: Apache Spark) > Add TempView APIs to Dataset >

[jira] [Assigned] (SPARK-42561) Add TempView APIs to Dataset

2023-02-24 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned SPARK-42561: Assignee: Rui Wang > Add TempView APIs to Dataset > > >

[jira] [Commented] (SPARK-42570) Fix DataFrameReader to use the default source

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693383#comment-17693383 ] Apache Spark commented on SPARK-42570: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42570) Fix DataFrameReader to use the default source

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42570: Assignee: (was: Apache Spark) > Fix DataFrameReader to use the default source >

[jira] [Assigned] (SPARK-42570) Fix DataFrameReader to use the default source

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42570: Assignee: Apache Spark > Fix DataFrameReader to use the default source >

[jira] [Created] (SPARK-42570) Fix DataFrameReader to use the default source

2023-02-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-42570: - Summary: Fix DataFrameReader to use the default source Key: SPARK-42570 URL: https://issues.apache.org/jira/browse/SPARK-42570 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-42565) Error log improve ment for the lock acquisition of RocksDB state store instance

2023-02-24 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42565: Assignee: Huanli Wang > Error log improve ment for the lock acquisition of RocksDB state

[jira] [Resolved] (SPARK-42565) Error log improve ment for the lock acquisition of RocksDB state store instance

2023-02-24 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42565. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40161

[jira] [Assigned] (SPARK-42568) SparkConnectStreamHandler should manage configs properly while creating plans.

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42568: Assignee: Apache Spark > SparkConnectStreamHandler should manage configs properly while

[jira] [Commented] (SPARK-42569) Throw unsupported exceptions for non-supported API

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693362#comment-17693362 ] Apache Spark commented on SPARK-42569: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42568) SparkConnectStreamHandler should manage configs properly while creating plans.

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42568: Assignee: (was: Apache Spark) > SparkConnectStreamHandler should manage configs

[jira] [Assigned] (SPARK-42569) Throw unsupported exceptions for non-supported API

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42569: Assignee: Rui Wang (was: Apache Spark) > Throw unsupported exceptions for non-supported

[jira] [Assigned] (SPARK-42569) Throw unsupported exceptions for non-supported API

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42569: Assignee: Apache Spark (was: Rui Wang) > Throw unsupported exceptions for non-supported

[jira] [Commented] (SPARK-42568) SparkConnectStreamHandler should manage configs properly while creating plans.

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693363#comment-17693363 ] Apache Spark commented on SPARK-42568: -- User 'ueshin' has created a pull request for this issue:

[jira] [Created] (SPARK-42569) Throw unsupported exceptions for non-supported API

2023-02-24 Thread Rui Wang (Jira)
Rui Wang created SPARK-42569: Summary: Throw unsupported exceptions for non-supported API Key: SPARK-42569 URL: https://issues.apache.org/jira/browse/SPARK-42569 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-42568) SparkConnectStreamHandler should manage configs properly while creating plans.

2023-02-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-42568: - Summary: SparkConnectStreamHandler should manage configs properly while creating plans. Key: SPARK-42568 URL: https://issues.apache.org/jira/browse/SPARK-42568

[jira] [Commented] (SPARK-42567) Track state store provider load time and log warning if it exceeds a threshold

2023-02-24 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693355#comment-17693355 ] Anish Shrigondekar commented on SPARK-42567: Sent the PR here:

[jira] [Assigned] (SPARK-42567) Track state store provider load time and log warning if it exceeds a threshold

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42567: Assignee: (was: Apache Spark) > Track state store provider load time and log warning

[jira] [Assigned] (SPARK-42567) Track state store provider load time and log warning if it exceeds a threshold

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42567: Assignee: Apache Spark > Track state store provider load time and log warning if it

[jira] [Commented] (SPARK-42567) Track state store provider load time and log warning if it exceeds a threshold

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693354#comment-17693354 ] Apache Spark commented on SPARK-42567: -- User 'anishshri-db' has created a pull request for this

[jira] [Created] (SPARK-42567) Track state store provider load time and log warning if it exceeds a threshold

2023-02-24 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-42567: -- Summary: Track state store provider load time and log warning if it exceeds a threshold Key: SPARK-42567 URL: https://issues.apache.org/jira/browse/SPARK-42567

[jira] [Commented] (SPARK-42567) Track state store provider load time and log warning if it exceeds a threshold

2023-02-24 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693337#comment-17693337 ] Anish Shrigondekar commented on SPARK-42567: I will send the fix out soon   cc -

[jira] [Commented] (SPARK-42566) RocksDB StateStore lock acquisition should happen after getting input iterator from inputRDD

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693329#comment-17693329 ] Apache Spark commented on SPARK-42566: -- User 'huanliwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-42566) RocksDB StateStore lock acquisition should happen after getting input iterator from inputRDD

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42566: Assignee: Apache Spark > RocksDB StateStore lock acquisition should happen after getting

[jira] [Assigned] (SPARK-42566) RocksDB StateStore lock acquisition should happen after getting input iterator from inputRDD

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42566: Assignee: (was: Apache Spark) > RocksDB StateStore lock acquisition should happen

[jira] [Updated] (SPARK-42566) RocksDB StateStore lock acquisition should happen after getting input iterator from inputRDD

2023-02-24 Thread Huanli Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huanli Wang updated SPARK-42566: Summary: RocksDB StateStore lock acquisition should happen after getting input iterator from

[jira] [Created] (SPARK-42566) RocksDB StateStore lock acquisition should happen before getting input iterator from inputRDD

2023-02-24 Thread Huanli Wang (Jira)
Huanli Wang created SPARK-42566: --- Summary: RocksDB StateStore lock acquisition should happen before getting input iterator from inputRDD Key: SPARK-42566 URL: https://issues.apache.org/jira/browse/SPARK-42566

[jira] [Assigned] (SPARK-42565) Error log improve ment for the lock acquisition of RocksDB state store instance

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42565: Assignee: Apache Spark > Error log improve ment for the lock acquisition of RocksDB

[jira] [Commented] (SPARK-42565) Error log improve ment for the lock acquisition of RocksDB state store instance

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693320#comment-17693320 ] Apache Spark commented on SPARK-42565: -- User 'huanliwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-42565) Error log improve ment for the lock acquisition of RocksDB state store instance

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42565: Assignee: (was: Apache Spark) > Error log improve ment for the lock acquisition of

[jira] [Updated] (SPARK-42286) Fix internal error for valid CASE WHEN expression with CAST when inserting into a table

2023-02-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-42286: --- Fix Version/s: 3.3.3 > Fix internal error for valid CASE WHEN expression with CAST when

[jira] [Updated] (SPARK-42286) Fix internal error for valid CASE WHEN expression with CAST when inserting into a table

2023-02-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-42286: --- Affects Version/s: 3.3.2 3.3.1 3.3.0 > Fix

[jira] [Updated] (SPARK-42286) Fix internal error for valid CASE WHEN expression with CAST when inserting into a table

2023-02-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-42286: --- Affects Version/s: (was: 3.3.0) > Fix internal error for valid CASE WHEN expression

[jira] [Updated] (SPARK-42565) Error log improve ment for the lock acquisition of RocksDB state store instance

2023-02-24 Thread Huanli Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huanli Wang updated SPARK-42565: Summary: Error log improve ment for the lock acquisition of RocksDB state store instance (was:

[jira] [Updated] (SPARK-42565) Error log improvement for RocksDB state store instance lock acquisition

2023-02-24 Thread Huanli Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huanli Wang updated SPARK-42565: Description:   {code:java} "23/02/23 23:57:44 INFO Executor: Running task 2.0 in stage 57.1 (TID

[jira] [Commented] (SPARK-42554) Spark Connect Scala Client

2023-02-24 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693304#comment-17693304 ] Herman van Hövell commented on SPARK-42554: --- I will backfill this with the work we have

[jira] [Created] (SPARK-42564) Implement Dataset.version and Dataset.time

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42564: - Summary: Implement Dataset.version and Dataset.time Key: SPARK-42564 URL: https://issues.apache.org/jira/browse/SPARK-42564 Project: Spark Issue

[jira] [Created] (SPARK-42565) Error log improvement for RocksDB state store instance lock acquisition

2023-02-24 Thread Huanli Wang (Jira)
Huanli Wang created SPARK-42565: --- Summary: Error log improvement for RocksDB state store instance lock acquisition Key: SPARK-42565 URL: https://issues.apache.org/jira/browse/SPARK-42565 Project: Spark

[jira] [Created] (SPARK-42563) Implement SparkSession.newSession

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42563: - Summary: Implement SparkSession.newSession Key: SPARK-42563 URL: https://issues.apache.org/jira/browse/SPARK-42563 Project: Spark Issue Type: New

[jira] [Created] (SPARK-42562) UnresolvedLambdaVariables in python do not need unique names

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42562: - Summary: UnresolvedLambdaVariables in python do not need unique names Key: SPARK-42562 URL: https://issues.apache.org/jira/browse/SPARK-42562 Project:

[jira] [Created] (SPARK-42561) Add TempView APIs to Dataset

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42561: - Summary: Add TempView APIs to Dataset Key: SPARK-42561 URL: https://issues.apache.org/jira/browse/SPARK-42561 Project: Spark Issue Type: New

[jira] [Created] (SPARK-42560) Implement ColumnName

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42560: - Summary: Implement ColumnName Key: SPARK-42560 URL: https://issues.apache.org/jira/browse/SPARK-42560 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-42559) Implement DataFrameNaFunctions

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42559: - Summary: Implement DataFrameNaFunctions Key: SPARK-42559 URL: https://issues.apache.org/jira/browse/SPARK-42559 Project: Spark Issue Type: New

[jira] [Created] (SPARK-42558) Implement DataFrameStatFunctions

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42558: - Summary: Implement DataFrameStatFunctions Key: SPARK-42558 URL: https://issues.apache.org/jira/browse/SPARK-42558 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-42556) Dataset.colregex should link a plan_id when it only matches a single column.

2023-02-24 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-42556: -- Description: When colregex returns a single column it should link the plans plan_id.

[jira] [Created] (SPARK-42557) Add Broadcast to functions

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42557: - Summary: Add Broadcast to functions Key: SPARK-42557 URL: https://issues.apache.org/jira/browse/SPARK-42557 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-42556) Dataset.colregex should link a plan_id when it only matches a single column.

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42556: - Summary: Dataset.colregex should link a plan_id when it only matches a single column. Key: SPARK-42556 URL: https://issues.apache.org/jira/browse/SPARK-42556

[jira] [Created] (SPARK-42555) Add JDBC to DataFrameReader

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42555: - Summary: Add JDBC to DataFrameReader Key: SPARK-42555 URL: https://issues.apache.org/jira/browse/SPARK-42555 Project: Spark Issue Type: New

[jira] [Created] (SPARK-42554) Spark Connect Scala Client

2023-02-24 Thread Jira
Herman van Hövell created SPARK-42554: - Summary: Spark Connect Scala Client Key: SPARK-42554 URL: https://issues.apache.org/jira/browse/SPARK-42554 Project: Spark Issue Type: Epic

[jira] [Updated] (SPARK-42539) User-provided JARs can override Spark's Hive metadata client JARs when using "builtin"

2023-02-24 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-42539: Description: Recently we observed that on version 3.2.0 and Java 8, it is possible for

[jira] [Assigned] (SPARK-41516) Allow jdbc dialects to override the query used to create a table

2023-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41516: --- Assignee: xiaoping.huang > Allow jdbc dialects to override the query used to create a

[jira] [Resolved] (SPARK-41516) Allow jdbc dialects to override the query used to create a table

2023-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41516. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39062

[jira] [Resolved] (SPARK-42533) SSL support for Scala Client

2023-02-24 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42533. --- Fix Version/s: 3.4.1 Assignee: Zhen Li Resolution: Fixed > SSL

[jira] [Assigned] (SPARK-42534) Fix DB2 Limit clause

2023-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42534: --- Assignee: Ivan Sadikov > Fix DB2 Limit clause > > >

[jira] [Resolved] (SPARK-42534) Fix DB2 Limit clause

2023-02-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42534. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40155

[jira] [Commented] (SPARK-42509) WindowGroupLimitExec supports codegen

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693180#comment-17693180 ] Apache Spark commented on SPARK-42509: -- User 'beliefer' has created a pull request for this issue:

[jira] [Updated] (SPARK-42553) NonReserved keyword "interval" can't be column name

2023-02-24 Thread jiang13021 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiang13021 updated SPARK-42553: --- Affects Version/s: 3.3.2 3.3.1 3.3.0 > NonReserved

[jira] [Commented] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

2023-02-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693103#comment-17693103 ] Yuming Wang commented on SPARK-42550: - I can't reproduce this issue: {noformat} spark-sql> select

[jira] [Created] (SPARK-42553) NonReserved keyword "interval" can't be column name

2023-02-24 Thread jiang13021 (Jira)
jiang13021 created SPARK-42553: -- Summary: NonReserved keyword "interval" can't be column name Key: SPARK-42553 URL: https://issues.apache.org/jira/browse/SPARK-42553 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-42488) Upgrade commons-crypto from 1.1.0 to 1.2.0

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42488: Assignee: (was: Apache Spark) > Upgrade commons-crypto from 1.1.0 to 1.2.0 >

[jira] [Assigned] (SPARK-42488) Upgrade commons-crypto from 1.1.0 to 1.2.0

2023-02-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42488: Assignee: Apache Spark > Upgrade commons-crypto from 1.1.0 to 1.2.0 >

[jira] [Updated] (SPARK-42552) Get ParseException when run sql: "SELECT 1 UNION SELECT 1;"

2023-02-24 Thread jiang13021 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiang13021 updated SPARK-42552: --- Summary: Get ParseException when run sql: "SELECT 1 UNION SELECT 1;" (was: Got ParseException when

[jira] [Updated] (SPARK-42552) Got ParseException when run sql: "SELECT 1 UNION SELECT 1;"

2023-02-24 Thread jiang13021 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiang13021 updated SPARK-42552: --- Description: When I run sql {code:java} scala> spark.sql("SELECT 1 UNION SELECT 1;") {code} I get

[jira] [Created] (SPARK-42552) Got ParseException when run sql: "SELECT 1 UNION SELECT 1;"

2023-02-24 Thread jiang13021 (Jira)
jiang13021 created SPARK-42552: -- Summary: Got ParseException when run sql: "SELECT 1 UNION SELECT 1;" Key: SPARK-42552 URL: https://issues.apache.org/jira/browse/SPARK-42552 Project: Spark

[jira] [Reopened] (SPARK-42488) Upgrade commons-crypto from 1.1.0 to 1.2.0

2023-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-42488: --- Assignee: (was: Yang Jie) This is reverted via

[jira] [Updated] (SPARK-42488) Upgrade commons-crypto from 1.1.0 to 1.2.0

2023-02-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42488: -- Fix Version/s: (was: 3.5.0) > Upgrade commons-crypto from 1.1.0 to 1.2.0 >