[jira] [Created] (SPARK-48111) Move tpcds-1g and docker-integration-tests to daily scheduled jobs

2024-05-03 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48111: Summary: Move tpcds-1g and docker-integration-tests to daily scheduled jobs Key: SPARK-48111 URL: https://issues.apache.org/jira/browse/SPARK-48111 Project: Spark

[jira] [Assigned] (SPARK-48088) Skip tests being failed in client 3.5 <> server 4.0

2024-05-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48088: Assignee: Hyukjin Kwon > Skip tests being failed in client 3.5 <> server 4.0 >

[jira] [Resolved] (SPARK-48088) Skip tests being failed in client 3.5 <> server 4.0

2024-05-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48088. -- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 46334

[jira] [Assigned] (SPARK-48054) Backward compatibility test for Spark Connect

2024-05-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48054: Assignee: Hyukjin Kwon > Backward compatibility test for Spark Connect >

[jira] [Resolved] (SPARK-48054) Backward compatibility test for Spark Connect

2024-05-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48054. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46298

[jira] [Resolved] (SPARK-48056) [CONNECT][PYTHON] Session not found error should automatically retry during reattach

2024-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48056. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46297

[jira] [Created] (SPARK-48090) Streaming exception catch failure in 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48090: Summary: Streaming exception catch failure in 3.5 client <> 4.0 server Key: SPARK-48090 URL: https://issues.apache.org/jira/browse/SPARK-48090 Project: Spark

[jira] [Created] (SPARK-48089) Streaming query listener not working in 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48089: Summary: Streaming query listener not working in 3.5 client <> 4.0 server Key: SPARK-48089 URL: https://issues.apache.org/jira/browse/SPARK-48089 Project: Spark

[jira] [Resolved] (SPARK-48085) ANSI enabled by default brings different results in the tests in 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48085. -- Resolution: Invalid We actually made this all compatible in master branch. I will avoid

[jira] [Updated] (SPARK-48085) ANSI enabled by default brings different results in the tests in 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48085: - Description: {code} == FAIL

[jira] [Updated] (SPARK-48088) Skip tests being failed in client 3.5 <> server 4.0

2024-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48088: - Affects Version/s: 3.5.2 (was: 4.0.0) > Skip tests being failed in

[jira] [Created] (SPARK-48088) Skip tests being failed in client 3.5 <> server 4.0

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48088: Summary: Skip tests being failed in client 3.5 <> server 4.0 Key: SPARK-48088 URL: https://issues.apache.org/jira/browse/SPARK-48088 Project: Spark Issue

[jira] [Created] (SPARK-48087) Python UDTF incompatibility in 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48087: Summary: Python UDTF incompatibility in 3.5 client <> 4.0 server Key: SPARK-48087 URL: https://issues.apache.org/jira/browse/SPARK-48087 Project: Spark

[jira] [Created] (SPARK-48085) ANSI enabled by default brings different results in the tests in 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48085: Summary: ANSI enabled by default brings different results in the tests in 3.5 client <> 4.0 server Key: SPARK-48085 URL: https://issues.apache.org/jira/browse/SPARK-48085

[jira] [Created] (SPARK-48084) pyspark.ml.connect.evaluation not working in 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48084: Summary: pyspark.ml.connect.evaluation not working in 3.5 client <> 4.0 server Key: SPARK-48084 URL: https://issues.apache.org/jira/browse/SPARK-48084 Project: Spark

[jira] [Created] (SPARK-48083) session.copyFromLocalToFs failure with 3.5 client <> 4.0 server

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48083: Summary: session.copyFromLocalToFs failure with 3.5 client <> 4.0 server Key: SPARK-48083 URL: https://issues.apache.org/jira/browse/SPARK-48083 Project: Spark

[jira] [Created] (SPARK-48086) Different Arrow versions in client and server

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48086: Summary: Different Arrow versions in client and server Key: SPARK-48086 URL: https://issues.apache.org/jira/browse/SPARK-48086 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48054) Backward compatibility test for Spark Connect

2024-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48054: - Parent: SPARK-48082 Issue Type: Sub-task (was: Improvement) > Backward compatibility

[jira] [Created] (SPARK-48082) Recover compatibility with Spark Connect client 3.5 <> Spark Connect server 4.0

2024-05-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48082: Summary: Recover compatibility with Spark Connect client 3.5 <> Spark Connect server 4.0 Key: SPARK-48082 URL: https://issues.apache.org/jira/browse/SPARK-48082

[jira] [Updated] (SPARK-45988) Fix `pyspark.pandas.tests.computation.test_apply_func` in Python 3.11

2024-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45988: - Fix Version/s: 3.5.2 > Fix `pyspark.pandas.tests.computation.test_apply_func` in Python 3.11 >

[jira] [Updated] (SPARK-45989) Fix `pyspark.pandas.tests.connect.computation.test_parity_apply_func` in Python 3.11

2024-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45989: - Fix Version/s: 3.5.2 > Fix `pyspark.pandas.tests.connect.computation.test_parity_apply_func` in

[jira] [Resolved] (SPARK-48075) Enforce type checking in from_avro and to_avro functions

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48075. -- Fix Version/s: 4.0.0 Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-48064) Improve error messages for routine related errors

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48064: Assignee: Allison Wang > Improve error messages for routine related errors >

[jira] [Resolved] (SPARK-48064) Improve error messages for routine related errors

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48064. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46310

[jira] [Resolved] (SPARK-43727) Parity returnType check in Spark Connect

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43727. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46300

[jira] [Assigned] (SPARK-48058) `UserDefinedFunction.returnType` parse the DDL string

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48058: Assignee: Ruifeng Zheng > `UserDefinedFunction.returnType` parse the DDL string >

[jira] [Resolved] (SPARK-48058) `UserDefinedFunction.returnType` parse the DDL string

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48058. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46300

[jira] [Resolved] (SPARK-48062) Add pyspark test for SimpleDataSourceStreamingReader

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48062. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46306

[jira] [Assigned] (SPARK-48062) Add pyspark test for SimpleDataSourceStreamingReader

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48062: Assignee: Chaoqin Li > Add pyspark test for SimpleDataSourceStreamingReader >

[jira] [Assigned] (SPARK-46894) Move PySpark error conditions into standalone JSON file

2024-04-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46894: Assignee: Nicholas Chammas > Move PySpark error conditions into standalone JSON file >

[jira] [Resolved] (SPARK-46894) Move PySpark error conditions into standalone JSON file

2024-04-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46894. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44920

[jira] [Resolved] (SPARK-48052) Recover pyspark-connect CI by parent classes

2024-04-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48052. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46294

[jira] [Assigned] (SPARK-48053) SparkSession.createDataFrame should warn for unsupported options

2024-04-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48053: Assignee: Hyukjin Kwon > SparkSession.createDataFrame should warn for unsupported

[jira] [Resolved] (SPARK-48053) SparkSession.createDataFrame should warn for unsupported options

2024-04-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48053. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46295

[jira] [Created] (SPARK-48054) Backward compatibility test for Spark Connect

2024-04-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48054: Summary: Backward compatibility test for Spark Connect Key: SPARK-48054 URL: https://issues.apache.org/jira/browse/SPARK-48054 Project: Spark Issue Type:

[jira] [Created] (SPARK-48053) SparkSession.createDataFrame should warn for unsupported options

2024-04-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48053: Summary: SparkSession.createDataFrame should warn for unsupported options Key: SPARK-48053 URL: https://issues.apache.org/jira/browse/SPARK-48053 Project: Spark

[jira] [Created] (SPARK-48052) Recover pyspark-connect CI by parent classes

2024-04-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-48052: Summary: Recover pyspark-connect CI by parent classes Key: SPARK-48052 URL: https://issues.apache.org/jira/browse/SPARK-48052 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48039) Update the error class for `group.apply`

2024-04-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48039. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46277

[jira] [Resolved] (SPARK-47292) safeMapToJValue should consider when map is null

2024-04-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47292. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46260

[jira] [Assigned] (SPARK-48014) Change the makeFromJava error in EvaluatePython to a user-facing error

2024-04-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48014: Assignee: Allison Wang > Change the makeFromJava error in EvaluatePython to a

[jira] [Resolved] (SPARK-48014) Change the makeFromJava error in EvaluatePython to a user-facing error

2024-04-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48014. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46250

[jira] [Resolved] (SPARK-48024) Enable `UDFParityTests.test_udf_timestamp_ntz`

2024-04-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48024. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46257

[jira] [Resolved] (SPARK-48002) Add Observed metrics test in PySpark StreamingQueryListeners

2024-04-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48002. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46237

[jira] [Assigned] (SPARK-48002) Add Observed metrics test in PySpark StreamingQueryListeners

2024-04-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48002: Assignee: Wei Liu > Add Observed metrics test in PySpark StreamingQueryListeners >

[jira] [Resolved] (SPARK-47993) Drop Python 3.8 support

2024-04-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47993. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46228

[jira] [Updated] (SPARK-47993) Drop Python 3.8 support

2024-04-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47993: - Labels: release-notes (was: release-note) > Drop Python 3.8 support > ---

[jira] [Resolved] (SPARK-47962) Improve doc test in pyspark dataframe

2024-04-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47962. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46189

[jira] [Resolved] (SPARK-47965) Avoid orNull in TypedConfigBuilder and OptionalConfigEntry

2024-04-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47965. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46197

[jira] [Assigned] (SPARK-47965) Avoid orNull in TypedConfigBuilder and OptionalConfigEntry

2024-04-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47965: Assignee: Hyukjin Kwon > Avoid orNull in TypedConfigBuilder and OptionalConfigEntry >

[jira] [Resolved] (SPARK-47964) Hide SQLContext and HiveContext

2024-04-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47964. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46194

[jira] [Updated] (SPARK-47965) Avoid orNull in TypedConfigBuilder

2024-04-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47965: - Issue Type: Improvement (was: Bug) > Avoid orNull in TypedConfigBuilder >

[jira] [Updated] (SPARK-47965) Avoid orNull in TypedConfigBuilder and OptionalConfigEntry

2024-04-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47965: - Summary: Avoid orNull in TypedConfigBuilder and OptionalConfigEntry (was: Avoid orNull in

[jira] [Updated] (SPARK-47965) Avoid orNull in TypedConfigBuilder

2024-04-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47965: - Priority: Minor (was: Major) > Avoid orNull in TypedConfigBuilder >

[jira] [Created] (SPARK-47965) Avoid orNull in TypedConfigBuilder

2024-04-23 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47965: Summary: Avoid orNull in TypedConfigBuilder Key: SPARK-47965 URL: https://issues.apache.org/jira/browse/SPARK-47965 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-47933) Parent Column class for Spark Connect and Spark Classic

2024-04-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47933: Assignee: Hyukjin Kwon > Parent Column class for Spark Connect and Spark Classic >

[jira] [Resolved] (SPARK-47933) Parent Column class for Spark Connect and Spark Classic

2024-04-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47933. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46155

[jira] [Resolved] (SPARK-47903) Add remaining scalar types to the Python variant library

2024-04-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47903. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46122

[jira] [Assigned] (SPARK-47903) Add remaining scalar types to the Python variant library

2024-04-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47903: Assignee: Harsh Motwani > Add remaining scalar types to the Python variant library >

[jira] [Resolved] (SPARK-47890) Add python and scala dataframe variant expression aliases.

2024-04-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47890. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46123

[jira] [Created] (SPARK-47933) Parent Column class for Spark Connect and Spark Classic

2024-04-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47933: Summary: Parent Column class for Spark Connect and Spark Classic Key: SPARK-47933 URL: https://issues.apache.org/jira/browse/SPARK-47933 Project: Spark

[jira] [Commented] (SPARK-47909) Parent DataFrame class for Spark Connect and Spark Classic

2024-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839436#comment-17839436 ] Hyukjin Kwon commented on SPARK-47909: -- Yes, I am working on it today :-). > Parent DataFrame

[jira] [Resolved] (SPARK-47909) Parent DataFrame class for Spark Connect and Spark Classic

2024-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47909. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46129

[jira] [Assigned] (SPARK-47909) Parent DataFrame class for Spark Connect and Spark Classic

2024-04-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47909: Assignee: Hyukjin Kwon > Parent DataFrame class for Spark Connect and Spark Classic >

[jira] [Created] (SPARK-47909) Parent DataFrame class for Spark Connect and Spark Classic

2024-04-18 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47909: Summary: Parent DataFrame class for Spark Connect and Spark Classic Key: SPARK-47909 URL: https://issues.apache.org/jira/browse/SPARK-47909 Project: Spark

[jira] [Assigned] (SPARK-47767) Show offset value in TakeOrderedAndProjectExec

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47767: Assignee: guihuawen > Show offset value in TakeOrderedAndProjectExec >

[jira] [Resolved] (SPARK-47767) Show offset value in TakeOrderedAndProjectExec

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47767. -- Resolution: Fixed Issue resolved by pull request 45931

[jira] [Assigned] (SPARK-47852) Support DataFrameQueryContext for reverse operations

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47852: Assignee: Haejoon Lee > Support DataFrameQueryContext for reverse operations >

[jira] [Resolved] (SPARK-47858) Refactoring the structure for DataFrame error context

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47858. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46063

[jira] [Resolved] (SPARK-47852) Support DataFrameQueryContext for reverse operations

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47852. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46063

[jira] [Assigned] (SPARK-47858) Refactoring the structure for DataFrame error context

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47858: Assignee: Haejoon Lee > Refactoring the structure for DataFrame error context >

[jira] [Resolved] (SPARK-47864) Enhance "Installation" page to cover all installable options

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47864. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46096

[jira] [Assigned] (SPARK-47864) Enhance "Installation" page to cover all installable options

2024-04-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47864: Assignee: Haejoon Lee > Enhance "Installation" page to cover all installable options >

[jira] [Resolved] (SPARK-47891) Improve docstring of mapInPandas

2024-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47891. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46108

[jira] [Resolved] (SPARK-47830) Reeanble ResourceProfileTests for pyspark-connect

2024-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47830. -- Fix Version/s: 4.0.0 Assignee: Hyukjin Kwon Resolution: Fixed fixed in

[jira] [Resolved] (SPARK-47885) Make pyspark.resource compatible with pyspark-connect

2024-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47885. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46100

[jira] [Resolved] (SPARK-47540) SPIP: Pure Python Package (Spark Connect)

2024-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47540. -- Fix Version/s: 4.0.0 Assignee: Hyukjin Kwon Resolution: Done > SPIP: Pure

[jira] [Resolved] (SPARK-47884) Switch ANSI SQL CI job to NON-ANSI SQL CI job

2024-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47884. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46099

[jira] [Updated] (SPARK-47807) Make pyspark.ml compatible with pyspark-connect

2024-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47807: - Summary: Make pyspark.ml compatible with pyspark-connect (was: Make pyspark.ml compatible

[jira] [Created] (SPARK-47885) Make pyspark.resource compatible with pyspark-connect

2024-04-17 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47885: Summary: Make pyspark.resource compatible with pyspark-connect Key: SPARK-47885 URL: https://issues.apache.org/jira/browse/SPARK-47885 Project: Spark Issue

[jira] [Resolved] (SPARK-46375) Add documentation for Python data source API

2024-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46375. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46089

[jira] [Assigned] (SPARK-46375) Add documentation for Python data source API

2024-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46375: Assignee: Allison Wang > Add documentation for Python data source API >

[jira] [Resolved] (SPARK-47877) Speed up test_parity_listener

2024-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47877. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46072

[jira] [Resolved] (SPARK-47760) Reeanble Avro function doctests

2024-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47760. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46055

[jira] [Resolved] (SPARK-47763) Reeanble Protobuf function doctests

2024-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47763. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46055

[jira] [Assigned] (SPARK-47818) Introduce plan cache in SparkConnectPlanner to improve performance of Analyze requests

2024-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47818: Assignee: Xi Lyu > Introduce plan cache in SparkConnectPlanner to improve performance of

[jira] [Resolved] (SPARK-47818) Introduce plan cache in SparkConnectPlanner to improve performance of Analyze requests

2024-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47818. -- Resolution: Fixed Issue resolved by pull request 46012

[jira] [Resolved] (SPARK-47862) Connect generated proots can't be pickled

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47862. -- Resolution: Fixed Issue resolved by pull request 46068

[jira] [Assigned] (SPARK-47862) Connect generated proots can't be pickled

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47862: Assignee: Martin Grund > Connect generated proots can't be pickled >

[jira] [Assigned] (SPARK-47371) XML: Ignore row tags in CDATA Tokenizer

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47371: Assignee: Yousof Hosny > XML: Ignore row tags in CDATA Tokenizer >

[jira] [Resolved] (SPARK-47371) XML: Ignore row tags in CDATA Tokenizer

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47371. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45487

[jira] [Resolved] (SPARK-47866) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47866. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46070

[jira] [Assigned] (SPARK-47866) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47866: Assignee: Dongjoon Hyun > Deflaky PythonForeachWriterSuite >

[jira] [Resolved] (SPARK-47851) Document pyspark-connect package

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47851. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46054

[jira] [Assigned] (SPARK-47851) Document pyspark-connect package

2024-04-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47851: Assignee: Hyukjin Kwon > Document pyspark-connect package >

[jira] [Created] (SPARK-47851) Document pyspark-connect package

2024-04-14 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47851: Summary: Document pyspark-connect package Key: SPARK-47851 URL: https://issues.apache.org/jira/browse/SPARK-47851 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-47757) Reeanble MemoryProfilerParityTests for pyspark-connect

2024-04-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47757: - Summary: Reeanble MemoryProfilerParityTests for pyspark-connect (was: Reeanble

[jira] [Updated] (SPARK-47830) Reeanble ResourceProfileTests for pyspark-connect

2024-04-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47830: - Summary: Reeanble ResourceProfileTests for pyspark-connect (was: Reeanble

[jira] [Resolved] (SPARK-47849) Change release script to release pyspark-connect

2024-04-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47849. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46049

[jira] [Assigned] (SPARK-47849) Change release script to release pyspark-connect

2024-04-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47849: Assignee: Hyukjin Kwon > Change release script to release pyspark-connect >

[jira] [Resolved] (SPARK-47757) Reeanble ResourceProfileTests for pyspark-connect

2024-04-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47757. -- Fix Version/s: 4.0.0 Assignee: Hyukjin Kwon Resolution: Fixed Fixed in

  1   2   3   4   5   6   7   8   9   10   >