[jira] [Created] (SPARK-43353) Migrate remaining session errors into error class

2023-05-02 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43353: --- Summary: Migrate remaining session errors into error class Key: SPARK-43353 URL: https://issues.apache.org/jira/browse/SPARK-43353 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-43348) Support Python 3.8 in PyPy3

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43348: - Assignee: Dongjoon Hyun > Support Python 3.8 in PyPy3 > --- >

[jira] [Assigned] (SPARK-43344) Upgrade mlflow to 3.2.1

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43344: - Assignee: Bjørn Jørgensen > Upgrade mlflow to 3.2.1 > --- > >

[jira] [Resolved] (SPARK-43344) Upgrade mlflow to 3.2.1

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43344. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41019

[jira] [Updated] (SPARK-43348) Support Python 3.8 in PyPy3

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43348: -- Description: {code:java} pypy3 version is: Python 3.8.16

[jira] [Updated] (SPARK-43348) Support Python 3.8 in PyPy3

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43348: -- Component/s: (was: Project Infra) > Support Python 3.8 in PyPy3 >

[jira] [Updated] (SPARK-43348) Support Python 3.8 in PyPy3

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43348: -- Component/s: PySpark > Support Python 3.8 in PyPy3 > --- > >

[jira] [Updated] (SPARK-43348) Support Python 3.8 in PyPy3

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43348: -- Summary: Support Python 3.8 in PyPy3 (was: Upgrade PyPy to Python 3.8) > Support Python 3.8

[jira] [Commented] (SPARK-43348) Upgrade PyPy to Python 3.8

2023-05-02 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718785#comment-17718785 ] Snoot.io commented on SPARK-43348: -- User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Created] (SPARK-43352) `setPythonSparkConfProperties` should use conf instead of sparkAppConf

2023-05-02 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-43352: - Summary: `setPythonSparkConfProperties` should use conf instead of sparkAppConf Key: SPARK-43352 URL: https://issues.apache.org/jira/browse/SPARK-43352 Project: Spark

[jira] [Created] (SPARK-43351) Support Golang in Spark Connect

2023-05-02 Thread BoYang (Jira)
BoYang created SPARK-43351: -- Summary: Support Golang in Spark Connect Key: SPARK-43351 URL: https://issues.apache.org/jira/browse/SPARK-43351 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-43350) Upgrade scalafmt from 3.7.2 to 3.7.3

2023-05-02 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43350: --- Summary: Upgrade scalafmt from 3.7.2 to 3.7.3 Key: SPARK-43350 URL: https://issues.apache.org/jira/browse/SPARK-43350 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43349) Fix flaky test for `DataFrame` creation

2023-05-02 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-43349: --- Summary: Fix flaky test for `DataFrame` creation Key: SPARK-43349 URL: https://issues.apache.org/jira/browse/SPARK-43349 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-43348) Upgrade PyPy to Python 3.8

2023-05-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-43348: - Summary: Upgrade PyPy to Python 3.8 Key: SPARK-43348 URL: https://issues.apache.org/jira/browse/SPARK-43348 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-43332) Allow ChannelBuilder extensions

2023-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43332. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40993

[jira] [Created] (SPARK-43347) Remove Python 3.7 Support

2023-05-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-43347: - Summary: Remove Python 3.7 Support Key: SPARK-43347 URL: https://issues.apache.org/jira/browse/SPARK-43347 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-43032) Add StreamingQueryManager API

2023-05-02 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-43032. -- Assignee: Wei Liu Resolution: Done Resolved by

[jira] [Resolved] (SPARK-43336) Casting between Timestamp and TimestampNTZ requires timezone

2023-05-02 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-43336. Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-43334) Error while serializing ExecutorPeakMetricsDistributions into API JSON response

2023-05-02 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718704#comment-17718704 ] Hudson commented on SPARK-43334: User 'thejdeep' has created a pull request for this issue:

[jira] [Updated] (SPARK-43346) Assign a name to the error class _LEGACY_ERROR_TEMP_1206

2023-05-02 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-43346: -- Description: Choose a proper name for the error class *_LEGACY_ERROR_TEMP_1206* defined in

[jira] [Created] (SPARK-43346) Assign a name to the error class _LEGACY_ERROR_TEMP_1206

2023-05-02 Thread Terry Kim (Jira)
Terry Kim created SPARK-43346: - Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_1206 Key: SPARK-43346 URL: https://issues.apache.org/jira/browse/SPARK-43346 Project: Spark Issue

[jira] [Updated] (SPARK-43345) Assign a name to the error class _LEGACY_ERROR_TEMP_0041

2023-05-02 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-43345: -- Description: Choose a proper name for the error class *_LEGACY_ERROR_TEMP_0041* defined in

[jira] [Created] (SPARK-43345) Assign a name to the error class _LEGACY_ERROR_TEMP_0041

2023-05-02 Thread Terry Kim (Jira)
Terry Kim created SPARK-43345: - Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_0041 Key: SPARK-43345 URL: https://issues.apache.org/jira/browse/SPARK-43345 Project: Spark Issue

[jira] [Created] (SPARK-43344) Upgrade mlflow to 3.2.1

2023-05-02 Thread Jira
Bjørn Jørgensen created SPARK-43344: --- Summary: Upgrade mlflow to 3.2.1 Key: SPARK-43344 URL: https://issues.apache.org/jira/browse/SPARK-43344 Project: Spark Issue Type: Dependency upgrade

[jira] [Created] (SPARK-43343) Spark Streaming is not able to read a .txt file whose name has [] special character

2023-05-02 Thread Siying Dong (Jira)
Siying Dong created SPARK-43343: --- Summary: Spark Streaming is not able to read a .txt file whose name has [] special character Key: SPARK-43343 URL: https://issues.apache.org/jira/browse/SPARK-43343

[jira] [Commented] (SPARK-43332) Allow ChannelBuilder extensions

2023-05-02 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718621#comment-17718621 ] Hudson commented on SPARK-43332: User 'nfx' has created a pull request for this issue:

[jira] [Updated] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43156: -- Labels: correctness (was: ) > Correctness COUNT bug in correlated scalar subselect with

[jira] [Commented] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718620#comment-17718620 ] Dongjoon Hyun commented on SPARK-43156: --- I added `correctness` label. > Correctness COUNT bug in

[jira] [Resolved] (SPARK-43156) Correctness COUNT bug in correlated scalar subselect with `COUNT(*) is null`

2023-05-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43156. --- Fix Version/s: 3.4.1 (was: 3.5.0) Resolution: Fixed Issue

[jira] [Created] (SPARK-43342) Spark in Kubernetes mode throws IllegalArgumentException when using static PVC

2023-05-02 Thread Oleg Frenkel (Jira)
Oleg Frenkel created SPARK-43342: Summary: Spark in Kubernetes mode throws IllegalArgumentException when using static PVC Key: SPARK-43342 URL: https://issues.apache.org/jira/browse/SPARK-43342

[jira] [Commented] (SPARK-43340) JsonProtocol is not backward compatible

2023-05-02 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718601#comment-17718601 ] Thomas Graves commented on SPARK-43340: --- Likely related to SPARK-39489 > JsonProtocol is not

[jira] [Created] (SPARK-43341) StructType.toDDL does not pick up on non-nullability of column in nested struct

2023-05-02 Thread Bram Boogaarts (Jira)
Bram Boogaarts created SPARK-43341: -- Summary: StructType.toDDL does not pick up on non-nullability of column in nested struct Key: SPARK-43341 URL: https://issues.apache.org/jira/browse/SPARK-43341

[jira] [Updated] (SPARK-43340) JsonProtocol is not backward compatible

2023-05-02 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated SPARK-43340: -- Description: Recently I was testing with some 3.0.2 eventlogs. The SHS-3.4+ does not

[jira] [Created] (SPARK-43340) JsonProtocol is not backward compatible

2023-05-02 Thread Ahmed Hussein (Jira)
Ahmed Hussein created SPARK-43340: - Summary: JsonProtocol is not backward compatible Key: SPARK-43340 URL: https://issues.apache.org/jira/browse/SPARK-43340 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43051) Allow emitting zero values when deserializing protobuf messages

2023-05-02 Thread Parth Upadhyay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43051: --- Description: Currently, when deserializing protobufs using from_protobuf, fields that are

[jira] [Updated] (SPARK-43051) Allow emitting zero values when deserializing protobuf messages

2023-05-02 Thread Parth Upadhyay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43051: --- Description: Currently, when deserializing protobufs using from_protobuf, fields that are

[jira] [Updated] (SPARK-43051) Allow emitting zero values when deserializing protobuf messages

2023-05-02 Thread Parth Upadhyay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43051: --- Summary: Allow emitting zero values when deserializing protobuf messages (was: Allow

[jira] [Created] (SPARK-43339) LEFT JOIN is treated as INNER JOIN when being in a middle of double join

2023-05-02 Thread Leonid Chistov (Jira)
Leonid Chistov created SPARK-43339: -- Summary: LEFT JOIN is treated as INNER JOIN when being in a middle of double join Key: SPARK-43339 URL: https://issues.apache.org/jira/browse/SPARK-43339

[jira] [Assigned] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

2023-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-16484: Assignee: Ryan Berti > Incremental Cardinality estimation operations with Hyperloglog >

[jira] [Resolved] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

2023-05-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16484. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40615

[jira] [Updated] (SPARK-43051) Allow materializing zero values when deserializing protobuf messages

2023-05-02 Thread Parth Upadhyay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43051: --- Description: Currently, when deserializing protobufs using from_protobuf, fields that are

[jira] [Updated] (SPARK-43051) Allow materializing zero values when deserializing protobuf messages

2023-05-02 Thread Parth Upadhyay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43051: --- Description: Currently, when deserializing protobufs using from_protobuf, fields that are

[jira] [Updated] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

2023-05-02 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-43338: -- Description: {code:java} private[sql] object CatalogManager { val SESSION_CATALOG_NAME: String =

[jira] [Created] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

2023-05-02 Thread melin (Jira)
melin created SPARK-43338: - Summary: Support modify the SESSION_CATALOG_NAME value Key: SPARK-43338 URL: https://issues.apache.org/jira/browse/SPARK-43338 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43337) Asc/desc arrow icons for sorting column does not get displayed in the table column

2023-05-02 Thread Maytas Monsereenusorn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maytas Monsereenusorn updated SPARK-43337: -- Summary: Asc/desc arrow icons for sorting column does not get displayed in