[jira] [Resolved] (SPARK-48075) Enforce type checking in from_avro and to_avro functions

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48075. -- Fix Version/s: 4.0.0 Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-48080) Promote `*MainAppResource` and `NonJVMResource` to `DeveloperApi`

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48080. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46332

[jira] [Assigned] (SPARK-48064) Improve error messages for routine related errors

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48064: Assignee: Allison Wang > Improve error messages for routine related errors >

[jira] [Resolved] (SPARK-48064) Improve error messages for routine related errors

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48064. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46310

[jira] [Resolved] (SPARK-43727) Parity returnType check in Spark Connect

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43727. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46300

[jira] [Assigned] (SPARK-48058) `UserDefinedFunction.returnType` parse the DDL string

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48058: Assignee: Ruifeng Zheng > `UserDefinedFunction.returnType` parse the DDL string >

[jira] [Resolved] (SPARK-48058) `UserDefinedFunction.returnType` parse the DDL string

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48058. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46300

[jira] [Updated] (SPARK-25875) Merge code to set up driver features for different languages

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25875: -- Fix Version/s: 3.0.0 > Merge code to set up driver features for different languages >

[jira] [Created] (SPARK-48080) Promote `*MainAppResource` and `NonJVMResource` to `DeveloperApi`

2024-05-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-48080: - Summary: Promote `*MainAppResource` and `NonJVMResource` to `DeveloperApi` Key: SPARK-48080 URL: https://issues.apache.org/jira/browse/SPARK-48080 Project: Spark

[jira] [Updated] (SPARK-48080) Promote `*MainAppResource` and `NonJVMResource` to `DeveloperApi`

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48080: --- Labels: pull-request-available (was: ) > Promote `*MainAppResource` and `NonJVMResource`

[jira] [Updated] (SPARK-48079) Upgrade maven-install/deploy-plugin to 3.1.2

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48079: --- Labels: pull-request-available (was: ) > Upgrade maven-install/deploy-plugin to 3.1.2 >

[jira] [Updated] (SPARK-47911) Introduces a universal BinaryFormatter to make binary output consistent

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47911: --- Labels: pull-request-available releasenotes (was: releasenotes) > Introduces a universal

[jira] [Updated] (SPARK-48078) Promote `o.a.s.d.k8s.Constants` to `DeveloperApi`

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48078: --- Labels: pull-request-available (was: ) > Promote `o.a.s.d.k8s.Constants` to `DeveloperApi`

[jira] [Updated] (SPARK-45923) Spark Kubernetes Operator

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45923: -- Summary: Spark Kubernetes Operator (was: SPIP: Spark Kubernetes Operator) > Spark Kubernetes

[jira] [Assigned] (SPARK-45923) SPIP: Spark Kubernetes Operator

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45923: - Assignee: Zhou Jiang > SPIP: Spark Kubernetes Operator >

[jira] [Resolved] (SPARK-48077) Promote `KubernetesClientUtils` to `DeveloperApi`

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48077. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46327

[jira] [Resolved] (SPARK-48076) Promote `KubernetesVolumeUtils` to `DeveloperApi`

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48076. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46326

[jira] [Updated] (SPARK-48077) Promote `KubernetesClientUtils` to `DeveloperApi`

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48077: --- Labels: pull-request-available (was: ) > Promote `KubernetesClientUtils` to `DeveloperApi`

[jira] [Created] (SPARK-48077) Promote `KubernetesClientUtils` to `DeveloperApi`

2024-05-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-48077: - Summary: Promote `KubernetesClientUtils` to `DeveloperApi` Key: SPARK-48077 URL: https://issues.apache.org/jira/browse/SPARK-48077 Project: Spark Issue

[jira] [Updated] (SPARK-48076) Promote `KubernetesVolumeUtils` to `DeveloperApi`

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48076: --- Labels: pull-request-available (was: ) > Promote `KubernetesVolumeUtils` to `DeveloperApi`

[jira] [Created] (SPARK-48076) Promote `KubernetesVolumeUtils` to `DeveloperApi`

2024-05-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-48076: - Summary: Promote `KubernetesVolumeUtils` to `DeveloperApi` Key: SPARK-48076 URL: https://issues.apache.org/jira/browse/SPARK-48076 Project: Spark Issue

[jira] [Resolved] (SPARK-48062) Add pyspark test for SimpleDataSourceStreamingReader

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48062. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46306

[jira] [Assigned] (SPARK-48062) Add pyspark test for SimpleDataSourceStreamingReader

2024-05-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48062: Assignee: Chaoqin Li > Add pyspark test for SimpleDataSourceStreamingReader >

[jira] [Updated] (SPARK-43923) Post listenerBus events during ExecutePlanRequest

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43923: -- Summary: Post listenerBus events during ExecutePlanRequest (was: [CONNECT] Post listenerBus

[jira] [Updated] (SPARK-47911) Introduces a universal BinaryFormatter to make binary output consistent

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47911: -- Labels: releasenotes (was: pull-request-available) > Introduces a universal BinaryFormatter

[jira] [Assigned] (SPARK-47911) Introduces a universal BinaryFormatter to make binary output consistent

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47911: - Assignee: Kent Yao > Introduces a universal BinaryFormatter to make binary output

[jira] [Resolved] (SPARK-47911) Introduces a universal BinaryFormatter to make binary output consistent

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47911. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46133

[jira] [Commented] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

2024-05-01 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842794#comment-17842794 ] L. C. Hsieh commented on SPARK-48073: - Yea, if that is only thing we can do, then I will go to make

[jira] [Updated] (SPARK-47934) Ensure trailing slashes in `HistoryServer` URL redirections

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47934: -- Summary: Ensure trailing slashes in `HistoryServer` URL redirections (was: Inefficient

[jira] [Updated] (SPARK-47934) Ensure trailing slashes in `HistoryServer` URL redirections

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47934: -- Parent: SPARK-45869 Issue Type: Sub-task (was: Improvement) > Ensure trailing

[jira] [Updated] (SPARK-47934) Ensure trailing slashes in `HistoryServer` URL redirections

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47934: -- Affects Version/s: 4.0.0 (was: 3.3.2)

[jira] [Assigned] (SPARK-47934) Inefficient Redirect Handling Due to Missing Trailing Slashes in URL Redirection

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47934: - Assignee: huangzhir > Inefficient Redirect Handling Due to Missing Trailing Slashes in

[jira] [Resolved] (SPARK-47934) Inefficient Redirect Handling Due to Missing Trailing Slashes in URL Redirection

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47934. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46157

[jira] [Resolved] (SPARK-48074) Improve the readability of JSON loggings

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48074. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46322

[jira] [Commented] (SPARK-48016) Fix a bug in try_divide function when with decimals

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842788#comment-17842788 ] Dongjoon Hyun commented on SPARK-48016: --- This is tested and landed back to branch-3.5 via the

[jira] [Updated] (SPARK-48016) Fix a bug in try_divide function when with decimals

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48016: -- Fix Version/s: 3.5.2 > Fix a bug in try_divide function when with decimals >

[jira] [Commented] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

2024-05-01 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842785#comment-17842785 ] Jungtaek Lim commented on SPARK-48073: -- I roughly remember that Encoder.bean() had changed to

[jira] [Updated] (SPARK-48065) SPJ: allowJoinKeysSubsetOfPartitionKeys is too strict

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48065: --- Labels: pull-request-available (was: ) > SPJ: allowJoinKeysSubsetOfPartitionKeys is too

[jira] [Updated] (SPARK-48075) Enforce type checking in from_avro and to_avro functions

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48075: --- Labels: pull-request-available (was: ) > Enforce type checking in from_avro and to_avro

[jira] [Assigned] (SPARK-48070) Support AdaptiveQueryExecSuite to skip check results

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48070: - Assignee: dzcxzl > Support AdaptiveQueryExecSuite to skip check results >

[jira] [Resolved] (SPARK-48070) Support AdaptiveQueryExecSuite to skip check results

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48070. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46316

[jira] [Comment Edited] (SPARK-48016) Fix a bug in try_divide function when with decimals

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842734#comment-17842734 ] Dongjoon Hyun edited comment on SPARK-48016 at 5/1/24 6:21 PM: --- This is

[jira] [Commented] (SPARK-48016) Fix a bug in try_divide function when with decimals

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842734#comment-17842734 ] Dongjoon Hyun commented on SPARK-48016: --- This is reverted from `branch-3.5` because the

[jira] [Updated] (SPARK-48016) Fix a bug in try_divide function when with decimals

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48016: -- Fix Version/s: (was: 3.5.2) > Fix a bug in try_divide function when with decimals >

[jira] [Updated] (SPARK-48068) `mypy` should have `--python-executable` parameter

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Description: We assumed that `PYTHON_EXECUTABLE` is used for `dev/lint-python` like the

[jira] [Updated] (SPARK-48068) `mypy` should have `--python-executable` parameter

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Component/s: Project Infra > `mypy` should have `--python-executable` parameter >

[jira] [Resolved] (SPARK-48068) `mypy` should have `--python-executable` parameter

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48068. --- Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution:

[jira] [Commented] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

2024-05-01 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842712#comment-17842712 ] L. C. Hsieh commented on SPARK-48073: - cc [~kabhwan] > StateStore schema incompatibility between

[jira] [Created] (SPARK-48073) StateStore schema incompatibility between 3.2 and 3.4

2024-05-01 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-48073: --- Summary: StateStore schema incompatibility between 3.2 and 3.4 Key: SPARK-48073 URL: https://issues.apache.org/jira/browse/SPARK-48073 Project: Spark Issue

[jira] [Updated] (SPARK-48068) `mypy` should have `--python-executable` parameter

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Summary: `mypy` should have `--python-executable` parameter (was: Fix `mypy` failure in

[jira] [Updated] (SPARK-48068) Fix `mypy` failure in Python 3.10 and 3.11

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Description: We assumed that `PYTHON_EXECUTABLE` is used for `dev/lint-python` like the

[jira] [Updated] (SPARK-48068) Fix `mypy` failure in Python 3.10 and 3.11

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Affects Version/s: 3.4.3 3.3.4 3.5.1

[jira] [Commented] (SPARK-47429) Rename errorClass to errorCondition and subClass to subCondition

2024-05-01 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842694#comment-17842694 ] Nicholas Chammas commented on SPARK-47429: -- I think one intermediate step we can take here is

[jira] [Updated] (SPARK-47429) Rename errorClass to errorCondition and subClass to subCondition

2024-05-01 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-47429: - Summary: Rename errorClass to errorCondition and subClass to subCondition (was: Rename

[jira] [Updated] (SPARK-47263) Assign classes to DEFAULT value errors

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47263: --- Labels: pull-request-available starter (was: starter) > Assign classes to DEFAULT value

[jira] [Comment Edited] (SPARK-47579) Spark core: Migrate logInfo with variables to structured logging framework

2024-05-01 Thread Anh Tuan Pham (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842651#comment-17842651 ] Anh Tuan Pham edited comment on SPARK-47579 at 5/1/24 1:48 PM: --- Hi

[jira] [Commented] (SPARK-48023) Stream-stream join with windowed+watermarked dropDuplicates suppresses all rows

2024-05-01 Thread Scott Schenkein (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842625#comment-17842625 ] Scott Schenkein commented on SPARK-48023: - [~siying] - if you want to simply the repro, you can

[jira] [Updated] (SPARK-48023) Stream-stream join with windowed+watermarked dropDuplicates suppresses all rows

2024-05-01 Thread Scott Schenkein (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Schenkein updated SPARK-48023: Description: When applying a streaming dropDuplicates-with-watermark to  a

[jira] [Updated] (SPARK-48072) Test output is not descriptive for parameterized DESCRIBE and EXPLAIN in SQLQuerySuite

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48072: --- Labels: pull-request-available (was: ) > Test output is not descriptive for parameterized

[jira] [Assigned] (SPARK-48054) Backward compatibility test for Spark Connect

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48054: -- Assignee: (was: Apache Spark) > Backward compatibility test for Spark Connect >

[jira] [Assigned] (SPARK-48054) Backward compatibility test for Spark Connect

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48054: -- Assignee: Apache Spark > Backward compatibility test for Spark Connect >

[jira] [Assigned] (SPARK-48054) Backward compatibility test for Spark Connect

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48054: -- Assignee: Apache Spark > Backward compatibility test for Spark Connect >

[jira] [Assigned] (SPARK-48054) Backward compatibility test for Spark Connect

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48054: -- Assignee: (was: Apache Spark) > Backward compatibility test for Spark Connect >

[jira] [Created] (SPARK-48072) Test output is not descriptive for parameterized DESCRIBE and EXPLAIN in SQLQuerySuite

2024-05-01 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-48072: Summary: Test output is not descriptive for parameterized DESCRIBE and EXPLAIN in SQLQuerySuite Key: SPARK-48072 URL: https://issues.apache.org/jira/browse/SPARK-48072

[jira] [Updated] (SPARK-43406) enable spark sql to drop multiple partitions in one call

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43406: --- Labels: pull-request-available (was: ) > enable spark sql to drop multiple partitions in

[jira] [Commented] (SPARK-47263) Assign classes to DEFAULT value errors

2024-05-01 Thread Ping Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842580#comment-17842580 ] Ping Xu commented on SPARK-47263: - [~maxgekk] I'll try to work on this issue. > Assign classes to

[jira] [Updated] (SPARK-44264) DeepSpeed Distributor

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44264: --- Labels: pull-request-available (was: ) > DeepSpeed Distributor > - > >

[jira] [Updated] (SPARK-48071) Use Python 3.10 in `Python Linter` step

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48071: --- Labels: pull-request-available (was: ) > Use Python 3.10 in `Python Linter` step >

[jira] [Updated] (SPARK-48071) Use Python 3.10 in `Python Linter` step

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48071: -- Issue Type: Improvement (was: Bug) > Use Python 3.10 in `Python Linter` step >

[jira] [Updated] (SPARK-48070) Support AdaptiveQueryExecSuite to skip check results

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48070: --- Labels: pull-request-available (was: ) > Support AdaptiveQueryExecSuite to skip check

[jira] [Resolved] (SPARK-48069) Handle PEP-632 by checking `ModuleNotFoundError` on `setuptools` in Python 3.12

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48069. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46315

[jira] [Created] (SPARK-48070) Support AdaptiveQueryExecSuite to skip check results

2024-05-01 Thread dzcxzl (Jira)
dzcxzl created SPARK-48070: -- Summary: Support AdaptiveQueryExecSuite to skip check results Key: SPARK-48070 URL: https://issues.apache.org/jira/browse/SPARK-48070 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48069) Handle PEP-632 by checking `ModuleNotFoundError` on `setuptools` in Python 3.12

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48069: -- Summary: Handle PEP-632 by checking `ModuleNotFoundError` on `setuptools` in Python 3.12

[jira] [Updated] (SPARK-48068) Fix `mypy` failure in Python 3.10 and 3.11

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Summary: Fix `mypy` failure in Python 3.10 and 3.11 (was: Fix `mypy` failure in Python

[jira] [Updated] (SPARK-48069) Handle PEP-632 by checking `ModuleNotFoundError` on `setuptools`

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48069: --- Labels: pull-request-available (was: ) > Handle PEP-632 by checking `ModuleNotFoundError`

[jira] [Created] (SPARK-48069) Handle PEP-632 by checking `ModuleNotFoundError` on `setuptools`

2024-05-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-48069: - Summary: Handle PEP-632 by checking `ModuleNotFoundError` on `setuptools` Key: SPARK-48069 URL: https://issues.apache.org/jira/browse/SPARK-48069 Project: Spark

[jira] [Assigned] (SPARK-48069) Handle PEP-632 by checking `ModuleNotFoundError` on `setuptools`

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48069: - Assignee: Dongjoon Hyun > Handle PEP-632 by checking `ModuleNotFoundError` on

[jira] [Assigned] (SPARK-48068) Fix `mypy` failure in Python 3.10+

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48068: - Assignee: Dongjoon Hyun > Fix `mypy` failure in Python 3.10+ >

[jira] [Updated] (SPARK-48068) Fix `mypy` failure in Python 3.10+

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Fix `mypy` failure in Python

[jira] [Updated] (SPARK-48068) Fix `mypy` failure in Python 3.10+

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Description: {code} $ python3 --version Python 3.10.13 $ dev/lint-python --mypy starting

[jira] [Updated] (SPARK-48068) Fix `mypy` failure in Python 3.10+

2024-05-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48068: -- Summary: Fix `mypy` failure in Python 3.10+ (was: Fix `mypy` failure in Python 3.10) > Fix

[jira] [Updated] (SPARK-48068) Fix `mypy` failure in Python 3.10

2024-05-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48068: --- Labels: pull-request-available (was: ) > Fix `mypy` failure in Python 3.10 >