[jira] [Resolved] (SPARK-41735) Any SparkThrowable (with an error class) not in error-classes.json is masked in SQLExecution.withNewExecutionId and end-user will see "org.apache.spark.SparkException:

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41735. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39794 [https://gith

[jira] [Assigned] (SPARK-41735) Any SparkThrowable (with an error class) not in error-classes.json is masked in SQLExecution.withNewExecutionId and end-user will see "org.apache.spark.SparkException:

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41735: --- Assignee: XiDuo You > Any SparkThrowable (with an error class) not in error-classes.json is

[jira] [Assigned] (SPARK-42232) Rename error class: UNSUPPORTED_FEATURE.JDBC_TRANSACTION

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42232: Assignee: (was: Apache Spark) > Rename error class: UNSUPPORTED_FEATURE.JDBC_TRANSACT

[jira] [Assigned] (SPARK-42232) Rename error class: UNSUPPORTED_FEATURE.JDBC_TRANSACTION

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42232: Assignee: Apache Spark > Rename error class: UNSUPPORTED_FEATURE.JDBC_TRANSACTION > -

[jira] [Commented] (SPARK-42232) Rename error class: UNSUPPORTED_FEATURE.JDBC_TRANSACTION

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681967#comment-17681967 ] Apache Spark commented on SPARK-42232: -- User 'itholic' has created a pull request f

[jira] [Commented] (SPARK-41855) `createDataFrame` doesn't handle None/NaN properly

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681974#comment-17681974 ] Apache Spark commented on SPARK-41855: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-42236) Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT`

2023-01-30 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42236: --- Summary: Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT` Key: SPARK-42236 URL: https://issues.apache.org/jira/browse/SPARK-42236 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Wei Guo (Jira)
Wei Guo created SPARK-42237: --- Summary: change binary to unsupported dataType in csv format Key: SPARK-42237 URL: https://issues.apache.org/jira/browse/SPARK-42237 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-42237: Attachment: image-2023-01-30-17-21-09-212.png > change binary to unsupported dataType in csv format >

[jira] [Updated] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-42237: Description: When a binary colunm is written into csv files, actual content of this colunm is {*}object.t

[jira] [Commented] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681979#comment-17681979 ] Apache Spark commented on SPARK-42237: -- User 'weiyuyilia' has created a pull reques

[jira] [Commented] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681977#comment-17681977 ] Apache Spark commented on SPARK-42237: -- User 'weiyuyilia' has created a pull reques

[jira] [Commented] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681978#comment-17681978 ] Wei Guo commented on SPARK-42237: - a pr is ready~ > change binary to unsupported dataTy

[jira] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Wei Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237 ] Wei Guo deleted comment on SPARK-42237: - was (Author: wayne guo): a pr is ready~ > change binary to unsupported dataType in csv format > --- > >

[jira] [Assigned] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42237: Assignee: (was: Apache Spark) > change binary to unsupported dataType in csv format >

[jira] [Assigned] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42237: Assignee: Apache Spark > change binary to unsupported dataType in csv format > --

[jira] [Updated] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42237: - Fix Version/s: (was: 3.4.0) > change binary to unsupported dataType in csv format >

[jira] [Updated] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42237: - Target Version/s: (was: 3.4.0) > change binary to unsupported dataType in csv format > ---

[jira] [Updated] (SPARK-42237) change binary to unsupported dataType in csv format

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42237: - Description: When a binary colunm is written into csv files, actual content of this colunm is {

[jira] [Commented] (SPARK-42168) CoGroup with window function returns incorrect result when partition keys differ in order

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682009#comment-17682009 ] Apache Spark commented on SPARK-42168: -- User 'EnricoMi' has created a pull request

[jira] [Assigned] (SPARK-42066) The DATATYPE_MISMATCH error class contains inappropriate and duplicating subclasses

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42066: --- Assignee: Haejoon Lee > The DATATYPE_MISMATCH error class contains inappropriate and duplic

[jira] [Resolved] (SPARK-42066) The DATATYPE_MISMATCH error class contains inappropriate and duplicating subclasses

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42066. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39625 [https://gith

[jira] [Updated] (SPARK-42198) spark.read fails to read filenames with accented characters

2023-01-30 Thread Tarique Anwer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tarique Anwer updated SPARK-42198: -- Description: Unable to read filenames with accented characters in the filename. *Sample error

[jira] [Comment Edited] (SPARK-42198) spark.read fails to read filenames with accented characters

2023-01-30 Thread Tarique Anwer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682041#comment-17682041 ] Tarique Anwer edited comment on SPARK-42198 at 1/30/23 12:15 PM: -

[jira] [Commented] (SPARK-42198) spark.read fails to read filenames with accented characters

2023-01-30 Thread Tarique Anwer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682041#comment-17682041 ] Tarique Anwer commented on SPARK-42198: --- I have updated the original comment to re

[jira] [Resolved] (SPARK-42214) Branch-3.4 daily test failed

2023-01-30 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-42214. - Assignee: Yikun Jiang Resolution: Fixed Branch-3.4 scheduled job recovered: https://githu

[jira] [Commented] (SPARK-42236) Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682045#comment-17682045 ] Apache Spark commented on SPARK-42236: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-42236) Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42236: Assignee: Apache Spark > Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT` > ---

[jira] [Assigned] (SPARK-42236) Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42236: Assignee: (was: Apache Spark) > Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT` >

[jira] [Commented] (SPARK-42236) Refine `NULLABLE_ARRAY_OR_MAP_ELEMENT`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682046#comment-17682046 ] Apache Spark commented on SPARK-42236: -- User 'itholic' has created a pull request f

[jira] [Created] (SPARK-42238) Rename UNSUPPORTED_FEATURE.NATURAL_CROSS_JOIN

2023-01-30 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42238: --- Summary: Rename UNSUPPORTED_FEATURE.NATURAL_CROSS_JOIN Key: SPARK-42238 URL: https://issues.apache.org/jira/browse/SPARK-42238 Project: Spark Issue Type: Sub-t

[jira] [Resolved] (SPARK-41490) Assign name to _LEGACY_ERROR_TEMP_2441

2023-01-30 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41490. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39700 [https://github.com

[jira] [Assigned] (SPARK-41490) Assign name to _LEGACY_ERROR_TEMP_2441

2023-01-30 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41490: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2441 >

[jira] [Updated] (SPARK-42238) Introduce `INCOMPATIBLE_JOIN_TYPES`

2023-01-30 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42238: Summary: Introduce `INCOMPATIBLE_JOIN_TYPES` (was: Rename UNSUPPORTED_FEATURE.NATURAL_CROSS_JOIN)

[jira] [Commented] (SPARK-42238) Introduce `INCOMPATIBLE_JOIN_TYPES`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682090#comment-17682090 ] Apache Spark commented on SPARK-42238: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-42238) Introduce `INCOMPATIBLE_JOIN_TYPES`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42238: Assignee: Apache Spark > Introduce `INCOMPATIBLE_JOIN_TYPES` > --

[jira] [Assigned] (SPARK-42238) Introduce `INCOMPATIBLE_JOIN_TYPES`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42238: Assignee: (was: Apache Spark) > Introduce `INCOMPATIBLE_JOIN_TYPES` > ---

[jira] [Resolved] (SPARK-42233) Improve error message for PIVOT_AFTER_GROUP_BY

2023-01-30 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42233. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39793 [https://github.com

[jira] [Created] (SPARK-42239) Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY

2023-01-30 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42239: --- Summary: Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY Key: SPARK-42239 URL: https://issues.apache.org/jira/browse/SPARK-42239 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42239) Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682108#comment-17682108 ] Apache Spark commented on SPARK-42239: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-42239) Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42239: Assignee: Apache Spark > Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY > --

[jira] [Assigned] (SPARK-42239) Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42239: Assignee: (was: Apache Spark) > Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY >

[jira] [Commented] (SPARK-41500) auto generate concat as Double when string minus an INTERVAL type

2023-01-30 Thread Narek Karapetian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682141#comment-17682141 ] Narek Karapetian commented on SPARK-41500: -- It is by design... You can set `{{

[jira] [Created] (SPARK-42240) Move ClientE2ETestSuite into a separate module to test shaded jvm client

2023-01-30 Thread Yang Jie (Jira)
Yang Jie created SPARK-42240: Summary: Move ClientE2ETestSuite into a separate module to test shaded jvm client Key: SPARK-42240 URL: https://issues.apache.org/jira/browse/SPARK-42240 Project: Spark

[jira] [Assigned] (SPARK-42240) Move ClientE2ETestSuite into a separate module to test shaded jvm client

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42240: Assignee: Apache Spark > Move ClientE2ETestSuite into a separate module to test shaded j

[jira] [Commented] (SPARK-42240) Move ClientE2ETestSuite into a separate module to test shaded jvm client

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682158#comment-17682158 ] Apache Spark commented on SPARK-42240: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-42240) Move ClientE2ETestSuite into a separate module to test shaded jvm client

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42240: Assignee: (was: Apache Spark) > Move ClientE2ETestSuite into a separate module to te

[jira] [Commented] (SPARK-41970) SparkPath

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682163#comment-17682163 ] Apache Spark commented on SPARK-41970: -- User 'databricks-david-lewis' has created a

[jira] [Created] (SPARK-42241) Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven

2023-01-30 Thread Yang Jie (Jira)
Yang Jie created SPARK-42241: Summary: Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven Key: SPARK-42241 URL: https://issues.apache.org/jira/br

[jira] [Commented] (SPARK-42230) Improve `lint` job by skipping PySpark and SparkR docs if unchanged

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682212#comment-17682212 ] Apache Spark commented on SPARK-42230: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-42241) Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42241: Assignee: Apache Spark > Correct the condition for `SparkConnectServerUtils#findSparkCon

[jira] [Assigned] (SPARK-42241) Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42241: Assignee: (was: Apache Spark) > Correct the condition for `SparkConnectServerUtils#f

[jira] [Commented] (SPARK-42241) Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682213#comment-17682213 ] Apache Spark commented on SPARK-42241: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-42241) Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682214#comment-17682214 ] Apache Spark commented on SPARK-42241: -- User 'LuciferYang' has created a pull reque

[jira] [Resolved] (SPARK-42239) Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY

2023-01-30 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42239. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39806 [https://github.com

[jira] [Assigned] (SPARK-42239) Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY

2023-01-30 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42239: Assignee: Haejoon Lee > Integrate MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY > ---

[jira] [Commented] (SPARK-40684) Update embedded vis-timeline javascript resources

2023-01-30 Thread Eugene Shinn (Truveta) (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682272#comment-17682272 ] Eugene Shinn (Truveta) commented on SPARK-40684: I think I filed [SPARK-

[jira] [Resolved] (SPARK-42221) Introduce a new conf for TimestampNTZ schema inference in JSON/CSV

2023-01-30 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-42221. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39777 [https:

[jira] [Created] (SPARK-42242) Upgrade snappy-java to 1.1.9.0

2023-01-30 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-42242: - Summary: Upgrade snappy-java to 1.1.9.0 Key: SPARK-42242 URL: https://issues.apache.org/jira/browse/SPARK-42242 Project: Spark Issue Type: Bug Co

[jira] [Assigned] (SPARK-42242) Upgrade snappy-java to 1.1.9.0

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42242: Assignee: (was: Apache Spark) > Upgrade snappy-java to 1.1.9.0 >

[jira] [Commented] (SPARK-42242) Upgrade snappy-java to 1.1.9.0

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682306#comment-17682306 ] Apache Spark commented on SPARK-42242: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-42242) Upgrade snappy-java to 1.1.9.0

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42242: Assignee: Apache Spark > Upgrade snappy-java to 1.1.9.0 > --

[jira] [Created] (SPARK-42243) Use `spark.sql.inferTimestampNTZInDataSources.enabled` to infer timestamp type on partition columns

2023-01-30 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-42243: -- Summary: Use `spark.sql.inferTimestampNTZInDataSources.enabled` to infer timestamp type on partition columns Key: SPARK-42243 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Commented] (SPARK-42243) Use `spark.sql.inferTimestampNTZInDataSources.enabled` to infer timestamp type on partition columns

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682311#comment-17682311 ] Apache Spark commented on SPARK-42243: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-42243) Use `spark.sql.inferTimestampNTZInDataSources.enabled` to infer timestamp type on partition columns

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42243: Assignee: Apache Spark (was: Gengliang Wang) > Use `spark.sql.inferTimestampNTZInDataSou

[jira] [Assigned] (SPARK-42243) Use `spark.sql.inferTimestampNTZInDataSources.enabled` to infer timestamp type on partition columns

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42243: Assignee: Gengliang Wang (was: Apache Spark) > Use `spark.sql.inferTimestampNTZInDataSou

[jira] [Assigned] (SPARK-42192) Migrate the `TypeError` from `pyspark/sql/dataframe.py` into `PySparkTypeError`.

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42192: Assignee: Haejoon Lee > Migrate the `TypeError` from `pyspark/sql/dataframe.py` into > `

[jira] [Resolved] (SPARK-42192) Migrate the `TypeError` from `pyspark/sql/dataframe.py` into `PySparkTypeError`.

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42192. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39785 [https://gi

[jira] [Assigned] (SPARK-42241) Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42241: Assignee: Yang Jie > Correct the condition for `SparkConnectServerUtils#findSparkConnect

[jira] [Resolved] (SPARK-42241) Correct the condition for `SparkConnectServerUtils#findSparkConnectJar` to find the correct connect server jar for maven

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42241. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39810 [https://gi

[jira] [Updated] (SPARK-41970) Introduce SparkPath to address paths and URIs

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41970: - Summary: Introduce SparkPath to address paths and URIs (was: SparkPath) > Introduce SparkPath t

[jira] [Resolved] (SPARK-37936) Use error classes in the parsing errors of intervals

2023-01-30 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-37936. - Resolution: Fixed > Use error classes in the parsing errors of intervals > -

[jira] [Resolved] (SPARK-42004) Migrate "XX000" sqlState onto `INTERNAL_ERROR`

2023-01-30 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-42004. - Fix Version/s: 3.4.0 Resolution: Fixed Done with https://github.com/apache/spark/pull/395

[jira] [Commented] (SPARK-41554) Decimal.changePrecision produces ArrayIndexOutOfBoundsException

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682333#comment-17682333 ] Apache Spark commented on SPARK-41554: -- User 'fe2s' has created a pull request for

[jira] [Created] (SPARK-42244) Refine error message by using Python types.

2023-01-30 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42244: --- Summary: Refine error message by using Python types. Key: SPARK-42244 URL: https://issues.apache.org/jira/browse/SPARK-42244 Project: Spark Issue Type: Sub-tas

[jira] [Commented] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682337#comment-17682337 ] Apache Spark commented on SPARK-42208: -- User 'xinrong-meng' has created a pull requ

[jira] [Assigned] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42208: Assignee: (was: Apache Spark) > Reuse UDF test cases under `pyspark.sql.tests` >

[jira] [Assigned] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42208: Assignee: Apache Spark > Reuse UDF test cases under `pyspark.sql.tests` > ---

[jira] [Commented] (SPARK-42244) Refine error message by using Python types.

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682338#comment-17682338 ] Apache Spark commented on SPARK-42244: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-42244) Refine error message by using Python types.

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42244: Assignee: Apache Spark > Refine error message by using Python types. > --

[jira] [Assigned] (SPARK-42244) Refine error message by using Python types.

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42244: Assignee: (was: Apache Spark) > Refine error message by using Python types. > ---

[jira] [Commented] (SPARK-42244) Refine error message by using Python types.

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682339#comment-17682339 ] Apache Spark commented on SPARK-42244: -- User 'itholic' has created a pull request f

[jira] [Created] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-30 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-42245: --- Summary: Upgrade scalafmt from 3.6.1 to 3.7.1 Key: SPARK-42245 URL: https://issues.apache.org/jira/browse/SPARK-42245 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-42202) Scala Client E2E test stop the server gracefully

2023-01-30 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42202. --- Fix Version/s: 3.4.0 Assignee: Zhen Li Resolution: Fixed > Scala Cli

[jira] [Commented] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682341#comment-17682341 ] Apache Spark commented on SPARK-42245: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42245: Assignee: (was: Apache Spark) > Upgrade scalafmt from 3.6.1 to 3.7.1 > --

[jira] [Assigned] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42245: Assignee: Apache Spark > Upgrade scalafmt from 3.6.1 to 3.7.1 > -

[jira] [Commented] (SPARK-42245) Upgrade scalafmt from 3.6.1 to 3.7.1

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682342#comment-17682342 ] Apache Spark commented on SPARK-42245: -- User 'panbingkun' has created a pull reques

[jira] [Created] (SPARK-42246) Reach Full Parity with Vanilla PySpark's UDF in Python

2023-01-30 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42246: Summary: Reach Full Parity with Vanilla PySpark's UDF in Python Key: SPARK-42246 URL: https://issues.apache.org/jira/browse/SPARK-42246 Project: Spark Issue

[jira] [Assigned] (SPARK-42163) Schema pruning fails on non-foldable array index or map key

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42163: --- Assignee: David Cashman > Schema pruning fails on non-foldable array index or map key > ---

[jira] [Resolved] (SPARK-42163) Schema pruning fails on non-foldable array index or map key

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42163. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39718 [https://gith

[jira] [Created] (SPARK-42247) `returnType` attribute of UDF when the user-specified return type has column name embeded

2023-01-30 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42247: Summary: `returnType` attribute of UDF when the user-specified return type has column name embeded Key: SPARK-42247 URL: https://issues.apache.org/jira/browse/SPARK-42247

[jira] [Assigned] (SPARK-42231) Rename error class: MISSING_STATIC_PARTITION_COLUMN

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-42231: --- Assignee: Haejoon Lee > Rename error class: MISSING_STATIC_PARTITION_COLUMN > -

[jira] [Resolved] (SPARK-42231) Rename error class: MISSING_STATIC_PARTITION_COLUMN

2023-01-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42231. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39797 [https://gith

[jira] [Created] (SPARK-42248) Assign name to _LEGACY_ERROR_TEMP_2141

2023-01-30 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42248: --- Summary: Assign name to _LEGACY_ERROR_TEMP_2141 Key: SPARK-42248 URL: https://issues.apache.org/jira/browse/SPARK-42248 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42249) Refining html strings in error messages

2023-01-30 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42249: --- Summary: Refining html strings in error messages Key: SPARK-42249 URL: https://issues.apache.org/jira/browse/SPARK-42249 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-42248) Assign name to _LEGACY_ERROR_TEMP_2141

2023-01-30 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-42248. - Resolution: Duplicate > Assign name to _LEGACY_ERROR_TEMP_2141 > ---

[jira] [Created] (SPARK-42250) batch_infer_udf with float fails when the batch size consists of single value

2023-01-30 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42250: Summary: batch_infer_udf with float fails when the batch size consists of single value Key: SPARK-42250 URL: https://issues.apache.org/jira/browse/SPARK-42250 Project

[jira] [Updated] (SPARK-42250) predict_batch_udf with float fails when the batch size consists of single value

2023-01-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42250: - Summary: predict_batch_udf with float fails when the batch size consists of single value (was:

[jira] [Commented] (SPARK-42250) predict_batch_udf with float fails when the batch size consists of single value

2023-01-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682368#comment-17682368 ] Apache Spark commented on SPARK-42250: -- User 'HyukjinKwon' has created a pull reque

  1   2   >