[jira] [Updated] (SPARK-48350) Support for exceptions thrown from parser/interpreter

2024-05-20 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48350: Summary: Support for exceptions thrown from parser/interpreter (was: Support

[jira] [Created] (SPARK-48352) set max file counter through spark conf

2024-05-20 Thread guihuawen (Jira)
guihuawen created SPARK-48352: - Summary: set max file counter through spark conf Key: SPARK-48352 URL: https://issues.apache.org/jira/browse/SPARK-48352 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-48350) Support for exceptions

2024-05-20 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48350: Summary: Support for exceptions (was: Support for error handling) > Supp

[jira] [Created] (SPARK-48351) JDBC Connectors - Add cast suite

2024-05-20 Thread Uros Stankovic (Jira)
Uros Stankovic created SPARK-48351: -- Summary: JDBC Connectors - Add cast suite Key: SPARK-48351 URL: https://issues.apache.org/jira/browse/SPARK-48351 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-48350) Support for error handling

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48350: --- Summary: Support for error handling Key: SPARK-48350 URL: https://issues.apache.org/jira/browse/SPARK-48350 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-48349) Support for debugging

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48349: --- Summary: Support for debugging Key: SPARK-48349 URL: https://issues.apache.org/jira/browse/SPARK-48349 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-48348) Support for BREAK/CONTINUE statements

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48348: --- Summary: Support for BREAK/CONTINUE statements Key: SPARK-48348 URL: https://issues.apache.org/jira/browse/SPARK-48348 Project: Spark Issue Type: Sub

[jira] [Created] (SPARK-48347) Support for WHILE statements

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48347: --- Summary: Support for WHILE statements Key: SPARK-48347 URL: https://issues.apache.org/jira/browse/SPARK-48347 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-48346) Support for IF ELSE statements

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48346: --- Summary: Support for IF ELSE statements Key: SPARK-48346 URL: https://issues.apache.org/jira/browse/SPARK-48346 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-48345) Checks for variable declarations

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48345: --- Summary: Checks for variable declarations Key: SPARK-48345 URL: https://issues.apache.org/jira/browse/SPARK-48345 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48238. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46611 [https

[jira] [Assigned] (SPARK-48238) Spark fail to start due to class o.a.h.yarn.server.webproxy.amfilter.AmIpFilter is not a jakarta.servlet.Filter

2024-05-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-48238: Assignee: Cheng Pan > Spark fail to start due to cl

[jira] [Created] (SPARK-48344) Changes to sql() API

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48344: --- Summary: Changes to sql() API Key: SPARK-48344 URL: https://issues.apache.org/jira/browse/SPARK-48344 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-48343) Interpreter support

2024-05-20 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48343: Description: Implement interpreter for SQL scripting: * Interpreter * Interpreter

[jira] [Created] (SPARK-48343) Interpreter support

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48343: --- Summary: Interpreter support Key: SPARK-48343 URL: https://issues.apache.org/jira/browse/SPARK-48343 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-48342) Parser support

2024-05-20 Thread David Milicevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Milicevic updated SPARK-48342: Description: Implement parse for SQL scripting with all supporting changes for upcoming

[jira] [Created] (SPARK-48342) Parser support

2024-05-20 Thread David Milicevic (Jira)
David Milicevic created SPARK-48342: --- Summary: Parser support Key: SPARK-48342 URL: https://issues.apache.org/jira/browse/SPARK-48342 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-48341) Allow Spark Connect plugins to use PlanTest in their tests

2024-05-20 Thread Tom van Bussel (Jira)
Tom van Bussel created SPARK-48341: -- Summary: Allow Spark Connect plugins to use PlanTest in their tests Key: SPARK-48341 URL: https://issues.apache.org/jira/browse/SPARK-48341 Project: Spark

[jira] [Resolved] (SPARK-48332) Upgrade `jdbc` related test dependencies

2024-05-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48332. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46653 [https

[jira] [Assigned] (SPARK-48332) Upgrade `jdbc` related test dependencies

2024-05-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48332: Assignee: BingKun Pan > Upgrade `jdbc` related test dependenc

[jira] [Resolved] (SPARK-48323) DB2: Map BooleanType to Boolean instead of Char(1)

2024-05-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48323. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46637 [https

[jira] [Updated] (SPARK-48340) Support TimestampNTZ infer schema miss prefer_timestamp_ntz

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48340: --- Labels: pull-request-available (was: ) > Support TimestampNTZ infer schema m

[jira] [Updated] (SPARK-48340) Support TimestampNTZ infer schema miss prefer_timestamp_ntz

2024-05-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-48340: -- Attachment: image-2024-05-20-18-38-39-769.png > Support TimestampNTZ infer schema m

[jira] [Created] (SPARK-48340) Support TimestampNTZ infer schema miss prefer_timestamp_ntz

2024-05-20 Thread angerszhu (Jira)
angerszhu created SPARK-48340: - Summary: Support TimestampNTZ infer schema miss prefer_timestamp_ntz Key: SPARK-48340 URL: https://issues.apache.org/jira/browse/SPARK-48340 Project: Spark

[jira] [Updated] (SPARK-48340) Support TimestampNTZ infer schema miss prefer_timestamp_ntz

2024-05-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-48340: -- Description: !image-2024-05-20-18-38-39-769.png|width=746,height=450! (was: !image-2024-05-20-18-38

[jira] [Updated] (SPARK-48290) AQE not working when joining dataframes with more than 2000 partitions

2024-05-20 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] André F. updated SPARK-48290: - Component/s: SQL > AQE not working when joining dataframes with more than 2000 partiti

[jira] [Updated] (SPARK-48339) Revert converting collated strings to regular strings when writing to hive metastore

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48339: --- Labels: pull-request-available (was: ) > Revert converting collated strings to regu

[jira] [Created] (SPARK-48339) Revert converting collated strings to regular strings when writing to hive metastore

2024-05-20 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-48339: - Summary: Revert converting collated strings to regular strings when writing to hive metastore Key: SPARK-48339 URL: https://issues.apache.org/jira/browse/SPARK-48339

[jira] [Updated] (SPARK-48338) Sql Scripting support for Spark SQL

2024-05-20 Thread Aleksandar Tomic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandar Tomic updated SPARK-48338: - Description: Design doc for this feature is in attachment. High level example of Sql

[jira] [Updated] (SPARK-48338) Sql Scripting support for Spark SQL

2024-05-20 Thread Aleksandar Tomic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandar Tomic updated SPARK-48338: - Attachment: Sql Scripting - OSS.odt > Sql Scripting support for Spark

[jira] [Created] (SPARK-48338) Sql Scripting support for Spark SQL

2024-05-20 Thread Aleksandar Tomic (Jira)
Aleksandar Tomic created SPARK-48338: Summary: Sql Scripting support for Spark SQL Key: SPARK-48338 URL: https://issues.apache.org/jira/browse/SPARK-48338 Project: Spark Issue Type: Epic

[jira] [Resolved] (SPARK-48242) Upgrade extra-enforcer-rules to 1.8.0

2024-05-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48242. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46538 [https

[jira] [Assigned] (SPARK-48242) Upgrade extra-enforcer-rules to 1.8.0

2024-05-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-48242: Assignee: BingKun Pan > Upgrade extra-enforcer-rules to 1.

[jira] [Assigned] (SPARK-48307) InlineCTE should keep not-inlined relations in the original WithCTE node

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48307: -- Assignee: Apache Spark > InlineCTE should keep not-inlined relations in the origi

[jira] [Assigned] (SPARK-48258) Implement DataFrame.checkpoint and DataFrame.localCheckpoint

2024-05-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48258: -- Assignee: (was: Apache Spark) > Implement DataFrame.checkpo

[jira] [Updated] (SPARK-48311) Nested pythonUDF in groupBy and aggregate result in Binding Exception

2024-05-20 Thread Sumit Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Singh updated SPARK-48311: Description: Steps to Reproduce  1. Data creation {code:java} from pyspark.sql import

[jira] [Resolved] (SPARK-48305) CurrentLike - Database/Schema, Catalog, User (all collations)

2024-05-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48305. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46613 [https

[jira] [Resolved] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

2024-05-20 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48335. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46657 [https

[jira] [Assigned] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

2024-05-20 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48335: - Assignee: Ruifeng Zheng > Make `_parse_datatype_string` compatible with Spark Conn

[jira] [Updated] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

2024-05-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48335: --- Labels: pull-request-available (was: ) > Make `_parse_datatype_string` compati

[jira] [Created] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

2024-05-19 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-48335: - Summary: Make `_parse_datatype_string` compatible with Spark Connect Key: SPARK-48335 URL: https://issues.apache.org/jira/browse/SPARK-48335 Project: Spark

[jira] [Updated] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

2024-05-19 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48335: -- Issue Type: Improvement (was: Bug) > Make `_parse_datatype_string` compatible with Sp

[jira] [Resolved] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

2024-05-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48333. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46654 [https

[jira] [Assigned] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

2024-05-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48333: Assignee: Ruifeng Zheng > Test `test_sorting_functions_with_column` with same `Col

[jira] [Created] (SPARK-48334) NettyServer doesn't shutdown if SparkContext initialize failed

2024-05-19 Thread IsisPolei (Jira)
IsisPolei created SPARK-48334: - Summary: NettyServer doesn't shutdown if SparkContext initialize failed Key: SPARK-48334 URL: https://issues.apache.org/jira/browse/SPARK-48334 Project: Spark

[jira] [Assigned] (SPARK-44924) Add configurations for FileStreamSource cached files

2024-05-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-44924: Assignee: kevin nacios > Add configurations for FileStreamSource cached fi

[jira] [Resolved] (SPARK-44924) Add configurations for FileStreamSource cached files

2024-05-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-44924. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45362 [https

[jira] [Updated] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

2024-05-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48333: --- Labels: pull-request-available (was: ) > Test `test_sorting_functions_with_col

[jira] [Created] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

2024-05-19 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-48333: - Summary: Test `test_sorting_functions_with_column` with same `Column` Key: SPARK-48333 URL: https://issues.apache.org/jira/browse/SPARK-48333 Project: Spark

[jira] [Updated] (SPARK-48332) Upgrade `jdbc` related test dependencies

2024-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48332: --- Labels: pull-request-available (was: ) > Upgrade `jdbc` related test dependenc

[jira] [Created] (SPARK-48332) Upgrade `jdbc` related test dependencies

2024-05-18 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48332: --- Summary: Upgrade `jdbc` related test dependencies Key: SPARK-48332 URL: https://issues.apache.org/jira/browse/SPARK-48332 Project: Spark Issue Type

[jira] [Updated] (SPARK-48330) Fix the python streaming data source timeout issue for large trigger interval

2024-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48330: --- Labels: pull-request-available (was: ) > Fix the python streaming data source time

[jira] [Updated] (SPARK-48330) Fix the python streaming data source timeout issue for large trigger interval

2024-05-18 Thread Chaoqin Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoqin Li updated SPARK-48330: --- Summary: Fix the python streaming data source timeout issue for large trigger interval (was: Fix

[jira] [Created] (SPARK-48330) Fix the python data source timeout issue for large trigger interval

2024-05-18 Thread Chaoqin Li (Jira)
Chaoqin Li created SPARK-48330: -- Summary: Fix the python data source timeout issue for large trigger interval Key: SPARK-48330 URL: https://issues.apache.org/jira/browse/SPARK-48330 Project: Spark

[jira] [Commented] (SPARK-48329) Default spark.sql.sources.v2.bucketing.pushPartValues.enabled to true

2024-05-18 Thread chesterxu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847569#comment-17847569 ] chesterxu commented on SPARK-48329: --- Hey there~  If no one was assigned, please check this PR: [https

[jira] (SPARK-48329) Default spark.sql.sources.v2.bucketing.pushPartValues.enabled to true

2024-05-18 Thread chesterxu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48329 ] chesterxu deleted comment on SPARK-48329: --- was (Author: JIRAUSER302535): Hey there~  If no one was assigned, please check this PR: https://github.com/apache/spark/pull/46650 > Defa

[jira] [Comment Edited] (SPARK-48329) Default spark.sql.sources.v2.bucketing.pushPartValues.enabled to true

2024-05-18 Thread chesterxu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847502#comment-17847502 ] chesterxu edited comment on SPARK-48329 at 5/18/24 11:26 AM: - Hey

[jira] [Updated] (SPARK-48329) Default spark.sql.sources.v2.bucketing.pushPartValues.enabled to true

2024-05-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48329: --- Labels: pull-request-available (was: ) > Default spark.sql.sources

[jira] [Commented] (SPARK-48329) Default spark.sql.sources.v2.bucketing.pushPartValues.enabled to true

2024-05-18 Thread chesterxu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847502#comment-17847502 ] chesterxu commented on SPARK-48329: --- Hey there, may I ask can I have a try for this? > Defa

[jira] [Assigned] (SPARK-48175) Store collation information in metadata and not in type for SER/DE

2024-05-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48175: --- Assignee: Stefan Kandic > Store collation information in metadata and not in type for

[jira] [Resolved] (SPARK-48175) Store collation information in metadata and not in type for SER/DE

2024-05-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48175. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46280 [https

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

2024-05-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Labels: correctness pull-request-available (was: pull-request-available) > Fix the d

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

2024-05-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Issue Type: Bug (was: Improvement) > Fix the data corruption issue when state store unl

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

2024-05-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Priority: Blocker (was: Major) > Fix the data corruption issue when state store unl

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

2024-05-17 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Fix Version/s: 3.4.4 > Fix the data corruption issue when state store unload and snapshott

[jira] [Created] (SPARK-48329) Default spark.sql.sources.v2.bucketing.pushPartValues.enabled to true

2024-05-17 Thread Szehon Ho (Jira)
Szehon Ho created SPARK-48329: - Summary: Default spark.sql.sources.v2.bucketing.pushPartValues.enabled to true Key: SPARK-48329 URL: https://issues.apache.org/jira/browse/SPARK-48329 Project: Spark

[jira] [Updated] (SPARK-48328) Upgrade `Arrow` to 16.1.0

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48328: --- Labels: pull-request-available (was: ) > Upgrade `Arrow` to 16.

[jira] [Created] (SPARK-48328) Upgrade `Arrow` to 16.1.0

2024-05-17 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48328: --- Summary: Upgrade `Arrow` to 16.1.0 Key: SPARK-48328 URL: https://issues.apache.org/jira/browse/SPARK-48328 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-48327) Concurrent Spark jobs execution on K8s cluster intermittently throws 'configmaps already exists' error

2024-05-17 Thread Praneet Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praneet Sharma updated SPARK-48327: --- Component/s: Kubernetes (was: Spark Core) > Concurrent Spark j

[jira] [Created] (SPARK-48327) Concurrent Spark jobs execution on K8s cluster intermittently throws 'configmaps already exists' error

2024-05-17 Thread Praneet Sharma (Jira)
Praneet Sharma created SPARK-48327: -- Summary: Concurrent Spark jobs execution on K8s cluster intermittently throws 'configmaps already exists' error Key: SPARK-48327 URL: https://issues.apache.org/jira/browse

[jira] [Commented] (SPARK-44810) XML: ArrayType and MapType support in from_xml

2024-05-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847457#comment-17847457 ] BingKun Pan commented on SPARK-44810: - Let me refactor it and delete unnecessary logic. Thank you

[jira] [Assigned] (SPARK-48294) Make nestedTypeMissingElementTypeError case insensitive

2024-05-17 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-48294: -- Assignee: Michael Zhang > Make nestedTypeMissingElementTypeError case insensit

[jira] [Resolved] (SPARK-48294) Make nestedTypeMissingElementTypeError case insensitive

2024-05-17 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48294. Fix Version/s: 3.5.2 (was: 4.0.0) Resolution: Fixed Issue

[jira] [Created] (SPARK-48326) Upgrade submission worker base Spark version to 4.0.0-preview2

2024-05-17 Thread Zhou JIANG (Jira)
Zhou JIANG created SPARK-48326: -- Summary: Upgrade submission worker base Spark version to 4.0.0-preview2 Key: SPARK-48326 URL: https://issues.apache.org/jira/browse/SPARK-48326 Project: Spark

[jira] [Commented] (SPARK-48220) Allow passing PyArrow Table to createDataFrame()

2024-05-17 Thread Ian Cook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847448#comment-17847448 ] Ian Cook commented on SPARK-48220: -- [~gurwls223] the PR for this is ready for review: https

[jira] [Commented] (SPARK-44810) XML: ArrayType and MapType support in from_xml

2024-05-17 Thread Sandip Agarwala (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847373#comment-17847373 ] Sandip Agarwala commented on SPARK-44810: - Yes, we don't plan to support root-level ArrayType

[jira] [Updated] (SPARK-48325) Always specify messages in ExecutorRunner.killProcess

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48325: --- Labels: pull-request-available (was: ) > Always specify messa

[jira] [Created] (SPARK-48325) Always specify messages in ExecutorRunner.killProcess

2024-05-17 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-48325: Summary: Always specify messages in ExecutorRunner.killProcess Key: SPARK-48325 URL: https://issues.apache.org/jira/browse/SPARK-48325 Project: Spark Issue Type

[jira] [Updated] (SPARK-48324) Codegen Support for `hll_sketch_estimate` and `hll_union`

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48324: --- Labels: pull-request-available (was: ) > Codegen Support for `hll_sketch_estim

[jira] [Created] (SPARK-48324) Codegen Support for `hll_sketch_estimate` and `hll_union`

2024-05-17 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-48324: --- Summary: Codegen Support for `hll_sketch_estimate` and `hll_union` Key: SPARK-48324 URL: https://issues.apache.org/jira/browse/SPARK-48324 Project: Spark

[jira] [Commented] (SPARK-44810) XML: ArrayType and MapType support in from_xml

2024-05-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847247#comment-17847247 ] BingKun Pan commented on SPARK-44810: - [~sandip.agarwala] Sorry to bother you, I want to know

[jira] [Updated] (SPARK-44810) XML: ArrayType and MapType support in from_xml

2024-05-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44810: Attachment: image-2024-05-17-19-23-07-534.png > XML: ArrayType and MapType support in from_

[jira] [Updated] (SPARK-44810) XML: ArrayType and MapType support in from_xml

2024-05-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-44810: Attachment: image-2024-05-17-19-22-31-844.png > XML: ArrayType and MapType support in from_

[jira] [Updated] (SPARK-48290) AQE not working when joining dataframes with more than 2000 partitions

2024-05-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] André F. updated SPARK-48290: - Issue Type: Bug (was: Question) > AQE not working when joining dataframes with more than 2

[jira] [Updated] (SPARK-48323) DB2: Map BooleanType to Boolean instead of Char(1)

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48323: --- Labels: pull-request-available (was: ) > DB2: Map BooleanType to Boolean instead of Cha

[jira] [Resolved] (SPARK-48321) Avoid using deprecated methods in dsl

2024-05-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48321. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46635 [https

[jira] [Created] (SPARK-48323) DB2: Map BooleanType to Boolean instead of Char(1)

2024-05-17 Thread Kent Yao (Jira)
Kent Yao created SPARK-48323: Summary: DB2: Map BooleanType to Boolean instead of Char(1) Key: SPARK-48323 URL: https://issues.apache.org/jira/browse/SPARK-48323 Project: Spark Issue Type: Sub

[jira] [Assigned] (SPARK-48159) Datetime expressions (all collations)

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48159: -- Assignee: (was: Apache Spark) > Datetime expressions (all collati

[jira] [Assigned] (SPARK-48159) Datetime expressions (all collations)

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48159: -- Assignee: Apache Spark > Datetime expressions (all collati

[jira] [Updated] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48322: --- Labels: pull-request-available (was: ) > Drop internal metadata in `DataFrame.sch

[jira] [Assigned] (SPARK-47972) Restrict CAST expression for collations

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47972: -- Assignee: (was: Apache Spark) > Restrict CAST expression for collati

[jira] [Assigned] (SPARK-47972) Restrict CAST expression for collations

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47972: -- Assignee: Apache Spark > Restrict CAST expression for collati

[jira] [Created] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

2024-05-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-48322: - Summary: Drop internal metadata in `DataFrame.schema` Key: SPARK-48322 URL: https://issues.apache.org/jira/browse/SPARK-48322 Project: Spark Issue Type

[jira] [Updated] (SPARK-48321) Avoid using deprecated methods in dsl

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48321: --- Labels: pull-request-available (was: ) > Avoid using deprecated methods in

[jira] [Created] (SPARK-48321) Avoid using deprecated methods in dsl

2024-05-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-48321: - Summary: Avoid using deprecated methods in dsl Key: SPARK-48321 URL: https://issues.apache.org/jira/browse/SPARK-48321 Project: Spark Issue Type

[jira] [Updated] (SPARK-48290) AQE not working when joining dataframes with more than 2000 partitions

2024-05-17 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-48290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] André F. updated SPARK-48290: - Component/s: (was: SQL) > AQE not working when joining dataframes with more than 2000 partiti

[jira] [Assigned] (SPARK-48319) Test `assert_true` and `raise_error` with the same error class as Spark Classic

2024-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48319: Assignee: Ruifeng Zheng > Test `assert_true` and `raise_error` with the same error cl

[jira] [Resolved] (SPARK-48319) Test `assert_true` and `raise_error` with the same error class as Spark Classic

2024-05-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48319. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46633 [https

[jira] [Updated] (SPARK-48320) Add external third-party ecosystem access guide to the doc

2024-05-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48320: --- Labels: pull-request-available (was: ) > Add external third-party ecosystem access gu

[jira] [Updated] (SPARK-48320) Add external third-party ecosystem access guide to the doc

2024-05-17 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-48320: Summary: Add external third-party ecosystem access guide to the doc (was: Add third-party access

  1   2   3   4   5   6   7   8   9   10   >