[jira] [Reopened] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-38341: - > Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date > ---

[jira] [Commented] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773146#comment-17773146 ] Wenchen Fan commented on SPARK-38341: - It seems like a common behavior to preserve t

[jira] [Commented] (SPARK-38341) Spark sql: 3.2.1 - Function of add_ Months returns an incorrect date

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773153#comment-17773153 ] Max Gekk commented on SPARK-38341: -- The SQL migration guide documents the behaviour cha

[jira] [Assigned] (SPARK-45463) Allow ShuffleDriverComponent to support reliable store with specified executorId

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45463: -- Assignee: Apache Spark > Allow ShuffleDriverComponent to support reliable store with

[jira] [Assigned] (SPARK-45463) Allow ShuffleDriverComponent to support reliable store with specified executorId

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45463: -- Assignee: (was: Apache Spark) > Allow ShuffleDriverComponent to support reliable

[jira] [Assigned] (SPARK-45455) [SQL][JDBC] Improve the rename interface of Postgres Dialect

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45455: -- Assignee: Apache Spark > [SQL][JDBC] Improve the rename interface of Postgres Dialect

[jira] [Assigned] (SPARK-45455) [SQL][JDBC] Improve the rename interface of Postgres Dialect

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45455: -- Assignee: (was: Apache Spark) > [SQL][JDBC] Improve the rename interface of Postg

[jira] [Assigned] (SPARK-45424) Regression in CSV schema inference when timestamps do not match specified timestampFormat

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45424: -- Assignee: Apache Spark > Regression in CSV schema inference when timestamps do not ma

[jira] [Assigned] (SPARK-45424) Regression in CSV schema inference when timestamps do not match specified timestampFormat

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45424: -- Assignee: (was: Apache Spark) > Regression in CSV schema inference when timestamp

[jira] [Assigned] (SPARK-45424) Regression in CSV schema inference when timestamps do not match specified timestampFormat

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45424: -- Assignee: (was: Apache Spark) > Regression in CSV schema inference when timestamp

[jira] [Assigned] (SPARK-45424) Regression in CSV schema inference when timestamps do not match specified timestampFormat

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45424: -- Assignee: Apache Spark > Regression in CSV schema inference when timestamps do not ma

[jira] [Assigned] (SPARK-45418) Change CURRENT_SCHEMA() column alias to match function name

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45418: -- Assignee: Apache Spark > Change CURRENT_SCHEMA() column alias to match function name

[jira] [Assigned] (SPARK-45418) Change CURRENT_SCHEMA() column alias to match function name

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45418: -- Assignee: (was: Apache Spark) > Change CURRENT_SCHEMA() column alias to match fun

[jira] [Assigned] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45459: -- Assignee: (was: Apache Spark) > Remove the last 2 extra spaces in the automatical

[jira] [Assigned] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45459: -- Assignee: Apache Spark > Remove the last 2 extra spaces in the automatically generate

[jira] [Assigned] (SPARK-45464) [CORE] Fix yarn distribution build

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45464: -- Assignee: (was: Apache Spark) > [CORE] Fix yarn distribution build >

[jira] [Assigned] (SPARK-45464) [CORE] Fix yarn distribution build

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45464: -- Assignee: Apache Spark > [CORE] Fix yarn distribution build > ---

[jira] [Assigned] (SPARK-45464) [CORE] Fix yarn distribution build

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45464: -- Assignee: Apache Spark > [CORE] Fix yarn distribution build > ---

[jira] [Assigned] (SPARK-45464) [CORE] Fix yarn distribution build

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45464: -- Assignee: (was: Apache Spark) > [CORE] Fix yarn distribution build >

[jira] [Assigned] (SPARK-45463) Allow ShuffleDriverComponent to support reliable store with specified executorId

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45463: -- Assignee: Apache Spark > Allow ShuffleDriverComponent to support reliable store with

[jira] [Assigned] (SPARK-45463) Allow ShuffleDriverComponent to support reliable store with specified executorId

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45463: -- Assignee: (was: Apache Spark) > Allow ShuffleDriverComponent to support reliable

[jira] [Assigned] (SPARK-45424) Regression in CSV schema inference when timestamps do not match specified timestampFormat

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45424: Assignee: Jia Fan > Regression in CSV schema inference when timestamps do not match specified >

[jira] [Resolved] (SPARK-45424) Regression in CSV schema inference when timestamps do not match specified timestampFormat

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45424. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45459: Assignee: BingKun Pan > Remove the last 2 extra spaces in the automatically generated > `sql-err

[jira] [Resolved] (SPARK-45459) Remove the last 2 extra spaces in the automatically generated `sql-error-conditions.md` file

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45459. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-45200) Spark 3.4.0 always using default log4j profile

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45200: --- Labels: pull-request-available (was: ) > Spark 3.4.0 always using default log4j profile > -

[jira] [Commented] (SPARK-44988) Parquet INT64 (TIMESTAMP(NANOS,false)) throwing Illegal Parquet type

2023-10-09 Thread Miles Granger (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773215#comment-17773215 ] Miles Granger commented on SPARK-44988: --- [~fanjia]that "worked" for me, but then o

[jira] [Created] (SPARK-45470) Avoid paste string value of hive orc compression kind

2023-10-09 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-45470: -- Summary: Avoid paste string value of hive orc compression kind Key: SPARK-45470 URL: https://issues.apache.org/jira/browse/SPARK-45470 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45470) Avoid paste string value of hive orc compression kind

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45470: --- Labels: pull-request-available (was: ) > Avoid paste string value of hive orc compression k

[jira] [Updated] (SPARK-42034) QueryExecutionListener and Observation API, df.observe do not work with `foreach` action.

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42034: --- Labels: pull-request-available sql-api (was: sql-api) > QueryExecutionListener and Observat

[jira] [Created] (SPARK-45471) Allow specifying ResourceProfile in Dataset API

2023-10-09 Thread Keio Kraaner (Jira)
Keio Kraaner created SPARK-45471: Summary: Allow specifying ResourceProfile in Dataset API Key: SPARK-45471 URL: https://issues.apache.org/jira/browse/SPARK-45471 Project: Spark Issue Type: N

[jira] [Updated] (SPARK-42132) DeduplicateRelations rule breaks plan when co-grouping the same DataFrame

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42132: --- Labels: correctness pull-request-available (was: correctness) > DeduplicateRelations rule b

[jira] [Updated] (SPARK-45468) More transparent proxy handling for HTTP redirects

2023-10-09 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-45468: - Summary: More transparent proxy handling for HTTP redirects (was: Add option to use pat

[jira] [Updated] (SPARK-45468) More transparent proxy handling for HTTP redirects

2023-10-09 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nobuaki Sukegawa updated SPARK-45468: - Description: Currently, proxies can be made transparent for hyperlinks in Spark web UIs

[jira] [Updated] (SPARK-45468) More transparent proxy handling for HTTP redirects

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45468: --- Labels: pull-request-available (was: ) > More transparent proxy handling for HTTP redirects

[jira] [Commented] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-10-09 Thread Nobuaki Sukegawa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773360#comment-17773360 ] Nobuaki Sukegawa commented on SPARK-45201: -- We've experienced the same issue an

[jira] [Updated] (SPARK-45383) Missing case for RelationTimeTravel in CheckAnalysis

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45383: --- Labels: pull-request-available (was: ) > Missing case for RelationTimeTravel in CheckAnalys

[jira] [Commented] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-10-09 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773397#comment-17773397 ] Sebastian Daberdaku commented on SPARK-45201: - Hello [~nsuke],  I am happy m

[jira] [Created] (SPARK-45472) RocksDB State Store Doesn't Need to Recheck checkpoint path existence

2023-10-09 Thread Siying Dong (Jira)
Siying Dong created SPARK-45472: --- Summary: RocksDB State Store Doesn't Need to Recheck checkpoint path existence Key: SPARK-45472 URL: https://issues.apache.org/jira/browse/SPARK-45472 Project: Spark

[jira] [Updated] (SPARK-45472) RocksDB State Store Doesn't Need to Recheck checkpoint path existence

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45472: --- Labels: pull-request-available (was: ) > RocksDB State Store Doesn't Need to Recheck checkp

[jira] [Updated] (SPARK-44919) Avro connector: convert a union of a single primitive type to a StructType

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44919: --- Labels: pull-request-available (was: ) > Avro connector: convert a union of a single primit

[jira] [Resolved] (SPARK-45383) Missing case for RelationTimeTravel in CheckAnalysis

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45383. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-45383) Missing case for RelationTimeTravel in CheckAnalysis

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45383: Assignee: Wenchen Fan > Missing case for RelationTimeTravel in CheckAnalysis > --

[jira] [Updated] (SPARK-44729) Add canonical links to the PySpark docs page

2023-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44729: -- Fix Version/s: 3.4.2 > Add canonical links to the PySpark docs page >

[jira] [Commented] (SPARK-44729) Add canonical links to the PySpark docs page

2023-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773473#comment-17773473 ] Dongjoon Hyun commented on SPARK-44729: --- This landed at branch-3.4 via https://git

[jira] [Resolved] (SPARK-45456) Upgrade maven to 3.9.5

2023-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45456. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43267 [https://

[jira] [Commented] (SPARK-44729) Add canonical links to the PySpark docs page

2023-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773474#comment-17773474 ] Dongjoon Hyun commented on SPARK-44729: --- This landed at branch-3.3 via https://git

[jira] [Commented] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

2023-10-09 Thread Eren Avsarogullari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773480#comment-17773480 ] Eren Avsarogullari commented on SPARK-45443: > But this idea only work for o

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

2023-10-09 Thread Eren Avsarogullari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773480#comment-17773480 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/9/23 9:16 PM: -

[jira] [Comment Edited] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

2023-10-09 Thread Eren Avsarogullari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773480#comment-17773480 ] Eren Avsarogullari edited comment on SPARK-45443 at 10/9/23 9:17 PM: -

[jira] [Updated] (SPARK-45221) Refine docstring of `DataFrameReader.parquet`

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45221: --- Labels: pull-request-available (was: ) > Refine docstring of `DataFrameReader.parquet` > --

[jira] [Created] (SPARK-45473) Incorrect error message for RoundBase

2023-10-09 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-45473: --- Summary: Incorrect error message for RoundBase Key: SPARK-45473 URL: https://issues.apache.org/jira/browse/SPARK-45473 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-43299) JVM Client throw StreamingQueryException when error handling is implemented

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43299: Assignee: Yihong He > JVM Client throw StreamingQueryException when error handling is imp

[jira] [Resolved] (SPARK-43299) JVM Client throw StreamingQueryException when error handling is implemented

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43299. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42859 [https://gi

[jira] [Updated] (SPARK-45473) Incorrect error message for RoundBase

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45473: --- Labels: pull-request-available (was: ) > Incorrect error message for RoundBase > --

[jira] [Commented] (SPARK-44434) Add more tests for Scala foreachBatch and streaming listeners

2023-10-09 Thread Brian Carlson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773501#comment-17773501 ] Brian Carlson commented on SPARK-44434: --- I'd like to work on this. > Add more tes

[jira] [Resolved] (SPARK-45419) Avoid reusing rocksdb sst files in a dfferent rocksdb instance by removing file version map entry of larger versions

2023-10-09 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-45419. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43174 [https://gi

[jira] [Commented] (SPARK-45443) Revisit TableCacheQueryStage to avoid replicated InMemoryRelation materialization

2023-10-09 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773507#comment-17773507 ] XiDuo You commented on SPARK-45443: --- > But this idea only work for one query Please s

[jira] [Assigned] (SPARK-45472) RocksDB State Store Doesn't Need to Recheck checkpoint path existence

2023-10-09 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-45472: Assignee: Siying Dong > RocksDB State Store Doesn't Need to Recheck checkpoint path exist

[jira] [Resolved] (SPARK-45472) RocksDB State Store Doesn't Need to Recheck checkpoint path existence

2023-10-09 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-45472. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43299 [https://gi

[jira] [Created] (SPARK-45474) Support top-level filtering in MasterPage JSON API

2023-10-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45474: - Summary: Support top-level filtering in MasterPage JSON API Key: SPARK-45474 URL: https://issues.apache.org/jira/browse/SPARK-45474 Project: Spark Issue Ty

[jira] [Updated] (SPARK-45474) Support top-level filtering in MasterPage JSON API

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45474: --- Labels: pull-request-available (was: ) > Support top-level filtering in MasterPage JSON API

[jira] [Created] (SPARK-45475) Should use DataFrame.foreachBatch instead of RDD.foreachBatch in JdbcUtils

2023-10-09 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45475: Summary: Should use DataFrame.foreachBatch instead of RDD.foreachBatch in JdbcUtils Key: SPARK-45475 URL: https://issues.apache.org/jira/browse/SPARK-45475 Project: S

[jira] [Updated] (SPARK-45475) Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45475: - Summary: Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils (wa

[jira] [Updated] (SPARK-45475) Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45475: --- Labels: pull-request-available (was: ) > Should use DataFrame.foreachPartition instead of R

[jira] [Resolved] (SPARK-45450) Fix imports according to PEP8: pyspark.pandas and pyspark (core)

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45450. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43257 [https://gi

[jira] [Created] (SPARK-45476) Raise exception directly instead of calling `resolveColumnsByPosition`

2023-10-09 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-45476: --- Summary: Raise exception directly instead of calling `resolveColumnsByPosition` Key: SPARK-45476 URL: https://issues.apache.org/jira/browse/SPARK-45476 Project: Spark

[jira] [Updated] (SPARK-45476) Raise exception directly instead of calling `resolveColumnsByPosition`

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45476: --- Labels: pull-request-available (was: ) > Raise exception directly instead of calling `resol

[jira] [Resolved] (SPARK-45470) Avoid paste string value of hive orc compression kind

2023-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45470. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43296 [https://

[jira] [Created] (SPARK-45477) Use `matrix.java/inputs.java` to replace the hardcoded Java version in `test results/unit tests log` naming

2023-10-09 Thread Yang Jie (Jira)
Yang Jie created SPARK-45477: Summary: Use `matrix.java/inputs.java` to replace the hardcoded Java version in `test results/unit tests log` naming Key: SPARK-45477 URL: https://issues.apache.org/jira/browse/SPARK-4547

[jira] [Updated] (SPARK-45477) Use `matrix.java/inputs.java` to replace the hardcoded Java version in `test results/unit tests log` naming

2023-10-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45477: --- Labels: pull-request-available (was: ) > Use `matrix.java/inputs.java` to replace the hardc

[jira] [Resolved] (SPARK-45458) Convert IllegalArgumentException to SparkIllegalArgumentException in bitwiseExpressions and add some UT

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45458. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43271 [https://github.com

[jira] [Assigned] (SPARK-45458) Convert IllegalArgumentException to SparkIllegalArgumentException in bitwiseExpressions and add some UT

2023-10-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45458: Assignee: BingKun Pan > Convert IllegalArgumentException to SparkIllegalArgumentException in > b

[jira] [Resolved] (SPARK-45205) Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by SPARK-35278

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45205. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45205) Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by SPARK-35278

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45205: --- Assignee: Qiang Yang > Since version 3.2.0, Spark SQL has taken longer to execute "show >

[jira] [Resolved] (SPARK-45449) Cache Invalidation Issue with JDBC Table

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45449. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-45449) Cache Invalidation Issue with JDBC Table

2023-10-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45449: --- Assignee: liangyongyuan > Cache Invalidation Issue with JDBC Table > --

[jira] [Resolved] (SPARK-45474) Support top-level filtering in MasterPage JSON API

2023-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45474. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43303 [https://

[jira] [Assigned] (SPARK-45474) Support top-level filtering in MasterPage JSON API

2023-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45474: - Assignee: Dongjoon Hyun > Support top-level filtering in MasterPage JSON API >