svn commit: r69426 - /dev/spark/v4.0.0-preview1-rc3-bin/

2024-05-28 Thread wenchen
Author: wenchen Date: Tue May 28 16:50:57 2024 New Revision: 69426 Log: Apache Spark v4.0.0-preview1-rc3 Added: dev/spark/v4.0.0-preview1-rc3-bin/ dev/spark/v4.0.0-preview1-rc3-bin/SparkR_4.0.0-preview1.tar.gz (with props) dev/spark/v4.0.0-preview1-rc3-bin/SparkR_4.0.0-preview1

(spark) 01/01: Preparing Spark release v4.0.0-preview1-rc3

2024-05-28 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to tag v4.0.0-preview1-rc3 in repository https://gitbox.apache.org/repos/asf/spark.git commit 7a7a8bc4bab591ac8b98b2630b38c57adf619b82 Author: Wenchen Fan AuthorDate: Tue May 28 16:23:00 2024 +

(spark) tag v4.0.0-preview1-rc3 created (now 7a7a8bc4bab5)

2024-05-28 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to tag v4.0.0-preview1-rc3 in repository https://gitbox.apache.org/repos/asf/spark.git at 7a7a8bc4bab5 (commit) This tag includes the following new commits: new 7a7a8bc4bab5 Preparing Spark

Re: [VOTE] SPARK 4.0.0-preview1 (RC2)

2024-05-28 Thread Wenchen Fan
ug fix > > > https://github.com/apache/spark/commit/6cd1ccc56321dfa52672cd25f4cfdf2bbc86b3ea > . > > The bug can lead to the unrecoverable job failure. > > > > Thanks, > > Yi > > > > On Tue, May 28, 2024 at 3:45 PM Wenchen Fan wrote: > > > > > Please vote on releasing

svn commit: r69425 - /dev/spark/KEYS

2024-05-28 Thread wenchen
Author: wenchen Date: Tue May 28 16:09:45 2024 New Revision: 69425 Log: Update KEYS Modified: dev/spark/KEYS Modified: dev/spark/KEYS == --- dev/spark/KEYS (original) +++ dev/spark/KEYS Tue May 28 16:09:45 2024

[VOTE] SPARK 4.0.0-preview1 (RC2)

2024-05-28 Thread Wenchen Fan
Please vote on releasing the following candidate as Apache Spark version 4.0.0-preview1. The vote is open until May 31 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 4.0.0-preview1 [ ] -1 Do not release this package

svn commit: r69418 - /dev/spark/v4.0.0-preview1-rc2-bin/

2024-05-28 Thread wenchen
Author: wenchen Date: Tue May 28 07:41:08 2024 New Revision: 69418 Log: Apache Spark v4.0.0-preview1-rc2 Added: dev/spark/v4.0.0-preview1-rc2-bin/ dev/spark/v4.0.0-preview1-rc2-bin/SparkR_4.0.0-preview1.tar.gz (with props) dev/spark/v4.0.0-preview1-rc2-bin/SparkR_4.0.0-preview1

svn commit: r69417 - /dev/spark/v4.0.0-preview1-rc2-bin/

2024-05-28 Thread wenchen
Author: wenchen Date: Tue May 28 06:35:58 2024 New Revision: 69417 Log: Deleting Removed: dev/spark/v4.0.0-preview1-rc2-bin/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail

(spark) branch branch-3.5 updated: [SPARK-41049][SQL][FOLLOW-UP] Mark map related expressions as stateful expressions

2024-05-27 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new f42c029fac5c [SPARK-41049][SQL][FOLLOW

(spark) branch master updated: [SPARK-41049][SQL][FOLLOW-UP] Mark map related expressions as stateful expressions

2024-05-27 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new af1ac1edc2a9 [SPARK-41049][SQL][FOLLOW-UP] Mark

svn commit: r69416 - in /dev/spark/v4.0.0-preview1-rc2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/R/articles/ _site/api/R/articles/sparkr-vignettes_files/ _site/api/R/articles/sparkr-vignettes_

2024-05-27 Thread wenchen
Author: wenchen Date: Tue May 28 05:29:59 2024 New Revision: 69416 Log: Apache Spark v4.0.0-preview1-rc2 docs [This commit notification would consist of 4816 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

svn commit: r69415 - /dev/spark/v4.0.0-preview1-rc2-bin/

2024-05-27 Thread wenchen
Author: wenchen Date: Tue May 28 04:31:46 2024 New Revision: 69415 Log: Apache Spark v4.0.0-preview1-rc2 Added: dev/spark/v4.0.0-preview1-rc2-bin/ dev/spark/v4.0.0-preview1-rc2-bin/SparkR_4.0.0-preview1.tar.gz (with props) dev/spark/v4.0.0-preview1-rc2-bin/SparkR_4.0.0-preview1

(spark) branch master updated: [SPARK-48239][INFRA][FOLLOWUP] install the missing `jq` library

2024-05-24 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 416d7f24fc35 [SPARK-48239][INFRA][FOLLOWUP

(spark) tag v4.0.0-preview1-rc2 created (now 7cfe5a6e44e8)

2024-05-24 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to tag v4.0.0-preview1-rc2 in repository https://gitbox.apache.org/repos/asf/spark.git at 7cfe5a6e44e8 (commit) This tag includes the following new commits: new 7cfe5a6e44e8 Preparing Spark

(spark) 01/01: Preparing Spark release v4.0.0-preview1-rc2

2024-05-24 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to tag v4.0.0-preview1-rc2 in repository https://gitbox.apache.org/repos/asf/spark.git commit 7cfe5a6e44e8d7079ae29ad3e2cee7231cd3dc66 Author: Wenchen Fan AuthorDate: Fri May 24 18:53:15 2024 +

(spark) tag v4.0.0-preview-rc1 deleted (was 9fec87d16a04)

2024-05-24 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to tag v4.0.0-preview-rc1 in repository https://gitbox.apache.org/repos/asf/spark.git *** WARNING: tag v4.0.0-preview-rc1 was deleted! *** was 9fec87d16a04 Preparing Spark release v4.0.0-preview

[jira] [Assigned] (SPARK-48364) Type casting for AbstractMapType

2024-05-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48364: --- Assignee: Uroš Bojanić > Type casting for AbstractMapT

[jira] [Resolved] (SPARK-48364) Type casting for AbstractMapType

2024-05-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48364. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46661 [https

(spark) branch master updated: [SPARK-48364][SQL] Add AbstractMapType type casting and fix RaiseError parameter map to work with collated strings

2024-05-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6be3560f3c89 [SPARK-48364][SQL] Add

[jira] [Resolved] (SPARK-48215) DateFormatClass (all collations)

2024-05-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48215. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46561 [https

[jira] [Assigned] (SPARK-48215) DateFormatClass (all collations)

2024-05-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48215: --- Assignee: Nebojsa Savic > DateFormatClass (all collati

(spark) branch master updated: [SPARK-48215][SQL] Extending support for collated strings on date_format expression

2024-05-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e04d3d7c430a [SPARK-48215][SQL] Extending

(spark) branch master updated: [SPARK-48031] Decompose viewSchemaMode config, add SHOW CREATE TABLE support

2024-05-20 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 617ac1aec748 [SPARK-48031] Decompose

[jira] [Resolved] (SPARK-48305) CurrentLike - Database/Schema, Catalog, User (all collations)

2024-05-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48305. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46613 [https

(spark) branch master updated: [SPARK-48305][SQL] Add collation support for CurrentLike expressions

2024-05-20 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6a17b794338b [SPARK-48305][SQL] Add collation

[jira] [Assigned] (SPARK-48175) Store collation information in metadata and not in type for SER/DE

2024-05-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48175: --- Assignee: Stefan Kandic > Store collation information in metadata and not in type for

[jira] [Resolved] (SPARK-48175) Store collation information in metadata and not in type for SER/DE

2024-05-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48175. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46280 [https

(spark) branch master updated: [SPARK-48175][SQL][PYTHON] Store collation information in metadata and not in type for SER/DE

2024-05-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6f6b4860268d [SPARK-48175][SQL][PYTHON] Store

(spark) branch master updated (15fb4787354a -> 3edd6c7e1d50)

2024-05-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 15fb4787354a [SPARK-48321][CONNECT][TESTS] Avoid using deprecated methods in dsl add 3edd6c7e1d50 [SPARK-48312

[jira] [Resolved] (SPARK-48308) Unify getting data schema without partition columns in FileSourceStrategy

2024-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48308. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46619 [https

[jira] [Assigned] (SPARK-48308) Unify getting data schema without partition columns in FileSourceStrategy

2024-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48308: --- Assignee: Johan Lasperas > Unify getting data schema without partition colu

(spark) branch master updated: [SPARK-48308][CORE] Unify getting data schema without partition columns in FileSourceStrategy

2024-05-16 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 57948c865e06 [SPARK-48308][CORE] Unify getting

[jira] [Resolved] (SPARK-48288) Add source data type to connector.Cast expression

2024-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48288. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46596 [https

(spark) branch master updated (fa83d0f8fce7 -> 4be0828e6e6a)

2024-05-16 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from fa83d0f8fce7 [SPARK-48296][SQL] Codegen Support for `to_xml` add 4be0828e6e6a [SPARK-48288] Add source data type

[jira] [Resolved] (SPARK-48252) Update CommonExpressionRef when necessary

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48252. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46552 [https

[jira] [Assigned] (SPARK-48252) Update CommonExpressionRef when necessary

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48252: --- Assignee: Wenchen Fan > Update CommonExpressionRef when necess

(spark) branch master updated: [SPARK-48252][SQL] Update CommonExpressionRef when necessary

2024-05-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ca3593288d57 [SPARK-48252][SQL] Update

(spark) branch branch-3.4 updated: [SPARK-48172][SQL] Fix escaping issues in JDBCDialects

2024-05-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new 0e7156d2d801 [SPARK-48172][SQL] Fix

[jira] [Resolved] (SPARK-48172) Fix escaping issues in JDBCDialects

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48172. - Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution: Fixed

(spark) branch branch-3.5 updated: [SPARK-48172][SQL] Fix escaping issues in JDBCDialects

2024-05-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 210ed2521d3d [SPARK-48172][SQL] Fix

(spark) branch master updated: [SPARK-48172][SQL] Fix escaping issues in JDBCDialects

2024-05-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9e386b472981 [SPARK-48172][SQL] Fix escaping

Re: [DISCUSS] clarify the definition of behavior changes

2024-05-15 Thread Wenchen Fan
migration guide would have helped. > > > > To summarize: the migration guide was invaluable, we appreciated every > entry, and we'd appreciate Wenchen's stricter definition of "behavior > changes" (especially for silent ones). > > > > *From: *Nimrod Ofek >

Re: [VOTE] SPARK 4.0.0-preview1 (RC1)

2024-05-15 Thread Wenchen Fan
kproject.jetty.servlet.ServletHandler.initialize(ServletHandler.java:749) > ~[spark-core_2.13-4.0.0-preview1.jar:4.0.0-preview1] > ... 38 more > > Thanks, > Cheng Pan > > > > On May 11, 2024, at 13:55, Wenchen Fan wrote: > > > > Please vote on releasing t

[jira] [Resolved] (SPARK-48277) Improve error message for ErrorClassesJsonReader.getErrorMessage

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48277. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46584 [https

(spark) branch master updated (8c0a7ba82c98 -> 5e87e9fbd6e6)

2024-05-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 8c0a7ba82c98 [SPARK-48160][SQL] Add collation support for XPATH expressions add 5e87e9fbd6e6 [SPARK-48277

[jira] [Assigned] (SPARK-48160) XPath expressions (all collations)

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48160: --- Assignee: Uroš Bojanić > XPath expressions (all collati

[jira] [Resolved] (SPARK-48160) XPath expressions (all collations)

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48160. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46508 [https

(spark) branch master updated: [SPARK-48160][SQL] Add collation support for XPATH expressions

2024-05-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8c0a7ba82c98 [SPARK-48160][SQL] Add collation

[jira] [Assigned] (SPARK-48162) Miscellaneous expressions (all collations)

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48162: --- Assignee: Uroš Bojanić > Miscellaneous expressions (all collati

[jira] [Resolved] (SPARK-48162) Miscellaneous expressions (all collations)

2024-05-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48162. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46461 [https

(spark) branch master updated: [SPARK-48162][SQL] Add collation support for MISC expressions

2024-05-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 723354039f1d [SPARK-48162][SQL] Add collation

[jira] [Updated] (SPARK-48271) Turn match error in RowEncoder into UNSUPPORTED_DATA_TYPE_FOR_ENCODER

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-48271: Summary: Turn match error in RowEncoder into UNSUPPORTED_DATA_TYPE_FOR_ENCODER (was: support

[jira] [Resolved] (SPARK-48263) Collate function support for non UTF8_BINARY strings

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48263. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46574 [https

(spark) branch master updated: [SPARK-48263] Collate function support for non UTF8_BINARY strings

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 91da2caa409c [SPARK-48263] Collate function

[jira] [Assigned] (SPARK-48263) Collate function support for non UTF8_BINARY strings

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48263: --- Assignee: Nebojsa Savic > Collate function support for non UTF8_BINARY stri

(spark) branch master updated: [SPARK-47301][SQL][TESTS][FOLLOWUP] Remove workaround for ParquetIOSuite

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 97bf1ee9f6f7 [SPARK-47301][SQL][TESTS][FOLLOWUP

[jira] [Resolved] (SPARK-48172) Fix escaping issues in JDBCDialects

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48172. - Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution: Fixed

(spark) branch branch-3.4 updated: [SPARK-48172][SQL] Fix escaping issues in JDBC Dialects

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new a848e2790cba [SPARK-48172][SQL] Fix

(spark) branch branch-3.5 updated: [SPARK-48172][SQL] Fix escaping issues in JDBC Dialects

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new f37fa436cd4e [SPARK-48172][SQL] Fix

(spark) branch master updated: [SPARK-48172][SQL] Fix escaping issues in JDBC Dialects

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 47006a493f98 [SPARK-48172][SQL] Fix escaping

[jira] [Resolved] (SPARK-48155) PropagateEmpty relation cause LogicalQueryStage only with broadcast without join then execute failed

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48155. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46523 [https

[jira] [Assigned] (SPARK-48155) PropagateEmpty relation cause LogicalQueryStage only with broadcast without join then execute failed

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48155: --- Assignee: angerszhu > PropagateEmpty relation cause LogicalQueryStage only with broadc

(spark) branch master updated: [SPARK-48155][SQL] AQEPropagateEmptyRelation for join should check if remain child is just BroadcastQueryStageExec

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e5ad5e94a8c8 [SPARK-48155][SQL

[jira] [Created] (SPARK-48271) support char/varchar in RowEncoder

2024-05-14 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-48271: --- Summary: support char/varchar in RowEncoder Key: SPARK-48271 URL: https://issues.apache.org/jira/browse/SPARK-48271 Project: Spark Issue Type: Improvement

(spark) branch master updated: [SPARK-46707][SQL][FOLLOWUP] Push down throwable predicate through aggregates

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6766c39b458a [SPARK-46707][SQL][FOLLOWUP] Push

[jira] [Resolved] (SPARK-48157) CSV expressions (all collations)

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48157. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46504 [https

[jira] [Assigned] (SPARK-48157) CSV expressions (all collations)

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48157: --- Assignee: Uroš Bojanić > CSV expressions (all collati

(spark) branch master updated: [SPARK-48157][SQL] Add collation support for CSV expressions

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e6c914f63079 [SPARK-48157][SQL] Add collation

[jira] [Resolved] (SPARK-48229) inputFile expressions (all collations)

2024-05-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48229. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46503 [https

(spark) branch master updated: [SPARK-48229][SQL] Add collation support for inputFile expressions

2024-05-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9241b8e8c0df [SPARK-48229][SQL] Add collation

[jira] [Assigned] (SPARK-48265) Infer window group limit batch should do constant folding

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48265: --- Assignee: angerszhu > Infer window group limit batch should do constant fold

[jira] [Resolved] (SPARK-48265) Infer window group limit batch should do constant folding

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48265. - Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

(spark) branch branch-3.5 updated: [SPARK-48265][SQL] Infer window group limit batch should do constant folding

2024-05-13 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 34588a82239a [SPARK-48265][SQL] Infer

(spark) branch master updated: [SPARK-48265][SQL] Infer window group limit batch should do constant folding

2024-05-13 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7974811218c9 [SPARK-48265][SQL] Infer window

(spark) branch master updated: [SPARK-48027][SQL][FOLLOWUP] Add comments for the other code branch

2024-05-13 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0ea808880e22 [SPARK-48027][SQL][FOLLOWUP] Add

[jira] [Resolved] (SPARK-48241) CSV parsing failure with char/varchar type columns

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48241. - Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 46565 [https

[jira] [Assigned] (SPARK-48241) CSV parsing failure with char/varchar type columns

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48241: --- Assignee: Jiayi Liu > CSV parsing failure with char/varchar type colu

(spark) branch branch-3.5 updated: [SPARK-48241][SQL][3.5] CSV parsing failure with char/varchar type columns

2024-05-13 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 19d12b249f0f [SPARK-48241][SQL][3.5

Re: [VOTE] SPIP: Stored Procedures API for Catalogs

2024-05-13 Thread Wenchen Fan
+1 On Tue, May 14, 2024 at 8:19 AM Zhou Jiang wrote: > +1 (non-binding) > > On Sat, May 11, 2024 at 2:10 PM L. C. Hsieh wrote: > >> Hi all, >> >> I’d like to start a vote for SPIP: Stored Procedures API for Catalogs. >> >> Please also refer to: >> >>- Discussion thread: >>

(spark) branch master updated: [SPARK-48241][SQL] CSV parsing failure with char/varchar type columns

2024-05-13 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b14abb3a2ed0 [SPARK-48241][SQL] CSV parsing

(spark) branch master updated (42f2132d1fc9 -> 3456d4f69a86)

2024-05-13 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 42f2132d1fc9 [SPARK-48206][SQL][TESTS] Add tests for window rewrites with RewriteWithExpression add

[jira] [Resolved] (SPARK-48206) Add tests for window expression rewrites in RewriteWithExpression

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48206. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46492 [https

[jira] [Assigned] (SPARK-48206) Add tests for window expression rewrites in RewriteWithExpression

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48206: --- Assignee: Kelvin Jiang > Add tests for window expression rewrites in RewriteWithExpress

(spark) branch master updated: [SPARK-48206][SQL][TESTS] Add tests for window rewrites with RewriteWithExpression

2024-05-13 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 42f2132d1fc9 [SPARK-48206][SQL][TESTS] Add

[jira] [Assigned] (SPARK-48031) Add schema evolution options to views

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48031: --- Assignee: Serge Rielau > Add schema evolution options to vi

[jira] [Resolved] (SPARK-48031) Add schema evolution options to views

2024-05-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48031. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46267 [https

Re: [DISCUSS] Spark - How to improve our release processes

2024-05-13 Thread Wenchen Fan
Hi Nicholas, Thanks for your help! I'm definitely interested in participating in this unification work. Let me know how I can help. Wenchen On Mon, May 13, 2024 at 1:41 PM Nicholas Chammas wrote: > Re: unification > > We also have a long-standing problem with how we mana

[jira] [Created] (SPARK-48260) disable output committer coordination in one test of ParquetIOSuite

2024-05-13 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-48260: --- Summary: disable output committer coordination in one test of ParquetIOSuite Key: SPARK-48260 URL: https://issues.apache.org/jira/browse/SPARK-48260 Project: Spark

[jira] [Created] (SPARK-48252) Update CommonExpressionRef when necessary

2024-05-13 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-48252: --- Summary: Update CommonExpressionRef when necessary Key: SPARK-48252 URL: https://issues.apache.org/jira/browse/SPARK-48252 Project: Spark Issue Type: Bug

Re: [VOTE] SPIP: Stored Procedures API for Catalogs

2024-05-12 Thread Wenchen Fan
+1 On Mon, May 13, 2024 at 10:30 AM Kent Yao wrote: > +1 > > Dongjoon Hyun 于2024年5月13日周一 08:39写道: > > > > +1 > > > > On Sun, May 12, 2024 at 3:50 PM huaxin gao > wrote: > >> > >> +1 > >> > >> On Sat, May 11, 2024 at 4:35 PM L. C. Hsieh wrote: > >>> > >>> +1 > >>> > >>> On Sat, May 11, 2024

Re: [DISCUSS] Spark - How to improve our release processes

2024-05-12 Thread Wenchen Fan
as Github Action jobs, but I think it's more important to do the unification now. Thanks, Wenchen On Fri, May 10, 2024 at 12:34 AM Hussein Awala wrote: > Hello, > > I can answer some of your common questions with other Apache projects. > > > Who currently has permissions

[VOTE] SPARK 4.0.0-preview1 (RC1)

2024-05-10 Thread Wenchen Fan
Please vote on releasing the following candidate as Apache Spark version 4.0.0-preview1. The vote is open until May 16 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 4.0.0-preview1 [ ] -1 Do not release this package

svn commit: r69098 - /dev/spark/v4.0.0-preview1-rc1-bin/

2024-05-10 Thread wenchen
Author: wenchen Date: Sat May 11 04:28:26 2024 New Revision: 69098 Log: Apache Spark v4.0.0-preview1-rc1 Added: dev/spark/v4.0.0-preview1-rc1-bin/ dev/spark/v4.0.0-preview1-rc1-bin/SparkR_4.0.0-preview1.tar.gz (with props) dev/spark/v4.0.0-preview1-rc1-bin/SparkR_4.0.0-preview1

svn commit: r69097 - /dev/spark/v4.0.0-preview1-rc1-bin/

2024-05-10 Thread wenchen
Author: wenchen Date: Sat May 11 03:59:33 2024 New Revision: 69097 Log: prepare for re-uploading Removed: dev/spark/v4.0.0-preview1-rc1-bin/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional

svn commit: r69092 - in /dev/spark/v4.0.0-preview1-rc1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/R/articles/ _site/api/R/articles/sparkr-vignettes_files/ _site/api/R/articles/sparkr-vignettes_

2024-05-10 Thread wenchen
Author: wenchen Date: Fri May 10 16:44:08 2024 New Revision: 69092 Log: Apache Spark v4.0.0-preview1-rc1 docs [This commit notification would consist of 4810 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

(spark) branch master updated: [SPARK-48143][SQL] Use lightweight exceptions for control-flow between UnivocityParser and FailureSafeParser

2024-05-10 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a6632ffa16f6 [SPARK-48143][SQL] Use lightweight

[jira] [Assigned] (SPARK-48146) Fix error with aggregate function in With child

2024-05-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48146: --- Assignee: Kelvin Jiang > Fix error with aggregate function in With ch

[jira] [Resolved] (SPARK-48146) Fix error with aggregate function in With child

2024-05-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48146. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46443 [https

(spark) branch master updated: [SPARK-48146][SQL] Fix aggregate function in With expression child assertion

2024-05-10 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7ef0440ef221 [SPARK-48146][SQL] Fix aggregate

[jira] [Resolved] (SPARK-48158) XML expressions (all collations)

2024-05-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48158. - Fix Version/s: 4.0.0 Assignee: Uroš Bojanić Resolution: Fixed > XML expressi

<    1   2   3   4   5   6   7   8   9   10   >