[jira] [Assigned] (SPARK-47411) StringInstr, FindInSet (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47411: --- Assignee: Milan Dankovic > StringInstr, FindInSet (all collati

[jira] [Resolved] (SPARK-47411) StringInstr, FindInSet (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47411. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45643 [https

(spark) branch master updated: [SPARK-47411][SQL] Support StringInstr & FindInSet functions to work with collated strings

2024-04-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 256fc51508e4 [SPARK-47411][SQL] Support

[jira] [Resolved] (SPARK-47900) Fix check for implicit collation

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47900. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46116 [https

(spark) branch master updated (2fb31dea1c53 -> b20356ef55a2)

2024-04-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 2fb31dea1c53 [SPARK-47930][BUILD] Upgrade RoaringBitmap to 1.0.6 add b20356ef55a2 [SPARK-47900] Fix check

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47413: --- Assignee: Gideon P > Substring, Right, Left (all collati

[jira] [Resolved] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47413. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46040 [https

(spark) branch master updated (2d0b56c3eac6 -> e1432ef6405a)

2024-04-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 2d0b56c3eac6 [SPARK-47932][SQL][TESTS] Avoid using legacy commons-lang add e1432ef6405a [SPARK-47413][SQL] - add

[jira] [Resolved] (SPARK-47902) Compute Current Time* expressions should be foldable

2024-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47902. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46120 [https

[jira] [Assigned] (SPARK-47902) Compute Current Time* expressions should be foldable

2024-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47902: --- Assignee: Aleksandar Tomic > Compute Current Time* expressions should be folda

(spark) branch master updated (9f34b8eca2f3 -> 458f70bd5213)

2024-04-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 9f34b8eca2f3 [SPARK-47845][SQL][PYTHON][CONNECT] Support Column type in split function for scala and python add

(spark) branch master updated (bb5ded8f1bb0 -> 76fb196a8ab7)

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from bb5ded8f1bb0 [SPARK-47371][SQL][FOLLOWUP] XML: Stop ignoring CDATA within row tags add 76fb196a8ab7 [SPARK

[jira] [Resolved] (SPARK-46935) Consolidate error documentation

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46935. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44971 [https

[jira] [Assigned] (SPARK-46935) Consolidate error documentation

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46935: --- Assignee: Nicholas Chammas > Consolidate error documentat

[jira] [Updated] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-47463: Fix Version/s: 3.5.2 > An error occurred while pushing down the filter of if express

(spark) branch branch-3.5 updated: [SPARK-47463][SQL][3.5] Use V2Predicate to wrap expression with return type of boolean

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new e439e29a7bf1 [SPARK-47463][SQL][3.5

[jira] [Resolved] (SPARK-47895) group by all should be idempotent

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47895. - Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution: Fixed

[jira] [Assigned] (SPARK-47895) group by all should be idempotent

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47895: --- Assignee: Wenchen Fan > group by all should be idempot

(spark) branch branch-3.4 updated: [SPARK-47895][SQL] group by all should be idempotent

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new 11c1c520fe57 [SPARK-47895][SQL] group

(spark) branch branch-3.5 updated: [SPARK-47895][SQL] group by all should be idempotent

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 7aea21eae377 [SPARK-47895][SQL] group

(spark) branch master updated: [SPARK-47895][SQL] group by all should be idempotent

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b5bb75ca240a [SPARK-47895][SQL] group by all

[jira] [Updated] (SPARK-47895) group by all should be idempotent

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-47895: Summary: group by all should be idempotent (was: group by ordinal should be idempotent) > gr

[jira] [Created] (SPARK-47895) group by ordinal should be idempotent

2024-04-17 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-47895: --- Summary: group by ordinal should be idempotent Key: SPARK-47895 URL: https://issues.apache.org/jira/browse/SPARK-47895 Project: Spark Issue Type: Bug

(spark) branch master updated (c4b0c260bb13 -> 60e75e6e0275)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from c4b0c260bb13 [SPARK-47839][SQL] Fix aggregate bug in RewriteWithExpression add 60e75e6e0275 [SPARK-47810][SQL

[jira] [Assigned] (SPARK-47839) Fix Aggregate bug in RewriteWithExpression

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47839: --- Assignee: Kelvin Jiang > Fix Aggregate bug in RewriteWithExpress

[jira] [Resolved] (SPARK-47839) Fix Aggregate bug in RewriteWithExpression

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47839. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46034 [https

(spark) branch master updated: [SPARK-47839][SQL] Fix aggregate bug in RewriteWithExpression

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c4b0c260bb13 [SPARK-47839][SQL] Fix aggregate

[jira] [Resolved] (SPARK-47846) Add support for Variant schema in from_json

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47846. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46046 [https

[jira] [Assigned] (SPARK-47846) Add support for Variant schema in from_json

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47846: --- Assignee: Harsh Motwani > Add support for Variant schema in from_j

(spark) branch master updated (37afc3fff8c6 -> 61a7901a2c83)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 37afc3fff8c6 [SPARK-47803][FOLLOWUP] Fix cast binary/decimal to variant add 61a7901a2c83 [SPARK-47846][SQL] Add

(spark) branch master updated: [SPARK-47803][FOLLOWUP] Fix cast binary/decimal to variant

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 37afc3fff8c6 [SPARK-47803][FOLLOWUP] Fix cast

(spark) branch master updated: [SPARK-47867][FOLLOWUP] Fix variant parsing in JacksonParser

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 21d8bbdc59d6 [SPARK-47867][FOLLOWUP] Fix

[jira] [Resolved] (SPARK-47360) Overlay, FormatString, Length, BitLength, OctetLength, SoundEx, Luhncheck (all collations)

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47360. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46003 [https

(spark) branch master updated: [SPARK-47360][SQL] Collation support: Overlay, FormatString, Length, BitLength, OctetLength, SoundEx, Luhncheck

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f99b1b1dd14d [SPARK-47360][SQL] Collation

[jira] [Resolved] (SPARK-47416) Add benchmark for stringpredicate expressions

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47416. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46078 [https

(spark) branch master updated: [SPARK-47416][SQL] Add new functions to CollationBenchmark

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 1b65b2aebf4e [SPARK-47416][SQL] Add new

(spark) branch master updated: [SPARK-47765][SQL][FOLLOWUP] Disable SET COLLATION when collations are disabled

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e2e6710fc8ea [SPARK-47765][SQL][FOLLOWUP

[jira] [Resolved] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47863. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46097 [https

[jira] [Assigned] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47863: --- Assignee: Vladimir Golubev > endsWith and startsWith don't work correctly for s

(spark) branch master updated (e327c1220abf -> 1b53f122d56c)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from e327c1220abf [SPARK-47884][INFRA] Switch ANSI SQL CI job to NON-ANSI SQL CI job add 1b53f122d56c [SPARK-47863

Re: [DISCUSS] Spark 4.0.0 release

2024-04-17 Thread Wenchen Fan
add more context on the plan for transformWithState) > > > > On Sat, Apr 13, 2024 at 3:15 AM Wenchen Fan wrote: > > Hi all, > > > > It's close to the previously proposed 4.0.0 release date (June 2024), > and I think it's time to prepare for it and discuss

[jira] [Assigned] (SPARK-47822) Prohibit Hash expressions from hashing Variant type

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47822: --- Assignee: Harsh Motwani > Prohibit Hash expressions from hashing Variant t

[jira] [Resolved] (SPARK-47822) Prohibit Hash expressions from hashing Variant type

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47822. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46017 [https

(spark) branch master updated: [SPARK-47822][SQL] Prohibit Hash Expressions from hashing the Variant Data Type

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4e754f778fdc [SPARK-47822][SQL] Prohibit Hash

[jira] [Resolved] (SPARK-47821) Add is_variant_null expression

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47821. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46011 [https

(spark) branch master updated (4913c344756d -> 76882d0c13db)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 4913c344756d [SPARK-47867][SQL] Support variant in JSON scan add 76882d0c13db [SPARK-47821][SQL] Implement

[jira] [Resolved] (SPARK-47867) Support Variant in JSON scan.

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47867. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46071 [https

(spark) branch master updated: [SPARK-47867][SQL] Support variant in JSON scan

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4913c344756d [SPARK-47867][SQL] Support variant

[jira] [Assigned] (SPARK-47417) Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Encode, ToBinary, FormatNumber, Sentences (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47417: --- Assignee: Nikola Mandic > Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Enc

[jira] [Resolved] (SPARK-47417) Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Encode, ToBinary, FormatNumber, Sentences (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47417. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45933 [https

(spark) branch master updated: [SPARK-47417][SQL] Collation support: Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Encode, ToBinary, FormatNumber, Sentences

2024-04-16 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ee2673f2e948 [SPARK-47417][SQL] Collation

[jira] [Resolved] (SPARK-47356) Add support for ConcatWs & Elt (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47356. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46061 [https

[jira] [Assigned] (SPARK-47356) Add support for ConcatWs & Elt (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47356: --- Assignee: Mihailo Milosevic > Add support for ConcatWs & Elt (all col

(spark) branch master updated: [SPARK-47356][SQL] Add support for ConcatWs & Elt (all collations)

2024-04-16 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4dad2170b05c [SPARK-47356][SQL] Add support

Re: [VOTE] Release Spark 3.4.3 (RC2)

2024-04-16 Thread Wenchen Fan
+1 On Mon, Apr 15, 2024 at 12:31 PM Dongjoon Hyun wrote: > I'll start with my +1. > > - Checked checksum and signature > - Checked Scala/Java/R/Python/SQL Document's Spark version > - Checked published Maven artifacts > - All CIs passed. > > Thanks, > Dongjoon. > > On 2024/04/15 04:22:26

[jira] [Resolved] (SPARK-47420) Fix CollationSupport test output

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47420. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46058 [https

(spark) branch master updated: [SPARK-47420][SQL] Fix test output

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6762d1f8beaf [SPARK-47420][SQL] Fix test output

[jira] [Resolved] (SPARK-47769) Add schema_of_variant_agg expression.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47769. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45934 [https

[jira] [Assigned] (SPARK-47769) Add schema_of_variant_agg expression.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47769: --- Assignee: Chenhao Li > Add schema_of_variant_agg express

(spark) branch master updated: [SPARK-47769][SQL] Add schema_of_variant_agg expression

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 348b1bcff75c [SPARK-47769][SQL] Add

[jira] [Resolved] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47463. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45589 [https

(spark) branch master updated: [SPARK-47463][SQL] Use V2Predicate to wrap expression with return type of boolean

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fa3ef03a0734 [SPARK-47463][SQL] Use V2Predicate

(spark) branch master updated (be080703688f -> c5b8e60e0d59)

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from be080703688f [SPARK-47673][SS] Implementing TTL for ListState add c5b8e60e0d59 [SPARK-46810][DOCS] Align error

[jira] [Resolved] (SPARK-46810) Clarify error class terminology

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46810. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44902 [https

[jira] [Assigned] (SPARK-46810) Clarify error class terminology

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46810: --- Assignee: Nicholas Chammas > Clarify error class terminol

[jira] [Resolved] (SPARK-47357) Add support for Upper, Lower, InitCap (all collations)

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47357. - Fix Version/s: 4.0.0 Assignee: Mihailo Milosevic Resolution: Fixed >

(spark) branch master updated (b8354bbe53c0 -> f3a6ca9e2c47)

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from b8354bbe53c0 [SPARK-47851][CONNECT][DOCS] Document pyspark-connect package add f3a6ca9e2c47 [SPARK-47357][SQL

[jira] [Resolved] (SPARK-47803) Support cast to variant.

2024-04-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47803. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45989 [https

(spark) branch master updated: [SPARK-47803][SQL] Support cast to variant

2024-04-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new dd4c7fc6ae91 [SPARK-47803][SQL] Support cast

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-14 Thread Wenchen Fan
+1 On Sun, Apr 14, 2024 at 6:28 AM Dongjoon Hyun wrote: > I'll start from my +1. > > Dongjoon. > > On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > > Please vote on SPARK-4 to use ANSI SQL mode by default. > > The technical scope is defined in the following PR which is > > one line of code

Re: [DISCUSS] SPARK-44444: Use ANSI SQL mode by default

2024-04-12 Thread Wenchen Fan
+1, the existing "NULL on error" behavior is terrible for data quality. I have one concern about error reporting with DataFrame APIs. Query execution is lazy and where the error happens can be far away from where the dataframe/column was created. We are improving it (PR

[DISCUSS] Spark 4.0.0 release

2024-04-12 Thread Wenchen Fan
collation support - Spark k8s operator versioning Please help to add more items to this list that are missed here. I would like to volunteer as the release manager for Apache Spark 4.0.0 if there is no objection. Thank you all for the great work that fills Spark 4.0! Wenchen Fan

[jira] [Resolved] (SPARK-47765) Add SET COLLATION to parser rules

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47765. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45946 [https

[jira] [Assigned] (SPARK-47765) Add SET COLLATION to parser rules

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47765: --- Assignee: Mihailo Milosevic > Add SET COLLATION to parser ru

(spark) branch master updated: [SPARK-47765][SQL] Add SET COLLATION to parser rules

2024-04-12 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new de00ac8a05ae [SPARK-47765][SQL] Add SET

[jira] [Resolved] (SPARK-47800) Add method for converting v2 identifier to table identifier

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47800. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45985 [https

[jira] [Assigned] (SPARK-47800) Add method for converting v2 identifier to table identifier

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47800: --- Assignee: Uros Stankovic > Add method for converting v2 identifier to table identif

(spark) branch master updated: [SPARK-47800][SQL] Create new method for identifier to tableIdentifier conversion

2024-04-12 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 27987536be38 [SPARK-47800][SQL] Create new

[jira] [Assigned] (SPARK-47410) refactor UTF8String and CollationFactory

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47410: --- Assignee: Uroš Bojanić > refactor UTF8String and CollationFact

[jira] [Resolved] (SPARK-47410) refactor UTF8String and CollationFactory

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47410. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45978 [https

(spark) branch master updated: [SPARK-47410][SQL] Refactor UTF8String and CollationFactory

2024-04-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 31036274fc1c [SPARK-47410][SQL] Refactor

[jira] [Resolved] (SPARK-47617) Add TPC-DS testing infrastructure for collations

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47617. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45739 [https

[jira] [Assigned] (SPARK-47617) Add TPC-DS testing infrastructure for collations

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47617: --- Assignee: Nikola Mandic > Add TPC-DS testing infrastructure for collati

(spark) branch master updated: [SPARK-47617][SQL] Add TPC-DS testing infrastructure for collations

2024-04-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6e371e1df50e [SPARK-47617][SQL] Add TPC-DS

[jira] [Assigned] (SPARK-47736) Add support for AbstractArrayType

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47736: --- Assignee: Mihailo Milosevic > Add support for AbstractArrayT

[jira] [Resolved] (SPARK-47736) Add support for AbstractArrayType

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47736. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45891 [https

(spark) branch master updated: [SPARK-47736][SQL] Add support for AbstractArrayType

2024-04-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a8b919f924db [SPARK-47736][SQL] Add support

[jira] [Created] (SPARK-47809) checkExceptionInExpression should check error for each codegen mode

2024-04-10 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-47809: --- Summary: checkExceptionInExpression should check error for each codegen mode Key: SPARK-47809 URL: https://issues.apache.org/jira/browse/SPARK-47809 Project: Spark

[jira] [Resolved] (SPARK-47001) Pushdown Verification in Optimizer.scala should support changed data types

2024-04-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47001. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45146 [https

(spark) branch master updated: [SPARK-47001][SQL] Pushdown verification in optimizer

2024-04-10 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d0605bf3bf7b [SPARK-47001][SQL] Pushdown

[jira] [Resolved] (SPARK-47274) Provide more useful context for PySpark DataFrame API errors

2024-04-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47274. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45377 [https

[jira] [Assigned] (SPARK-47274) Provide more useful context for PySpark DataFrame API errors

2024-04-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47274: --- Assignee: Haejoon Lee > Provide more useful context for PySpark DataFrame API err

(spark) branch master updated: [SPARK-47274][PYTHON][SQL] Provide more useful context for PySpark DataFrame API errors

2024-04-10 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 86ae0d2bc198 [SPARK-47274][PYTHON][SQL] Provide

[jira] [Resolved] (SPARK-47802) Revert mapping ( star ) to named_struct ( star )

2024-04-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47802. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45987 [https

[jira] [Assigned] (SPARK-47802) Revert mapping ( star ) to named_struct ( star )

2024-04-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47802: --- Assignee: Serge Rielau > Revert mapping ( star ) to named_struct ( s

(spark) branch master updated (8da9a70f701a -> b637bb92a2f8)

2024-04-10 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 8da9a70f701a [SPARK-47725][INFRA][FOLLOW-UP] Do not run scheduled job in forked repository add b637bb92a2f8

(spark) branch master updated: [SPARK-47693][SQL] Add optimization for lowercase comparison of UTF8String used in UTF8_BINARY_LCASE collation

2024-04-10 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 627f6082edca [SPARK-47693][SQL] Add

[jira] [Assigned] (SPARK-47775) Support remaining scalar types in the variant spec.

2024-04-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47775: --- Assignee: Chenhao Li > Support remaining scalar types in the variant s

[jira] [Resolved] (SPARK-47775) Support remaining scalar types in the variant spec.

2024-04-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47775. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45945 [https

(spark) branch master updated: [SPARK-47775][SQL] Support remaining scalar types in the variant spec

2024-04-10 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d817c9a60f51 [SPARK-47775][SQL] Support

<    1   2   3   4   5   6   7   8   9   10   >