[jira] [Assigned] (SPARK-47764) Cleanup shuffle dependencies for Spark Connect SQL executions

2024-04-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47764: --- Assignee: Bo Zhang > Cleanup shuffle dependencies for Spark Connect SQL executi

[jira] [Resolved] (SPARK-47764) Cleanup shuffle dependencies for Spark Connect SQL executions

2024-04-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47764. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45930 [https

(spark) branch master updated: [SPARK-47692][SQL] Fix default StringType meaning in implicit casting

2024-04-24 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 461ffa1a681b [SPARK-47692][SQL] Fix default

[jira] [Resolved] (SPARK-47418) Optimize string predicate expressions for UTF8_BINARY_LCASE collation

2024-04-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47418. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46181 [https

(spark) branch master updated: [SPARK-47418][SQL] Add hand-crafted implementations for lowercase unicode-aware contains, startsWith and endsWith and optimize UTF8_BINARY_LCASE

2024-04-24 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 890f78d03020 [SPARK-47418][SQL] Add hand

[jira] [Assigned] (SPARK-47873) Write collated strings to hive as regular strings

2024-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47873: --- Assignee: Stefan Kandic > Write collated strings to hive as regular stri

[jira] [Resolved] (SPARK-47873) Write collated strings to hive as regular strings

2024-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47873. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46083 [https

(spark) branch master updated: [SPARK-47873][SQL] Write collated strings to Hive metastore using the regular string type

2024-04-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9d9f292ac941 [SPARK-47873][SQL] Write collated

[jira] [Created] (SPARK-47956) sanity check for unresolved LCA reference

2024-04-23 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-47956: --- Summary: sanity check for unresolved LCA reference Key: SPARK-47956 URL: https://issues.apache.org/jira/browse/SPARK-47956 Project: Spark Issue Type

(spark) branch master updated: [SPARK-47352][SQL] Fix Upper, Lower, InitCap collation awareness

2024-04-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b9f2270f5b0b [SPARK-47352][SQL] Fix Upper

[jira] [Resolved] (SPARK-47352) Fix Upper, Lower, InitCap collation awareness

2024-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47352. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46104 [https

[jira] [Resolved] (SPARK-47412) StringLPad, StringRPad (all collations)

2024-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47412. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46041 [https

[jira] [Assigned] (SPARK-47412) StringLPad, StringRPad (all collations)

2024-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47412: --- Assignee: Gideon P > StringLPad, StringRPad (all collati

(spark) branch master updated: [SPARK-47412][SQL] Add Collation Support for LPad/RPad

2024-04-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 885e98ecbe64 [SPARK-47412][SQL] Add Collation

(spark) branch master updated: [SPARK-47633][SQL] Include right-side plan output in `LateralJoin#allAttributes` for more consistent canonicalization

2024-04-23 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new eba6364dbb36 [SPARK-47633][SQL] Include right

[jira] [Assigned] (SPARK-47411) StringInstr, FindInSet (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47411: --- Assignee: Milan Dankovic > StringInstr, FindInSet (all collati

[jira] [Resolved] (SPARK-47411) StringInstr, FindInSet (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47411. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45643 [https

(spark) branch master updated: [SPARK-47411][SQL] Support StringInstr & FindInSet functions to work with collated strings

2024-04-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 256fc51508e4 [SPARK-47411][SQL] Support

[jira] [Resolved] (SPARK-47900) Fix check for implicit collation

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47900. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46116 [https

(spark) branch master updated (2fb31dea1c53 -> b20356ef55a2)

2024-04-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 2fb31dea1c53 [SPARK-47930][BUILD] Upgrade RoaringBitmap to 1.0.6 add b20356ef55a2 [SPARK-47900] Fix check

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47413: --- Assignee: Gideon P > Substring, Right, Left (all collati

[jira] [Resolved] (SPARK-47413) Substring, Right, Left (all collations)

2024-04-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47413. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46040 [https

(spark) branch master updated (2d0b56c3eac6 -> e1432ef6405a)

2024-04-22 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 2d0b56c3eac6 [SPARK-47932][SQL][TESTS] Avoid using legacy commons-lang add e1432ef6405a [SPARK-47413][SQL] - add

[jira] [Resolved] (SPARK-47902) Compute Current Time* expressions should be foldable

2024-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47902. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46120 [https

[jira] [Assigned] (SPARK-47902) Compute Current Time* expressions should be foldable

2024-04-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47902: --- Assignee: Aleksandar Tomic > Compute Current Time* expressions should be folda

(spark) branch master updated (9f34b8eca2f3 -> 458f70bd5213)

2024-04-21 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 9f34b8eca2f3 [SPARK-47845][SQL][PYTHON][CONNECT] Support Column type in split function for scala and python add

(spark) branch master updated (bb5ded8f1bb0 -> 76fb196a8ab7)

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from bb5ded8f1bb0 [SPARK-47371][SQL][FOLLOWUP] XML: Stop ignoring CDATA within row tags add 76fb196a8ab7 [SPARK

[jira] [Resolved] (SPARK-46935) Consolidate error documentation

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46935. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44971 [https

[jira] [Assigned] (SPARK-46935) Consolidate error documentation

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46935: --- Assignee: Nicholas Chammas > Consolidate error documentat

[jira] [Updated] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-47463: Fix Version/s: 3.5.2 > An error occurred while pushing down the filter of if express

(spark) branch branch-3.5 updated: [SPARK-47463][SQL][3.5] Use V2Predicate to wrap expression with return type of boolean

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new e439e29a7bf1 [SPARK-47463][SQL][3.5

[jira] [Resolved] (SPARK-47895) group by all should be idempotent

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47895. - Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution: Fixed

[jira] [Assigned] (SPARK-47895) group by all should be idempotent

2024-04-18 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47895: --- Assignee: Wenchen Fan > group by all should be idempot

(spark) branch branch-3.4 updated: [SPARK-47895][SQL] group by all should be idempotent

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.4 by this push: new 11c1c520fe57 [SPARK-47895][SQL] group

(spark) branch branch-3.5 updated: [SPARK-47895][SQL] group by all should be idempotent

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.5 by this push: new 7aea21eae377 [SPARK-47895][SQL] group

(spark) branch master updated: [SPARK-47895][SQL] group by all should be idempotent

2024-04-18 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b5bb75ca240a [SPARK-47895][SQL] group by all

[jira] [Updated] (SPARK-47895) group by all should be idempotent

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-47895: Summary: group by all should be idempotent (was: group by ordinal should be idempotent) > gr

[jira] [Created] (SPARK-47895) group by ordinal should be idempotent

2024-04-17 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-47895: --- Summary: group by ordinal should be idempotent Key: SPARK-47895 URL: https://issues.apache.org/jira/browse/SPARK-47895 Project: Spark Issue Type: Bug

(spark) branch master updated (c4b0c260bb13 -> 60e75e6e0275)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from c4b0c260bb13 [SPARK-47839][SQL] Fix aggregate bug in RewriteWithExpression add 60e75e6e0275 [SPARK-47810][SQL

[jira] [Assigned] (SPARK-47839) Fix Aggregate bug in RewriteWithExpression

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47839: --- Assignee: Kelvin Jiang > Fix Aggregate bug in RewriteWithExpress

[jira] [Resolved] (SPARK-47839) Fix Aggregate bug in RewriteWithExpression

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47839. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46034 [https

(spark) branch master updated: [SPARK-47839][SQL] Fix aggregate bug in RewriteWithExpression

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c4b0c260bb13 [SPARK-47839][SQL] Fix aggregate

[jira] [Resolved] (SPARK-47846) Add support for Variant schema in from_json

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47846. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46046 [https

[jira] [Assigned] (SPARK-47846) Add support for Variant schema in from_json

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47846: --- Assignee: Harsh Motwani > Add support for Variant schema in from_j

(spark) branch master updated (37afc3fff8c6 -> 61a7901a2c83)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 37afc3fff8c6 [SPARK-47803][FOLLOWUP] Fix cast binary/decimal to variant add 61a7901a2c83 [SPARK-47846][SQL] Add

(spark) branch master updated: [SPARK-47803][FOLLOWUP] Fix cast binary/decimal to variant

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 37afc3fff8c6 [SPARK-47803][FOLLOWUP] Fix cast

(spark) branch master updated: [SPARK-47867][FOLLOWUP] Fix variant parsing in JacksonParser

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 21d8bbdc59d6 [SPARK-47867][FOLLOWUP] Fix

[jira] [Resolved] (SPARK-47360) Overlay, FormatString, Length, BitLength, OctetLength, SoundEx, Luhncheck (all collations)

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47360. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46003 [https

(spark) branch master updated: [SPARK-47360][SQL] Collation support: Overlay, FormatString, Length, BitLength, OctetLength, SoundEx, Luhncheck

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f99b1b1dd14d [SPARK-47360][SQL] Collation

[jira] [Resolved] (SPARK-47416) Add benchmark for stringpredicate expressions

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47416. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46078 [https

(spark) branch master updated: [SPARK-47416][SQL] Add new functions to CollationBenchmark

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 1b65b2aebf4e [SPARK-47416][SQL] Add new

(spark) branch master updated: [SPARK-47765][SQL][FOLLOWUP] Disable SET COLLATION when collations are disabled

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e2e6710fc8ea [SPARK-47765][SQL][FOLLOWUP

[jira] [Resolved] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47863. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46097 [https

[jira] [Assigned] (SPARK-47863) endsWith and startsWith don't work correctly for some collations

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47863: --- Assignee: Vladimir Golubev > endsWith and startsWith don't work correctly for s

(spark) branch master updated (e327c1220abf -> 1b53f122d56c)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from e327c1220abf [SPARK-47884][INFRA] Switch ANSI SQL CI job to NON-ANSI SQL CI job add 1b53f122d56c [SPARK-47863

Re: [DISCUSS] Spark 4.0.0 release

2024-04-17 Thread Wenchen Fan
add more context on the plan for transformWithState) > > > > On Sat, Apr 13, 2024 at 3:15 AM Wenchen Fan wrote: > > Hi all, > > > > It's close to the previously proposed 4.0.0 release date (June 2024), > and I think it's time to prepare for it and discuss

[jira] [Assigned] (SPARK-47822) Prohibit Hash expressions from hashing Variant type

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47822: --- Assignee: Harsh Motwani > Prohibit Hash expressions from hashing Variant t

[jira] [Resolved] (SPARK-47822) Prohibit Hash expressions from hashing Variant type

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47822. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46017 [https

(spark) branch master updated: [SPARK-47822][SQL] Prohibit Hash Expressions from hashing the Variant Data Type

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4e754f778fdc [SPARK-47822][SQL] Prohibit Hash

[jira] [Resolved] (SPARK-47821) Add is_variant_null expression

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47821. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46011 [https

(spark) branch master updated (4913c344756d -> 76882d0c13db)

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from 4913c344756d [SPARK-47867][SQL] Support variant in JSON scan add 76882d0c13db [SPARK-47821][SQL] Implement

[jira] [Resolved] (SPARK-47867) Support Variant in JSON scan.

2024-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47867. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46071 [https

(spark) branch master updated: [SPARK-47867][SQL] Support variant in JSON scan

2024-04-17 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4913c344756d [SPARK-47867][SQL] Support variant

[jira] [Assigned] (SPARK-47417) Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Encode, ToBinary, FormatNumber, Sentences (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47417: --- Assignee: Nikola Mandic > Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Enc

[jira] [Resolved] (SPARK-47417) Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Encode, ToBinary, FormatNumber, Sentences (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47417. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45933 [https

(spark) branch master updated: [SPARK-47417][SQL] Collation support: Ascii, Chr, Base64, UnBase64, Decode, StringDecode, Encode, ToBinary, FormatNumber, Sentences

2024-04-16 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ee2673f2e948 [SPARK-47417][SQL] Collation

[jira] [Resolved] (SPARK-47356) Add support for ConcatWs & Elt (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47356. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46061 [https

[jira] [Assigned] (SPARK-47356) Add support for ConcatWs & Elt (all collations)

2024-04-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47356: --- Assignee: Mihailo Milosevic > Add support for ConcatWs & Elt (all col

(spark) branch master updated: [SPARK-47356][SQL] Add support for ConcatWs & Elt (all collations)

2024-04-16 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4dad2170b05c [SPARK-47356][SQL] Add support

Re: [VOTE] Release Spark 3.4.3 (RC2)

2024-04-16 Thread Wenchen Fan
+1 On Mon, Apr 15, 2024 at 12:31 PM Dongjoon Hyun wrote: > I'll start with my +1. > > - Checked checksum and signature > - Checked Scala/Java/R/Python/SQL Document's Spark version > - Checked published Maven artifacts > - All CIs passed. > > Thanks, > Dongjoon. > > On 2024/04/15 04:22:26

[jira] [Resolved] (SPARK-47420) Fix CollationSupport test output

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47420. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46058 [https

(spark) branch master updated: [SPARK-47420][SQL] Fix test output

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6762d1f8beaf [SPARK-47420][SQL] Fix test output

[jira] [Resolved] (SPARK-47769) Add schema_of_variant_agg expression.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47769. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45934 [https

[jira] [Assigned] (SPARK-47769) Add schema_of_variant_agg expression.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47769: --- Assignee: Chenhao Li > Add schema_of_variant_agg express

(spark) branch master updated: [SPARK-47769][SQL] Add schema_of_variant_agg expression

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 348b1bcff75c [SPARK-47769][SQL] Add

[jira] [Resolved] (SPARK-47463) An error occurred while pushing down the filter of if expression for iceberg datasource.

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47463. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45589 [https

(spark) branch master updated: [SPARK-47463][SQL] Use V2Predicate to wrap expression with return type of boolean

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fa3ef03a0734 [SPARK-47463][SQL] Use V2Predicate

(spark) branch master updated (be080703688f -> c5b8e60e0d59)

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from be080703688f [SPARK-47673][SS] Implementing TTL for ListState add c5b8e60e0d59 [SPARK-46810][DOCS] Align error

[jira] [Resolved] (SPARK-46810) Clarify error class terminology

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46810. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44902 [https

[jira] [Assigned] (SPARK-46810) Clarify error class terminology

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46810: --- Assignee: Nicholas Chammas > Clarify error class terminol

[jira] [Resolved] (SPARK-47357) Add support for Upper, Lower, InitCap (all collations)

2024-04-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47357. - Fix Version/s: 4.0.0 Assignee: Mihailo Milosevic Resolution: Fixed >

(spark) branch master updated (b8354bbe53c0 -> f3a6ca9e2c47)

2024-04-15 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from b8354bbe53c0 [SPARK-47851][CONNECT][DOCS] Document pyspark-connect package add f3a6ca9e2c47 [SPARK-47357][SQL

[jira] [Resolved] (SPARK-47803) Support cast to variant.

2024-04-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47803. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45989 [https

(spark) branch master updated: [SPARK-47803][SQL] Support cast to variant

2024-04-14 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new dd4c7fc6ae91 [SPARK-47803][SQL] Support cast

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-14 Thread Wenchen Fan
+1 On Sun, Apr 14, 2024 at 6:28 AM Dongjoon Hyun wrote: > I'll start from my +1. > > Dongjoon. > > On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > > Please vote on SPARK-4 to use ANSI SQL mode by default. > > The technical scope is defined in the following PR which is > > one line of code

Re: [DISCUSS] SPARK-44444: Use ANSI SQL mode by default

2024-04-12 Thread Wenchen Fan
+1, the existing "NULL on error" behavior is terrible for data quality. I have one concern about error reporting with DataFrame APIs. Query execution is lazy and where the error happens can be far away from where the dataframe/column was created. We are improving it (PR

[DISCUSS] Spark 4.0.0 release

2024-04-12 Thread Wenchen Fan
collation support - Spark k8s operator versioning Please help to add more items to this list that are missed here. I would like to volunteer as the release manager for Apache Spark 4.0.0 if there is no objection. Thank you all for the great work that fills Spark 4.0! Wenchen Fan

[jira] [Resolved] (SPARK-47765) Add SET COLLATION to parser rules

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47765. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45946 [https

[jira] [Assigned] (SPARK-47765) Add SET COLLATION to parser rules

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47765: --- Assignee: Mihailo Milosevic > Add SET COLLATION to parser ru

(spark) branch master updated: [SPARK-47765][SQL] Add SET COLLATION to parser rules

2024-04-12 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new de00ac8a05ae [SPARK-47765][SQL] Add SET

[jira] [Resolved] (SPARK-47800) Add method for converting v2 identifier to table identifier

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47800. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45985 [https

[jira] [Assigned] (SPARK-47800) Add method for converting v2 identifier to table identifier

2024-04-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47800: --- Assignee: Uros Stankovic > Add method for converting v2 identifier to table identif

(spark) branch master updated: [SPARK-47800][SQL] Create new method for identifier to tableIdentifier conversion

2024-04-12 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 27987536be38 [SPARK-47800][SQL] Create new

[jira] [Assigned] (SPARK-47410) refactor UTF8String and CollationFactory

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47410: --- Assignee: Uroš Bojanić > refactor UTF8String and CollationFact

[jira] [Resolved] (SPARK-47410) refactor UTF8String and CollationFactory

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47410. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45978 [https

(spark) branch master updated: [SPARK-47410][SQL] Refactor UTF8String and CollationFactory

2024-04-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 31036274fc1c [SPARK-47410][SQL] Refactor

[jira] [Resolved] (SPARK-47617) Add TPC-DS testing infrastructure for collations

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47617. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45739 [https

[jira] [Assigned] (SPARK-47617) Add TPC-DS testing infrastructure for collations

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47617: --- Assignee: Nikola Mandic > Add TPC-DS testing infrastructure for collati

(spark) branch master updated: [SPARK-47617][SQL] Add TPC-DS testing infrastructure for collations

2024-04-11 Thread wenchen
This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6e371e1df50e [SPARK-47617][SQL] Add TPC-DS

[jira] [Assigned] (SPARK-47736) Add support for AbstractArrayType

2024-04-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47736: --- Assignee: Mihailo Milosevic > Add support for AbstractArrayT

<    1   2   3   4   5   6   7   8   9   10   >