Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Xiao Li
+1 On Sat, Apr 13, 2024 at 17:21 huaxin gao wrote: > +1 > > On Sat, Apr 13, 2024 at 4:36 PM L. C. Hsieh wrote: > >> +1 >> >> On Sat, Apr 13, 2024 at 4:12 PM Hyukjin Kwon >> wrote: >> > >> > +1 >> > >> > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: >> >> >> >> +1. >> >> >> >> This feature

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Denny Lee
+1 (non-binding) On Sat, Apr 13, 2024 at 7:49 PM huaxin gao wrote: > +1 > > On Sat, Apr 13, 2024 at 4:36 PM L. C. Hsieh wrote: > >> +1 >> >> On Sat, Apr 13, 2024 at 4:12 PM Hyukjin Kwon >> wrote: >> > >> > +1 >> > >> > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: >> >> >> >> +1. >> >> >>

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread huaxin gao
+1 On Sat, Apr 13, 2024 at 4:36 PM L. C. Hsieh wrote: > +1 > > On Sat, Apr 13, 2024 at 4:12 PM Hyukjin Kwon wrote: > > > > +1 > > > > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: > >> > >> +1. > >> > >> This feature is very helpful for guarding against correctness issues, > such as null

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Holden Karau
+1 -- even if it's not perfect now is the time to change default values On Sat, Apr 13, 2024 at 4:11 PM Hyukjin Kwon wrote: > +1 > > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: > >> +1. >> >> This feature is very helpful for guarding against correctness issues, >> such as null results due

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread L. C. Hsieh
+1 On Sat, Apr 13, 2024 at 4:12 PM Hyukjin Kwon wrote: > > +1 > > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: >> >> +1. >> >> This feature is very helpful for guarding against correctness issues, such >> as null results due to invalid input or math overflows. It’s been there for >> a

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Hyukjin Kwon
+1 On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: > +1. > > This feature is very helpful for guarding against correctness issues, such > as null results due to invalid input or math overflows. It’s been there for > a while now and it’s a good time to enable it by default as Spark enters > the

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Chao Sun
+1. This feature is very helpful for guarding against correctness issues, such as null results due to invalid input or math overflows. It’s been there for a while now and it’s a good time to enable it by default as Spark enters the next major release. On Sat, Apr 13, 2024 at 3:27 PM Dongjoon

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Gengliang Wang
+1 On Sat, Apr 13, 2024 at 3:26 PM Dongjoon Hyun wrote: > I'll start from my +1. > > Dongjoon. > > On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > > Please vote on SPARK-4 to use ANSI SQL mode by default. > > The technical scope is defined in the following PR which is > > one line of code

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Dongjoon Hyun
I'll start from my +1. Dongjoon. On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > Please vote on SPARK-4 to use ANSI SQL mode by default. > The technical scope is defined in the following PR which is > one line of code change and one line of migration guide. > > - DISCUSSION: >

[VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Dongjoon Hyun
Please vote on SPARK-4 to use ANSI SQL mode by default. The technical scope is defined in the following PR which is one line of code change and one line of migration guide. - DISCUSSION: https://lists.apache.org/thread/ztlwoz1v1sn81ssks12tb19x37zozxlz - JIRA:

Re: [DISCUSS] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Dongjoon Hyun
Thank you for your opinions, Gangling, Liang-Chi, Wenchen, Huaxin, Serge, Nicholas. To Nicholas, Apache Spark community already decided not to pursuit PostgreSQL dialect. > I’m flagging this since Spark’s behavior differs in these cases from > Postgres, > as described in the ticket. Please

Support Avro rolling version upgrades using schema manager

2024-04-13 Thread Nimrod Ofek
Hi, Currently, Avro records are supported in Spark - but with the limitation that we must specify the input and output schema versions. For writing out an avro record that is fine - but for reading avro records, that is usually a problem since there are upgrades and changes - and the current