;> different users have different levels of tolerance for different kinds of
>>> errors. I’d expect these sorts of configurations to be set up at an
>>> infrastructure level, e.g. to maintain consistent standards throughout a
>>> whole organization.
>>>
>>
uations that could produce invalid results. My intuition is yes, because
>>> different users have different levels of tolerance for different kinds of
>>> errors. I’d expect these sorts of configurations to be set up at an
>>> infrastructure level, e.g. to maintain consisten
ation.
>>
>>
>>
>> *From: *Gengliang Wang
>> *Date: *Thursday, August 1, 2019 at 3:07 AM
>> *To: *Marco Gaido
>> *Cc: *Wenchen Fan , Hyukjin Kwon <
>> gurwls...@gmail.com>, Russell Spitzer , Ryan
>> Blue , Reynold Xin , Matt Cheah <
ang Wang
> *Date: *Thursday, August 1, 2019 at 3:07 AM
> *To: *Marco Gaido
> *Cc: *Wenchen Fan , Hyukjin Kwon ,
> Russell Spitzer , Ryan Blue ,
> Reynold Xin , Matt Cheah ,
> Takeshi Yamamuro , Spark dev list <
> dev@spark.apache.org>
> *Subject: *Re: [Discuss] Follow ANSI
enchen Fan , Hyukjin Kwon ,
Russell Spitzer , Ryan Blue ,
Reynold Xin , Matt Cheah , Takeshi
Yamamuro , Spark dev list
Subject: Re: [Discuss] Follow ANSI SQL on table insertion
Hi all,
Let me explain a little bit on the proposal.
By default, we follow the store assignment rule
;
>>
>> -Matt Cheah
>>
>>
>>
>> *From: *Reynold Xin
>> *Date: *Wednesday, July 31, 2019 at 9:58 AM
>> *To: *Matt Cheah
>> *Cc: *Russell Spitzer , Takeshi Yamamuro <
>> linguin@gmail.com>, Gengliang Wang ,
>> Ryan Blue ,
.com >, Ryan Blue <
> >rb...@netflix.com
> >, Spark dev list < dev@spark.apache.org >, Hyukjin Kwon < gurwls...@gmail.com
> >, Wenchen Fan < cloud0...@gmail.com >
> *Subject:* Re: [Discuss] Follow ANSI SQL on table insertion
>
>
>
>
>
&
Date: Wednesday, July 31, 2019 at 9:58 AM
To: Matt Cheah
Cc: Russell Spitzer , Takeshi Yamamuro
, Gengliang Wang , Ryan
Blue , Spark dev list , Hyukjin Kwon
, Wenchen Fan
Subject: Re: [Discuss] Follow ANSI SQL on table insertion
Matt what do you mean by maximizing 3, while allowing
n Fan < cloud0...@gmail.com >
> *Cc:* Russell Spitzer < russell.spit...@gmail.com >, Takeshi Yamamuro <
> linguin@gmail.com
> >, Gengliang Wang < gengliang.w...@databricks.com >, Ryan Blue <
> >rb...@netflix.com
> >, Spark dev list <
, or perhaps the behavior can be flagged
by the destination writer at write time.
-Matt Cheah
From: Hyukjin Kwon
Date: Monday, July 29, 2019 at 11:33 PM
To: Wenchen Fan
Cc: Russell Spitzer , Takeshi Yamamuro
, Gengliang Wang , Ryan
Blue , Spark dev list
Subject: Re: [Discuss] Follow ANSI SQL
;>
>>>>>>>> In addition, part of the justification for using the ANSI standard
>>>>>>>> is to avoid breaking existing jobs. But the new behavior is only
>>>>>>>> applied in
>>>>>>>> DataSource
>>>> That still leaves the use case that motivated this proposal, which
>>>>>>> is that floating point literals are parsed as decimals and fail simple
>>>>>>> insert statements. We already came up with two alternati
; wrote:
>>>>>>
>>>>>>> I have heard about many complaints about the old table insertion
>>>>>>> behavior. Blindly casting everything will leak the user mistake to a
>>>>>>> late
>>>>>>> stage of
he query
>>>>>> earlier and ask users to fix the mistake.
>>>>>>
>>>>>> In the meanwhile, I agree that the new table insertion behavior we
>>>>>> introduced for Data Source V2 is too strict. It may fa
ata Source V2 is too strict. It may fail valid queries
>>>>> unexpectedly.
>>>>>
>>>>> In general, I support the direction of following the ANSI SQL
>>>>> standard. But I'd like to do it with 2 steps:
>>>>> 1. only add
ig as well and by
>>>> default we still return null.
>>>>
>>>> After doing this, the default behavior will be slightly different from
>>>> the SQL standard (cast can return null), and users can turn on the ANSI
>>>> mode to fully follow
lightly different from
>>> the SQL standard (cast can return null), and users can turn on the ANSI
>>> mode to fully follow the SQL standard. This is much better than before and
>>> should prevent a lot of user mistakes. It's also a reasonable choice to me
>>> to
istakes. It's also a reasonable choice to me
>> to not throw exceptions at runtime by default, as it's usually bad for
>> long-running jobs.
>>
>> Thanks,
>> Wenchen
>>
>> On Thu, Jul 25, 2019 at 11:37 PM Gengliang Wang <
>> gengliang.w...@dat
9 at 11:37 PM Gengliang Wang <
> gengliang.w...@databricks.com> wrote:
>
>> Hi everyone,
>>
>> I would like to discuss the table insertion behavior of Spark. In the
>> current data source V2, only UpCast is allowed for table insertion. I think
>> following ANSI
everyone,
>
> I would like to discuss the table insertion behavior of Spark. In the
> current data source V2, only UpCast is allowed for table insertion. I think
> following ANSI SQL is a better idea.
> For more information, please read the Discuss: Follow ANSI SQL on ta
Hi everyone,
I would like to discuss the table insertion behavior of Spark. In the
current data source V2, only UpCast is allowed for table insertion. I think
following ANSI SQL is a better idea.
For more information, please read the Discuss: Follow ANSI SQL on table
insertion
<ht
21 matches
Mail list logo
