Ah yes, on rereading the original email I see that the sync discussion was
different. Thanks for the clarification! I’ll file a JIRA about PERMISSIVE.
2019년 9월 9일 (월) 오전 6:05, Wenchen Fan 님이 작성:
> Hi Nicholas,
>
> You are talking about a different thing. The PERMISSIVE mode is the
> failure mode
Hi Nicholas,
You are talking about a different thing. The PERMISSIVE mode is the failure
mode for reading text-based data source (json, csv, etc.). It's not the
general failure mode for Spark table insertion.
I agree with you that the PERMISSIVE mode is hard to use. Feel free to open
a JIRA
A quick question about failure modes, as a casual observer of the DSv2
effort:
I was considering filing a JIRA ticket about enhancing the DataFrameReader
to include the failure *reason* in addition to the corrupt record when the
mode is PERMISSIVE. So if you are loading a CSV, for example, and a
Here are my notes from the latest sync. Feel free to reply with
clarifications if I’ve missed anything.
*Attendees*:
Ryan Blue
John Zhuge
Russell Spitzer
Matt Cheah
Gengliang Wang
Priyanka Gomatam
Holden Karau
*Topics*:
- DataFrameWriterV2 insert vs append (recap)
- ANSI and strict modes
