Re: Behavior of SaveMode.Append when table is not present

2018-11-09 Thread Ryan Blue
Right now, it is up to the source implementation to decide what to do. I think path-based tables (with no metastore component) treat an append as an implicit create. If you're thinking that relying on sources to interpret SaveMode is bad for consistent behavior, I agree. That's why the community

Behavior of SaveMode.Append when table is not present

2018-11-08 Thread Shubham Chaurasia
Hi, For SaveMode.Append, the doc https://spark.apache.org/docs/latest/sql-data-sources-load-save-functions.html#save-modes says *When saving a DataFrame to a data source, if data/table already exists, contents of the DataFrame are expected to be appended to existing data* However it does not