Hi Rahul,
With Spark data frame, the schema is deduced automatically. If you write a
dataframe with schema that is backwards compatible (for eg: with new column
added at the end), it should work seamlessly.
Are you seeing any problems with this approach ?
Thanks,Balaji.V
On Wednesday, December 2, 2020, 10:16:52 PM PST, Rahul Narayanan
<[email protected]> wrote:
Hi Team,
We are interested in writing new columns and maybe removing some columns in the
future in our dataset. I have read hudi supports schema evolution if it is
backward compatible. To do a poc I tried writing a spark data frame to hudi
using schema but it’s failing. How to write a spark data frame to hudi
specifying the schema explicitly
Thanks in advance
