sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-855198402
Hi @vinothchandar, @nsivabalan and me are working on Slack trying to get our
schema evolution usecase running. Currently only Inserts, no Upserts for CoW.
I'm trying out his patch in
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-666277215
> Do you mean using SCHEMA_v1 at step 4. Otherwise, both reader and writer
schema is same. right ?
I could have been more precise. At Step 4 we consume events without the new
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-666232604
Thanks @bvaradar for opening the tickets!
I will continue commenting in this tickets.
This is an automated message
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-664807718
Is there anything we can do further to resolve this issue?
This is an automated message from the Apache Git Service.
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-663373241
> 4. We ingest old events again (there are some upserts). ?? What schema
is being used here?
At this step I used SCHEMA_V2
We use Deltastreamer in continues mode and
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-663065111
Yes, to be correct you have to specify both reader and writer schema to
GenericDatumReader. Hudi currenty only passes one schema here
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-662921730
Hi @bvaradar,
I've created a test using pure avro DataFileWriter to reproduce my
DeltaStreamer-test and it kind of works.
You can find it here
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-661711698
@bvaradar, yes i am appending the field to end of the schema (as reproduced
in the test). The definition of the event is outside my scope, we just consume
this events ;)
sbernauer commented on issue #1845:
URL: https://github.com/apache/hudi/issues/1845#issuecomment-660853366
Thanks for your fast reply!
The PR adds a new Test and improves 2 existing tests. The mentioned 4 new
cols in TestHoodieAvroUtils increase the number of tested cases, the tests
