[GitHub] [hudi] sathyaprakashg commented on pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2021-05-08 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-835222588 > > @sathyaprakashg and others: trying to understand the use-case here. I understand its related to deltastreamer receiving events in old schema after Hudi's dataset schema

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2021-05-08 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-835216648 > @sathyaprakashg and others: trying to understand the use-case here. I understand its related to deltastreamer receiving events in old schema after Hudi's dataset schema go

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2021-04-22 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-824608661 @n3nash I rebased the PR. Thanks @sbernauer for help -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2020-09-17 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-694081153 @n3nash Changes are done. Please review it when you get time This is an automated message from the Apache Gi

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: HUDI-1129 Deltastreamer Add support for schema evolution

2020-09-02 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-686046387 Below issue was causing build failure and i fixed it now by implementing a method `HoodieAvroUtils.addNamespaceToFixedField`. Not sure whether this is the best way to fix th

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: HUDI-1129 Deltastreamer Add support for schema evolution

2020-09-01 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-685308504 @n3nash I am working on fixing build issue and will have that fix pushed soon. I would like to point out that with this new approach, we are stroing writer schema part of pa

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: HUDI-1129 Deltastreamer Add support for schema evolution

2020-09-01 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-684687937 @vinothchandar I updated the code to use reader and writer schema for avro payload. Please review it This i

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: HUDI-1129 Deltastreamer Add support for schema evolution

2020-08-31 Thread GitBox
sathyaprakashg commented on pull request #2012: URL: https://github.com/apache/hudi/pull/2012#issuecomment-684007672 @vinothchandar Sure, I will start looking into using writer and reader schema. Once that looks good, then i will remove the new Schema provide class i added --