bvaradar commented on issue #1979:
URL: https://github.com/apache/hudi/issues/1979#issuecomment-682061419
Will close the ticket for now. Please reopen if we need to discuss more on
this topic.
This is an automated message fr
bvaradar commented on issue #1979:
URL: https://github.com/apache/hudi/issues/1979#issuecomment-680998998
@hughfdjackson : Good point about incrementally reading multiple commits.
The variation you suggested seems to make sense.
---
bvaradar commented on issue #1979:
URL: https://github.com/apache/hudi/issues/1979#issuecomment-679522323
@hughfdjackson : In general getting incremental read to discard duplicates
is not possible for MOR table types as we defer the merging of records to
compaction.
I was thinking a
bvaradar commented on issue #1979:
URL: https://github.com/apache/hudi/issues/1979#issuecomment-678595102
Right, this dataset is essentially a log but if you are only worried about
incremental query, then you will be reading only the records added by the new
commits. Also, note that your
bvaradar commented on issue #1979:
URL: https://github.com/apache/hudi/issues/1979#issuecomment-676490788
One option to make this to work currently is to add columns that gets
updated also as part of the composite record key. We can use key uniqueness
constraint of Hudi to achieve the res
