sydneyhoran commented on issue #8519: URL: https://github.com/apache/hudi/issues/8519#issuecomment-1529939285
When we either turn off tombstones in Debezium, or filter them out in DebeziumSource.java, no null/tombstones are coming in (which is good). But we still get a "commit failed" and upon further inspection of the log I have found that before this error it says `Delta Sync found errors when writing. Errors/Total=14/9282` It seems that DeltaSync/WriteStatus.java is treating the deletes as "Errors". When I set `--commit-on-errors` to true, it allows the job to run but what happens to those "error" records? Shouldn't they be telling Deltastreamer to "delete" those records? ``` 23/05/01 13:49:26 ERROR org.apache.hudi.utilities.deltastreamer.DeltaSync: Delta Sync found errors when writing. Errors/Total=14/9282 23/05/01 13:49:26 ERROR org.apache.hudi.utilities.deltastreamer.DeltaSync: Printing out the top 100 errors 23/05/01 13:49:26 ERROR org.apache.hudi.utilities.deltastreamer.DeltaSync: Global error : 23/05/01 13:49:26 ERROR org.apache.hudi.utilities.deltastreamer.DeltaSync: Global error : 23/05/01 13:49:26 ERROR org.apache.hudi.utilities.deltastreamer.DeltaSync: Global error : ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
