[GitHub] [incubator-hudi] bhasudha commented on issue #1653: [SUPPORT]: Hudi Deltastreamer OffsetoutofRange Exception reading from Kafka topic (12 partitions)

2020-05-22 Thread GitBox
bhasudha commented on issue #1653: URL: https://github.com/apache/incubator-hudi/issues/1653#issuecomment-632516157 @prashanthpdesai sorry I assumed you were referring to your own check pointing. Your understanding is right. Checkpoints are written to hoodie commit metadata after each

[GitHub] [incubator-hudi] bhasudha commented on issue #1653: [SUPPORT]: Hudi Deltastreamer OffsetoutofRange Exception reading from Kafka topic (12 partitions)

2020-05-21 Thread GitBox
bhasudha commented on issue #1653: URL: https://github.com/apache/incubator-hudi/issues/1653#issuecomment-632405067 @prashanthpdesai quick questions. Where do you checkpoint the offsets between mini batches? and how do you configure that to deltastreamer? Do you have the