[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2022-03-20 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-1073504409 @pratyakshsharma -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2022-03-20 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-1073504190 The purpose of introducing timestamps: Mainly when users want to consume from a certain location, deltastreamer can only specify checkpoint sites in the past. For example,

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-07-16 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-881820983 @nsivabalan Thank you for your concern and patience to help! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-07-16 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-881278532 @nsivabalan I have completed the changes as you requested, please take a look~ Thank you very much for your help! -- This is an automated message from the Apache Git

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-07-16 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-881203019 > Let's try to land this in by weekend. Its been hanging for quite sometime. ok. Sorry, I'll deal with it now, please excuse me -- This is an automated

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-07-07 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-875975684 woking -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-06-18 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-863968623 deltaSync should reset this(...kafka.checkpoint.type) configuration (similar to how we reset checkpoints) In this way, we may need to store this in the metadata file. If

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-06-02 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-852970149 I am currently facing a problem and would like to hear your opinion After we add this type, hoodie.deltastreamer.source.kafka.checkpoint.type=timestamp I am currently

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-05-21 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-845791067 > @liujinhui1994 : were you able to make progress on this. would be nice to have this in before next release. Sorry, I was too busy with work before~ I just sorted out

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-04-01 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-811856655 > Myself and Nishith discussed on this. Here is our proposal. > Let's rely on Deltastreamer.Config.checkpoint to pass in any type of checkpoint. > We can add another

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-03-15 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-799512747 no problem -- Original -- From: Sivabalan Narayanan ***@***.*** Date: Mon,Mar 15,2021 11:28 PM To: apache/hudi

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-02-28 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-787662595 The current implementation is mainly in KafkaOffsetGen @wangxianghu This is an automated message from the

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-02-25 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-786452099 I will add the unit test, and then please review This is an automated message from the Apache Git Service.

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-02-20 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-782589121 I have verified, please help review @wangxianghu @yanghua @nsivabalan This is an automated message

[GitHub] [hudi] liujinhui1994 commented on pull request #2438: [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

2021-02-20 Thread GitBox
liujinhui1994 commented on pull request #2438: URL: https://github.com/apache/hudi/pull/2438#issuecomment-782588952 @yanghua @wangxianghu @nsivabalan I have verified, please help review This is an automated