Hi Gray Thanks for reply. It is the latter. We can use the callback to publish write commit message to external system. HTTP is the way currently implemented, Kafka is another implementation I’d like to do next.
Thanks, Mathieu > 2020年7月24日 上午10:54，Gary Li <yanjia.gary...@gmail.com> 写道： > > Hi xianghu, > > Thanks for implementing the callback service. That’s very helpful! > > I am curious what is the difference between the asynchronous mode you are > referring to and the existing timeline service. Like, if I am spontaneously > looking for the new commit, I can go to the timeline to check if there is > any. > > Or, do you mean something like use the callback to publish messages to > external pub/sub services? So the user can consume from there? > > Thanks, > Gary > > > On Thu, Jul 23, 2020 at 7:23 PM vino yang <yanghua1...@gmail.com> wrote: > >> +1 as well >> >> We can introduce an async mode to provide a pub-sub model so that we can >> integrate with other systems. >> >> Best, >> Vino >> >> 957029...@qq.com <957029...@qq.com> 于2020年7月23日周四 下午9:05写道： >> >>> +1. >>> it's great. >>> >>> >>> >>> 957029...@qq.com >>> >>> From: wangxianghu >>> Date: 2020-07-23 21:02 >>> To: dev@hudi.apache.org >>> Subject: Implement an asynchronous write commit callback >>> Hi all, >>> Currently, a write callback service implemented in HTTP has been merged, >>> it will issue an HTTP request to inform downstream that a write commit >> has >>> been executed successfully！User can leverage this notification to do some >>> cascaded incremental processes. >>> >>> But, this implementation is synchronous, user must consume this >>> notification immediately. This might not be what user wants. >>> >>> So, I'd like to implement an asynchronous one，implemented by Kafka for >>> example. user can consume the callback message as long as they want, no >>> need to act immediately. >>> >>> WDYT? any feedback are appreciated :) >>> >>> Best >>> Mathieu >>> >>> >>
