Awesome thank you very much I will check an try with this.. I am pumping data with a simple Python producer script to send data to just 1 topic
El vie., 20 jul. 2018 a las 11:04, Thakrar, Jayesh (< [email protected]>) escribió: > Below are some examples. > > They might be old, but the concept is the same - essentially you are > implementing an interface. > > You have to be conscious of not to do heavy or high latency processing to > be non-impactful to the data flow. > > > > > https://www.ashishpaliwal.com/blog/2013/06/flume-cookbook-implementing-custom-interceptors/ > > > > > https://medium.com/@bkvarda/building-a-custom-flume-interceptor-8c7a55070038 > > > > > > > > *From: *antonio saldivar <[email protected]> > *Date: *Friday, July 20, 2018 at 9:57 AM > *To: *"Thakrar, Jayesh" <[email protected]> > *Cc: *"[email protected]" <[email protected]> > *Subject: *Re: Measure latency from Source to Sink > > > > Hi > > > > Actually I am running the app I a single node this is for a POC, I was not > aware of the custom interceptors do you have an example of this? > > > > Best Regards > > > > El vie., 20 jul. 2018 a las 9:20, Thakrar, Jayesh (< > [email protected]>) escribió: > > See if you can use a custom interceptors for this. > The only fuzzy thing is that the clocks would be different so I would be a > little skeptical of its accuracy. > I have heard of some companies who have a special topic in which they > insert test msgs and then read them back - using the same machine for > producer and consumer, thereby doing away with this dilemma. > The assumption is that any delay in the cluster would be felt everywhere. > Another option is to create topics with one replica on all brokers (i.e. # > of topics = # of brokers) - so you can be sensitive to a single broker > having an issue. > > On 7/19/18, 3:09 PM, "antonio saldivar" <[email protected]> wrote: > > Hello > > I am developing an application using kafka and flink, I need to be > able to > measure the latency from the producer and when it comes out to the > sink. > > I can append the timestamp in Millisecond when I send the trxn from the > producer but at the end how to append the timestamp when it comes out > the > sink. > > Someone can help me with an example > > Thank you > Best Regards > >
