Hello all, I have been trying to calculate, as precisely as possible, how much time a tuple spends waiting in the input queue of a Bolt until it is provided in the BaseRichBolt.execute() function.
The way I try to get this wait time (latency) is by adding a timestamp attribute in the tuple, the time it gets produced (I call Java's System.currentTimeInMillis()). When it gets picked up from a downstream Bolt (call of execute() function), I retrieve the timestamp attribute and calculate the difference in milliseconds from the current timestamp. However, due to clock skewness in my local cluster (I guess that this is the cause), I usually retrieve difference values that do not make sense (sometimes even negative wait times). I have previously tried NTP but I still do not get values that make sense. When I check the Storm UI reported metrics, I see that there two latency metrics: Process latency and one more. Is there any way I can retrieve those from the java code inside a Bolt's execute method? Is there a better way to get a more precise latency metric from inside the execute() function of a Bolt? Thanks, Nick -- Nikolaos Romanos Katsipoulakis, University of Pittsburgh, PhD candidate