sunithabeeram opened a new pull request #3979: Track Indexed timestamp across consuming segments URL: https://github.com/apache/incubator-pinot/pull/3979 *Motivation*: We would like to track freshness of the realtime data being consumed and available for queries. The true indication of this could be a timestamp for when the message was accepted into the stream-provider's (such as Kafka's) queue. However, this isn't something that is exposed as a client API. This change exposes the lastIndexed timestamp of a consuming segment as a proxy for freshness. Exposing this through the query path helps comparison against client defined SLAs and provides ability to reflect freshness as part of Pinot's availability. *Changes*: The PR exposes lastIndexedTimestamp on the MutableSegment interface. The min timestamp across consuming segments is obtained by the server query-executor before the segments are pruned out. This along with the number of consuming segments queried is returned to the broker in the response. *Testing done*: Unit and integration tests added.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
