erankor commented on issue #6963: Tuning Kafka Indexing Service for real time URL: https://github.com/apache/incubator-druid/issues/6963#issuecomment-459277567 @gianm, another question on this, just to clarify - if I set either intermediateHandoffPeriod or taskDuration to 5 min, but leave segmentGranularity on 1 hour, it means the same data will be indexed over and over again, right? To put it differently - the events of the first 5 min of the hour will be indexed 12 times until the full 1 hour segment is finalized. If that is indeed the case, maybe there is an advantage to changing the segmentGranularity instead, it will put less load on the KIS tasks. If having multiple granularities is not recommended, I can update the script we use for merging segments so that it will only merge on hour boundary. Thank you! Eran
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
