abhishekrb19 commented on code in PR #18082: URL: https://github.com/apache/druid/pull/18082#discussion_r2152696262
########## docs/ingestion/supervisor.md: ########## @@ -411,6 +413,10 @@ This value is for the ideal situation in which there is at most one set of tasks In some circumstances, it is possible to have multiple sets of tasks publishing simultaneously. This would happen if the time-to-publish (generate segment, push to deep storage, load on Historical) is greater than `taskDuration`. This is a valid and correct scenario but requires additional worker capacity to support. In general, it is a good idea to have `taskDuration` be large enough that the previous set of tasks finishes publishing before the current set begins. +## Multi-Supervisor Support +Druid supports multiple stream supervisors ingesting into the same datasource. This means you can have any number of the configured stream supervisors (Kafka, Kinesis, etc.) ingesting into the same datasource at the same time. Review Comment: Should we also cross-reference this [topic](https://druid.apache.org/docs/latest/ingestion/kafka-ingestion/#ingest-from-multiple-topics) with `topicPattern` and vice-versa? I'm also wondering with support for multiple supervisors, would anyone still prefer to use `topicPattern`, even if all the input topics are on the same Kafka brokers? `topicPattern` is limited to Kafka and only works when all input topics reside on the same Kafka brokers. Multiple supervisors on the other hand has other added benefits in relation to scaling and mangement. If there's value in keeping both, we could call out the differences. Otherwise, we could consider deprecating `topicPattern` in favor of this new feature once it graduates from experimental to GA. Thoughts, @jtuglu-netflix @kfaraz? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
