aho135 commented on PR #17707: URL: https://github.com/apache/druid/pull/17707#issuecomment-2648964443
> Thank you for these changes, @aho135! > > I think we would benefit from a change where we check if the spec has changed. If it hasn't we still restart the supervisor, but do not go to the metadata store and add an unnecessary entry in the spec history. Otherwise, the flow remains unchanged. I think @kfaraz has suggested this as well. I believe the supervisor restart itself would be helpful to rollover tasks easily or to get out of an idle supervisor state etc. > > I also wanted to understand if the problem was with the metadata operations associated with it including an unneeded entry, or if the supervisor operation is also problematic. > > If it is just the first case, is a feature flag really needed? I believe we should skip the metadata operation and history update as there is no benefit in both cases > > If you still believe that the supervisor operation is wasteful, and want to introduce a flag, please add the relevant docs in `docs/api-reference/supervisor-api.md`. Thanks for the review @AmatyaAvadhanula! My original motivation for this change was to avoid unnecessary restarts of the Supervisor if possible. Our use case is that we maintain a repository of schemas and do periodic releases. It is often unclear which schemas were actually modified. We want to be able to submit them all, and just restart the Supervisors which had schema updates. This is so we can avoid the undesirable side effects of task restart, such as small segments. With this use case in mind, I think that having the feature flag does make sense. I will add an update in the relevant doc -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
