georgew5656 commented on code in PR #16510: URL: https://github.com/apache/druid/pull/16510#discussion_r1626119628
########## docs/development/extensions-contrib/k8s-jobs.md: ########## @@ -217,6 +217,66 @@ data: druid.peon.mode=remote druid.indexer.task.encapsulatedTask=true ``` +#### Dynamic Pod Template Selection Config +The Dynamic Pod Template Selection feature enhances the K8s extension by enabling more flexible and dynamic selection of pod templates based on task properties. This process is governed by the `ExecutionBehaviorStrategy`. Below are the two strategies implemented: + +|Property|Description|Default| +|--------|-----------|-------| +|`DefaultExecutionBehaviorStrategy`| This strategy categorizes tasks based on their type for execution purposes, implementing the existing behavior that maps pod templates according to task type. | true | +|`DynamicTaskExecutionBehaviorStrategy`| This strategy dynamically evaluates a series of selectors, with each selector corresponding to a potential task category.| false | + +`DynamicTaskExecutionBehaviorStrategy`, the strategy implementing this new feature, is based on conditional selectors that match against task properties. Conditions are specified in the dynamic config, and the selection process uses these conditions to determine the category of a task based on context tags and task fields. The identified category is then used to map to different Peon Pod templates, allowing tailored resource allocation and management according to the task’s requirements. + +Example Configuration: + +We define two categories in the configuration—`low-throughput` and `medium-throughput`—each associated with specific task conditions. + +- Low Throughput Category: Tasks that have a context tag `billingCategory=streaming_ingestion` and a datasource of `wikipedia` will be classified under the `low-throughput` category. This classification directs such tasks to utilize a predefined pod template optimized for low throughput requirements. Review Comment: can we make it a bit more clear in the docs that the categorySelectors are ordered, e.g. the first mapping has the highest priority? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org