Hey Yue,
Very interesting idea. I am not a kubernetes expert, but this seems like a
neat concept. I guess the idea is only one MM would be needed? (Or maybe a
handful, if one can't manage every pod?) If so, great. Hopefully someone
that is more of a kubernetes expert will be able to chime in on the PR.
On Fri, Feb 26, 2021 at 12:46 AM Yue Zhang wrote:
> Hi Druid,
>
> I’d like to contribute a new Community Extension named
> druid-kubernetes-middlemanager-extensions. Proposal Link:
> https://github.com/apache/druid/issues/10824 and PR link :
> https://github.com/apache/druid/pull/10910
>
> When enable this feature, Druid can launch Peon as a pod to do data
> ingestion. The advantages are as follows:
>
> Cost saving: there is no need to let MiddleManager take up a lot of
> resources in advance, and just require resources whenever it will use. Now
> we can use a single 2Cores and 2Gi Memory middelmanager pod to control
> Dozens or even hundreds peon pods.
> Improve resource utilization: Now different kinds of tasks can use
> different configs including CPU resources and Memory resources if necessary.
>
> If you are interested in this feature, please let me know. Looking
> forward to your reply.
>
> Best wishes,
>
> Yue Zhang
>
>
>
