ScalingRealizer and
> > > > > AutoScalerEventHandler interface.
> > > > >
> > > > > The ScalingRealizer handles scaling action.
> > > > >
> > > > > The AutoScalerEventHandler interface handles loggable events.
> > > &g
he.org/confluence/x/x4qzDw
> > > >
> > > > Best,
> > > > Rui
> > > >
> > > > On Thu, Aug 24, 2023 at 10:55 AM Matt Wang wrote:
> > > >>
> > > >> Sorry for the late reply, I still have a small question here:
> >
he late reply, I still have a small question here:
> > >> Does the interface function of handlerRecommendedParallelism
> > >> in AutoScalerEventHandler conflict with
> > >> handlerScalingFailure/handlerScalingReport (one of the
> > >> handles the event of scale f
success).
> >>
> >>
> >>
> >> --
> >>
> >> Best,
> >> Matt Wang
> >>
> >>
> >> Replied Message
> >> | From | Rui Fan<1996fan...@gmail.com> |
> >> | Date | 08/21/2023 17:41 |
configmap,
>> can it support the use of zookeeper (zookeeper is a relatively
>> common use of flink HA)?
>> 2. Regarding each scaling information, can it be persisted in
>> the shared file system through the filesystem? I think it will
>> be a more valuable requiremen
ess).
>
>
>
> --
>
> Best,
> Matt Wang
>
>
> Replied Message
> | From | Rui Fan<1996fan...@gmail.com> |
> | Date | 08/21/2023 17:41 |
> | To | |
> | Cc | Maximilian Michels ,
> Gyula Fóra ,
> Matt Wang |
> | Subject | Re: [DISCU
success).
--
Best,
Matt Wang
Replied Message
| From | Rui Fan<1996fan...@gmail.com> |
| Date | 08/21/2023 17:41 |
| To | |
| Cc | Maximilian Michels ,
Gyula Fóra ,
Matt Wang |
| Subject | Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes |
Hi Max, Gyula and Matt,
viewing
>>> Autoscaling info on the UI in the future, which can provide
>>> some foundations in advance;
>>> 3. A solution mentioned in FLIP is to initialize the
>>> AutoScalerEventHandler object every time an event is
>>> processed. W
>> AutoScalerEventHandler object every time an event is
>> processed. What is the main purpose of this solution?
>>
>>
>>
>> --
>>
>> Best,
>> Matt Wang
>>
>>
>> Replied Message
>> | From | Rui Fan<1996fa
processed. What is the main purpose of this solution?
>
>
>
> --
>
> Best,
> Matt Wang
>
>
> Replied Message
> | From | Rui Fan<1996fan...@gmail.com> |
> | Date | 08/7/2023 11:34 |
> | To | |
> | Cc | m...@apache.org ,
> Gyula Fóra |
&
|
| To | |
| Cc | m...@apache.org ,
Gyula Fóra |
| Subject | Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes |
Hi Ron:
Thanks for the feedback! The goal is indeed to turn the autoscaler into
a general tool that can support other resource management.
Hi Max, Gyula:
My pr
Hi Ron:
Thanks for the feedback! The goal is indeed to turn the autoscaler into
a general tool that can support other resource management.
Hi Max, Gyula:
My proposed `AutoScalerStateStore` is similar to Map, it can really be
improved.
> public interface AutoScalerStateStore {
> Map getStat
Hi, Rui
Thanks for driving the FLIP.
The tuning of streaming jobs by autoscaler is very important. Although the
mainstream trend now is cloud-native, many companies still run their Flink
jobs on Yarn for historical reasons. If we can decouple autoscaler from K8S
and turn it into a common tool tha
Hi Rui!
Thanks for the proposal.
I agree with Max on that the state store abstractions could be improved and
be more specific as we know what goes into the state. It could simply be
public interface AutoScalerStateStore {
Map getState(KEY jobKey)
void updateState(KEY jobKey, Map state);
Hi Max,
After careful consideration, I prefer to keep the AutoScalerStateStore
instead of AutoScalerEventHandler taking over the work of
AutoScalerStateStore. And the following are some reasons:
1. Keeping the AutoScalerStateStore to make StateStore easy to plug in.
Currently, the kubernetes-ope
Hi Max,
Thanks for your quick response!
> 1. Handle state in the AutoScalerEventHandler which will receive
> all related scaling and metric collection events, and can keep
> track of any state.
If I understand correctly, you mean that updating state is just part of
handling events, right?
If ye
Hi Rui,
Thanks for the proposal. I think it makes a lot of sense to decouple
the autoscaler from Kubernetes-related dependencies. A couple of notes
when I read the proposal:
1. You propose AutoScalerEventHandler, AutoScalerStateStore,
AutoScalerStateStoreFactory, and AutoScalerEventHandler.
Autos
Hi all,
I and Samrat(cc'ed) created the FLIP-334[1] to decoupling the autoscaler
and kubernetes.
Currently, the flink-autoscaler is tightly integrated with Kubernetes.
There are compelling reasons to extend the use of flink-autoscaler to
more types of Flink jobs:
1. With the recent merge of the E
18 matches
Mail list logo
