Hi everyone, I wanted to bring up a potential enhancement to Flink's 
autoscaling capabilities that could help address some operational challenges 
we've been experiencing. **Background:** Currently, we have a use case where we 
need to apply a flag to stop our Flink jobs during deployments to ensure smooth 
blue/green deployment transition. This helps us prevent duplicates and 
out-of-order messages that can occur. **Proposal:** I'm thinking about 
implementing pre/post scaling hooks that could provide more control over 
scaling operations. The key benefits would be: Prevent processing issues : 
Allow jobs to gracefully pause processing during scaling to avoid duplicates 
and message ordering issues Plugin influence : Enable plugins to influence the 
amount and timing of scaling operations, this can also help solve the 
ScalingRealizer usecase defined in 
https://lists.apache.org/thread/bp9wymtrfmsjxk3hql3r4gp6yd0qyhxr , by allowing 
the pre hook to potentially influence the scaling in, just an idea Flexible 
control : The hooks serve as an extension to the existing autoscaler rather 
than having to deploy a standalone. Thoughts? Best regards, Diljeet(DJ) Singh

Reply via email to