Hi All,
Thanks for bring-up this discussion, Robert!
Congratulations on becoming the release manager of 1.12， Dian and Robert !
----------
Here are some of my thoughts of the features for native integration with
Kubernetes in Flink 1.12:
1. Support user-specified pod templates
Description:
The current approach of introducing new configuration options for each
aspect of pod specification a user might wish is becoming unwieldy, we have
to maintain more and more Flink side Kubernetes configuration options and
users have to learn the gap between the declarative model used by
Kubernetes and the configuration model used by Flink. It's a great
improvement to allow users to specify pod templates as central places for
all customization needs for the jobmanager and taskmanager pods.
Benefits:
Users can leverage many of the advanced K8s features that the Flink
community does not support explicitly, such as volume mounting, DNS
configuration, pod affinity/anti-affinity setting, etc.
2. Support running PyFlink on Kubernetes
Description:
Support running PyFlink on Kubernetes, including session cluster and
application cluster.
Benefits:
Running python application in a containerized environment.
3. Support built-in init-Container
Description:
We need a built-in init-Container to help solve dependency management
in a containerized environment, especially in the application mode.
Benefits:
Separate the base Flink image from dynamic dependencies.
4. Support accessing secured services via K8s secrets
Description:
Kubernetes Secrets
<https://kubernetes.io/docs/concepts/configuration/secret/> can be used to
provide credentials for a Flink application to access secured services. It
helps people who want to use a user-specified K8s Secret through an
environment variable.
Benefits:
Improve user experience.
5. Support configuring replica of JobManager Deployment in ZooKeeper HA
setups
Description:
Make the *replica* of Deployment configurable in the ZooKeeper HA
setups.
Benefits:
Achieve faster failover.
6. Support to configure limit for CPU requirement
Description:
To leverage the Kubernetes feature of container request/limit CPU.
Benefits:
Reduce cost.
Regards,
Canbin Zheng
Harold.Miao <miaohong...@gmail.com> 于2020年7月23日周四 下午12:44写道：
> I'm excited to hear about this feature, very, very, very highly encouraged
>
>
> Prasanna kumar <prasannakumarram...@gmail.com> 于2020年7月23日周四 上午12:10写道：
>
> > Hi Flink Dev Team,
> >
> > Dynamic AutoScaling Based on the incoming data load would be a great
> > feature.
> >
> > We should be able have some rule say If the load increased by 20% , add
> > extra resource should be added.
> > Or time based say during these peak hours the pipeline should scale
> > automatically by 50%.
> >
> > This will help a lot in cost reduction.
> >
> > EMR cluster provides a similar feature for SPARK based application.
> >
> > Thanks,
> > Prasanna.
> >
> > On Wed, Jul 22, 2020 at 5:40 PM Robert Metzger <rmetz...@apache.org>
> > wrote:
> >
> > > Hi all,
> > >
> > > Now that the 1.11 release is out, it is time to plan for the next major
> > > Flink release.
> > >
> > > Some items:
> > >
> > > 1.
> > >
> > > Dian Fu and me volunteer to be the release managers for Flink 1.12.
> > >
> > >
> > >
> > > 1.
> > >
> > > Timeline: We propose to stick to our approximate 4 month release
> > cycle,
> > > thus the release should be done by late October. Given that there’s
> a
> > > holiday week in China at the beginning of October, I propose to do
> the
> > > feature freeze on master by late September.
> > >
> > > 2.
> > >
> > > Collecting features: It would be good to have a rough overview of
> the
> > > features that will likely be ready to be merged by late September,
> and
> > > that
> > > we want in the release.
> > > Based on the discussion, we will update the Roadmap on the Flink
> > website
> > > again!
> > >
> > >
> > >
> > > 1.
> > >
> > > Test instabilities and blockers: I would like to avoid a situation
> > where
> > > we have many blocking issues or build instabilities at the time of
> the
> > > feature freeze. To achieve that, we will try to check every build
> > > instability within a week, to decide if it is a blocker (make sure
> to
> > > use
> > > the “test-stability” label for those tickets!)
> > > Blocker issues will need to have somebody assigned (responsible)
> > within
> > > a week, and we want to see progress on all blocker issues
> (downgrade,
> > > resolution, a good plan how to proceed if it is more complicated)
> > >
> > > 2.
> > >
> > > Quality and stability of new features: In order to have a short
> > feature
> > > freeze phase, we encourage developers to only merge well-tested and
> > > documented features. In our experience, the feature freeze works
> best
> > if
> > > new features are complete, and the community can focus fully on
> > > addressing
> > > newly found bugs and voting the release.
> > > By having a smooth release process, the next merge-window for the
> next
> > > release will come sooner.
> > >
> > >
> > > Let me know what you think about our items, and share which features
> you
> > > want in Flink 1.12.
> > >
> > > Best,
> > >
> > > Robert & Dian
> > >
> >
>
>
> --
>
> Best Regards,
> Harold Miao
>
