I think the launcher can go into this FLIP.
On Tue, Aug 25, 2026 at 8:44 PM Dale Lane <[email protected]> wrote: > > 3. Launcher placement - Here I'd go the opposite way of the current doc. > > Currently, every deployment shape keeps its bootstrap in the Flink > > distribution, and that convention is what decouples operator version from > > Flink version. My opinion is that the MiniCluster launcher should follow > > it. In practice, the launcher is close to `flink run -t local`, only > adding > > HA-aware resubmission and pod exit-code semantics on top, so it is core > > code either way. I'd also say it makes sense to ship it in the > distribution > > (ideally with a bin/ script, so the same setup runs locally too) because > > this means the operator depends on no MiniCluster Java API at all, and it > > can manage the deployment in the same unified way as the other shapes. > > I really like this idea, thanks - that's much better than what I'd > originally thought of. It simplifies a few things, such as migration > between MiniCluster and Deployment by letting you just use the same image > for both as-is. I'll have a proper think through what it would need and > update the doc. > > Do you think a Flink launcher would need a separate/dedicated FLIP to get > broader scrutiny? > > Kind regards > > D > -- > dalelane.co.uk > > On Tuesday, 25 August 2026 at 14:10, Dennis-Mircea Ciupitu < > [email protected]> wrote: > > > Hi Dale, > > > > My response to the open points is the following: > > > > 1. MiniCluster vs 2 containers in one pod - Robert's numbers and > > argument make it clear to me. So, in order to ensure the lowest > possible > > resource consumption, we need to go with the MiniCluster approach and > not > > rely on something intermediary that does not reach this end goal and > is > > only better suited from the implementation/design perspective. > > 2. Separate CRD vs a third mode - Because the MiniCluster approach is > > the way to go, I'd say that we should have an implementation that > > integrates with it in the best way possible. For me, I see a separate > CRD > > as the best way to proceed here, as MiniCluster is different from many > > points of view from a normal Flink cluster, and by going with a third > mode > > I think we just complicate ourselves and we introduce too much blast > radius. > > 3. Launcher placement - Here I'd go the opposite way of the current > doc. > > Currently, every deployment shape keeps its bootstrap in the Flink > > distribution, and that convention is what decouples operator version > from > > Flink version. My opinion is that the MiniCluster launcher should > follow > > it. In practice, the launcher is close to `flink run -t local`, only > adding > > HA-aware resubmission and pod exit-code semantics on top, so it is > core > > code either way. I'd also say it makes sense to ship it in the > distribution > > (ideally with a bin/ script, so the same setup runs locally too) > because > > this means the operator depends on no MiniCluster Java API at all, > and it > > can manage the deployment in the same unified way as the other shapes. > > 4. Promoting the MiniCluster - Here I say if the entrypoint lands in > > Flink core, then I'd follow Gyula's and Robert's guidance on this > side, if > > there is something that can be done further here in order to promote > it > > properly. > > 5. Resource and memory mapping - Deferring the exact ratios to > > implementation is fine with me. > > > > > > Best regards, > > Dennis > > > > > > On Mon, Aug 24, 2026 at 2:00 PM Dale Lane < > [email protected]> > > wrote: > > > > > > Whether it should be a different CRD I think should > > > > depend on the overlap with the FlinkDeployment. If > > > > we are exposing essentially the same spec (podTemplate, > > > > jobSpec, flinkConfig, ingress..) then it feels like we > > > > should reuse the CRD and simply introduce a new > > > > deployment mode next to native and standalone. > > > > > > @Gyula, @Dennis - How would you like to proceed with this question? > > > > > > I can absolutely see arguments for/against both approaches. In the end > I > > > leaned more towards a separate CRD, partly from an API purity > perspective > > > (feeling that the separate topology deserved a separate API, and avoids > > > describing the pod spec and status as "jobManager" or "taskManager" > when > > > it's both) and partly because it dodged the automated topology > migration. > > > > > > But maybe we don't mind the "a MiniCluster includes a Job Manager so we > > > can just describe it as jobManager (and ignore the taskManager > references)" > > > API smell. It could change the manual migration path I describe to > > > transition jobs between MiniCluster and full distributed clusters to > just > > > flipping a mode value. And we can always reject/block such a mode > switch in > > > the meantime unless/until the Operator is ready to support topology > > > transition/migration. > > > > > > Kind regards > > > > > > Dale > > > -- > > > dalelane.co.uk > > > > > > Sent with Proton Mail secure email. > > > > > > On Thursday, 20 August 2026 at 10:47, Gyula Fóra <[email protected] > > > > > wrote: > > > > > > > Thanks @Robert Metzger for the explanation, > > > > > > > > Overall I think this is a good idea and hopefully we can reuse most > of > > > > the controller flow that we already have. > > > > It would be nice to share the spec/status as much as reasonably > possible. > > > > > > > > Whether it should be a different CRD I think should depend on the > > > > overlap with the FlinkDeployment. If we are exposing essentially the > > > > same spec (podTemplate, jobSpec, flinkConfig, ingress..) then it > feels > > > > like we should reuse the CRD and simply introduce a new deployment > > > > mode next to native and standalone. > > > > > > > > Cheers > > > > Gyula > > > > > > > > On Thu, Aug 20, 2026 at 11:26 AM Dale Lane > > > > <[email protected]> wrote: > > > > > > > > > > This bit that I added to the FLIP is perhaps worthy of a bit more > > > scrutiny, as it's based on my assumptions (misunderstandings?) of how > the > > > FKO project works. > > > > > > > > > > In the section "Promoting MiniCluster to a stable API" I asserted: > > > > > > > > > > > For the Flink Kubernetes Operator project to provide a launcher > > > > > > as an example project would depend on the core Flink project > > > > > > promoting some MiniCluster components to @PublicEvolving ... > > > > > > This is not a technical requirement, as the current > > > > > > proof-of-concept demonstrates that the existing API is sufficient > > > > > > - the promotion recommended here is of MiniCluster’s existing > > > > > > submission surface. The requirement is about the Operator project > > > > > > managing risk by building upon a stable contract. ... > > > > > > > > > > Is that really a requirement or have I just invented that? :) > > > > > > > > > > Looking more closely at the existing code, I can see we already > have > > > plenty of uses of flink-runtime classes that don't have @Public / > > > @PublicEvolving annotations, so my usage of flink-runtime classes like > the > > > MiniCluster and MiniClusterConfiguration wouldn't be without precedent. > > > > > > > > > > While I still think it'd be lovely to build this feature solely on > > > stable APIs, maybe I'm creating an unreasonably high bar to clear by > > > framing it this so strongly. > > > > > > > > > > What do you think? What are the norms here? > > > > > > > > > > Kind regards > > > > > > > > > > Dale > > > > > -- > > > > > dalelane.co.uk > > > > > > > > > > > > > > > On Wednesday, 5 August 2026 at 16:02, Dale Lane < > > > [email protected]> wrote: > > > > > > > > > > > I'd like to start a discussion on > > > > > > FLIP-XXX : Running Flink jobs in MiniCluster using the Kubernetes > > > Operator > > > > > > > > > > https://docs.google.com/document/d/1dtGjPYcsBkx1vxHPs1QnDtPxeH_Acz_pl8gx4b1BLB4/edit?usp=sharing > > > > > > > > > > > > The aim of the FLIP is to extend the Flink Kubernetes Operator to > > > offer a single-pod, light-weight deployment option for low-throughput > jobs. > > > > > > > > > > > > From the motivation: > > > > > > A single-pod, self-contained Flink job that starts fast and > needs no > > > multi-pod coordination could be a good fit for low-throughput jobs that > > > aren't suitable for session clusters because they need isolation. > > > > > > > > > > > > Looking forward to feedback, both on the general motivation (Have > > > you seen a need for very small lightweight Flink jobs where fast > > > crash-consistent resume is good enough without a full distributed Flink > > > cluster?) and the suggested implementation approach (Do you think a new > > > custom resource kind is the best way to represent this capability?) > > > > > > > > > > > > Kind regards > > > > > > > > > > > > Dale > > > > > > -- > > > > > > dalelane.co.uk > > > > > > > > > > > > > > > >
