Thanks for the reply!
Just to clarify, when I talked about standalone deployments I was referring to
standalone Kubernetes deployments. We currently have no interest in running
Flink outside of K8s. I was mostly just curious about the differences in the
native integration vs. standalone deployment options on Kubernetes.
As a side note, we're planning on using Flink in Application Mode.
Alex
On Jan 7, 2021, at 4:17 PM, Israel Ekpo
mailto:israele...@gmail.com>> wrote:
If you are just getting started, running it on Kubernetes could simplify that
logistics and resources needed for getting started.
It also allows you to possibly reuse infrastructure that you may already be
using for other projects and purposes.
If you are just getting started and just learning, the setup and teardown time
is much faster with Kubernetes (in my experience) when compared to setting it
as Standalone on VMs
Network connectivity to other components within my ecosystem is also easier
when with K8s.
Other options are available here for your reference
https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/#overview-and-reference-architecture
If you are just starting out, I would recommend deploying your containers on
the K8S cluster via YAML manifests first and then convert to Helm charts (once
the vanilla YAML is stable) so that you can bundle the various components as a
single deployment.
On Thu, Jan 7, 2021 at 4:57 PM Alex Adriaanse
mailto:a...@oseberg.io>> wrote:
I'm trying to setup a Flink 1.12 deployment on a Kubernetes cluster using
custom Docker images (since the official ones aren't out yet). Since the
documentation states that "We generally recommend new users to deploy Flink on
Kubernetes using native Kubernetes deployments", I'm trying out the native
Kubernetes integration.
At the same time, I'm also trying to move to GitOps where we use a tool like
Argo CD to keep Kubernetes resources in sync with declarations stored in a Git
repository. From what I can tell there's no way to do GitOps with native
Kubernetes deployments, since the Kubernetes resources it produces are
generated dynamically and are managed via CLI rather than being suitable for
storage in a Git repository. Is that correct?
Would it be better in my case to just use the standalone Kubernetes deployment
option and have our CD tool deploy the YAML files listed in Flink's
documentation via Git? Or should I use the native Kubernetes integration to
create an initial set of resources in our cluster, export those to a Git
repository, and then use a GitOps workflow to modify those resources going
forward rather than using the Flink CLI? In other words, I'd only use the Flink
CLI to bootstrap the Git repository.
Besides ease of setup, are there any other benefits to the native Kubernetes
integration compared to standalone deployments?
Thanks!
Alex