Hi Pankaj, Just like Xintong has said, the biggest difference of Flink on Kubernetes and native integration is dynamic resource allocation. Since the latter has en embedded K8s client and will communicate with K8s Api server directly to allocate/release JM/TM pods.
Both for the two ways to run Flink on K8s, you do not need to reserve the whole cluster for Flink. Flink could run with other workloads(e.g. Spark, tensorflow, etc.). The K8s cluster could guarantee the isolation. Best, Yang Pankaj Chand <pankajchanda...@gmail.com> 于2020年3月16日周一 下午5:51写道： > Hi Xintong, > > Thank you for the explanation! > > If I run Flink "natively" on Kubernetes, will I also be able to run Spark > on the same Kubernetes cluster, or will it make the Kubernetes cluster be > reserved for Flink only? > > Thank you! > > Pankaj > > On Mon, Mar 16, 2020 at 5:41 AM Xintong Song <tonysong...@gmail.com> > wrote: > >> Forgot to mention that "running Flink natively on Kubernetes" is newly >> introduced and is only available for Flink 1.10 and above. >> >> >> Thank you~ >> >> Xintong Song >> >> >> >> On Mon, Mar 16, 2020 at 5:40 PM Xintong Song <tonysong...@gmail.com> >> wrote: >> >>> Hi Pankaj, >>> >>> "Running Flink on Kubernetes" refers to the old way that basically >>> deploys a Flink standalone cluster on Kubernetes. We leverage scripts to >>> run Flink Master and TaskManager processes inside Kubernetes container. In >>> this way, Flink is not ware of whether it's running in containers or >>> directly on physical machines, and will not interact with the Kubernetes >>> Master. Flink Master reactively accept all registered TaskManagers, whose >>> number is decided by the Kubernetes replica. >>> >>> "Running Flink natively on Kubernetes" refers deploy Flink as a >>> Kubernetes Job. Flink Master will interact with Kubernetes Master, and >>> actively requests for pods/containers, like on Yarn/Mesos. >>> >>> Thank you~ >>> >>> Xintong Song >>> >>> >>> >>> On Mon, Mar 16, 2020 at 4:03 PM Pankaj Chand <pankajchanda...@gmail.com> >>> wrote: >>> >>>> Hi all, >>>> >>>> I want to run Flink, Spark and other processing engines on a single >>>> Kubernetes cluster. >>>> >>>> From the Flink documentation, I did not understand the difference >>>> between: >>>> (1) Running Flink on Kubernetes, Versus (2) Running Flink natively on >>>> Kubernetes. >>>> >>>> Could someone please explain the difference between the two, and when >>>> would you use which option? >>>> >>>> Thank you, >>>> >>>> Pankaj >>>> >>>
