Weave allows encryption of the vpn, and your Flink containers can be secured using kerberos https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/config.html#kerberos-based-security.
> Le 24 mars 2017 à 11:16, Chakravarthy varaga <chakravarth...@gmail.com> a > écrit : > > Hi, > > Thanks for your inputs. It kind of makes sense to use a container > orchestrator to plough through networking under the hood. > How do you tackle security? > > I don't see a way to authorize users for job management. I understand > few orchestrators provide name space isolation and security policies on > these. How do this work if the flink cluster is standalone on AWS ? > > > Best Regards > CVP > > On Fri, Mar 24, 2017 at 8:49 AM, Philippe Caparroy > <philippe.capar...@orange.fr <mailto:philippe.capar...@orange.fr>> wrote: > Hi, > > If I can give my 2 cents. > > One simple solution to your problem is using weave (https://www.weave.works/ > <https://www.weave.works/>) a Docker network plugin. > > We’ve been working for more then year with dockerized > (Flink+zookeeper+Yarn+spark+Kafka+hadoop+elasticsearch ) cluster using weave. > > Design your docker container so that you can set the cluster size on startup > (number of task manager stand job managers should be a docker arg). > > Weave will act as a switch with dns server embedded. Your containers will > only have to be configured with hosts names such as : > flink.taskmanager-1.weave.local, link.taskmanager-2.weave.local, > flink.jobmanager-1.weave.local, and so on … > > with flink Yarn it’s even simpler, but you have to dockerize a Yarn cluster. > > It works perfectly on bare metal machines and in the cloud (digital-ocean, > aws,…). > > > >> Le 24 mars 2017 à 08:50, Chakravarthy varaga <chakravarth...@gmail.com >> <mailto:chakravarth...@gmail.com>> a écrit : >> >> Hi, >> >> I request someone to help here. >> >> Best Regards >> CVP >> >> On Thu, Mar 23, 2017 at 10:13 PM, Chakravarthy varaga >> <chakravarth...@gmail.com <mailto:chakravarth...@gmail.com>> wrote: >> I'm looking forward to hearing some updates on this... >> >> Any help here is highly appreciated !! >> >> On Thu, Mar 23, 2017 at 4:20 PM, Chakravarthy varaga >> <chakravarth...@gmail.com <mailto:chakravarth...@gmail.com>> wrote: >> Hi Team, >> >> We are doing a PoC to deploy Flink cluster on AWS. All runtime >> components will be dockerized. >> >> I have few questions in relation to discover & security: >> >> 1. How does Job Manager discover task managers? Do they talk to over >> TCP ? >> >> 2. If the runtime components TM, JM are containerized how are the IPs >> resolved dynamically? Basically do I have to configure the JM with the >> hostnames of the TMs. If so, if the TMs are on ephemeral IPs and on restart >> of TM how does the job manager know the TM's (IP/Host). Before I go into DNS >> and subnets, I'd like to understand how they disvoer & talk to each other ! >> >> 3. I went through some Flink materials on the web on security >> precisely on kerebros. However how do I ensure that user level >> authentication is applied on job management. For ex., only certain users are >> allowed to start/stop jobs ? This question is in relation to if flink is >> deployed as standalone-cluster >> >> Thanks & Regards >> CVP >> >> > >
