Hi Welly! In the end, all remote communication in Flink will go through Netty (Flink direct shuffles do, and Akka uses Netty as well).
Netty authenticates connections and encrypts data via SSL, implementing Java's SSLContext. As far as I know, the available algorithms for encryption / signatures are the usual ones in the TLS standard (can also be configured via JVM arguments). Greetings, Stephan On Tue, Jan 12, 2016 at 9:38 AM, Ufuk Celebi <u...@apache.org> wrote: > Hey Welly! > > I’m not aware of any concrete plans, but is it possible that you share > your requirements on a high level? > > – Ufuk > > > On 12 Jan 2016, at 08:24, Welly Tambunan <if05...@gmail.com> wrote: > > > > Hi Stephen, > > > > Do you have any plan on which encryption method and mechanism will be > used on Flink ? Could you share about the detail on this ? > > > > We have very strict requirement from client that every communication > need to be encryption. So any detail would be really appreciated for > answering their security concern. > > > > > > Cheers > > > > On Mon, Jan 11, 2016 at 9:46 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wrote: > > Thanks Steven for your details response. Things are more clear to me now. > > > > A follow up Qs - > > Looks like most of the security support depends on Hadoop ? What happens > if anyone wants to use Flink with Hadoop (in a cluster where Hadoop is not > there) ? > > > > Regards, > > Sourav > > > > On Sun, Jan 10, 2016 at 12:41 PM, Stephan Ewen <se...@apache.org> wrote: > > Hi Sourav! > > > > There is user-authentication support in Flink via the Hadoop / Kerberos > infrastructure. If you run Flink on YARN, it should seamlessly work that > Flink acquires the Kerberos tokens of the user that submits programs, and > authenticate itself at YARN, HDFS, and HBase with that. > > > > If you run Flink standalone, Flink can still authenticate at HDFS/HBase > via Kerberos, with a bit of manual help by the user (running kinit on the > workers). > > > > With Kafka 0.9 and Flink's upcoming connector ( > https://github.com/apache/flink/pull/1489), streaming programs can > authenticate themselves as stream brokers via SSL (and read via encrypted > connections). > > > > > > What we have on the roadmap for the coming months it the following: > > - Encrypt in-flight data streams that are exchanged between worker > nodes (TaskManagers). > > - Encrypt the coordination messages between client/master/workers. > > Note that these refer to encryption between Flink's own components only, > which would use transient keys generated just for a specific job or session > (hence would not need any user involvement). > > > > > > Let us know if that answers your questions, and if that meets your > requirements. > > > > Greetings, > > Stephan > > > > > > On Fri, Jan 8, 2016 at 3:23 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wrote: > > Hi, > > > > Can anyone point me to ant documentation on support for Security in > Flink ? > > > > The type of information I'm looking for are - > > > > 1. How do I do user level authentication to ensure that a job is > submitted/deleted/modified by the right user ? Is it possible though the > web client ? > > 2. Authentication across multiple slave nodes (where the task managers > are running) and driver program so that they can communicate with each other > > 3. Support for SSL/encryption for data exchanged happening across the > slave nodes > > 4. Support for pluggable authentication with existing solution like LDAP > > > > If not there today is there a roadmap for these security features ? > > > > Regards, > > Sourav > > > > > > > > > > > > -- > > Welly Tambunan > > Triplelands > > > > http://weltam.wordpress.com > > http://www.triplelands.com > >
