Hi Welly, There is no fixed timeline yet but we plan to make progress in terms of authentication and encryption after the 1.0.0 release.
Cheers, Max On Wed, Jan 13, 2016 at 8:34 AM, Welly Tambunan <if05...@gmail.com> wrote: > Hi Stephan, > > Thanks a lot for the explanation. > > Is there any timeline on when this will be released ? I guess this one will > be the important for our case if we want Flink to be deployed in production. > > Cheers > > On Tue, Jan 12, 2016 at 6:19 PM, Stephan Ewen <se...@apache.org> wrote: >> >> Hi Sourav! >> >> If you want to use Flink in a cluster where neither Hadoop/YARN (not soon >> Mesos) is available, then I assume you have installed Flink in a standalone >> mode on the cluster already. >> >> There is no support in Flink currently to manage user authentication. Few >> thoughts on how that may evolve >> >> 1) It should be not too hard to add authentication to the web dashboard. >> That way, if the cluster is otherwise blocked off (the master's RPC ports >> are firewalled), one would have restricted job starts. >> >> 2) We plan to add authenticated / encrypted connections soon. With that, >> the client that submits the program would need to have access to the >> keystore or key and the corresponding password to connect. >> >> Greetings, >> Stephan >> >> >> >> On Mon, Jan 11, 2016 at 3:46 PM, Sourav Mazumder >> <sourav.mazumde...@gmail.com> wrote: >>> >>> Thanks Steven for your details response. Things are more clear to me now. >>> >>> A follow up Qs - >>> Looks like most of the security support depends on Hadoop ? What happens >>> if anyone wants to use Flink with Hadoop (in a cluster where Hadoop is not >>> there) ? >>> >>> Regards, >>> Sourav >>> >>> On Sun, Jan 10, 2016 at 12:41 PM, Stephan Ewen <se...@apache.org> wrote: >>>> >>>> Hi Sourav! >>>> >>>> There is user-authentication support in Flink via the Hadoop / Kerberos >>>> infrastructure. If you run Flink on YARN, it should seamlessly work that >>>> Flink acquires the Kerberos tokens of the user that submits programs, and >>>> authenticate itself at YARN, HDFS, and HBase with that. >>>> >>>> If you run Flink standalone, Flink can still authenticate at HDFS/HBase >>>> via Kerberos, with a bit of manual help by the user (running kinit on the >>>> workers). >>>> >>>> With Kafka 0.9 and Flink's upcoming connector >>>> (https://github.com/apache/flink/pull/1489), streaming programs can >>>> authenticate themselves as stream brokers via SSL (and read via encrypted >>>> connections). >>>> >>>> >>>> What we have on the roadmap for the coming months it the following: >>>> - Encrypt in-flight data streams that are exchanged between worker >>>> nodes (TaskManagers). >>>> - Encrypt the coordination messages between client/master/workers. >>>> Note that these refer to encryption between Flink's own components only, >>>> which would use transient keys generated just for a specific job or session >>>> (hence would not need any user involvement). >>>> >>>> >>>> Let us know if that answers your questions, and if that meets your >>>> requirements. >>>> >>>> Greetings, >>>> Stephan >>>> >>>> >>>> On Fri, Jan 8, 2016 at 3:23 PM, Sourav Mazumder >>>> <sourav.mazumde...@gmail.com> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Can anyone point me to ant documentation on support for Security in >>>>> Flink ? >>>>> >>>>> The type of information I'm looking for are - >>>>> >>>>> 1. How do I do user level authentication to ensure that a job is >>>>> submitted/deleted/modified by the right user ? Is it possible though the >>>>> web >>>>> client ? >>>>> 2. Authentication across multiple slave nodes (where the task managers >>>>> are running) and driver program so that they can communicate with each >>>>> other >>>>> 3. Support for SSL/encryption for data exchanged happening across the >>>>> slave nodes >>>>> 4. Support for pluggable authentication with existing solution like >>>>> LDAP >>>>> >>>>> If not there today is there a roadmap for these security features ? >>>>> >>>>> Regards, >>>>> Sourav >>>> >>>> >>> >> > > > > -- > Welly Tambunan > Triplelands > > http://weltam.wordpress.com > http://www.triplelands.com
