Re: Data Transfer between TM should be encrypted
patil <[hidden email]> wrote: >>>>>> >>>>>> Hi Stephan, >>>>>> >>>>>> Thank you for your reply. >>>>>> >>>>>> Till when can I expect this feature to be integrated in master or >>>>>> release version ? >>>>>> >>>>>> We are going to get production data (financial data) in October end , >>>>>> so want to have this feature before that. >>>>>> >>>>>> Regards, >>>>>> Vinay Patil >>>>>> >>>>>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User >>>>>> Mailing List archive.] <[hidden email]> wrote: >>>>>> >>>>>> Hi! >>>>>> >>>>>> The way that the JIRA issue you linked will achieve this is by >>>>>> hooking into the network stream pipeline directly, and encrypt the raw >>>>>> network byte stream. We built the network stack on Netty, and will use >>>>>> Netty's SSL/TLS handlers for that. >>>>>> >>>>>> That should be much more efficient than manual encryption/decryption >>>>>> in each user function. >>>>>> >>>>>> Stephan >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote: >>>>>> >>>>>> Hi Ufuk, >>>>>> >>>>>> This is regarding this issue >>>>>> https://issues.apache.org/jira /browse/FLINK-4404 >>>>>> <https://issues.apache.org/jira/browse/FLINK-4404> >>>>>> >>>>>> How can we achieve this, I am able to decrypt the data from Kafka >>>>>> coming in, but I want to make sure that the data is encrypted when >>>>>> flowing >>>>>> between TM's. >>>>>> >>>>>> One approach I can think of is to decrypt the data at the start of >>>>>> each operator and encrypt it at the end of each operator, but I feel this >>>>>> is not an efficient approach. >>>>>> >>>>>> I just want to check if there are alternatives to this and can this >>>>>> be achieved by doing some configurations. >>>>>> >>>>>> Regards, >>>>>> Vinay Patil >>>>>> >>>>>> -- >>>>>> View this message in context: Data Transfer between TM should be >>>>>> encrypted >>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html> >>>>>> Sent from the Apache Flink User Mailing List archive. mailing list >>>>>> archive >>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> >>>>>> at Nabble.com. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -------------- >>>>>> If you reply to this email, your message will be added to the >>>>>> discussion below: >>>>>> http://apache-flink-user-maili ng-list-archive.2336050.n4. >>>>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- >>>>>> tp8781p8782.html >>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html> >>>>>> To start a new topic under Apache Flink User Mailing List archive., >>>>>> email [hidden email] >>>>>> To unsubscribe from Apache Flink User Mailing List archive., click >>>>>> here. >>>>>> NAML >>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer=instant_html%21nabble%3Aemail.naml=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>>>>> >>>>>> >>>>>> >>>
Re: Data Transfer between TM should be encrypted
I agree with Robert. We should probably aim for the release to be end of November, so that it will come this year even in the presence of some delay. Best, Stephan On Thu, Oct 13, 2016 at 11:36 AM, Robert Metzger <rmetz...@apache.org> wrote: > Hi, > the release dates depend on the community, when features are ready and so > on. > There was no discussion yet when we plan to do the release, because most > of the features we want to have in are not yet done yet. > I think its likely that we'll have a 1.2 release by end of this year. > > Regards, > Robert > > On Thu, Oct 13, 2016 at 5:07 AM, vinay patil <vinay18.pa...@gmail.com> > wrote: > >> Hi Robert, >> >> Thank you for that information, can you please let me know when 1.2 is >> planned to release ? >> >> Regards, >> Vinay Patil >> >> On Wed, Oct 12, 2016 at 4:17 AM, rmetzger0 [via Apache Flink User Mailing >> List archive.] <[hidden email] >> <http:///user/SendEmail.jtp?type=node=9508=0>> wrote: >> >>> Hi, >>> I think that pull request will be merged for 1.2. >>> >>> On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <[hidden email] >>> <http:///user/SendEmail.jtp?type=node=9473=0>> wrote: >>> >>>> Hi Stephan, >>>> >>>> https://github.com/apache/flink/pull/2518 >>>> Is this pull request going to be part of 1.2 release ? Just wanted to >>>> get an idea on timelines so that I can pass on to the team. >>>> >>>> >>>> Regards, >>>> Vinay Patil >>>> >>>> On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden >>>> email] <http:///user/SendEmail.jtp?type=node=9392=0>> wrote: >>>> >>>>> Hi Vinay, >>>>> >>>>> There are some delays and we expect the PR to be created next week. >>>>> >>>>> Regards >>>>> Vijay >>>>> >>>>> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email] >>>>> <http:///user/SendEmail.jtp?type=node=9392=1>> wrote: >>>>> >>>>> >>>>> Hi Vijay, >>>>> >>>>> Did you raise the PR for this task, I don't mind testing it out as >>>>> well. >>>>> >>>>> Regards, >>>>> Vinay Patil >>>>> >>>>> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote: >>>>> >>>>> Hi Vijay, >>>>> >>>>> That's a good news for me. Eagerly waiting for this change so that I >>>>> can integrate and test it before going live. >>>>> >>>>> Regards, >>>>> Vinay Patil >>>>> >>>>> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache >>>>> Flink User Mailing List archive.] <[hidden email]> wrote: >>>>> >>>>> Hi Stephan, >>>>> >>>>> The dev work is almost complete except the Yarn mode deployment stuff >>>>> that needs to be patched. We are expecting to send a PR in a week or two. >>>>> >>>>> Regards >>>>> Vijay >>>>> >>>>> >>>>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]> >>>>> wrote: >>>>> >>>>> >>>>> Let me loop in Vijay, I think he is the one working on this and can >>>>> probably give the best estimate when it can be expected. >>>>> >>>>> @vijay: For the SSL/TLS transport encryption - do you have an estimate >>>>> for the timeline of that feature? >>>>> >>>>> >>>>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote: >>>>> >>>>> Hi Stephan, >>>>> >>>>> Thank you for your reply. >>>>> >>>>> Till when can I expect this feature to be integrated in master or >>>>> release version ? >>>>> >>>>> We are going to get production data (financial data) in October end , >>>>> so want to have this feature before that. >>>>> >>>>> Regards, >>>>> Vinay Patil >>>>> >>>>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User >>>>> Mailing List archive.] <[hidden email]> wrot
Re: Data Transfer between TM should be encrypted
Hi, the release dates depend on the community, when features are ready and so on. There was no discussion yet when we plan to do the release, because most of the features we want to have in are not yet done yet. I think its likely that we'll have a 1.2 release by end of this year. Regards, Robert On Thu, Oct 13, 2016 at 5:07 AM, vinay patil <vinay18.pa...@gmail.com> wrote: > Hi Robert, > > Thank you for that information, can you please let me know when 1.2 is > planned to release ? > > Regards, > Vinay Patil > > On Wed, Oct 12, 2016 at 4:17 AM, rmetzger0 [via Apache Flink User Mailing > List archive.] <[hidden email] > <http:///user/SendEmail.jtp?type=node=9508=0>> wrote: > >> Hi, >> I think that pull request will be merged for 1.2. >> >> On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <[hidden email] >> <http:///user/SendEmail.jtp?type=node=9473=0>> wrote: >> >>> Hi Stephan, >>> >>> https://github.com/apache/flink/pull/2518 >>> Is this pull request going to be part of 1.2 release ? Just wanted to >>> get an idea on timelines so that I can pass on to the team. >>> >>> >>> Regards, >>> Vinay Patil >>> >>> On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden >>> email] <http:///user/SendEmail.jtp?type=node=9392=0>> wrote: >>> >>>> Hi Vinay, >>>> >>>> There are some delays and we expect the PR to be created next week. >>>> >>>> Regards >>>> Vijay >>>> >>>> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email] >>>> <http:///user/SendEmail.jtp?type=node=9392=1>> wrote: >>>> >>>> >>>> Hi Vijay, >>>> >>>> Did you raise the PR for this task, I don't mind testing it out as well. >>>> >>>> Regards, >>>> Vinay Patil >>>> >>>> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote: >>>> >>>> Hi Vijay, >>>> >>>> That's a good news for me. Eagerly waiting for this change so that I >>>> can integrate and test it before going live. >>>> >>>> Regards, >>>> Vinay Patil >>>> >>>> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache >>>> Flink User Mailing List archive.] <[hidden email]> wrote: >>>> >>>> Hi Stephan, >>>> >>>> The dev work is almost complete except the Yarn mode deployment stuff >>>> that needs to be patched. We are expecting to send a PR in a week or two. >>>> >>>> Regards >>>> Vijay >>>> >>>> >>>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]> >>>> wrote: >>>> >>>> >>>> Let me loop in Vijay, I think he is the one working on this and can >>>> probably give the best estimate when it can be expected. >>>> >>>> @vijay: For the SSL/TLS transport encryption - do you have an estimate >>>> for the timeline of that feature? >>>> >>>> >>>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote: >>>> >>>> Hi Stephan, >>>> >>>> Thank you for your reply. >>>> >>>> Till when can I expect this feature to be integrated in master or >>>> release version ? >>>> >>>> We are going to get production data (financial data) in October end , >>>> so want to have this feature before that. >>>> >>>> Regards, >>>> Vinay Patil >>>> >>>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User >>>> Mailing List archive.] <[hidden email]> wrote: >>>> >>>> Hi! >>>> >>>> The way that the JIRA issue you linked will achieve this is by hooking >>>> into the network stream pipeline directly, and encrypt the raw network byte >>>> stream. We built the network stack on Netty, and will use Netty's SSL/TLS >>>> handlers for that. >>>> >>>> That should be much more efficient than manual encryption/decryption in >>>> each user function. >>>> >>>> Stephan >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]>
Re: Data Transfer between TM should be encrypted
Hi, I think that pull request will be merged for 1.2. On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <vinay18.pa...@gmail.com> wrote: > Hi Stephan, > > https://github.com/apache/flink/pull/2518 > Is this pull request going to be part of 1.2 release ? Just wanted to get > an idea on timelines so that I can pass on to the team. > > > Regards, > Vinay Patil > > On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden email] > <http:///user/SendEmail.jtp?type=node=9392=0>> wrote: > >> Hi Vinay, >> >> There are some delays and we expect the PR to be created next week. >> >> Regards >> Vijay >> >> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email] >> <http:///user/SendEmail.jtp?type=node=9392=1>> wrote: >> >> >> Hi Vijay, >> >> Did you raise the PR for this task, I don't mind testing it out as well. >> >> Regards, >> Vinay Patil >> >> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote: >> >> Hi Vijay, >> >> That's a good news for me. Eagerly waiting for this change so that I can >> integrate and test it before going live. >> >> Regards, >> Vinay Patil >> >> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache >> Flink User Mailing List archive.] <[hidden email]> wrote: >> >> Hi Stephan, >> >> The dev work is almost complete except the Yarn mode deployment stuff >> that needs to be patched. We are expecting to send a PR in a week or two. >> >> Regards >> Vijay >> >> >> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]> >> wrote: >> >> >> Let me loop in Vijay, I think he is the one working on this and can >> probably give the best estimate when it can be expected. >> >> @vijay: For the SSL/TLS transport encryption - do you have an estimate >> for the timeline of that feature? >> >> >> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote: >> >> Hi Stephan, >> >> Thank you for your reply. >> >> Till when can I expect this feature to be integrated in master or release >> version ? >> >> We are going to get production data (financial data) in October end , so >> want to have this feature before that. >> >> Regards, >> Vinay Patil >> >> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User >> Mailing List archive.] <[hidden email]> wrote: >> >> Hi! >> >> The way that the JIRA issue you linked will achieve this is by hooking >> into the network stream pipeline directly, and encrypt the raw network byte >> stream. We built the network stack on Netty, and will use Netty's SSL/TLS >> handlers for that. >> >> That should be much more efficient than manual encryption/decryption in >> each user function. >> >> Stephan >> >> >> >> >> >> >> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote: >> >> Hi Ufuk, >> >> This is regarding this issue >> https://issues.apache.org/jira /browse/FLINK-4404 >> <https://issues.apache.org/jira/browse/FLINK-4404> >> >> How can we achieve this, I am able to decrypt the data from Kafka coming >> in, but I want to make sure that the data is encrypted when flowing between >> TM's. >> >> One approach I can think of is to decrypt the data at the start of each >> operator and encrypt it at the end of each operator, but I feel this is not >> an efficient approach. >> >> I just want to check if there are alternatives to this and can this be >> achieved by doing some configurations. >> >> Regards, >> Vinay Patil >> >> -------------- >> View this message in context: Data Transfer between TM should be >> encrypted >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html> >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> >> at Nabble.com. >> >> >> >> >> -- >> If you reply to this email, your message will be added to the discussion >> below: >> http://apache-flink-user-maili ng-list-archive.2336050.n4. >> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html >> <http://apac
Re: Data Transfer between TM should be encrypted
Hi Stephan, https://github.com/apache/flink/pull/2518 Is this pull request going to be part of 1.2 release ? Just wanted to get an idea on timelines so that I can pass on to the team. Regards, Vinay Patil On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan < vijikar...@yahoo.com> wrote: > Hi Vinay, > > There are some delays and we expect the PR to be created next week. > > Regards > Vijay > > On Wednesday, September 14, 2016 5:41 PM, vinay patil < > vinay18.pa...@gmail.com> wrote: > > > Hi Vijay, > > Did you raise the PR for this task, I don't mind testing it out as well. > > Regards, > Vinay Patil > > On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote: > > Hi Vijay, > > That's a good news for me. Eagerly waiting for this change so that I can > integrate and test it before going live. > > Regards, > Vinay Patil > > On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache Flink > User Mailing List archive.] <[hidden email]> wrote: > > Hi Stephan, > > The dev work is almost complete except the Yarn mode deployment stuff that > needs to be patched. We are expecting to send a PR in a week or two. > > Regards > Vijay > > > On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]> wrote: > > > Let me loop in Vijay, I think he is the one working on this and can > probably give the best estimate when it can be expected. > > @vijay: For the SSL/TLS transport encryption - do you have an estimate for > the timeline of that feature? > > > On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote: > > Hi Stephan, > > Thank you for your reply. > > Till when can I expect this feature to be integrated in master or release > version ? > > We are going to get production data (financial data) in October end , so > want to have this feature before that. > > Regards, > Vinay Patil > > On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User > Mailing List archive.] <[hidden email]> wrote: > > Hi! > > The way that the JIRA issue you linked will achieve this is by hooking > into the network stream pipeline directly, and encrypt the raw network byte > stream. We built the network stack on Netty, and will use Netty's SSL/TLS > handlers for that. > > That should be much more efficient than manual encryption/decryption in > each user function. > > Stephan > > > > > > > On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote: > > Hi Ufuk, > > This is regarding this issue > https://issues.apache.org/jira /browse/FLINK-4404 > <https://issues.apache.org/jira/browse/FLINK-4404> > > How can we achieve this, I am able to decrypt the data from Kafka coming > in, but I want to make sure that the data is encrypted when flowing between > TM's. > > One approach I can think of is to decrypt the data at the start of each > operator and encrypt it at the end of each operator, but I feel this is not > an efficient approach. > > I just want to check if there are alternatives to this and can this be > achieved by doing some configurations. > > Regards, > Vinay Patil > > -- > View this message in context: Data Transfer between TM should be encrypted > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html> > Sent from the Apache Flink User Mailing List archive. mailing list archive > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at > Nabble.com. > > > > > -- > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-maili ng-list-archive.2336050.n4. > nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html> > To start a new topic under Apache Flink User Mailing List archive., email > [hidden > email] > To unsubscribe from Apache Flink User Mailing List archive., click here. > NAML > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer=instant_html%21nabble%3Aemail.naml=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > >
Re: Data Transfer between TM should be encrypted
Hi Vinay, There are some delays and we expect the PR to be created next week. RegardsVijay On Wednesday, September 14, 2016 5:41 PM, vinay patil <vinay18.pa...@gmail.com> wrote: Hi Vijay, Did you raise the PR for this task, I don't mind testing it out as well. Regards,Vinay Patil On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote: Hi Vijay, That's a good news for me. Eagerly waiting for this change so that I can integrate and test it before going live. Regards,Vinay Patil On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache Flink User Mailing List archive.] <[hidden email]> wrote: Hi Stephan, The dev work is almost complete except the Yarn mode deployment stuff that needs to be patched. We are expecting to send a PR in a week or two. RegardsVijay On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]> wrote: Let me loop in Vijay, I think he is the one working on this and can probably give the best estimate when it can be expected. @vijay: For the SSL/TLS transport encryption - do you have an estimate for the timeline of that feature? On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote: Hi Stephan, Thank you for your reply. Till when can I expect this feature to be integrated in master or release version ? We are going to get production data (financial data) in October end , so want to have this feature before that. Regards,Vinay Patil On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User Mailing List archive.] <[hidden email]> wrote: Hi! The way that the JIRA issue you linked will achieve this is by hooking into the network stream pipeline directly, and encrypt the raw network byte stream. We built the network stack on Netty, and will use Netty's SSL/TLS handlers for that. That should be much more efficient than manual encryption/decryption in each user function. Stephan On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote: Hi Ufuk, This is regarding this issuehttps://issues.apache.org/jira /browse/FLINK-4404 How can we achieve this, I am able to decrypt the data from Kafka coming in, but I want to make sure that the data is encrypted when flowing between TM's. One approach I can think of is to decrypt the data at the start of each operator and encrypt it at the end of each operator, but I feel this is not an efficient approach. I just want to check if there are alternatives to this and can this be achieved by doing some configurations. Regards,Vinay Patil View this message in context: Data Transfer between TM should be encrypted Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com. If you reply to this email, your message will be added to the discussion below: http://apache-flink-user-maili ng-list-archive.2336050.n4. nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html To start a new topic under Apache Flink User Mailing List archive., email [hidden email] To unsubscribe from Apache Flink User Mailing List archive., click here. NAML View this message in context: Re: Data Transfer between TM should be encrypted Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com. If you reply to this email, your message will be added to the discussion below: http://apache-flink-user-maili ng-list-archive.2336050.n4. nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html To start a new topic under Apache Flink User Mailing List archive., email [hidden email] To unsubscribe from Apache Flink User Mailing List archive., click here. NAML View this message in context: Re: Data Transfer between TM should be encrypted Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Data Transfer between TM should be encrypted
Hi Vijay, Did you raise the PR for this task, I don't mind testing it out as well. Regards, Vinay Patil On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <vinay18.pa...@gmail.com> wrote: > Hi Vijay, > > That's a good news for me. Eagerly waiting for this change so that I can > integrate and test it before going live. > > Regards, > Vinay Patil > > On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache Flink > User Mailing List archive.] <ml-node+s2336050n8801...@n4.nabble.com> > wrote: > >> Hi Stephan, >> >> The dev work is almost complete except the Yarn mode deployment stuff >> that needs to be patched. We are expecting to send a PR in a week or two. >> >> Regards >> Vijay >> >> >> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email] >> <http:///user/SendEmail.jtp?type=node=8801=0>> wrote: >> >> >> Let me loop in Vijay, I think he is the one working on this and can >> probably give the best estimate when it can be expected. >> >> @vijay: For the SSL/TLS transport encryption - do you have an estimate >> for the timeline of that feature? >> >> >> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email] >> <http:///user/SendEmail.jtp?type=node=8801=1>> wrote: >> >> Hi Stephan, >> >> Thank you for your reply. >> >> Till when can I expect this feature to be integrated in master or release >> version ? >> >> We are going to get production data (financial data) in October end , so >> want to have this feature before that. >> >> Regards, >> Vinay Patil >> >> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User >> Mailing List archive.] <[hidden email]> wrote: >> >> Hi! >> >> The way that the JIRA issue you linked will achieve this is by hooking >> into the network stream pipeline directly, and encrypt the raw network byte >> stream. We built the network stack on Netty, and will use Netty's SSL/TLS >> handlers for that. >> >> That should be much more efficient than manual encryption/decryption in >> each user function. >> >> Stephan >> >> >> >> >> >> >> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote: >> >> Hi Ufuk, >> >> This is regarding this issue >> https://issues.apache.org/jira /browse/FLINK-4404 >> <https://issues.apache.org/jira/browse/FLINK-4404> >> >> How can we achieve this, I am able to decrypt the data from Kafka coming >> in, but I want to make sure that the data is encrypted when flowing between >> TM's. >> >> One approach I can think of is to decrypt the data at the start of each >> operator and encrypt it at the end of each operator, but I feel this is not >> an efficient approach. >> >> I just want to check if there are alternatives to this and can this be >> achieved by doing some configurations. >> >> Regards, >> Vinay Patil >> >> -- >> View this message in context: Data Transfer between TM should be >> encrypted >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html> >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> >> at Nabble.com. >> >> >> >> >> -- >> If you reply to this email, your message will be added to the discussion >> below: >> http://apache-flink-user-maili ng-list-archive.2336050.n4. >> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html> >> To start a new topic under Apache Flink User Mailing List archive., email >> [hidden >> email] >> To unsubscribe from Apache Flink User Mailing List archive., click here. >> NAML >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer=instant_html%21nabble%3Aemail.naml=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> >> >> >>
Re: Data Transfer between TM should be encrypted
Hi Vijay, That's a good news for me. Eagerly waiting for this change so that I can integrate and test it before going live. Regards, Vinay Patil On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache Flink User Mailing List archive.] <ml-node+s2336050n8801...@n4.nabble.com> wrote: > Hi Stephan, > > The dev work is almost complete except the Yarn mode deployment stuff that > needs to be patched. We are expecting to send a PR in a week or two. > > Regards > Vijay > > > On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email] > <http:///user/SendEmail.jtp?type=node=8801=0>> wrote: > > > Let me loop in Vijay, I think he is the one working on this and can > probably give the best estimate when it can be expected. > > @vijay: For the SSL/TLS transport encryption - do you have an estimate for > the timeline of that feature? > > > On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email] > <http:///user/SendEmail.jtp?type=node=8801=1>> wrote: > > Hi Stephan, > > Thank you for your reply. > > Till when can I expect this feature to be integrated in master or release > version ? > > We are going to get production data (financial data) in October end , so > want to have this feature before that. > > Regards, > Vinay Patil > > On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User > Mailing List archive.] <[hidden email]> wrote: > > Hi! > > The way that the JIRA issue you linked will achieve this is by hooking > into the network stream pipeline directly, and encrypt the raw network byte > stream. We built the network stack on Netty, and will use Netty's SSL/TLS > handlers for that. > > That should be much more efficient than manual encryption/decryption in > each user function. > > Stephan > > > > > > > On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote: > > Hi Ufuk, > > This is regarding this issue > https://issues.apache.org/jira /browse/FLINK-4404 > <https://issues.apache.org/jira/browse/FLINK-4404> > > How can we achieve this, I am able to decrypt the data from Kafka coming > in, but I want to make sure that the data is encrypted when flowing between > TM's. > > One approach I can think of is to decrypt the data at the start of each > operator and encrypt it at the end of each operator, but I feel this is not > an efficient approach. > > I just want to check if there are alternatives to this and can this be > achieved by doing some configurations. > > Regards, > Vinay Patil > > -- > View this message in context: Data Transfer between TM should be encrypted > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html> > Sent from the Apache Flink User Mailing List archive. mailing list archive > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at > Nabble.com. > > > > > -------------- > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-maili ng-list-archive.2336050.n4. > nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html> > To start a new topic under Apache Flink User Mailing List archive., email > [hidden > email] > To unsubscribe from Apache Flink User Mailing List archive., click here. > NAML > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer=instant_html%21nabble%3Aemail.naml=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > -- > View this message in context: Re: Data Transfer between TM should be > encrypted > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html> > > Sent from the Apache Flink User Mailing List archive. mailing list archive > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at > Nabble.com. > > > > > > > -- > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-mailing-list-archive.2336050. > n4.nabble
Re: Data Transfer between TM should be encrypted
Hi Stephan, The dev work is almost complete except the Yarn mode deployment stuff that needs to be patched. We are expecting to send a PR in a week or two. RegardsVijay On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <se...@apache.org> wrote: Let me loop in Vijay, I think he is the one working on this and can probably give the best estimate when it can be expected. @vijay: For the SSL/TLS transport encryption - do you have an estimate for the timeline of that feature? On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <vinay18.pa...@gmail.com> wrote: Hi Stephan, Thank you for your reply. Till when can I expect this feature to be integrated in master or release version ? We are going to get production data (financial data) in October end , so want to have this feature before that. Regards,Vinay Patil On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User Mailing List archive.] <[hidden email]> wrote: Hi! The way that the JIRA issue you linked will achieve this is by hooking into the network stream pipeline directly, and encrypt the raw network byte stream. We built the network stack on Netty, and will use Netty's SSL/TLS handlers for that. That should be much more efficient than manual encryption/decryption in each user function. Stephan On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote: Hi Ufuk, This is regarding this issuehttps://issues.apache.org/jira /browse/FLINK-4404 How can we achieve this, I am able to decrypt the data from Kafka coming in, but I want to make sure that the data is encrypted when flowing between TM's. One approach I can think of is to decrypt the data at the start of each operator and encrypt it at the end of each operator, but I feel this is not an efficient approach. I just want to check if there are alternatives to this and can this be achieved by doing some configurations. Regards,Vinay Patil View this message in context: Data Transfer between TM should be encrypted Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com. If you reply to this email, your message will be added to the discussion below: http://apache-flink-user-maili ng-list-archive.2336050.n4. nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html To start a new topic under Apache Flink User Mailing List archive., email [hidden email] To unsubscribe from Apache Flink User Mailing List archive., click here. NAML View this message in context: Re: Data Transfer between TM should be encrypted Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Data Transfer between TM should be encrypted
Let me loop in Vijay, I think he is the one working on this and can probably give the best estimate when it can be expected. @vijay: For the SSL/TLS transport encryption - do you have an estimate for the timeline of that feature? On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <vinay18.pa...@gmail.com> wrote: > Hi Stephan, > > Thank you for your reply. > > Till when can I expect this feature to be integrated in master or release > version ? > > We are going to get production data (financial data) in October end , so > want to have this feature before that. > > Regards, > Vinay Patil > > On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User > Mailing List archive.] <[hidden email] > <http:///user/SendEmail.jtp?type=node=8786=0>> wrote: > >> Hi! >> >> The way that the JIRA issue you linked will achieve this is by hooking >> into the network stream pipeline directly, and encrypt the raw network byte >> stream. We built the network stack on Netty, and will use Netty's SSL/TLS >> handlers for that. >> >> That should be much more efficient than manual encryption/decryption in >> each user function. >> >> Stephan >> >> >> >> >> >> >> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email] >> <http:///user/SendEmail.jtp?type=node=8782=0>> wrote: >> >>> Hi Ufuk, >>> >>> This is regarding this issue >>> https://issues.apache.org/jira/browse/FLINK-4404 >>> >>> How can we achieve this, I am able to decrypt the data from Kafka coming >>> in, but I want to make sure that the data is encrypted when flowing between >>> TM's. >>> >>> One approach I can think of is to decrypt the data at the start of each >>> operator and encrypt it at the end of each operator, but I feel this is not >>> an efficient approach. >>> >>> I just want to check if there are alternatives to this and can this be >>> achieved by doing some configurations. >>> >>> Regards, >>> Vinay Patil >>> >>> -- >>> View this message in context: Data Transfer between TM should be >>> encrypted >>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html> >>> Sent from the Apache Flink User Mailing List archive. mailing list >>> archive >>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> >>> at Nabble.com. >>> >> >> >> >> -- >> If you reply to this email, your message will be added to the discussion >> below: >> http://apache-flink-user-mailing-list-archive.2336050.n4. >> nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html >> To start a new topic under Apache Flink User Mailing List archive., email >> [hidden >> email] <http:///user/SendEmail.jtp?type=node=8786=1> >> To unsubscribe from Apache Flink User Mailing List archive., click here. >> NAML >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer=instant_html%21nabble%3Aemail.naml=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > > -- > View this message in context: Re: Data Transfer between TM should be > encrypted > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html> > > Sent from the Apache Flink User Mailing List archive. mailing list archive > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at > Nabble.com. >
Data Transfer between TM should be encrypted
Hi Ufuk, This is regarding this issue https://issues.apache.org/jira/browse/FLINK-4404 How can we achieve this, I am able to decrypt the data from Kafka coming in, but I want to make sure that the data is encrypted when flowing between TM's. One approach I can think of is to decrypt the data at the start of each operator and encrypt it at the end of each operator, but I feel this is not an efficient approach. I just want to check if there are alternatives to this and can this be achieved by doing some configurations. Regards, Vinay Patil -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
