This is something I wrote specifically for the challenges that we faced when taking spark ml models to production http://www.tothenew.com/blog/when-you-take-your-machine-learning-models-to-production-for-real-time-predictions/
On Sat, Sep 23, 2017 at 1:33 PM, Jörn Franke <jornfra...@gmail.com> wrote: > As far as I know there is currently no encryption in-memory in Spark. > There are some research projects to create secure enclaves in-memory based > on Intel sgx, but there is still a lot to do in terms of performance and > security objectives. > The more interesting question is why would you need this for your > organization. There are very few scenarios where it could be needed and if > you have attacker’s in the cluster you have anyway other problems. > > On 23. Sep 2017, at 09:41, Irfan Kabli <irfan.kabli...@gmail.com> wrote: > > Dear All, > > We are looking to position MLLib in our organisation for machine learning > tasks and are keen to understand if their are any challenges that you might > have seen with MLLib in production. We will be going with the pure > open-source approach here, rather than using one of the hadoop > distributions out their in the market. > > Furthemore, with a multi-tenant hadoop cluster, and data in memory, would > spark support encrypting the data in memory with DataFrames. > > -- > Best Regards, > Irfan Kabli > >
