Hi Damien - Interesting questions...
I suspect that development environments are quite varying in configuration but for the most part that they are not typical production deployment configurations. With recent focus on the KnoxShell DSL and SDK classes it makes sense to try and determine what the programming model is for the use of those aspects of Knox. However, the question you ask is how Knox fits into the traditional hadoop programming model, environment and flow. If you have anything particular in mind, I would be interested in hearing what you think. Perimeter security is certainly achievable but I guess there are valid questions as to what sort of deployments are generally available for such development. If you need access to the actual data does it push you to development in production-like environments? Again, I'm not sure what you have in mind here but interested to hear more. thanks, --larry On Tue, Mar 14, 2017 at 5:54 PM, Damien Claveau <[email protected]> wrote: > Hi, > > First time emailing the user mailing list. > > We currently use Knox successfully on several Kerberized clusters in > production, > > and mainly use it to integrate with external client applications (such as ETL > and Viz tools), > > We would like to promote and generalize the concept of a single Rest access > point for all services, > > then, in an ideal world, ban access from the outside world to the RPC and > Thrift interfaces of the core hadoop services. > > > The question is ... > > Even if we can deploy binaries, scripts, workflows to hdfs and submit or > schedule them through Knox, > > At the very beginning, the developpers of course have to code apps (say Spark > jobs) > that are designed to run natively inside the cluster (and will use Java > client libs to access the Thrift interfaces). > > How do you deal with that need ? > Do they develop on sandboxed environments or their own laptop without Knox, > and so Knox only applies to the production/target clusters ? > Is the promise of a "Perimeter Level Security" really achievable ? > > Thank you for your feedback. > > Damien Claveau > > France > > > > >
