Hi, we are using Spark standalone cluster with Dnsmasq for Spark executors to be able to resolve container hostnames and communicate back to Driver (run by Zeppelin).
On Thu, Apr 2, 2015 at 10:23 AM, RJ Nowling <rnowl...@gmail.com> wrote: > Are you using Spark standalone, YARN, or Mesos? > > I'm having issues with Zeppelin in a Docker container submitting jobs to > Mesos. Zeppelin works fine with Mesos when not in a Docker container. > > Thanks! > > > On Apr 1, 2015, at 8:17 PM, Alex B. <abezzu...@nflabs.com> wrote: > > Thank you for keeping us posted! > > We have recently successfully used this kind of approach: run a separate > Zeppelin instances per-user in Docker containers and have a single Nginx > that does reverse-proxy and url re-write (to different ports on the same > host machine) + a cookie-based auth. > > Alex. > > On Thu, Apr 2, 2015 at 4:31 AM, RJ Nowling <rnowl...@gmail.com> wrote: > >> Wanted to follow up again. The 2 docker container approach is proving to >> be a bit unstable and I'm running into problems integration with Mesos. >> >> I'm going to try a different approach using iptables: >> >> http://unix.stackexchange.com/questions/111906/allowing-users-to-access-certain-ports-on-server >> >> I'll try doing the following: >> * Run a zeppelin instance under each user on separate ports >> * Limit the access to the ports to that user >> * Ask users to use SSH for port forwarding >> >> On Tue, Mar 31, 2015 at 11:43 AM, RJ Nowling <rnowl...@gmail.com> wrote: >> >>> Hi all, >>> >>> I was able to get nginx working for basic authentication. I used docker >>> containers for private networking so that the zeppelin ports aren't open to >>> the public. >>> >>> https://github.com/rnowling/zeppelin-authentication >>> >>> The problem right now is that nginx requires authenticating twice -- for >>> the two ports, I'm assuming. >>> >>> If anyone has ideas for how to fix that, please let me know! >>> >>> RJ >>> >>> On Mon, Mar 30, 2015 at 9:03 AM, Ram Venkatesh < >>> rvenkat...@hortonworks.com> wrote: >>> >>>> I agree - have also been thinking about the same problem in the >>>> context of running zeppelin against a secure kerberized hadoop cluster. >>>> >>>> There are a couple concerns here: >>>> 1. authentication, both for the zeppelin server and for interpreters >>>> that need it >>>> 2. isolating the code for different users from each other - for example >>>> by running an interpreter for each user >>>> 3. sharing interpreters across note evaluations - this is likely an >>>> issue even without multiple users but multiple tabs for the same user. It >>>> is already taken care of by the interpreter scheduler (by returning a FIFO >>>> or parallel scheduler with the specified max concurrency option), is this >>>> correct? >>>> >>>> Thanks! >>>> Ram >>>> >>>> On Mar 30, 2015, at 6:35 AM, RJ Nowling <rnowl...@gmail.com> wrote: >>>> >>>> Since this may be an oft-requested feature, I'll go ahead and create >>>> a JIRA to document the interest. >>>> >>>> On Mon, Mar 30, 2015 at 8:33 AM, Corneau Damien <cornead...@apache.org> >>>> wrote: >>>> >>>>> Actually the branch wasn't merged in the end >>>>> >>>>> On Mon, Mar 30, 2015 at 10:31 PM, RJ Nowling <rnowl...@gmail.com> >>>>> wrote: >>>>> >>>>>> Is there documentation for the simple authentication? >>>>>> >>>>>> We'll be creating separate unix users for each zeppelin instance >>>>>> with appropriate file system permissions. >>>>>> >>>>>> On Fri, Mar 27, 2015 at 9:39 PM, Corneau Damien < >>>>>> cornead...@apache.org> wrote: >>>>>> >>>>>>> If I remember well, we merged some kind of simple authentication in >>>>>>> Zeppelin. But it should be more about accessing the instance than having >>>>>>> multiple users. One way could be launching multiple zeppelin instances, >>>>>>> I >>>>>>> think Kevin is doing something similar. >>>>>>> >>>>>>> One thing to realize however is that it doesnt create separation at >>>>>>> the data layer >>>>>>> On Mar 28, 2015 1:53 AM, "RJ Nowling" <rnowl...@gmail.com> wrote: >>>>>>> >>>>>>>> I'll report back if I'm successful with the approach I've mentioned. >>>>>>>> >>>>>>>> On Fri, Mar 27, 2015 at 11:51 AM, RJ Nowling <rnowl...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> A search of the user list suggests that Zeppelin can support a >>>>>>>>> separate SparkContext for each notebook but the user who tried it ran >>>>>>>>> into >>>>>>>>> issues. >>>>>>>>> >>>>>>>>> On Fri, Mar 27, 2015 at 11:32 AM, Denny Lee <denny.g....@gmail.com >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> I haven't done this myself but was recently starting to do >>>>>>>>>> research on doing this. Perhaps this blog post may be of help: >>>>>>>>>> http://nginx.com/blog/websocket-nginx/ ? >>>>>>>>>> >>>>>>>>>> On Fri, Mar 27, 2015 at 9:24 AM Silvio Fiorito < >>>>>>>>>> silvio.fior...@granturing.com> wrote: >>>>>>>>>> >>>>>>>>>>> I haven’t tried this myself yet but something I’ve been >>>>>>>>>>> thinking as well. Will the nginx reverse proxy support web sockets >>>>>>>>>>> as well? >>>>>>>>>>> >>>>>>>>>>> Ideally we’d have isolated SparkContexts so users aren’t >>>>>>>>>>> trampling over each other. Honestly I think it’d be good to have >>>>>>>>>>> the option >>>>>>>>>>> of starting a new SparkContext per notebook as well or using the >>>>>>>>>>> model >>>>>>>>>>> Databricks has where you “attach” a notebook to a cluster. >>>>>>>>>>> >>>>>>>>>>> From: RJ Nowling >>>>>>>>>>> Reply-To: "users@zeppelin.incubator.apache.org" >>>>>>>>>>> Date: Friday, March 27, 2015 at 12:19 PM >>>>>>>>>>> To: "users@zeppelin.incubator.apache.org" >>>>>>>>>>> Subject: Multi-user approach >>>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I'm looking into ways to support multiple users with >>>>>>>>>>> Zeppelin. I want to provide isolation between users. >>>>>>>>>>> >>>>>>>>>>> I'm considering the following approach: >>>>>>>>>>> * Run Zeppelin under each user's account with its own set of >>>>>>>>>>> ports >>>>>>>>>>> * Use nginx as a reverse proxy for providing authentication >>>>>>>>>>> >>>>>>>>>>> Has anyone done anything similar? Any better alternatives? >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> RJ >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>> >>> >> > > > -- > -- > Kind regards, > Alexander. > > -- -- Kind regards, Alexander.
