I believe you need to co-locate your Zeppelin on the same node where Spark is installed. You need to specify the SPARK HOME. The master I used was YARN.
Zeppelin exposes a notebook interface. A notebook can have many paragraphs. You run the paragraphs. You can mix multiple contexts in the same notebook. So first paragraph can be scala, second can be sql that uses DF from first paragraph etc. If you use a select query, the output is automatically displayed as a chart. As RDDs are bound to the context that creates them, I don't think Zeppelin can use those RDDs. I don't know if notebooks can be reused within other notebooks. It would be a nice way of doing some common preparatory work (like building these RDDs). Regards Sab On Thu, Mar 10, 2016 at 2:28 PM, ayan guha <guha.a...@gmail.com> wrote: > Hi All > > I am writing this in order to get a fair understanding of how zeppelin can > be integrated with Spark. > > Our use case is to load few tables from a DB to Spark, run some > transformation. Once done, we want to expose data through Zeppelin for > analytics. I have few question around that to sound off any gross > architectural flaws. > > Questions: > > 1. How Zeppelin connects to Spark? Thriftserver? Thrift JDBC? > > 2. What is the scope of Spark application when it is used from Zeppelin? > For example, if I have few subsequent actions in zeppelin like > map,filter,reduceByKey, filter,collect. I assume this will translate to an > application and get submitted to Spark. However, If I want to use reuse > some part of the data (for example) after first map transformation in > earlier application. Can I do it? Or will it be another application and > another spark submit? > > In our use case data will already be loaded in RDDs. So how Zeppelin can > access it? > > 3. How can I control access on specific rdds to specific users in Zeppelin > (assuming we have implemented some way of login mechanism in Zeppelin and > we have a mapping between Zeppelin users and their LDAP accounts). Is it > even possible? > > 4. If Zeppelin is not a good choice, yet, for the use case, what are the > other alternatives? > > appreciate any help/pointers/guidance. > > > -- > Best Regards, > Ayan Guha > -- Architect - Big Data Ph: +91 99805 99458 Manthan Systems | *Company of the year - Analytics (2014 Frost and Sullivan India ICT)* +++
