[GitHub] incubator-zeppelin pull request: ZEPPELIN-773 : Livy interperter

Mon, 11 Apr 2016 00:20:23 -0700 

GitHub user prabhjyotsingh opened a pull request:

    https://github.com/apache/incubator-zeppelin/pull/827

    ZEPPELIN-773 : Livy interperter

    ### What is this PR for?
    As Zeppelin evolves its notebook, for large scale data analysis, multiple 
zeppelin users are expected to use and connect to the same set of data 
repositories within an enterprise. Since Zeppelin notebooks could affect data, 
state and its lineage, it is important to have separation of users, provide 
them with appropriate sandboxes, in addition to capturing the right audit 
details. Further, the IT within the organization would prefer to support fewer 
Zeppelin instances (preferably one) to support its customers. Therefore, the 
objectives of creating a multi-tenant zeppelin are:
    ● Supporting workloads of multiple customers
    ● Supporting multiple LOBs (lines of business), on a single data systems
    ● Support fine grained audits
    As a natural evolution of Zeppelin Authentication and Authorization design, 
partly user awareness in downstream data systems such as Spark/Hive and others, 
is essential to achieve the above stated objectives.
    
    
    ### What type of PR is it?
    Feature
    
    ### Todos
    * [ ] - Test case
    * [ ] - Review Comments
    * [ ] - Documentation
    
    ### What is the Jira issue?
    ZEPPELIN-773
    
    ### How should this be tested?
     - Install Livy by following steps on https://github.com/cloudera/livy
     - Start the Livy server
     - Now by using Zeppelin-Livy interpreter, run any of the spark, pyspark or 
R commands.
    
    ### Screenshots (if appropriate)
    <img width="1436" alt="screen shot 2016-04-11 at 12 41 35 pm" 
src="https://cloud.githubusercontent.com/assets/674497/14419479/b514979c-ffe3-11e5-9dea-df9854d8409c.png";>
    
    
    <img width="1434" alt="screen shot 2016-04-11 at 12 41 59 pm" 
src="https://cloud.githubusercontent.com/assets/674497/14419478/b514922e-ffe3-11e5-9c98-93c5b99de106.png";>
    
    
    <img width="1440" alt="screen shot 2016-04-11 at 12 48 13 pm" 
src="https://cloud.githubusercontent.com/assets/674497/14419480/b515d8c8-ffe3-11e5-8c20-4c988c621f51.png";>
    
    
    ### Questions:
    * Does the licenses files need update? n/a
    * Is there breaking changes for older versions? n/a
    * Does this needs documentation? yes


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/prabhjyotsingh/incubator-zeppelin 
livyInterperter

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-zeppelin/pull/827.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #827
    
----
commit fadc6d92ffabc0dc4cb2079702ad292df3fd070a
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-08T16:29:13Z

    userName to be present in InterpreterContext/RemoteInterpreterContext

commit d928203cf6d69b3c6427aa35f047b25a399b3198
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-08T16:48:37Z

    revert shiri.ini
    if fromMessage.principal.equals("anonymous") then set user as null

commit 320790cf4581ba14dfa6791220818058ba3bc7ba
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-09T09:00:40Z

    fix for CI, missing change signature

commit 57ca5771f968ed93fcc566b24cfa699aaf7fb301
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-16T16:23:32Z

    review change create such class AuthenticationInfo, and pass it into 
InterpreterContext

commit ba91da4121668f5082aa6e0d73a69ecd2b35fa32
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-16T16:23:53Z

    Merge remote-tracking branch 'origin/master' into UserInInterpreterContext

commit 34dcc32035128cc7ecd0b4c2ed8ad7342b034ecd
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-16T16:44:06Z

    instead of null pass "new AuthenticationInfo()"

commit a5a991d2df197d6242d96e2451b4ea0112a7c56e
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-16T18:38:10Z

    check for selenium

commit 95e7c13086de56df000a6cd014fd02d475bd977d
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-16T18:38:55Z

    test for selenium

commit 0709b9c51c6bc7aaa4dafa5c6d52a0eb4530fefe
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-16T18:58:37Z

    moving AuthenticationInfo to org.apache.zeppelin.display.AuthenticationInfo

commit 40534974ae1ce813214d6e13dc5d5558a4815f07
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-17T19:45:52Z

    test livy

commit ace28a8000a8f8a65c66344bccec1b868d9a85be
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-17T21:35:26Z

    working livy

commit d0519d5a02ff5979a37dec9b8c9976db57db400b
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-19T13:05:43Z

    Working livy interpreter with spark, pyspark, R

commit 10311d3bafa2234852e84ca5f75424646c909a4e
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-20T06:43:00Z

    This works in all cases

commit 07f0846d789213200ddedd54352c35d3e446902a
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-20T18:51:35Z

    with lspark sql

commit 8f4ec479d8fbbbc938e9bbfa8eb3bf6be6ed9fe5
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-20T18:54:08Z

    removing unrequired logs

commit 9cb081993dd964d87f91e15864c3a5fcd998a6c0
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-21T06:29:14Z

    Fix for 1st request failing

commit de2fd3cd60811d891234c88030c1bf80378ee553
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-22T14:43:35Z

    have spark streaming

commit b53fd8bfb7da935fb511444f4c70e63b6490e616
Author: Rohit Choudhary <[email protected]>
Date:   2016-02-23T03:19:13Z

    Don't need so many interpreters

commit 1f9a111188bcfc67c0d02826963d92d49ba8ef1d
Author: Rohit Choudhary <[email protected]>
Date:   2016-02-23T04:29:33Z

    remove references to LivyInterpreters

commit ee2dcebe3551b09f5986cb6a3414680d4929762e
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-23T06:27:44Z

    working spark sql

commit 68f438de8b4d76d1019f261dfb9a47b8d9c7b45b
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-23T06:46:13Z

    Merge remote-tracking branch 'origin/master' into livyInterperter

commit 426bbe80c318621afd7157278fc4b07448cf8e6c
Author: Rohit Choudhary <[email protected]>
Date:   2016-02-23T17:54:32Z

    Merge branch 'livyInterperter' of 
https://github.com/prabhjyotsingh/incubator-zeppelin into livyInterperter

commit 4f513a5b5aa797e9dc8a7fd72dbed17c70a182c7
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-24T04:03:14Z

    site.xml

commit 948615a3ebb8309cc4c44208fbffd213d5e4a6b9
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-24T05:28:36Z

    Merge remote-tracking branch 'origin/master' into livyInterperter
    
    # Conflicts:
    #   testing/startSparkCluster.sh

commit 9d89b0d235855636d93a37bcd74f4951ced39d3c
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-26T05:47:03Z

    Merge remote-tracking branch 'origin/master' into livyInterperter

commit b8e17794d957a51a63eebf16a5338b3db1c6651e
Author: Prabhjyot Singh <[email protected]>
Date:   2016-02-26T05:59:33Z

    adding back LivySparkRInterpreter.java , pyspark

commit dadc2571956abd40af48cb1ad3ee5d82382843c5
Author: Prabhjyot Singh <[email protected]>
Date:   2016-03-02T18:55:21Z

    reverting zeppelinConfiguration

commit 0fbb74bcf6fc4532f15656d6e679d1166d55fa5a
Author: Prabhjyot Singh <[email protected]>
Date:   2016-03-07T11:22:52Z

    Merge remote-tracking branch 'origin/master' into livyInterperter
    
    # Conflicts:
    #   conf/zeppelin-site.xml.template

commit ea05fe9a1ab8f8a5fcd49354d916edf9bc52a84a
Author: Prabhjyot Singh <[email protected]>
Date:   2016-03-08T06:00:58Z

    Merge remote-tracking branch 'origin/master' into livyInterperter

commit ee1c9f4cfb79c2d34d9c0bc34e81d48fac1d8908
Author: Prabhjyot Singh <[email protected]>
Date:   2016-03-09T05:12:14Z

    Merge remote-tracking branch 'origin/master' into livyInterperter

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---

Reply via email to