zeppelin git commit: [ZEPPELIN-2431] Corrected deletion of notes by incorrect interpreter.json

2017-04-27 Thread cloverhearts 
Repository: zeppelin
Updated Branches:
  refs/heads/master c8cd1cf50 -> 874403256


[ZEPPELIN-2431] Corrected deletion of notes by incorrect interpreter.json

## What is this PR for?

We sometimes can not delete a note, or we will be accompanied by an NPE for 
deleting a note.

This problem occurs when:
When interpreter.json 's note binding is wrong, or there is a problem.
If you are configuring an interpreter that is not through zeppelin's user 
interface.
As a result, it happens when synchronization of notes deletion and setting 
retention is not normal.
Therefore, we should add handling for note deletion and exception handling for 
nonexistent interpreter bindings.
It reduces the synchronization problem of interpreter.json.

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2431

### How should this be tested?
1. zeppelin stop
2. edit con/interpreter.json and `interpreterBindings`
  fix any notes or incorrect information.
  for example
  ```
},
  "interpreterBindings": {
"2CFS9YSM5": [
  "2CFRR1D3TINVALIDINVALIDINVALID", <-- edit
  "2CFZ1JKUR",
  "2CEAJK1VW",
  "2CGSESWBH",
  "2CERNPGW5",
   }
  ```
3. zeppelin start
4. You can try remove invalid interpreter bind note on web. (on example = 
`2CFS9YSM5`)

result :
If the modifications to this PR are not reflected,
It will not be deleted or an error will appear on the server.
Also, the interpreterBindings information in interpreter.json does not respond 
to delete events.

### Screenshots (if appropriate)
problem animation
![cantremove](https://cloud.githubusercontent.com/assets/10525473/25327785/7031f960-2910-11e7-88c7-d322da21290c.gif)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: CloverHearts 

Closes #2278 from cloverhearts/fix/invalidsyncInterpreterJson and squashes the 
following commits:

35da524c [CloverHearts] notebook interpreter binding synchronization process


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/87440325
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/87440325
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/87440325

Branch: refs/heads/master
Commit: 8744032563718dfe9aac0c9b968254fcdca7b004
Parents: c8cd1cf
Author: CloverHearts 
Authored: Mon Apr 24 16:16:47 2017 +0900
Committer: CloverHearts 
Committed: Fri Apr 28 14:16:44 2017 +0900

--
 .../zeppelin/interpreter/InterpreterSettingManager.java  | 8 ++--
 .../src/main/java/org/apache/zeppelin/notebook/Notebook.java | 6 +-
 2 files changed, 11 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/87440325/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java
--
diff --git 
a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java
 
b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java
index bebbf35..1b05a76 100644
--- 
a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java
+++ 
b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java
@@ -887,14 +887,18 @@ public class InterpreterSettingManager {
 saveToFile();
   }
 
-  public void removeNoteInterpreterSettingBinding(String user, String noteId) {
+  public void removeNoteInterpreterSettingBinding(String user, String noteId) 
throws IOException {
 synchronized (interpreterSettings) {
   List settingIds = (interpreterBindings.containsKey(noteId) ?
   interpreterBindings.remove(noteId) :
   Collections.emptyList());
   for (String settingId : settingIds) {
-this.removeInterpretersForNote(get(settingId), user, noteId);
+InterpreterSetting setting = get(settingId);
+if (setting != null) {
+  this.removeInterpretersForNote(setting, user, noteId);
+}
   }
+  saveToFile();
 }
   }
 

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/87440325/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
--
diff --git 
a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java 
b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
index de48bef..720a3be 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
@@ -338,7

zeppelin git commit: ZEPPELIN-2261. Support to connect with livy through https

2017-04-27 Thread zjffdu 
Repository: zeppelin
Updated Branches:
  refs/heads/branch-0.7 59c1e1b0a -> c52c0aced


ZEPPELIN-2261. Support to connect with livy through https

### What is this PR for?
Livy server support https, but the currently livy interpreter doesn't support 
it. This PR is for for the supporting to connect with livy through https

### What type of PR is it?
[Improvement]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2261

### How should this be tested?
Tested manually on livy server with ssl enabled.

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang 

Closes #2139 from zjffdu/ZEPPELIN-2261 and squashes the following commits:

52fc204 [Jeff Zhang] address comment
53230c3 [Jeff Zhang] [ZEPPELIN-2261]. Support to connect with livy through https

(cherry picked from commit 9d40013a994ca65f99831a416fb235d15ee17fdb)
Signed-off-by: Jeff Zhang 


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/c52c0ace
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/c52c0ace
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/c52c0ace

Branch: refs/heads/branch-0.7
Commit: c52c0aced7bd265b20de2071d40e62690dd9e9af
Parents: 59c1e1b
Author: Jeff Zhang 
Authored: Mon Mar 20 16:11:38 2017 +0800
Committer: Jeff Zhang 
Committed: Fri Apr 28 08:50:17 2017 +0800

--
 docs/interpreter/livy.md| 10 +++
 .../zeppelin/livy/BaseLivyInterprereter.java| 83 
 .../zeppelin/livy/LivySQLInterpreterTest.java   |  1 +
 3 files changed, 79 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/c52c0ace/docs/interpreter/livy.md
--
diff --git a/docs/interpreter/livy.md b/docs/interpreter/livy.md
index 6f04244..ce1c34a 100644
--- a/docs/interpreter/livy.md
+++ b/docs/interpreter/livy.md
@@ -130,6 +130,16 @@ Example: `spark.driver.memory` to 
`livy.spark.driver.memory`
   
   Adding extra libraries to livy interpreter
 
+  
+zeppelin.livy.ssl.trustStore
+
+client trustStore file. Used when livy ssl is enabled
+  
+  
+zeppelin.livy.ssl.trustStorePassword
+
+password for trustStore file. Used when livy ssl is enabled
+
 
 
 **We remove livy.spark.master in zeppelin-0.7. Because we sugguest user to use 
livy 0.3 in zeppelin-0.7. And livy 0.3 don't allow to specify 
livy.spark.master, it enfornce yarn-cluster mode.**

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/c52c0ace/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
--
diff --git 
a/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java 
b/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
index ecb5d77..7f9897e 100644
--- a/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
+++ b/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
@@ -21,6 +21,10 @@ import com.google.gson.Gson;
 import com.google.gson.GsonBuilder;
 import com.google.gson.annotations.SerializedName;
 import org.apache.commons.lang.StringUtils;
+import org.apache.http.client.HttpClient;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.conn.ssl.SSLContexts;
+import org.apache.http.impl.client.HttpClients;
 import org.apache.zeppelin.interpreter.*;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -28,11 +32,16 @@ import org.springframework.http.HttpEntity;
 import org.springframework.http.HttpHeaders;
 import org.springframework.http.HttpMethod;
 import org.springframework.http.ResponseEntity;
+import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
 import org.springframework.security.kerberos.client.KerberosRestTemplate;
 import org.springframework.web.client.HttpClientErrorException;
 import org.springframework.web.client.RestClientException;
 import org.springframework.web.client.RestTemplate;
 
+import javax.net.ssl.SSLContext;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.security.KeyStore;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
@@ -56,6 +65,7 @@ public abstract class BaseLivyInterprereter extends 
Interpreter {
   protected boolean displayAppInfo;
   private AtomicBoolean sessionExpired = new AtomicBoolean(false);
   protected LivyVersion livyVersion;
+  private RestTemplate restTemplate;
 
   // keep tracking the mapping between paragraphId and statementId, so that we 
can cancel the

zeppelin git commit: ZEPPELIN-2151. Add integration test for livy cancel api

2017-04-27 Thread zjffdu 
Repository: zeppelin
Updated Branches:
  refs/heads/branch-0.7 b9bc2b810 -> 59c1e1b0a


ZEPPELIN-2151. Add integration test for livy cancel api

### What is this PR for?

Just add integration test for livy cancel api. Only do it for livy 0.3 as the 
cancel api is only available in livy 0.3

### What type of PR is it?
[ Improvement]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2151

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang 

Closes #2056 from zjffdu/ZEPPELIN-2151 and squashes the following commits:

b7ca7b3 [Jeff Zhang] ZEPPELIN-2151. Add integration test for livy cancel api

(cherry picked from commit 152147122b9797baef20a382eb880eadcf7cdc0f)
Signed-off-by: Jeff Zhang 


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/59c1e1b0
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/59c1e1b0
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/59c1e1b0

Branch: refs/heads/branch-0.7
Commit: 59c1e1b0abc68ac978fc2c4a0284e852a93f4fc7
Parents: b9bc2b8
Author: Jeff Zhang 
Authored: Wed Feb 22 19:59:47 2017 +0800
Committer: Jeff Zhang 
Committed: Fri Apr 28 08:48:51 2017 +0800

--
 .../zeppelin/livy/BaseLivyInterprereter.java|  2 +-
 .../apache/zeppelin/livy/LivyInterpreterIT.java | 85 ++--
 2 files changed, 80 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/59c1e1b0/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
--
diff --git 
a/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java 
b/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
index 7f92127..ecb5d77 100644
--- a/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
+++ b/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterprereter.java
@@ -55,7 +55,7 @@ public abstract class BaseLivyInterprereter extends 
Interpreter {
   private int pullStatusInterval;
   protected boolean displayAppInfo;
   private AtomicBoolean sessionExpired = new AtomicBoolean(false);
-  private LivyVersion livyVersion;
+  protected LivyVersion livyVersion;
 
   // keep tracking the mapping between paragraphId and statementId, so that we 
can cancel the
   // statement after we execute it.

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/59c1e1b0/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java
--
diff --git a/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java 
b/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java
index aec2742..7df9b20 100644
--- a/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java
+++ b/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java
@@ -82,13 +82,13 @@ public class LivyInterpreterIT {
 }
 InterpreterGroup interpreterGroup = new InterpreterGroup("group_1");
 interpreterGroup.put("session_1", new ArrayList());
-LivySparkInterpreter sparkInterpreter = new 
LivySparkInterpreter(properties);
+final LivySparkInterpreter sparkInterpreter = new 
LivySparkInterpreter(properties);
 sparkInterpreter.setInterpreterGroup(interpreterGroup);
 interpreterGroup.get("session_1").add(sparkInterpreter);
 AuthenticationInfo authInfo = new AuthenticationInfo("user1");
 MyInterpreterOutputListener outputListener = new 
MyInterpreterOutputListener();
 InterpreterOutput output = new InterpreterOutput(outputListener);
-InterpreterContext context = new InterpreterContext("noteId", 
"paragraphId", "livy.spark",
+final InterpreterContext context = new InterpreterContext("noteId", 
"paragraphId", "livy.spark",
 "title", "text", authInfo, null, null, null, null, null, output);
 sparkInterpreter.open();
 
@@ -158,6 +158,31 @@ public class LivyInterpreterIT {
   assertEquals(InterpreterResult.Code.ERROR, result.code());
   assertEquals(InterpreterResult.Type.TEXT, 
result.message().get(0).getType());
   assertTrue(result.message().get(0).getData().contains("incomplete 
statement"));
+
+  // cancel
+  if 
(sparkInterpreter.livyVersion.newerThanEquals(LivyVersion.LIVY_0_3_0)) {
+Thread cancelThread = new Thread() {
+  @Override
+  public void run() {
+// invoke cancel after 3 seconds to wait job starting
+try {
+  Thread.sleep(3000);
+} catch (InterruptedException e) {
+  e.printStackTrace();
+

zeppelin git commit: ZEPPELIN-1595. Make ZeppelinContext extensible

2017-04-27 Thread zjffdu 
Repository: zeppelin
Updated Branches:
  refs/heads/master 0f1701da8 -> c8cd1cf50


ZEPPELIN-1595. Make ZeppelinContext extensible

### What is this PR for?
For now, ZeppelinContext only support Spark Interpreter. I'd like to make it 
extensible, so that it can support other interpreters as well. For now, user 
need to implement the following 3 methods to extend ZeppelinContext

* public List getSupportedClasses()
* public abstract Map getInterpreterClassMap();
* protected abstract String showData(Object obj);

### What type of PR is it?
[Feature | Refactoring]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1595

### How should this be tested?
Outline the steps to test the PR here.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang 

Closes #1574 from zjffdu/ZEPPELIN-1595 and squashes the following commits:

754c1e0 [Jeff Zhang] ZEPPELIN-1595. Make ZeppelinContext extensible


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/c8cd1cf5
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/c8cd1cf5
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/c8cd1cf5

Branch: refs/heads/master
Commit: c8cd1cf5065605269cf83ebc8fe7b265337e34a2
Parents: 0f1701d
Author: Jeff Zhang 
Authored: Mon Oct 31 15:33:51 2016 +0800
Committer: Jeff Zhang 
Committed: Fri Apr 28 08:23:34 2017 +0800

--
 .../apache/zeppelin/rinterpreter/RStatics.java  |6 +-
 .../apache/zeppelin/rinterpreter/RContext.scala |6 +-
 .../zeppelin/spark/PySparkInterpreter.java  |4 +-
 .../apache/zeppelin/spark/SparkInterpreter.java |   19 +-
 .../zeppelin/spark/SparkSqlInterpreter.java |2 +-
 .../zeppelin/spark/SparkZeppelinContext.java|  280 +
 .../apache/zeppelin/spark/ZeppelinContext.java  | 1019 --
 .../apache/zeppelin/spark/ZeppelinRContext.java |6 +-
 .../main/resources/python/zeppelin_pyspark.py   |2 +-
 .../interpreter/BaseZeppelinContext.java|  821 ++
 10 files changed, 1120 insertions(+), 1045 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/c8cd1cf5/r/src/main/java/org/apache/zeppelin/rinterpreter/RStatics.java
--
diff --git a/r/src/main/java/org/apache/zeppelin/rinterpreter/RStatics.java 
b/r/src/main/java/org/apache/zeppelin/rinterpreter/RStatics.java
index 361fe47..1ea35ce 100644
--- a/r/src/main/java/org/apache/zeppelin/rinterpreter/RStatics.java
+++ b/r/src/main/java/org/apache/zeppelin/rinterpreter/RStatics.java
@@ -25,7 +25,7 @@ package org.apache.zeppelin.rinterpreter;
 import org.apache.spark.SparkContext;
 import org.apache.spark.api.java.JavaSparkContext;
 import org.apache.spark.sql.SQLContext;
-import org.apache.zeppelin.spark.ZeppelinContext;
+import org.apache.zeppelin.spark.SparkZeppelinContext;
 
 /**
  * RStatics provides static class methods that can be accessed through the 
SparkR bridge
@@ -33,7 +33,7 @@ import org.apache.zeppelin.spark.ZeppelinContext;
  */
 public class RStatics {
   private static SparkContext sc = null;
-  private static ZeppelinContext z = null;
+  private static SparkZeppelinContext z = null;
   private static SQLContext sql = null;
   private static RContext rCon = null;
 
@@ -42,7 +42,7 @@ public class RStatics {
 return sc;
   }
 
-  public static ZeppelinContext setZ(ZeppelinContext newZ) {
+  public static SparkZeppelinContext setZ(SparkZeppelinContext newZ) {
 z = newZ;
 return z;
   }

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/c8cd1cf5/r/src/main/scala/org/apache/zeppelin/rinterpreter/RContext.scala
--
diff --git a/r/src/main/scala/org/apache/zeppelin/rinterpreter/RContext.scala 
b/r/src/main/scala/org/apache/zeppelin/rinterpreter/RContext.scala
index ffab160..39ba5ae 100644
--- a/r/src/main/scala/org/apache/zeppelin/rinterpreter/RContext.scala
+++ b/r/src/main/scala/org/apache/zeppelin/rinterpreter/RContext.scala
@@ -28,7 +28,7 @@ import org.apache.zeppelin.interpreter._
 import org.apache.zeppelin.rinterpreter.rscala.RClient._
 import org.apache.zeppelin.rinterpreter.rscala._
 import org.apache.zeppelin.scheduler._
-import org.apache.zeppelin.spark.{SparkInterpreter, ZeppelinContext}
+import org.apache.zeppelin.spark.{SparkInterpreter, SparkZeppelinContext}
 import org.slf4j._
 
 import scala.collection.JavaConversions._
@@ -45,7 +45,7 @@ private[rinterpreter] class RContext(private val sockets: 
ScalaSockets,
   val backend: RBackendHelper =

zeppelin git commit: [HOTFIX] Add livy build in .travis

2017-04-27 Thread zjffdu 
Repository: zeppelin
Updated Branches:
  refs/heads/branch-0.7 c4cf06869 -> b9bc2b810


[HOTFIX] Add livy build in .travis

livy integration test is ignored, this PR add livy integration test in travis

[Hot Fix]

* [ ] - Task

* No jira created

CI pass

* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang 

Closes #2279 from zjffdu/hotfix_livy and squashes the following commits:

674c987 [Jeff Zhang] [HOTFIX] Add livy build in .travis

(cherry picked from commit 0f1701da8bb98678ae8e486796b2c1e650125106)
Signed-off-by: Jeff Zhang 


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/b9bc2b81
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/b9bc2b81
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/b9bc2b81

Branch: refs/heads/branch-0.7
Commit: b9bc2b810dea27fcef44062f56c4446eacb70b64
Parents: c4cf068
Author: Jeff Zhang 
Authored: Fri Apr 21 17:38:51 2017 +0800
Committer: Jeff Zhang 
Committed: Fri Apr 28 08:05:04 2017 +0800

--
 .travis.yml  | 15 ++-
 .../org/apache/zeppelin/livy/LivyInterpreterIT.java  |  8 ++--
 2 files changed, 12 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b9bc2b81/.travis.yml
--
diff --git a/.travis.yml b/.travis.yml
index a5a9acb..c7e97d8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -73,18 +73,15 @@ matrix:
 - jdk: "oraclejdk7"
   env: SCALA_VER="2.11" SPARK_VER="1.6.3" HADOOP_VER="2.6" 
PROFILE="-Pspark-1.6 -Phadoop-2.6 -Ppyspark -Psparkr -Pscala-2.11" 
BUILD_FLAG="package -DskipTests -DskipRat" TEST_FLAG="test -DskipRat" 
MODULES="-pl 
.,zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-display,spark-dependencies,spark"
 TEST_PROJECTS="-Dtest=ZeppelinSparkClusterTest,org.apache.zeppelin.spark.* 
-DfailIfNoTests=false"
 
-# Test python/pyspark with python 2
-- jdk: "oraclejdk7"
-  env: PYTHON="2" SCALA_VER="2.10" SPARK_VER="1.6.1" HADOOP_VER="2.6" 
PROFILE="-Pspark-1.6 -Phadoop-2.6 -Ppyspark" BUILD_FLAG="package -am 
-DskipTests -DskipRat" TEST_FLAG="test -DskipRat" MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python" 
TEST_PROJECTS="-Dtest=org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
-
-# Test python/pyspark with python 3
-- jdk: "oraclejdk7"
-  env: PYTHON="3" SCALA_VER="2.11" SPARK_VER="2.0.0" HADOOP_VER="2.6" 
PROFILE="-Pspark-2.0 -Phadoop-2.6 -Ppyspark -Pscala-2.11" BUILD_FLAG="package 
-am -DskipTests -DskipRat" TEST_FLAG="test -DskipRat" MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python" 
TEST_PROJECTS="-Dtest=org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
+# Test python/pyspark with python 2, livy 0.2
+- sudo: required
+  jdk: "oraclejdk7"
+  env: PYTHON="2" SCALA_VER="2.10" SPARK_VER="1.6.1" HADOOP_VER="2.6" 
LIVY_VER="0.2.0" PROFILE="-Pspark-1.6 -Phadoop-2.6 -Plivy-0.2" 
BUILD_FLAG="package -am -DskipTests -DskipRat" TEST_FLAG="verify -DskipRat" 
MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python,livy" 
TEST_PROJECTS="-Dtest=LivySQLInterpreterTest,org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
 
-# Test livy with spark 1.5.2 and hadoop 2.6
+# Test python/pyspark with python 3, livy 0.3
 - sudo: required
   jdk: "oraclejdk7"
-  env: SCALA_VER="2.10" $LIVY_VER="0.2.0" SPARK_VER="1.5.2" 
HADOOP_VER="2.6" PROFILE="-Pspark-1.5 -Phadoop-2.6" BUILD_FLAG="package 
-DskipTests -DskipRat" TEST_FLAG="verify -DskipRat" MODULES="-pl 
zeppelin-interpreter,livy" TEST_PROJECTS="-DfailIfNoTests=false"
+  env: PYTHON="3" SCALA_VER="2.11" SPARK_VER="2.0.0" HADOOP_VER="2.6" 
LIVY_VER="0.3.0" PROFILE="-Pspark-2.0 -Phadoop-2.6 -Pscala-2.11 -Plivy-0.3" 
BUILD_FLAG="package -am -DskipTests -DskipRat" TEST_FLAG="verify -DskipRat" 
MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python,livy" 
TEST_PROJECTS="-Dtest=LivySQLInterpreterTest,org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
 
 before_install:
   # check files included in commit range, clear bower_components if a 
bower.json file has changed.

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b9bc2b81/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java

zeppelin git commit: [HOTFIX] Add livy build in .travis

2017-04-27 Thread zjffdu 
Repository: zeppelin
Updated Branches:
  refs/heads/master 5cd806dc3 -> 0f1701da8


[HOTFIX] Add livy build in .travis

### What is this PR for?

livy integration test is ignored, this PR add livy integration test in travis

### What type of PR is it?
[Hot Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* No jira created

### How should this be tested?
CI pass

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang 

Closes #2279 from zjffdu/hotfix_livy and squashes the following commits:

674c987 [Jeff Zhang] [HOTFIX] Add livy build in .travis


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/0f1701da
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/0f1701da
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/0f1701da

Branch: refs/heads/master
Commit: 0f1701da8bb98678ae8e486796b2c1e650125106
Parents: 5cd806d
Author: Jeff Zhang 
Authored: Fri Apr 21 17:38:51 2017 +0800
Committer: Jeff Zhang 
Committed: Thu Apr 27 16:36:18 2017 +0800

--
 .travis.yml | 16 +---
 .../org/apache/zeppelin/livy/LivyInterpreterIT.java |  8 ++--
 2 files changed, 15 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0f1701da/.travis.yml
--
diff --git a/.travis.yml b/.travis.yml
index f77906f..72104db 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -73,13 +73,15 @@ matrix:
 - jdk: "oraclejdk7"
   env: SCALA_VER="2.11" SPARK_VER="1.6.3" HADOOP_VER="2.6" 
PROFILE="-Pspark-1.6 -Phadoop-2.6 -Pscala-2.11" SPARKR="true" 
BUILD_FLAG="package -DskipTests -DskipRat" TEST_FLAG="test -DskipRat" 
MODULES="-pl 
.,zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-display,spark-dependencies,spark"
 TEST_PROJECTS="-Dtest=ZeppelinSparkClusterTest,org.apache.zeppelin.spark.* 
-DfailIfNoTests=false"
 
-# Test python/pyspark with python 2
-- jdk: "oraclejdk7"
-  env: PYTHON="2" SCALA_VER="2.10" SPARK_VER="1.6.1" HADOOP_VER="2.6" 
PROFILE="-Pspark-1.6 -Phadoop-2.6" BUILD_FLAG="package -am -DskipTests 
-DskipRat" TEST_FLAG="test -DskipRat" MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python" 
TEST_PROJECTS="-Dtest=org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
-
-# Test python/pyspark with python 3
-- jdk: "oraclejdk7"
-  env: PYTHON="3" SCALA_VER="2.11" SPARK_VER="2.0.0" HADOOP_VER="2.6" 
PROFILE="-Pspark-2.0 -Phadoop-2.6 -Pscala-2.11" BUILD_FLAG="package -am 
-DskipTests -DskipRat" TEST_FLAG="test -DskipRat" MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python" 
TEST_PROJECTS="-Dtest=org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
+# Test python/pyspark with python 2, livy 0.2
+- sudo: required
+  jdk: "oraclejdk7"
+  env: PYTHON="2" SCALA_VER="2.10" SPARK_VER="1.6.1" HADOOP_VER="2.6" 
LIVY_VER="0.2.0" PROFILE="-Pspark-1.6 -Phadoop-2.6 -Plivy-0.2" 
BUILD_FLAG="package -am -DskipTests -DskipRat" TEST_FLAG="verify -DskipRat" 
MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python,livy" 
TEST_PROJECTS="-Dtest=LivySQLInterpreterTest,org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
+
+# Test python/pyspark with python 3, livy 0.3
+- sudo: required
+  jdk: "oraclejdk7"
+  env: PYTHON="3" SCALA_VER="2.11" SPARK_VER="2.0.0" HADOOP_VER="2.6" 
LIVY_VER="0.3.0" PROFILE="-Pspark-2.0 -Phadoop-2.6 -Pscala-2.11 -Plivy-0.3" 
BUILD_FLAG="package -am -DskipTests -DskipRat" TEST_FLAG="verify -DskipRat" 
MODULES="-pl 
.,zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python,livy" 
TEST_PROJECTS="-Dtest=LivySQLInterpreterTest,org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.*
 -Dpyspark.test.exclude='' -DfailIfNoTests=false"
 
 before_install:
   # check files included in commit range, clear bower_components if a 
bower.json file has changed.

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0f1701da/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java
--
diff --git a/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java 
b/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java
index 06da238..007c0ed 100644
--- a/livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterIT.java
+++