Russell Spitzer created ZEPPELIN-2416: -----------------------------------------
Summary: Make Spark Properties Optional in Spark Interperter Key: ZEPPELIN-2416 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2416 Project: Zeppelin Issue Type: Improvement Components: Interpreters Affects Versions: 0.7.1 Reporter: Russell Spitzer Priority: Trivial I was doing some experimentation connecting Zepplin to DataStax Enterprise and I noticed that the Spark Interperter requires the "master" interpreter parameter to be present in order to work. There are several locations which do null-unsafe operations against the return value of getProperty("master") See https://github.com/apache/zeppelin/blob/v0.7.1/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L310 Then the value of this property is set in the SparkConf even if the value is already set there. https://github.com/apache/zeppelin/blob/v0.7.1/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L335 and https://github.com/apache/zeppelin/blob/v0.7.1/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L449 For this and other parameters it might make sense to allow values that are already set to be used and not overwritten. This would allow Zepplin to work better with "spark-defaults" and those vendors which preset these parameters in the environment. I would propose that all interpreter properties be optional. Operationally this means that the "master" is allowed to be unset if it can be automatically inherited from the environment in the SparkConf. Other parameters would simply need a null check, where they would not be set if they are already null (unset). -- This message was sent by Atlassian JIRA (v6.3.15#6346)