Re: Upgrade spark to 1.6.0

2016-02-03 Thread Felix Cheung 
I think his build command only works with Cloudera CDH 5.4.8, as you can see. 
Mismatch Akka version is very common if the Hadoop distribution is different. 
What version of Spark and Hadoop distribution are you running with?






On Tue, Feb 2, 2016 at 1:36 PM -0800, "Daniel Valdivia" 
 wrote:





Hello,

An update on the matter, using compile string

mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 
-Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo 
-DskipTests

I end up getting the following error stack trace upon executing a new JSON

akka.ConfigurationException: Akka JAR version [2.2.3] does not match the 
provided config version [2.3.11] at 
akka.actor.ActorSystem$Settings.(ActorSystem.scala:181) at 
akka.actor.ActorSystemImpl.(ActorSystem.scala:470) at 
akka.actor.ActorSystem$.apply(ActorSystem.scala:111) at 
akka.actor.ActorSystem$.apply(ActorSystem.scala:104) at 
org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
 at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53) at 
org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52) at 
org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1964)
 at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) at 
org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1955) at 
org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55) at 
org.apache.spark.SparkEnv$.create(SparkEnv.scala:266) at 
org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193) at 
org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288) at 
org.apache.spark.SparkContext.(SparkContext.scala:457) at 
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:339)
 at 
org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:145)
 at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:465) 
at 
org.apache.zeppelin.interpreter.ClassloaderInterpreter.open(ClassloaderInterpreter.java:74)
 at 
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:68)
 at 
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:92)
 at 
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
 at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at 
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:134) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at 
java.util.concurrent.FutureTask.run(FutureTask.java:262) at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:745)

There's some mentioning of this problem on SO, but seems like it was fixed

http://stackoverflow.com/questions/32294276/how-to-connect-zeppelin-to-spark-1-5-built-from-the-sources
 


any idea on how to deal with this AKKA library problem?

> On Feb 2, 2016, at 12:02 PM, Daniel Valdivia  wrote:
>
> Hi,
>
> Thanks for the suggestion, I'm running maven with Ben's command
>
> Cheers!
>
>> On Feb 1, 2016, at 7:47 PM, Benjamin Kim > > wrote:
>>
>> Hi Felix,
>>
>> After installing Spark 1.6, I built Zeppelin using:
>>
>> mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 
>> -Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo 
>> -DskipTests
>>
>> This worked for me.
>>
>> Cheers,
>> Ben
>>
>>
>>> On Feb 1, 2016, at 7:44 PM, Felix Cheung >> > wrote:
>>>
>>> Hi
>>>
>>> You can see the build command line example here for spark 1.6 profile
>>>
>>> https://github.com/apache/incubator-zeppelin/blob/master/README.md 
>>> 
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Feb 1, 2016 at 3:59 PM -0800, "Daniel Valdivia" 
>>> > wrote:
>>>
>>> Hi,
>>>
>>> I'd like to ask if there's an easy way to upgrade spark to 1.6.0 from the 
>>> current 1.4.x that's bundled with the current release of zepellin, would 
>>> updating the pom.xml and compiling suffice ?
>>>
>>> Cheers
>>
>

Re: Upgrade spark to 1.6.0

2016-02-03 Thread Daniel Valdivia 
I don't need any specific version of Hadoop, I actually removed it from the 
build command and still get the error, I just need spark 1.6


> On Feb 3, 2016, at 9:05 AM, Felix Cheung  wrote:
> 
> I think his build command only works with Cloudera CDH 5.4.8, as you can see. 
> Mismatch Akka version is very common if the Hadoop distribution is different. 
> What version of Spark and Hadoop distribution are you running with?
> 
> 
> 
> 
> 
> On Tue, Feb 2, 2016 at 1:36 PM -0800, "Daniel Valdivia" 
> > wrote:
> 
> Hello,
> 
> An update on the matter, using compile string
> 
> mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 
> -Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo 
> -DskipTests
> 
> I end up getting the following error stack trace upon executing a new JSON
> 
> akka.ConfigurationException: Akka JAR version [2.2.3] does not match the 
> provided config version [2.3.11] at 
> akka.actor.ActorSystem$Settings.(ActorSystem.scala:181) at 
> akka.actor.ActorSystemImpl.(ActorSystem.scala:470) at 
> akka.actor.ActorSystem$.apply(ActorSystem.scala:111) at 
> akka.actor.ActorSystem$.apply(ActorSystem.scala:104) at 
> org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
>  at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53) at 
> org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52) at 
> org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1964)
>  at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) at 
> org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1955) at 
> org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55) at 
> org.apache.spark.SparkEnv$.create(SparkEnv.scala:266) at 
> org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193) at 
> org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288) at 
> org.apache.spark.SparkContext.(SparkContext.scala:457) at 
> org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:339)
>  at 
> org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:145)
>  at 
> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:465)  
> at 
> org.apache.zeppelin.interpreter.ClassloaderInterpreter.open(ClassloaderInterpreter.java:74)
>  at 
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:68)
>  at 
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:92)
>  at 
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
>  at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at 
> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:134) at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at 
> java.util.concurrent.FutureTask.run(FutureTask.java:262) at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  at java.lang.Thread.run(Thread.java:745)
> 
> There's some mentioning of this problem on SO, but seems like it was fixed
> 
> http://stackoverflow.com/questions/32294276/how-to-connect-zeppelin-to-spark-1-5-built-from-the-sources
>  
> 
> 
> any idea on how to deal with this AKKA library problem?
> 
>> On Feb 2, 2016, at 12:02 PM, Daniel Valdivia > > wrote:
>> 
>> Hi,
>> 
>> Thanks for the suggestion, I'm running maven with Ben's command
>> 
>> Cheers!
>> 
>>> On Feb 1, 2016, at 7:47 PM, Benjamin Kim >> > wrote:
>>> 
>>> Hi Felix,
>>> 
>>> After installing Spark 1.6, I built Zeppelin using:
>>> 
>>> mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 
>>> -Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo 
>>> -DskipTests
>>> 
>>> This worked for me.
>>> 
>>> Cheers,
>>> Ben
>>> 
>>> 
 On Feb 1, 2016, at 7:44 PM, Felix Cheung > wrote:
 
 Hi
 
 You can see the build command line example here for spark 1.6 profile
 
 https://github.com/apache/incubator-zeppelin/blob/master/README.md 
 
 
 
 
 
 
 On Mon, Feb 1, 2016 at 3:59 PM -0800, "Daniel Valdivia"

Re: Upgrade spark to 1.6.0

2016-02-02 Thread Daniel Valdivia 
Hello,

An update on the matter, using compile string

mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 
-Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo 
-DskipTests

I end up getting the following error stack trace upon executing a new JSON

akka.ConfigurationException: Akka JAR version [2.2.3] does not match the 
provided config version [2.3.11] at 
akka.actor.ActorSystem$Settings.(ActorSystem.scala:181) at 
akka.actor.ActorSystemImpl.(ActorSystem.scala:470) at 
akka.actor.ActorSystem$.apply(ActorSystem.scala:111) at 
akka.actor.ActorSystem$.apply(ActorSystem.scala:104) at 
org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
 at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53) at 
org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52) at 
org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1964)
 at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) at 
org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1955) at 
org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55) at 
org.apache.spark.SparkEnv$.create(SparkEnv.scala:266) at 
org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193) at 
org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288) at 
org.apache.spark.SparkContext.(SparkContext.scala:457) at 
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:339)
 at 
org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:145)
 at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:465) 
at 
org.apache.zeppelin.interpreter.ClassloaderInterpreter.open(ClassloaderInterpreter.java:74)
 at 
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:68)
 at 
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:92)
 at 
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
 at org.apache.zeppelin.scheduler.Job.run(Job.java:169) at 
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:134) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at 
java.util.concurrent.FutureTask.run(FutureTask.java:262) at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:745)

There's some mentioning of this problem on SO, but seems like it was fixed

http://stackoverflow.com/questions/32294276/how-to-connect-zeppelin-to-spark-1-5-built-from-the-sources
 


any idea on how to deal with this AKKA library problem?

> On Feb 2, 2016, at 12:02 PM, Daniel Valdivia  wrote:
> 
> Hi,
> 
> Thanks for the suggestion, I'm running maven with Ben's command
> 
> Cheers!
> 
>> On Feb 1, 2016, at 7:47 PM, Benjamin Kim > > wrote:
>> 
>> Hi Felix,
>> 
>> After installing Spark 1.6, I built Zeppelin using:
>> 
>> mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 
>> -Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo 
>> -DskipTests
>> 
>> This worked for me.
>> 
>> Cheers,
>> Ben
>> 
>> 
>>> On Feb 1, 2016, at 7:44 PM, Felix Cheung >> > wrote:
>>> 
>>> Hi
>>> 
>>> You can see the build command line example here for spark 1.6 profile
>>> 
>>> https://github.com/apache/incubator-zeppelin/blob/master/README.md 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Mon, Feb 1, 2016 at 3:59 PM -0800, "Daniel Valdivia" 
>>> > wrote:
>>> 
>>> Hi,
>>> 
>>> I'd like to ask if there's an easy way to upgrade spark to 1.6.0 from the 
>>> current 1.4.x that's bundled with the current release of zepellin, would 
>>> updating the pom.xml and compiling suffice ?
>>> 
>>> Cheers
>> 
>

Re: Upgrade spark to 1.6.0

2016-02-01 Thread Felix Cheung 
Hi
You can see the build command line example here for spark 1.6 profile
https://github.com/apache/incubator-zeppelin/blob/master/README.md





On Mon, Feb 1, 2016 at 3:59 PM -0800, "Daniel Valdivia" 
 wrote:





Hi,

I'd like to ask if there's an easy way to upgrade spark to 1.6.0 from the 
current 1.4.x that's bundled with the current release of zepellin, would 
updating the pom.xml and compiling suffice ?

Cheers

Re: Upgrade spark to 1.6.0

2016-02-01 Thread Benjamin Kim 
Hi Felix,

After installing Spark 1.6, I built Zeppelin using:

mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 
-Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo 
-DskipTests

This worked for me.

Cheers,
Ben


> On Feb 1, 2016, at 7:44 PM, Felix Cheung  wrote:
> 
> Hi
> 
> You can see the build command line example here for spark 1.6 profile
> 
> https://github.com/apache/incubator-zeppelin/blob/master/README.md
> 
> 
> 
> 
> 
> On Mon, Feb 1, 2016 at 3:59 PM -0800, "Daniel Valdivia" 
> > wrote:
> 
> Hi,
> 
> I'd like to ask if there's an easy way to upgrade spark to 1.6.0 from the 
> current 1.4.x that's bundled with the current release of zepellin, would 
> updating the pom.xml and compiling suffice ?
> 
> Cheers