Re: spark-shell with Yarn failed

2015-07-18 Thread Chester @work
it might be a network issue. The error states failed to bind the server IP address Chester Sent from my iPhone On Jul 18, 2015, at 11:46 AM, Amjad ALSHABANI ashshab...@gmail.com wrote: Does anybody have any idea about the error I m having.. I am really clueless... And appreciate any idea

Re: What is akka-actor_2.10-2.2.3-shaded-protobuf.jar?

2014-10-17 Thread Chester @work
They should be the same except the package names are changed to avoid protopuf conflict. You can use it just like other Akka jars Chester Sent from my iPhone On Oct 17, 2014, at 5:56 AM, Ruebenacker, Oliver A oliver.ruebenac...@altisource.com wrote: Hello, My SBT pulls in,

Re: SPARK UI - Details post job processiong

2014-09-26 Thread Chester @work
I am working on a PR that allows one to send the same spark listener event message back to the application in yarn cluster mode. So far I have put this function in our application, our UI will receive and display the same spark job event message such as progress, job start, completed etc

Re: Running Spark On Yarn without Spark-Submit

2014-08-29 Thread Chester @work
Archit We are using yarn-cluster mode , and calling spark via Client class directly from servlet server. It works fine. To establish a communication channel to give further requests, It should be possible with yarn client, but not with yarn server. Yarn client mode, spark driver

Re: jar changed on src filesystem

2014-07-17 Thread Chester@work
Since you are running in yarn-cluster mode, and you are supply the spark assembly jar file. There is no need to install spark on each node. Is it possible two spark jars have different version ? Chester Sent from my iPad On Jul 16, 2014, at 22:49, cmti95035 cmti95...@gmail.com wrote: Hi,

Re: Unable to run Spark 1.0 SparkPi on HDP 2.0

2014-07-07 Thread Chester @work
In Yarn cluster mode, you can either have spark on all the cluster nodes or supply the spark jar yourself. In the 2nd case, you don't need install spark on cluster at all. As you supply the spark assembly as we as your app jar together. I hope this make it clear Chester Sent from my iPhone