Re: Running Multiple Versions of Spark on the same cluster (YARN)

= For previously versions --conf spark.yarn.jar= Tiago Tiago Albineli Motta Desenvolvedor de Software - Globo.com ICQ: 32107100 http://programandosemcafeina.blogspot.com On Sat, Dec 17, 2016 at 5:36 AM, Jorge Machado <jom...@me.com> wrote: > Hi Everyone, > > I ha

Re: Best approach to schedule Spark jobs

Here at Globo.com we use Airflow to schedule and manage our spark pipeline. We use the Yarn API in the Airflow Dags to controls things like garantee that the job is not running before start another batch. Tiago Albineli Motta Desenvolvedor de Software - Globo.com ICQ: 32107100 http

Re: Error in starting Spark Streaming Context

{ } Tiago Albineli Motta Desenvolvedor de Software - Globo.com ICQ: 32107100 http://programandosemcafeina.blogspot.com On Wed, Jul 29, 2015 at 12:59 PM, Sadaf <sa...@platalytics.com> wrote: > Hi > > I am new to Spark Streaming and writing a code for twitter connector. when > i > ru

Re: Error in starting Spark Streaming Context

the DStream using the returned from StreamingContext.getOrCreate returned. So you should call *fetchTweets *inside *managingContext*. That worked for me. Tiago Tiago Albineli Motta Desenvolvedor de Software - Globo.com ICQ: 32107100 http://programandosemcafeina.blogspot.com On Thu, Oct 22