Re: simultaneous actions

M, Mennour Rostom <mennou...@gmail.com> > wrote: > >> Hi, >> >> I am running my app in a single machine first before moving it in the >> cluster; actually simultaneous actions are not working for me now; is this >> comming from the fact that I am using a single mac

Re: simultaneous actions

Hi, I am running my app in a single machine first before moving it in the cluster; actually simultaneous actions are not working for me now; is this comming from the fact that I am using a single machine ? yet I am using FAIR scheduler. 2016-01-17 21:23 GMT+01:00 Mark Hamstra &l

Re: simultaneous actions

stacktrace? details? On Mon, Jan 18, 2016 at 5:58 AM, Mennour Rostom <mennou...@gmail.com> wrote: > Hi, > > I am running my app in a single machine first before moving it in the > cluster; actually simultaneous actions are not working for me now; is this > comming from the

Re: simultaneous actions

ll be > used to calculate simultaneous actions against the same RDD. It is likely > that many of the same Workers and Executors will be used as the Scheduler > tries to preserve data locality, but that is not guaranteed. In fact, what > is most likely to happen is that the share

Re: simultaneous actions

erform operations sequentially. >> >> As an alternative, I would suggest to restructure your RDD transformations >> to compute the required results in one single operation. >> >> On 15 January 2016 at 06:18, Jonathan Coveney <jcove...@gmail.com >> <m

Re: simultaneous actions

Same SparkContext means same pool of Workers. It's up to the Scheduler, not the SparkContext, whether the exact same Workers or Executors will be used to calculate simultaneous actions against the same RDD. It is likely that many of the same Workers and Executors will be used as the Scheduler

Re: simultaneous actions

Executors will be >> used to calculate simultaneous actions against the same RDD. It is likely >> that many of the same Workers and Executors will be used as the Scheduler >> tries to preserve data locality, but that is not guaranteed. In fact, what >> is most likely to

Re: simultaneous actions

the required results in one single operation. >>> >>> On 15 January 2016 at 06:18, Jonathan Coveney <jcove...@gmail.com> >>> wrote: >>> >>>> Threads >>>> >>>> >>>> El viernes, 15 de enero de 2016, Kira <mennou...@

Re: simultaneous actions

ns on distributed data. Even if you were to call >>>> actions from several threads at once, the individual executors of your >>>> spark environment would still have to perform operations sequentially. >>>> >>>> As an alternative, I would suggest to restructur

simultaneous actions

Hi, Can we run *simultaneous* actions on the *same RDD* ?; if yes how can this be done ? Thank you, Regards -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/simultaneous-actions-tp25977.html Sent from the Apache Spark User List mailing list archive

Re: simultaneous actions

Threads El viernes, 15 de enero de 2016, Kira <mennou...@gmail.com> escribió: > Hi, > > Can we run *simultaneous* actions on the *same RDD* ?; if yes how can this > be done ? > > Thank you, > Regards > > > > -- > View this message in context: > http:/

Re: simultaneous actions

operation. > > On 15 January 2016 at 06:18, Jonathan Coveney <jcove...@gmail.com> wrote: >> >> Threads >> >> >> El viernes, 15 de enero de 2016, Kira <mennou...@gmail.com> escribió: >>> >>> Hi, >>> >>> Can we

Re: simultaneous actions

com> wrote: > Threads > > > El viernes, 15 de enero de 2016, Kira <mennou...@gmail.com> escribió: > >> Hi, >> >> Can we run *simultaneous* actions on the *same RDD* ?; if yes how can this >> be done ? >> >> Thank you, >> Regards >&g

Re: simultaneous actions

es, 15 de enero de 2016, Kira <mennou...@gmail.com >> <javascript:_e(%7B%7D,'cvml','mennou...@gmail.com');>> escribió: >> >>> Hi, >>> >>> Can we run *simultaneous* actions on the *same RDD* ?; if yes how can >>> this >>> be done ? &g

Re: simultaneous actions

gt; wrote: > Threads > > > El viernes, 15 de enero de 2016, Kira <mennou...@gmail.com > <mailto:mennou...@gmail.com>> escribió: > Hi, > > Can we run *simultaneous* actions on the *same RDD* ?; if yes how can this > be done ? > > Thank you, >

Re: simultaneous actions

ucture your RDD transformations > to compute the required results in one single operation. > > On 15 January 2016 at 06:18, Jonathan Coveney <jcove...@gmail.com> wrote: > >> Threads >> >> >> El viernes, 15 de enero de 2016, Kira <mennou...@gmail.c

Re: simultaneous actions

>> As an alternative, I would suggest to restructure your RDD >> transformations to compute the required results in one single operation. >> >> On 15 January 2016 at 06:18, Jonathan Coveney <jcove...@gmail.com> wrote: >> >>> Threads >>&

Re: simultaneous actions

It makes sense if you're parallelizing jobs that have relatively few tasks, and have a lot of execution slots available. It makes sense to turn them loose all at once and try to use the parallelism available. There are downsides, eventually: for example, N jobs accessing one cached RDD may