Thanks. That looks pretty similar to what I'm doing, with the
difference being getPeers vs getMemoryStatus. Seems like they're both
backed by the same blockManagerInfo, but getPeers is filtering in a
way that looks close to what I need. Is there a reason to prefer
getMemoryStatus?
On Thu, Mar 3
You can take a look at
"org.apache.spark.streaming.scheduler.ReceiverTracker#getExecutors"
On Thu, Mar 3, 2016 at 3:10 PM, Reynold Xin wrote:
> What do you mean by consistent? Throughout the life cycle of an app, the
> executors can come and go and as a result really has no consistency. Do you
>
What do you mean by consistent? Throughout the life cycle of an app, the
executors can come and go and as a result really has no consistency. Do you
just need it for a specific job?
On Thu, Mar 3, 2016 at 3:08 PM, Cody Koeninger wrote:
> I need getPreferredLocations to choose a consistent exec
I need getPreferredLocations to choose a consistent executor for a
given partition in a stream. In order to do that, I need to know what
the current executors are.
I'm currently grabbing them from the block manager master .getPeers(),
which works, but I don't know if that's the most reasonable wa
