Broadcast Variable question
Is there any concurrent access to broadcast variables in the workers? My use case is that the broadcast variable is a large dataset object which is *only* read. However, there is some cache in this object which is written (so it speeds up generation of entries in this dataset). Everything will work fine if there is only a single access at a time to this object. So, my question is how many threads in each worker access broadcast variables. Thanks in advance, Eduardo
Re: broadcast variable question
That's great! Thanks El martes, 28 de julio de 2015, Ted Yu yuzhih...@gmail.com escribió: If I understand correctly, there would be one value in the executor. Cheers On Tue, Jul 28, 2015 at 4:23 PM, Jonathan Coveney jcove...@gmail.com javascript:_e(%7B%7D,'cvml','jcove...@gmail.com'); wrote: i am running in coarse grained mode, let's say with 8 cores per executor. If I use a broadcast variable, will all of the tasks in that executor share the same value? Or will each task broadcast its own value ie in this case, would there be one value in the executor shared by the 8 tasks, or would there be eight copies, 1 per task?
broadcast variable question
i am running in coarse grained mode, let's say with 8 cores per executor. If I use a broadcast variable, will all of the tasks in that executor share the same value? Or will each task broadcast its own value ie in this case, would there be one value in the executor shared by the 8 tasks, or would there be eight copies, 1 per task?
Re: broadcast variable question
If I understand correctly, there would be one value in the executor. Cheers On Tue, Jul 28, 2015 at 4:23 PM, Jonathan Coveney jcove...@gmail.com wrote: i am running in coarse grained mode, let's say with 8 cores per executor. If I use a broadcast variable, will all of the tasks in that executor share the same value? Or will each task broadcast its own value ie in this case, would there be one value in the executor shared by the 8 tasks, or would there be eight copies, 1 per task?
