Hi,
Shooting in the dark...it's executed on executors (it's old tech RDD-based
so not many extra optimizations like in Spark SQL now).
Can you show the code as I'm scared to hear that you're trying to broadcast
inside a transformation which I'd believe is impossible.
Jacek
On 26 Jan 2017 12:18
After more reading, I know the state is distributed across the cluster. But If
I need to lookup a map in the updatefunction, I need to broadcast it.
Just want to make sure I am on the right path.
Appreciate your help. Thanks
On Wed, Jan 25, 2017 at 2:33 PM, shyla deshpande
Is it executed on the driver or executor. If I need to lookup a map in the
updatefunction, I need to broadcast it, if mapWithState executed runs on
executor.
Thanks
