WARN MemoryStore: Not enough space to cache broadcast_4 in memory! (computed 60.2 MB so far) WARN MemoryStore: Persisting block broadcast_4 to disk instead.
Can I increase the memory allocation for broadcast variables? I have a few broadcast variables that I create with sc.broadcast() . Are these labeled starting from 0 or from 1 (in reference to "broadcast_N")? I want to debug/track down which one is offending. As a feature request, it would be good if there were an optional argument (or perhaps a requireed argument) added to sc.broadcast() so that we could give it an internal label. Then it would work the same as the sc.accumulator() "name" argument. It would enable more useful warn/error messages. Arun
