Agree, I'm not pushing for it unless there's other evidence. The closure check does entail serialization, not just checking serializability, note. I don't like flags either but this one sounded like it could actually be something a user wanted to vary, globally, for runs of the same code.
On Tue, Jan 22, 2019 at 11:25 AM Reynold Xin <r...@databricks.com> wrote: > Typically very large closures include some array, and the serialization > itself should be much more expensive than the closure check. Does anybody > have actual data on this could be a problem? We don't want to add a config > flag if for virtually any case it doesn't make sense to change. > >
