Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/3518#issuecomment-70394250
Hey @ilganeli - I took a slightly deeper look this time. I still don't
totally follow how this all hooks together, but I wonder if it's possible to
write a single utility function that is much simpler. It would just do the
following:
```
/**
* Given an object reference, recursively traverses all fields of the
reference,
* fields of objects within those fields, and so on. If any of those
references
* are neither Serializable nor Externalizable, prints the path from the
root object
* to the reference.
*/
def findNonSerailizableReferences(root: AnyRef): String {
}
```
And it would do something like:
1. Start with the root reference.
2. Traverse the graph of all referred-to objects, maintaining path
information. Path information means both the sequence of parent pointers and
the field name.
3. Check whether Serializable.class.isAssignableFrom(c) or
Externalizable.class.isAssignableFrom(c) for any object encountered, where c is
the class of the object.
4. When the first object that isn't serializable is encountered, print the
path to that object.
This wouldn't work for custom serializers, it would only work for the Java
serializer. However, that's all we support for closure's anyways.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
