Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/14579
@nchammas to answer your question above
(https://github.com/apache/spark/pull/14579#issuecomment-239185438) - in short
no.
The semantics of the utility method will be the same as for the `closing`
example. `persisted` will return a wrapper class that implements the context
manager methods. When entering the `with` statement, the `__enter__` method is
called, which returns the underlying `rdd` instance - this is in turn bound to
the variable following `as`. So it would look something like this:
```python
class persisted():
def __init__(self, thing):
self.thing = thing
def __enter__(self):
return self.thing
def __exit__(self, *exc_info):
self.thing.unpersist()
```
If someone tries to do `persisted(rdd).map(...)` it will throw an
`AttributeError`.
The "file-like" version is what is currently implemented in this PR, and it
works if `__enter__` returns `self` which is what `file` et al do. Of course
those classes seem to not tend to have other methods that return `self` so
don't suffer the same chaining issue we run into with `RDD`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
