Github user witgo closed the pull request at:
https://github.com/apache/spark/pull/2376
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/2376#issuecomment-57078975
I temporarily can not reproduce it, and close this PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/2376#issuecomment-55913971
I tried to reproduce it with the way the test case, but was
unsuccessful.But like the code below does appear the problem, do not know why
```scala
def
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2376#issuecomment-55509430
It seems like the issue here is that unnecessary objects are being included
in the closure, since presumably this bug would also manifest itself through
serialization
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2376#issuecomment-55480695
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20248/consoleFull)
for PR 2376 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/2376#issuecomment-55482040
What is the problem here? Can you give an example or test case?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/2376#issuecomment-55482720
@rxin Code like this:
```
val topicModel = Big object
val broadcastModel = data.context.broadcast(topicModel)
corpus =
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/2376#issuecomment-55482915
Can you add a unit test? You can just create a RDD and serialize it to test
the size.
---
If your project is set up for it, you can reply to this email and have your
reply
