Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
Btw, of course if we can prove this helps the failed test, then maybe we
can convince @davies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
@asurve I think @davies meant it is a general OOM issue. Looks like we can
only deal with it by adjusting JVM's memory related parameters.
---
If your project is set up for it, you can reply to
Github user asurve commented on the issue:
https://github.com/apache/spark/pull/16865
What is an alternative option to avoid this issue?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
Ok. Close this for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user davies commented on the issue:
https://github.com/apache/spark/pull/16865
I still think it's not worth it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
ping @davies Do you still think this is not helpful generally?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
BTW, as the generated classes can be big, the total memory occupied of the
cache entries accumulated in executor is still considerable.
---
If your project is set up for it, you can reply to this
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
Yeah, it makes sense. However, many other pieces in executor can better
respond to memory pressure. But this cache can't.
I don't say it definitely solves this issue because maybe other
Github user davies commented on the issue:
https://github.com/apache/spark/pull/16865
@viirya This is a general OOM, should not be caused by cached bytecode,
they are way smaller comparing other things in executor, I think this patch
will not help either.
---
If your project is set
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16865
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72671/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16865
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16865
**[Test build #72671 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72671/testReport)**
for PR 16865 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
My colleagues recently have a case that their codes throw an exception like
this:
12:45:10at
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16865
**[Test build #72671 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72671/testReport)**
for PR 16865 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user davies commented on the issue:
https://github.com/apache/spark/pull/16865
I understand the motivation here, could you show the benefit of this change
for a real use case?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
Jenkins seems inaccessible now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16865
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16865
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72632/
Test FAILed.
---
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16865
cc @davies
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16865
**[Test build #72632 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72632/testReport)**
for PR 16865 at commit
21 matches
Mail list logo
