Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21977
Seems okay.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
Looks like tests are passing now.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95175/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95175 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95175/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95167/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95167 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95167/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95175 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95175/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95167 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95167/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95165 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95165/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95165/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95165 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95165/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
This is close. The Java and Scala tests were passing and I think I fixed
the remaining issue for the Python tests. Unfortunately, Scala tests are
failing again and I was trying to run tests a couple
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21977
Is this PR close to getting merged? Or do we have some problems that are
hard to solve?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95108/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95108 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95108/testReport)**
for PR 21977 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95108 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95108/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95064/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95064 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95064/testReport)**
for PR 21977 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95064 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95064/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95050/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95050 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95050/testReport)**
for PR 21977 at commit
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
@BryanCutler, thanks for taking a look at this.
Despite the problems you hit when the limit was set too low, I think we do
want to use that limit. It was the most reliable one from our
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95050 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95050/testReport)**
for PR 21977 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21977
@holdenk pyarrow uses a C++ based memory pool, so I'm not sure how that
exactly works with rlimit but I ran some tests and looks like an error is
thrown when the limit is set.
**with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95039/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95039 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95039/testReport)**
for PR 21977 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #95039 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95039/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
Looks like the test problems were caused by accessing the SparkConf through
either SparkContext or SparkSession on the executor side. The Scala tests are
passing and I've fixed a couple more
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94980/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94980 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94980/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94980 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94980/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94804/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94804 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94804/testReport)**
for PR 21977 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94804 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94804/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
@holdenk, what could be the cause?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/21977
Actually looking at the Jenkins failures, could be the cause of some of
them as well (certainly the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94750/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94750 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94750/testReport)**
for PR 21977 at commit
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
You're right on the line number. Maybe it was that I hadn't done a full
rebuild in this branch locally before running the test. I'll look into the
other error if that's consistent in the Jenkins
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21977
it fails consistently for me locally too, with your branch, but with this
failure:
```
[info] File
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
@squito, the last Jenkins test had a different error message and two of the
tests didn't show the stdout/stderr output. I updated the other tests to show
that output, and hopefully we get a run with
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21977
@rdblue looking ...
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94750 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94750/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94710/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94710 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94710/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94703/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94703 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94703/testReport)**
for PR 21977 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94710 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94710/testReport)**
for PR 21977 at commit
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
@squito, I updated `YarnClusterSuite` in d58ad7a to capture the output of
the child processes to find out what is causing the test failures. I think it
is related to the commit you reviewed,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21977
**[Test build #94703 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94703/testReport)**
for PR 21977 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
@gatorsmile, we tried both RLIMIT_HEAP and RLIMIT_RSS but those limits
didn't consistently work.
---
-
To unsubscribe, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21977
Why not using `resource.RLIMIT_RSS`?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/21977
What about R, do we also need a similar setting for R? I was thinking that
with project hydrogen, more and more external processes will be run inside the
Spark's executor (MPP), all these
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
> So if users don't set this conf, the behavior is the same as before,
right?
Yes.
> My concern is that meaning of the overhead parameter becomes pretty
confusing.
I think
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21977
> We've found that python requires a lot less memory than it actually uses
because it doesn't know when to GC
yes, totally agree, sorry I wasn't clear in my initial comment -- overall I
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
@squito, this is much more clear for our user base. Right now, they can
control the YARN container allocation to make room for python by increasing the
overhead, but that does nothing to actually
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21977
Does this work by setting some container configs? Maybe we can apply this
to k8s later, cc @liyinan926
---
-
To unsubscribe,
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21977
Yes, this is for YARN only. I've also opened follow-up issues for Mesos and
Kubernetes integration.
---
-
To unsubscribe,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21977
cc @jiangxb1987 @cloud-fan @jerryshao @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21977
@rdblue Is this for YARN only?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94386/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21977
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
1 - 100 of 204 matches
Mail list logo
