Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
This was discussed way before RC1 was cut. Not adding this to 2.4 makes the
timeline for this API to be out way too long in the future, and we want people
to try it out.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22192
why do we merge a new API to branch-2.4, when RC1 is already out?
---
-
To unsubscribe, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
merging to master / 2.4.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96276/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #96276 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96276/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #96276 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96276/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96211/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #96211 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96211/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96212/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #96212 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96212/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #96212 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96212/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #96211 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96211/testReport)**
for PR 22192 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
Hi, just want to follow up on this and see if anyone has additional
comments/questions/issues. Please feel free to ping me and I'll respond as soon
as I can!
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95947/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95947 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95947/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95947 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95947/testReport)**
for PR 22192 at commit
Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95915/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95915 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95915/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95915 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95915/testReport)**
for PR 22192 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/22192
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please.
It's that old "java.lang.reflect.InvocationTargetException: null" error
we've seen many times.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #4333 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4333/testReport)**
for PR 22192 at commit
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
I feel like it should be unrelated as well. It's strange that I failed
tests in the same suite twice in a row, and that no other recent build has
failed that suite, but I tried running locally on a
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #4333 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4333/testReport)**
for PR 22192 at commit
Github user squito commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please.
I took a look at the failures, pretty certain its unrelated, and I filed
https://issues.apache.org/jira/browse/SPARK-25400 to increase the timeouts in
one of those
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95887/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95887 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95887/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95887 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95887/testReport)**
for PR 22192 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
Logs suddenly cut off again without any exceptions, don't think this one is
a code error as well. Could someone retest this please?
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95862/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95862 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95862/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95862 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95862/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95814/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95814 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95814/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95810/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95810 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95810/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95814 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95814/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95806/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95806 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95806/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95810 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95810/testReport)**
for PR 22192 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
it's ok to keep them in the same suite where they are used.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
If I want to add more testing plugins (i.e. to test that a plugin failing
to shut down doesn't affect other plugins), is it stylistically ok for me to
keep throwing them at the bottom of
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
^This commit is to show what that looks like. I'm still looking into how to
properly barrier the plugins after calling `init` from a new thread (since
`join`ing the thread requires the thread to die
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95806 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95806/testReport)**
for PR 22192 at commit
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
I tested @vanzin's suggestion of storing the classloader, changing it to
replClassLoader, and changing back after init (and then changing again for
shutdown). It's working well (and this time I made
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95772 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95772/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95772/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95772 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95772/testReport)**
for PR 22192 at commit
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
@mridulm I'm inclined to agree. I was doing some extra testing on separate
thread and it seems to be performing well. I'll push this shortly. I'm going to
also test setting the class loader and then
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/22192
@vanzin @NiharS I am uncomfortable with that change as well - which is why
I wanted the initialization to be pushed into a separate thread (and then join)
- if we really need to set the context
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
> It seems like the line
Thread.currentThread().setContextClassLoader(replClassLoader) is causing the
pyspark failures
What if you restore the original class loader after initializing the
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
> I'll change the config documentation to specify YARN only, hopefully it's
not a huge issue.
I don't think you're really understanding what I'm saying above. I'm not
saying this feature
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
I'll change the config documentation to specify YARN only, hopefully it's
not a huge issue.
It seems like the line
`Thread.currentThread().setContextClassLoader(replClassLoader)` is causing
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
> I'm looking up other spark-submit options that can be used to provide the
jar to other nodes
I don't think there is anything like that currently; so using `--jars` for
this will only work
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
You're right, I ran in local-cluster and it exited very quickly citing
executors shutting down after not being able to find my test plugin. Although,
the logs say that it does use a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95681/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95681 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95681/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95681 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95681/testReport)**
for PR 22192 at commit
Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
I believe this is another glitch, not from my changes. Could someone retest
this please?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95584/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95584 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95584/testReport)**
for PR 22192 at commit
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
Tried on a different machine and those tests pass, here's hoping
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95584 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95584/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95567/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95567 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95567/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95565/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95565 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95565/testReport)**
for PR 22192 at commit
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
These tests are failing locally for me, both with my code and without on a
clean pull of master
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95559/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95559 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95559/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95567 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95567/testReport)**
for PR 22192 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95565 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95565/testReport)**
for PR 22192 at commit
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
Can't be a coincidence...it's failing two pyspark streaming tests,
test_binary_records_stream and test_text_file_stream (same two earlier, I
misanalyzed it when I said it was ML stuff). I'm running
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95544/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95544 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95544/testReport)**
for PR 22192 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95559 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95559/testReport)**
for PR 22192 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95557/
Test FAILed.
---
1 - 100 of 134 matches
Mail list logo
