Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8758#issuecomment-143400142
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user rekhajoshm commented on the pull request:
https://github.com/apache/spark/pull/8758#issuecomment-143400118
The error is not related to this pull
```
[error] SERVER ERROR: Service Temporarily Unavailable
Github user rekhajoshm commented on the pull request:
https://github.com/apache/spark/pull/8758#issuecomment-143400127
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8758#issuecomment-143400137
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user viirya closed the pull request at:
https://github.com/apache/spark/pull/8903
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/8903#issuecomment-143398042
@cloud-fan You are right. This patch still can't eliminate the extra
conversion. Close it now.
---
If your project is set up for it, you can reply to this email and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8758#issuecomment-143400329
[Test build #43047 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43047/consoleFull)
for PR 8758 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143240249
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143240216
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8914#issuecomment-143243915
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143246283
[Test build #43025 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43025/console)
for PR 8917 at commit
Github user buckhx commented on a diff in the pull request:
https://github.com/apache/spark/pull/4897#discussion_r40440718
--- Diff: python/pyspark/context.py ---
@@ -711,6 +721,30 @@ def addPyFile(self, path):
# for tests in local mode
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143241460
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user nyaapa commented on the pull request:
https://github.com/apache/spark/pull/7345#issuecomment-143269671
*let me remind you about this discussion and ipv6 support at all.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143241488
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user buckhx commented on the pull request:
https://github.com/apache/spark/pull/4897#issuecomment-143251274
The tests fail because one of them attempts to pip install a package and
doesn't have permissions to do so. Is there a way to enable that? Or just leave
the pip
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143243961
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143243962
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8914#issuecomment-143243883
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143246410
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143246407
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/8881#issuecomment-143269767
I cannot remember very clearly, `count()` may introduce incorrect result
when you use `None` instead of `RDD.empty` as I remembered.
In the early version some
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8914#issuecomment-143244694
[Test build #43026 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43026/consoleFull)
for PR 8914 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8917#issuecomment-143243183
[Test build #43025 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43025/consoleFull)
for PR 8917 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8912#issuecomment-143190405
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8912#issuecomment-143190406
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8912#issuecomment-143190324
[Test build #43018 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43018/console)
for PR 8912 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8918#issuecomment-143192934
[Test build #43019 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43019/console)
for PR 8918 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143284017
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143295346
[Test build #43028 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43028/consoleFull)
for PR 8919 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143295384
[Test build #43029 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43029/consoleFull)
for PR 8919 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8914#issuecomment-143297138
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143344646
[Test build #43032 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43032/console)
for PR 8910 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143344733
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143344736
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143354584
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143354587
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143354540
[Test build #43033 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43033/console)
for PR 8910 at commit
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/8889#discussion_r40476685
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -553,60 +553,28 @@ private[hive] class
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/8889#discussion_r40476708
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -553,60 +553,28 @@ private[hive] class
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/8889#discussion_r40477126
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
---
@@ -1223,4 +1223,26 @@ class SQLQuerySuite extends QueryTest
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143359655
ah that makes sense, I guess I forgot the Client was a public API.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143356542
@vanzin so in my own code (where I do try and switch between yarn and non
yarn mode) I clear the SPARK_YARN_MODE as done in the test.
I could update
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143356986
Makes sense, do you think I should put that change in the SparkContext (on
startup of non-yarn client or stop of any client) or in the yarnclient stop
code?
---
If
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143356828
Yeah, having the Spark code clean up after itself is easier because it
means people don't have to remember to do it, and it doesn't need to be
documented.
---
If your
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/8889#issuecomment-143358045
LGTM except for a few minor issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143359407
I don't think so. `Client.scala`, for better or for worse, is still a
public API. So you can submit a `yarn-cluster` job by calling `Client.scala`
directly, and that
Github user tedyu commented on the pull request:
https://github.com/apache/spark/pull/8896#issuecomment-143360758
What if we write our own version of ObjectOutputStream with weak references
instead of using strong references ?
---
If your project is set up for it, you can reply to
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143349393
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143349374
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143349924
There's code in different places that set `SPARK_YARN_MODE`, but there's no
code to unset it. So, to follow your example, if you start a context with
yarn-client and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143349972
[Test build #43033 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43033/consoleFull)
for PR 8910 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8879#issuecomment-143354947
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8879#issuecomment-143354966
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/8879#issuecomment-143356109
@srowen so I refactored the shared code, but for serialization reasons I
put it in the companion object. Let me know if this looks good to you :)
---
If your project
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/8889#discussion_r40476942
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -553,60 +553,28 @@ private[hive] class
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143358255
I did a cursory lookup for where it is set, and I think the places that
need to be changed are `SparkContext.stop()` and YARN's `Client.scala`.
Doing it in
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143359153
@vanzin so it _seems_ like if I do it in SparkContext shutdown that should
be sufficient for all cases?
---
If your project is set up for it, you can reply to this
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/8907#discussion_r40451726
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala ---
@@ -145,7 +145,13 @@ final class Decimal extends Ordered[Decimal] with
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/8910#discussion_r40457129
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -314,5 +314,28 @@ object YarnSparkHadoopUtil {
def
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143304877
LGTM pending a couple of minor changes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8668#issuecomment-143307328
FYI: #8914 makes some changes to these tests to avoid the races I alluded
to in my last comment.
---
If your project is set up for it, you can reply to this email and
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8908#issuecomment-143318263
Also could you change the PR title to `[SPARK-10807][SPARKR]` ? This
matches the format we use for all PRs --
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/8903#issuecomment-143320012
+1. It seems to me that if the two flags always have the same value, we
don't need two flags.
---
If your project is set up for it, you can reply to this email and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143324304
[Test build #43030 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43030/consoleFull)
for PR 8910 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143325990
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143325969
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8911#issuecomment-143328680
[Test build #43031 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43031/consoleFull)
for PR 8911 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143331196
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143331056
[Test build #43027 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43027/console)
for PR 8910 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143334636
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143334652
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143335842
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143335895
[Test build #43032 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43032/consoleFull)
for PR 8910 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143335845
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8870
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143323853
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143323932
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143331198
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/8910#discussion_r40468010
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -39,7 +39,7 @@ import org.apache.hadoop.yarn.util.ConverterUtils
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8870#issuecomment-143320666
LGTM - Thanks @NarineK -- Merging this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user harishreedharan commented on the pull request:
https://github.com/apache/spark/pull/8867#issuecomment-143322902
@SaintBacchus Right. That is what I was talking about above. I think your
fix takes care of this issue.
---
If your project is set up for it, you can reply to
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143331411
[Test build #43030 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43030/console)
for PR 8910 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143331462
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143331463
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-14675
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143335735
[Test build #43029 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43029/console)
for PR 8919 at commit
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/8747#issuecomment-143313189
I tried to generate as less code as possible and move the code to real java
classes, to debug and maintain it easier.
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143284102
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143283745
Thanks @srowen for your review, I've updated the codes according to your
comments.
---
If your project is set up for it, you can reply to this email and have your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8910#issuecomment-143289715
[Test build #43027 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43027/consoleFull)
for PR 8910 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143293763
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143293685
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8914#issuecomment-143296899
[Test build #43026 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43026/console)
for PR 8914 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143299299
[Test build #43028 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43028/console)
for PR 8919 at commit
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/8908#issuecomment-143306461
Thanks @olarayej -- I just had a couple of minor style comments
One more thing is that it will be good to add a unit test for this. You
could just add a test
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8914#issuecomment-143306821
LGTM but Andrew should take a look. I looked at this code recently and
really found it odd that it expected things to just be updated "atomically", so
good thing I was
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/5423#issuecomment-143313000
Hi guys, current folder structure under yarn is:
```
yarn ---
history ---
src ---
main ---
test ---
src
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143294898
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8919#issuecomment-143294928
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
1 - 100 of 306 matches
Mail list logo
