[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-06-03 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-108749998 @jacek-lewandowski can you close this issue? It didnt' close properly because of the way github auto-closes patches into release branches. --- If your project is set u

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4221 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-74952993 Yeah our auto-close doesn't work on PR's into release branches like this. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-18 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-74858242 Mind closing this PR @jacek-lewandowski ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-08 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-73394982 I've merged this into `branch-1.2` (1.2.2). Thanks! @jacek-lewandowski, you can close this PR now that it's been merged, since GitHub can't auto close. --- I

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-73394939 [Test build #27019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27019/consoleFull) for PR 4221 at commit [`87951a2`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-73394942 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-08 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-73392012 I'm going to merge this into `branch-1.2` (1.2.2) now since I'd like to see if resolves https://issues.apache.org/jira/browse/SPARK-5227. --- If your project is set u

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-73392001 [Test build #27019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27019/consoleFull) for PR 4221 at commit [`87951a2`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-08 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-73391966 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread jacek-lewandowski
Github user jacek-lewandowski closed the pull request at: https://github.com/apache/spark/pull/4220 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72549252 Let's leave this PR open for a few days until we complete the 1.2.1 release voting, then I'll merge it into `branch-1.2` (1.2.2). --- If your project is set up for it,

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72549138 I've merged this PR into `branch-1.1` (1.1.2), so feel free to close it (GitHub won't auto-close PRs that are opened against maintenance branches). --- If your project

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4222 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72548525 I've merged this into `master` (1.3.0). Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72540322 @jacek-lewandowski This looks good to me, so I'll merge it in a few minutes. It looks like your PRs against the other branches have also been updated, so I'll pull tho

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72529009 @JoshRosen is it ok to go now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72432199 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72432191 [Test build #26498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26498/consoleFull) for PR 4221 at commit [`87951a2`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72428826 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72428814 [Test build #26500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26500/consoleFull) for PR 4222 at commit [`03da61b`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72428104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72428095 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26497/consoleFull) for PR 4220 at commit [`6c48a1f`](https://github.com/a

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72421545 [Test build #26500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26500/consoleFull) for PR 4222 at commit [`03da61b`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72420654 [Test build #26498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26498/consoleFull) for PR 4221 at commit [`87951a2`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-02-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72420242 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26497/consoleFull) for PR 4220 at commit [`6c48a1f`](https://github.com/ap

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-31 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72309821 Agreed, I'll make the changes and add the clarifying comments. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72308691 > Maybe a more clear solution would be to modify `Utils.getSystemProperties` to use `stringPropertyNames` to make a copy and return just a simple Map instead?

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72308249 `Utils.getSystemProperties` is fine but not as a replacement of `stringProperties`. There were actually two problems - one was related to `ConcurrentModificatio

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72306613 I did a little digging and it looks like this ConcurrentModificationException error was fixed previously (8f1098a3f0de8c9b2eb9ede91a1b01da10a525ea) but that the fix may

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72252306 @jacek-lewandowski a `Properties` is just a `Hashtable` so you can make a new `Properties` and `addAll`. No need to even clone. --- If your project is set up for it, you

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72249820 Could we get some closure on this? The broken `ResetSystemProperties` causes `core/` tests to fill up my disk currently (and run a lot more slowly than needed), because of

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72274441 It serializes the object and then deserializes so I suppose this is a deep copy. For the `stringPropertyNames` - you can, but this will not be a 1:1 co

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72268511 @jacek-lewandowski I think Sean meant that you can do `new Properties().putAll(oldProperties)` instead of cloning. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72272628 Look at this simple example: ```scala val parent = new Properties() parent.setProperty("test1", "A") val child = new Properties(parent) c

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72273382 Ah OK, interesting. On another note, does the clone make a deep copy? that's necessary I think? Anyway, worst case, you can iterate over `stringPropertyNames` and `put`

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72273213 I see, makes sense, thanks for the details. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72286197 @JoshRosen who investigated this a bunch for tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72268328 @srowen - unfortunately they are something more - they inherit from the `HashTable` but they makes a hierarchy by referencing the parent `Properties` which are

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-30 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72168541 @JoshRosen can you take a look and maybe merge it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72118648 So is it going to be merged? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72096204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72096193 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26327/consoleFull) for PR 4220 at commit [`74b4489`](https://github.com/a

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72089151 [Test build #26321 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26321/consoleFull) for PR 4221 at commit [`01dd5cb`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72089157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72082457 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26327/consoleFull) for PR 4220 at commit [`74b4489`](https://github.com/ap

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72082093 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72081475 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26323/consoleFull) for PR 4220 at commit [`74b4489`](https://github.com/a

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72081490 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72080808 [Test build #26322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26322/consoleFull) for PR 4222 at commit [`8faf2ea`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72080823 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72079687 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72069561 [Test build #26322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26322/consoleFull) for PR 4222 at commit [`8faf2ea`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72069575 [Test build #26321 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26321/consoleFull) for PR 4221 at commit [`01dd5cb`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72069566 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26323/consoleFull) for PR 4220 at commit [`74b4489`](https://github.com/ap

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72051450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72051431 [Test build #26319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26319/consoleFull) for PR 4222 at commit [`4b6f366`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4220#discussion_r23776203 --- Diff: core/src/test/scala/org/apache/spark/util/ResetSystemProperties.scala --- @@ -42,7 +43,7 @@ private[spark] trait ResetSystemProperties extends Befor

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72033735 [Test build #26319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26319/consoleFull) for PR 4222 at commit [`4b6f366`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72021290 [Test build #26312 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26312/consoleFull) for PR 4221 at commit [`4a0c50d`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72021294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72014861 [Test build #26315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26315/consoleFull) for PR 4222 at commit [`00f11a1`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72014870 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72009138 [Test build #26315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26315/consoleFull) for PR 4222 at commit [`00f11a1`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72008957 Explanation of the second commit is in https://github.com/apache/spark/pull/4220 --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-72008939 Explanation of the second commit is in https://github.com/apache/spark/pull/4220 --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-72008603 [Test build #26312 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26312/consoleFull) for PR 4221 at commit [`4a0c50d`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72004360 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26308/consoleFull) for PR 4220 at commit [`d821b15`](https://github.com/a

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72004368 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71993275 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26308/consoleFull) for PR 4220 at commit [`d821b15`](https://github.com/ap

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71992373 It looks like there is `ResetSystemProperties` trait which works completely wrong. Its aim is to save system properties before the test and the reset them after

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-29 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71988851 I found that there is a small difference in the outcome of my code in comparison to the original code. I don't know how it affect this test suite yet, but: the

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71923968 Kinda diverging into off-topic territory, but... > It seems like having multiple apps futzing w/ the system properties at the same time is bound to create problem

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71923634 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71923618 [Test build #26248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26248/consoleFull) for PR 4222 at commit [`51987d2`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71923273 I kinda see what is going on with the tests now. A [test case in SparkSubmitSuite](https://github.com/apache/spark/blob/master/core/src/test/scala/org/apache/spark/deploy/

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71915276 [Test build #26248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26248/consoleFull) for PR 4222 at commit [`51987d2`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71914714 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71896740 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26234/consoleFull) for PR 4220 at commit [`685780e`](https://github.com/a

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71896752 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71885151 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26234/consoleFull) for PR 4220 at commit [`685780e`](https://github.com/ap

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71884820 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71855275 retest this please hopefully those test failures were random, lets see. btw, I think that if you want the exact same patch applied to multiple branches, the stand

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71774621 LGTM. I was worried about `System.getProperty()`'s thread-safety, but I assume it's ultimately synchronized since the underlying store is a `Properties` object. --- If y

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4220#discussion_r23663471 --- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala --- @@ -17,6 +17,10 @@ package org.apache.spark +import java.util.concu

Re: [GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread Sean Owen
The test failures look unrelated, and are a Jenkins error. You should just make one PR for master; it will be back-ported as needed. On Tue, Jan 27, 2015 at 1:58 PM, jacek-lewandowski wrote: > Github user jacek-lewandowski commented on the pull request: > > https://github.com/apache/spark/pu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71651118 What is going on with these tests??? I've created three PRs - for 1.1, 1.2 and 1.3 and all of them failed in a very strange way. --- If your project is set up

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-71649841 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-71649835 [Test build #26161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26161/consoleFull) for PR 4221 at commit [`94aeacf`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71646575 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71646568 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26159/consoleFull) for PR 4220 at commit [`685780e`](https://github.com/a

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71644827 [Test build #26160 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26160/consoleFull) for PR 4222 at commit [`51987d2`](https://gith

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71644832 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71640081 @srowen exactly - that was the idea :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71639954 It does look safer, since `stringPropertyNames` will create a copy of the keys. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4221#issuecomment-71639726 [Test build #26161 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26161/consoleFull) for PR 4221 at commit [`94aeacf`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71639706 [Test build #26160 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26160/consoleFull) for PR 4222 at commit [`51987d2`](https://githu

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71639169 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26159/consoleFull) for PR 4220 at commit [`685780e`](https://github.com/ap

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread jacek-lewandowski
GitHub user jacek-lewandowski opened a pull request: https://github.com/apache/spark/pull/4221 SPARK-5425: Use synchronised methods in system properties to create SparkConf You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacek-l

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-27 Thread jacek-lewandowski
GitHub user jacek-lewandowski opened a pull request: https://github.com/apache/spark/pull/4222 SPARK-5425: Use synchronised methods in system properties to create SparkConf You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacek-l

  1   2   >