[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/7459 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-123400607 Nope, merging into master, thanks for your changes! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-123208446 @andrewor14 any other comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122640067 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122640048 [Test build #37762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37762/console) for PR 7459 at commit [`7c9858d`](https://github.com/apache/spark/commit/7c9858db0f8374c8f124b4a964190ad2ff5ad898). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122633283 [Test build #37762 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37762/consoleFull) for PR 7459 at commit [`7c9858d`](https://github.com/apache/spark/commit/7c9858db0f8374c8f124b4a964190ad2ff5ad898). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122633262 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122633267 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/7459#discussion_r34953374 --- Diff: core/src/test/scala/org/apache/spark/util/ClosureCleanerSuite2.scala --- @@ -171,14 +167,20 @@ class ClosureCleanerSuite2 extends SparkFunSuite with BeforeAndAfterAll with Pri val closure2 = () => localValue val closure3 = () => someSerializableValue val closure4 = () => someSerializableMethod() -val outerClasses1 = getOuterClasses(closure1) -val outerClasses2 = getOuterClasses(closure2) -val outerClasses3 = getOuterClasses(closure3) -val outerClasses4 = getOuterClasses(closure4) -val outerObjects1 = getOuterObjects(closure1) -val outerObjects2 = getOuterObjects(closure2) -val outerObjects3 = getOuterObjects(closure3) -val outerObjects4 = getOuterObjects(closure4) + +val outerClassesAndObject1 = getOuterClassesAndObjects(closure1) --- End diff -- instead of relying on underscores `outerClassesAndObjects._1` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/7459#discussion_r34953369 --- Diff: core/src/test/scala/org/apache/spark/util/ClosureCleanerSuite2.scala --- @@ -171,14 +167,20 @@ class ClosureCleanerSuite2 extends SparkFunSuite with BeforeAndAfterAll with Pri val closure2 = () => localValue val closure3 = () => someSerializableValue val closure4 = () => someSerializableMethod() -val outerClasses1 = getOuterClasses(closure1) -val outerClasses2 = getOuterClasses(closure2) -val outerClasses3 = getOuterClasses(closure3) -val outerClasses4 = getOuterClasses(closure4) -val outerObjects1 = getOuterObjects(closure1) -val outerObjects2 = getOuterObjects(closure2) -val outerObjects3 = getOuterObjects(closure3) -val outerObjects4 = getOuterObjects(closure4) + +val outerClassesAndObject1 = getOuterClassesAndObjects(closure1) --- End diff -- you should just do ``` val (outerClasses1, outerObjects1) = getOuterClassesAndObjects(closure1) ... ``` to be more concise --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122509603 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122509580 [Test build #37690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37690/console) for PR 7459 at commit [`2ec5ce1`](https://github.com/apache/spark/commit/2ec5ce16566444f72ebd2f7f99ed1d8fc78c45e6). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class IntArrayParam(parent: Params, name: String, doc: String, isValid: Array[Int] => Boolean)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122485381 [Test build #37690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37690/consoleFull) for PR 7459 at commit [`2ec5ce1`](https://github.com/apache/spark/commit/2ec5ce16566444f72ebd2f7f99ed1d8fc78c45e6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122485082 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122485080 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122485016 @dragos @andrewor14 thanks for suggestions. I updated this and removed the two methods. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7459#discussion_r34918452
--- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala ---
@@ -205,8 +222,7 @@ private[spark] object ClosureCleaner extends Logging {
// A list of enclosing objects and their respective classes, from
innermost to outermost
// An outer object at a given index is of type outer class at the same
index
-val outerClasses = getOuterClasses(func)
-val outerObjects = getOuterObjects(func)
--- End diff --
+1 the old methods should go away
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122368693 Thanks for cleaning up this piece of code @viirya! As @dragos pointed out we should just replace the old methods, which are no longer used. As long as we have sufficient unit tests for the new method (which we do for `ClosureCleaner`) we can proceed with the changes with confidence. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/7459#discussion_r34903106
--- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala ---
@@ -205,8 +222,7 @@ private[spark] object ClosureCleaner extends Logging {
// A list of enclosing objects and their respective classes, from
innermost to outermost
// An outer object at a given index is of type outer class at the same
index
-val outerClasses = getOuterClasses(func)
-val outerObjects = getOuterObjects(func)
--- End diff --
Indeed. I don't remove them now just because there are too many unit test
for them. I will take time to remove them and modify these tests. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/7459#discussion_r34899049
--- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala ---
@@ -205,8 +222,7 @@ private[spark] object ClosureCleaner extends Logging {
// A list of enclosing objects and their respective classes, from
innermost to outermost
// An outer object at a given index is of type outer class at the same
index
-val outerClasses = getOuterClasses(func)
-val outerObjects = getOuterObjects(func)
--- End diff --
You should remove these methods (or, if they're useful in their own right,
re-implement them in terms of the new method). Now there's three
implementations of the same logic.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122200530 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122200477 [Test build #37584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37584/console) for PR 7459 at commit [`dc110d1`](https://github.com/apache/spark/commit/dc110d149e0cc24a19e49bac56ea793186b1e10a). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122177379 [Test build #37584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37584/consoleFull) for PR 7459 at commit [`dc110d1`](https://github.com/apache/spark/commit/dc110d149e0cc24a19e49bac56ea793186b1e10a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122176847 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7459#issuecomment-122176915 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9128][Core] Get outerclasses and object...
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/7459 [SPARK-9128][Core] Get outerclasses and objects with only one method calling in ClosureCleaner JIRA: https://issues.apache.org/jira/browse/SPARK-9128 Currently, in `ClosureCleaner`, the outerclasses and objects are retrieved using two different methods. However, the logic of the two methods is the same, and we can get both the outerclasses and objects with only one method calling. You can merge this pull request into a Git repository by running: $ git pull https://github.com/viirya/spark-1 remove_extra_closurecleaner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7459.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7459 commit dc110d149e0cc24a19e49bac56ea793186b1e10a Author: Liang-Chi Hsieh Date: 2015-07-17T05:01:30Z Add method to get outerclasses and objects at the same time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
