[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-86789060 Thanks! Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5175 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-86779685 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-86203918 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29166/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-86203901 [Test build #29166 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29166/consoleFull) for PR 5175 at commit [`248d5c8`](https://github.com/apache/spark/commit/248d5c8dc63f98706450be5342cc98be083987bf). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-86176114 [Test build #29166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29166/consoleFull) for PR 5175 at commit [`248d5c8`](https://github.com/apache/spark/commit/248d5c8dc63f98706450be5342cc98be083987bf). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85817949 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29139/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85817935 [Test build #29139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29139/consoleFull) for PR 5175 at commit [`3fb7cce`](https://github.com/apache/spark/commit/3fb7cced8cb306c75e0dfd17c19e52e67ae4159b). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user ankurdave commented on a diff in the pull request: https://github.com/apache/spark/pull/5175#discussion_r27092877 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/VertexRDD.scala --- @@ -122,6 +122,14 @@ abstract class VertexRDD[VD]( def mapValues[VD2: ClassTag](f: (VertexId, VD) => VD2): VertexRDD[VD2] /** + * For each VertexId present in both `this` and `other`, minus will act as a set difference + * operation returning only those unique VertexId's present in `this`. + * + * @param other an RDD to run the set operation against + */ + def minus(other: RDD[(VertexId, VD)]): VertexRDD[VD] --- End diff -- It would be good to add a version that takes a VertexRDD, similar to the other methods below. That way we could avoid calling createUsingIndex when we already have two VertexRDDs that share the same index. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user ankurdave commented on a diff in the pull request:
https://github.com/apache/spark/pull/5175#discussion_r27092490
--- Diff:
graphx/src/main/scala/org/apache/spark/graphx/impl/VertexPartitionBaseOps.scala
---
@@ -88,6 +88,16 @@ private[graphx] abstract class VertexPartitionBaseOps
this.withMask(newMask)
}
+ /** Hides the VertexId's that are the same between `this` and `other`. */
+ def minus(other: Self[VD]): Self[VD] = {
+other.withMask((self.mask & other.mask) ^ self.mask)
--- End diff --
This can be simplified to `self.mask.andNot(other.mask)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85791046 [Test build #29139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29139/consoleFull) for PR 5175 at commit [`3fb7cce`](https://github.com/apache/spark/commit/3fb7cced8cb306c75e0dfd17c19e52e67ae4159b). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user brennonyork commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85788983 jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85752976 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29120/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85752927 [Test build #29120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29120/consoleFull) for PR 5175 at commit [`6575d92`](https://github.com/apache/spark/commit/6575d927cd36076db7797a12d45d6bb98f1bf43e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85741545 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29121/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user brennonyork commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85739639 /cc @maropu @ankurdave @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5175#issuecomment-85737823 [Test build #29120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29120/consoleFull) for PR 5175 at commit [`6575d92`](https://github.com/apache/spark/commit/6575d927cd36076db7797a12d45d6bb98f1bf43e). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6510][GraphX]: Add Graph#minus method t...
GitHub user brennonyork opened a pull request: https://github.com/apache/spark/pull/5175 [SPARK-6510][GraphX]: Add Graph#minus method to act as Set#difference Adds a `Graph#minus` method which will return only unique `VertexId`'s from the calling `VertexRDD`. For example: ``` Set((0L,0),(1L,1)).minus(Set((1L,1),(2L,2))) > Set((0L,0)) ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/brennonyork/spark SPARK-6510 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/5175.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5175 commit 7227c0ffd8a2ea93a3dcb28440c912921ff14380 Author: Brennon York Date: 2015-03-24T22:59:28Z beginning work on minus functionality commit aaa030b3ff04738f5ffd38b6fec3f92043359b3a Author: Brennon York Date: 2015-03-24T23:14:54Z completed graph#minus functionality commit 6575d927cd36076db7797a12d45d6bb98f1bf43e Author: Brennon York Date: 2015-03-24T23:16:09Z updated mima exclude --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
