[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/7469#issuecomment-122718973 Oh, thanks @srowen. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7469#issuecomment-122638035 Thanks @ankurdave -- you can follow this by resolving the issue (done already now) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/7469#issuecomment-122379908 Thanks for finding the leak here. Merged into master and branch-1.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/7469 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7469#issuecomment-122303342
[Test build #1095 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1095/console)
for PR 7469 at commit
[`8d87997`](https://github.com/apache/spark/commit/8d879979241050be5918d9630bc5bdec9f62be79).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `logDebug("isMulticlass = " + metadata.isMulticlass)`
* ` * (i.e., if isMulticlass &&
isSpaceSufficientForAllCategoricalSplits),`
* `logDebug("isMulticlass = " + metadata.isMulticlass)`
* `abstract class UnsafeProjection extends Projection `
* `case class FromUnsafeProjection(fields: Seq[DataType]) extends
Projection `
* `abstract class BaseProjection extends Projection `
* `class SpecificProjection extends $`
* `class SpecificProjection extends $`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7469#issuecomment-122296378 [Test build #1095 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1095/consoleFull) for PR 7469 at commit [`8d87997`](https://github.com/apache/spark/commit/8d879979241050be5918d9630bc5bdec9f62be79). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/7469#issuecomment-122296197 @ankurdave @jegonzal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7469#issuecomment-122296071 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7469#issuecomment-122295869 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9109] [GraphX] Keep the cached edge in ...
GitHub user tien-dungle opened a pull request:
https://github.com/apache/spark/pull/7469
[SPARK-9109] [GraphX] Keep the cached edge in the graph
The change here is to keep the cached RDDs in the graph object so that when
the graph.unpersist() is called these RDDs are correctly unpersisted.
import org.apache.spark.graphx._
import org.apache.spark.rdd.RDD
import org.slf4j.LoggerFactory
import org.apache.spark.graphx.util.GraphGenerators
// Create an RDD for the vertices
val users: RDD[(VertexId, (String, String))] =
sc.parallelize(Array((3L, ("rxin", "student")), (7L, ("jgonzal",
"postdoc")),
(5L, ("franklin", "prof")), (2L, ("istoica",
"prof"
// Create an RDD for edges
val relationships: RDD[Edge[String]] =
sc.parallelize(Array(Edge(3L, 7L, "collab"),Edge(5L, 3L, "advisor"),
Edge(2L, 5L, "colleague"), Edge(5L, 7L, "pi")))
// Define a default user in case there are relationship with missing user
val defaultUser = ("John Doe", "Missing")
// Build the initial Graph
val graph = Graph(users, relationships, defaultUser)
graph.cache().numEdges
graph.unpersist()
sc.getPersistentRDDs.foreach( r => println( r._2.toString))
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tien-dungle/spark SPARK-9109_Graphx-unpersist
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7469.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7469
commit 8d879979241050be5918d9630bc5bdec9f62be79
Author: tien-dungle
Date: 2015-07-17T14:25:25Z
Keep the cached edge in the graph
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
