[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/547#issuecomment-91048962 Thanks so much for the review @fhueske, @StephanEwen! I'll make the suggested changes and merge :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/547#issuecomment-90883704 I like the change as well! Good to go! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/547#discussion_r27955651
--- Diff:
flink-staging/flink-gelly/src/main/java/org/apache/flink/graph/Graph.java ---
@@ -1118,30 +1119,52 @@ public boolean filter(EdgeK, EV edge) {
}
/**
-* Create a Vertex-Centric iteration on the graph.
-*
+* Runs a Vertex-Centric iteration on the graph.
+* No configuration options are provided.
+
* @param vertexUpdateFunction the vertex update function
* @param messagingFunction the messaging function
* @param maximumNumberOfIterations maximum number of iterations to
perform
-* @return
+*
+* @return the updated Graph after the vertex-centric iteration has
converged or
+* after maximumNumberOfIterations.
*/
- public M VertexCentricIterationK, VV, M, EV
createVertexCentricIteration(
+ public M GraphK, VV, EV runVertexCentricIteration(
VertexUpdateFunctionK, VV, M vertexUpdateFunction,
MessagingFunctionK, VV, M, EV messagingFunction,
int maximumNumberOfIterations) {
- return VertexCentricIteration.withEdges(edges,
vertexUpdateFunction,
- messagingFunction, maximumNumberOfIterations);
+
--- End diff --
Function call could be forwarded to the configured
`runVertexCentricIteration` to reduce code duplication.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/547#discussion_r27955499
--- Diff: docs/gelly_guide.md ---
@@ -343,29 +343,86 @@ Vertex-centric Iterations
Gelly wraps Flink's [Spargel API](spargel_guide.html) to provide methods
for vertex-centric iterations.
Like in Spargel, the user only needs to implement two functions: a
`VertexUpdateFunction`, which defines how a vertex will update its value
based on the received messages and a `MessagingFunction`, which allows a
vertex to send out messages for the next superstep.
-These functions are given as parameters to Gelly's
`createVertexCentricIteration`, which returns a `VertexCentricIteration`.
-The user can configure this iteration (set the name, the parallelism,
aggregators, etc.) and then run the computation, using the
`runVertexCentricIteration` method:
+These functions and the maximum number of iterations to run are given as
parameters to Gelly's `runVertexCentricIteration`.
+This method will execute the vertex-centric iteration on the input Graph
and return a new Graph, with updated vertex values:
{% highlight java %}
GraphLong, Double, Double graph = ...
-// create the vertex-centric iteration
-VertexCentricIterationLong, Double, Double, Double iteration =
- graph.createVertexCentricIteration(
+// run Single-Source-Shortest-Paths vertex-centric iteration
+GraphLong, Double, Double result =
+ graph.runVertexCentricIteration(
new VertexDistanceUpdater(), new
MinDistanceMessenger(), maxIterations);
+// user-defined functions
+public static final class VertexDistanceUpdater {...}
+public static final class MinDistanceMessenger {...}
+
+{% endhighlight %}
+
+### Configuring a Vertex-Centric Iteration
+A vertex-centric iteration can be configured using an
`IterationConfiguration` object.
+Currently, the following parameters can be specified:
+
+* strongName/strong: The name for the vertex-centric iteration. The
name is displayed in logs and messages
+and can be specified using the `setName()` method.
+
+* strongParallelism/strong: The parallelism for the iteration. It can
be set using the `setParallelism()` method.
+
+* strongSolution set in unmanaged memory/strong: Defines whether the
solution set is kept in managed memory (Flink's internal way of keeping object
in serialized form) or as a simple object map. By default, the solution set
runs in managed memory. This property can be set using the
`setSolutionSetUnmanagedMemory()` method.
--- End diff --
Typo: ... of keeping objectS in ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/547#issuecomment-90863498 LGTM I browsed the diffs and didn't find anything crucial. Just a typo and one possible simplification. The change makes vertex iterations much easier to use as indicated by the implemented algorithms. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1694] [gelly] added IterationConfigurat...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/547#issuecomment-90745094 Hi, any opinions on this? I would like to merge it soon because it changes the api and vertex-centric related issues depend on it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
