Hello Vasia and Greg,
Thank you for the feedback!
I am probably misusing the Gelly API in some way, but I thought I could run
the undirected version after calling getUndirected()?
While not going into the concept of local clustering coefficients, I
thought that from a Gelly API point-of-view, both my code and data set were
properly established.
However:
- I believe that the graph was already undirected;
- I am getting NaN results after executing the algorithm.
This is the code I am using to obtain an (undirected) graph instance upon
which I call LocalClusteringCoefficient:
import
org.apache.flink.graph.library.clustering.undirected.LocalClusteringCoefficient.Result;
import
org.apache.flink.graph.library.clustering.undirected.LocalClusteringCoefficient;
/** other imports and method definitions **/
// Generate edge tuples from the input file.
final DataSet<Tuple2<LongValue, LongValue>> edgeTuples =
env.readCsvFile(inputPath)
.fieldDelimiter("\t") // node IDs are separated by spaces
.ignoreComments("#") // comments start with "%"
.types(LongValue.class, LongValue.class);
// Generate actual Edge<Long, Double> instances.
@SuppressWarnings("serial")
final DataSet<Edge<LongValue, Double>> edges = edgeTuples.map(
new MapFunction<Tuple2<LongValue, LongValue>, Edge<LongValue,
Double>>() {
@Override
public Edge<LongValue, Double> map(Tuple2<LongValue, LongValue>
arg0) throws Exception {
return new Edge<LongValue, Double>(arg0.f0, arg0.f1, 1.0d);
}
});
// Generate the basic graph.
@SuppressWarnings("serial")
final Graph<LongValue, Double, Double> graph = Graph.fromDataSet(
edges,
new MapFunction<LongValue, Double>() {
@Override
public Double map(LongValue arg0) throws Exception {
// For testing purposes, just setting each vertex value to 1.0.
return 1.0;
}
},
env).*getUndirected(*);
// Execute the LocalClusteringCoefficient algorithm.
final DataSet<Result<LongValue>> localClusteringCoefficients =
graph.run(new LocalClusteringCoefficient<LongValue, Double, Double>());
// Get the values as per Vasia's help:
@SuppressWarnings("serial")
DataSet<Double> *CLUSTERING_COEFFICIENTS* =
localClusteringCoefficients.map(new MapFunction<Result<LongValue>,
Double>() {
@Override
public Double map(Result<LongValue> arg0) throws Exception {
return arg0.getLocalClusteringCoefficientScore();
}
});
I believe this is the correct way to get a DataSet<Double> of coefficients
from a DataSet<Result<LongValue>> ?
Among the coefficients are a lot of NaN values:
*CLUSTERING_COEFFICIENTS*.print();
NaN
0.0
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
NaN
Apologies for the verbosity in advance, but just to provide detail,
printing the graph edges yields this (notice that for each pair or vertices
there are two links, which are original and the reverse version derived
from getUndirected()).
*Greg:* I therefore believe the *graph is undirected*:
graph.getEdgesAsTuple3().print();
(5113,6008,1.0)
(6008,5113,1.0)
(5113,6774,1.0)
(6774,5113,1.0)
(5113,32938,1.0)
(32938,5113,1.0)
(5113,6545,1.0)
(6545,5113,1.0)
(5113,7088,1.0)
(7088,5113,1.0)
(5113,37929,1.0)
(37929,5113,1.0)
(5113,26562,1.0)
(26562,5113,1.0)
(5113,6107,1.0)
(6107,5113,1.0)
(5113,7171,1.0)
(7171,5113,1.0)
(5113,6192,1.0)
(6192,5113,1.0)
(5113,7763,1.0)
(7763,5113,1.0)
(9748,5113,1.0)
(5113,9748,1.0)
(10191,5113,1.0)
(5113,10191,1.0)
(6064,5113,1.0)
(5113,6064,1.0)
(6065,5113,1.0)
(5113,6065,1.0)
(6279,5113,1.0)
(5113,6279,1.0)
(4907,5113,1.0)
(5113,4907,1.0)
(6465,5113,1.0)
(5113,6465,1.0)
(6707,5113,1.0)
(5113,6707,1.0)
(7089,5113,1.0)
(5113,7089,1.0)
(7172,5113,1.0)
(5113,7172,1.0)
(14310,5113,1.0)
(5113,14310,1.0)
(6252,5113,1.0)
(5113,6252,1.0)
(33855,5113,1.0)
(5113,33855,1.0)
(7976,5113,1.0)
(5113,7976,1.0)
(26284,5113,1.0)
(5113,26284,1.0)
(8056,5113,1.0)
(5113,8056,1.0)
(10371,5113,1.0)
(5113,10371,1.0)
(16785,5113,1.0)
(5113,16785,1.0)
(19801,5113,1.0)
(5113,19801,1.0)
(6715,5113,1.0)
(5113,6715,1.0)
(31724,5113,1.0)
(5113,31724,1.0)
(32443,5113,1.0)
(5113,32443,1.0)
(10370,5113,1.0)
(5113,10370,1.0)
Any insight into what I may be doing wrong would be greatly appreciated.
Thanks for your time,
Kind regards,
Miguel E. Coimbra
Email: miguel.e.coim...@gmail.com <miguel.e.coim...@ist.utl.pt>
Skype: miguel.e.coimbra
On 20 January 2017 at 19:31, Greg Hogan <c...@greghogan.com> wrote:
> Hi Miguel,
>
> The '--output print' option describes the values and also displays the
> local clustering coefficient value.
>
> You're running the undirected algorithm on a directed graph. In 1.2 there
> is an option '--simplify true' that will add reverse edges and remove
> duplicate edges and self-loops. Alternatively, it looks like you could
> simply add reverse edges to your input file (with an optional ' | sort |
> uniq' following):
>
> $ cat edges.txt | awk ' { print $1, $2; print $2, $1 } '
>
> The drivers are being reworked for 1.3 to better reuse code and options
> which will better support additional drivers and algorithms and make
> documentation simpler.
>
> Greg
>
> On Fri, Jan 20, 2017 at 2:06 PM, Vasiliki Kalavri <
> vasilikikala...@gmail.com> wrote:
>
>> Hi Miguel,
>>
>> the LocalClusteringCoefficient algorithm returns a DataSet of type Result,
>> which basically wraps a vertex id, its degree, and the number of triangles
>> containing this vertex. The number 11 you see is indeed the degree of
>> vertex 5113. The Result type contains the method
>> getLocalClusteringCoefficientScore() which allows you to retrieve the
>> clustering coefficient score for a vertex. The method simply divides the
>> numbers of triangles by the number of potential edges between neighbors.
>>
>> I'm sorry that you this is not clear in the docs. We should definitely
>> improve them to explain what is the output and how to retrieve the actual
>> clustering coefficient values. I have opened a JIRA for this [1].
>>
>> Cheers,
>> -Vasia.
>>
>> [1]: https://issues.apache.org/jira/browse/FLINK-5597
>>
>> On 20 January 2017 at 19:31, Miguel Coimbra <miguel.e.coim...@gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>> In the documentation of the LocalClusteringCoefficient algorithm, it is
>>> said:
>>>
>>>
>>> *The local clustering coefficient measures the connectedness of each
>>> vertex’s neighborhood.Scores range from 0.0 (no edges between neighbors) to
>>> 1.0 (neighborhood is a clique).*
>>>
>>> https://ci.apache.org/projects/flink/flink-docs-release-1.1/
>>> apis/batch/libs/gelly.html#local-clustering-coefficient
>>> <https://ci.apache.org/projects/flink/flink-docs-master/dev/libs/gelly/library_methods.html#local-clustering-coefficient>
>>>
>>> However, upon running the algorithm (undirected version), I obtained
>>> values above 1.
>>>
>>> The result I got was this. As you can see, vertex 5113 has a score of
>>> 11:
>>> (the input edges for the graph are shown further below - around *35
>>> edges*):
>>>
>>> (4907,(1,0))
>>> *(5113,(11,0))*
>>> (6008,(0,0))
>>> (6064,(1,0))
>>> (6065,(1,0))
>>> (6107,(0,0))
>>> (6192,(0,0))
>>> (6252,(1,0))
>>> (6279,(1,0))
>>> (6465,(1,0))
>>> (6545,(0,0))
>>> (6707,(1,0))
>>> (6715,(1,0))
>>> (6774,(0,0))
>>> (7088,(0,0))
>>> (7089,(1,0))
>>> (7171,(0,0))
>>> (7172,(1,0))
>>> (7763,(0,0))
>>> (7976,(1,0))
>>> (8056,(1,0))
>>> (9748,(1,0))
>>> (10191,(1,0))
>>> (10370,(1,0))
>>> (10371,(1,0))
>>> (14310,(1,0))
>>> (16785,(1,0))
>>> (19801,(1,0))
>>> (26284,(1,0))
>>> (26562,(0,0))
>>> (31724,(1,0))
>>> (32443,(1,0))
>>> (32938,(0,0))
>>> (33855,(1,0))
>>> (37929,(0,0))
>>>
>>> This was from a small isolated test with these edges:
>>>
>>> 5113 6008
>>> 5113 6774
>>> 5113 32938
>>> 5113 6545
>>> 5113 7088
>>> 5113 37929
>>> 5113 26562
>>> 5113 6107
>>> 5113 7171
>>> 5113 6192
>>> 5113 7763
>>> 9748 5113
>>> 10191 5113
>>> 6064 5113
>>> 6065 5113
>>> 6279 5113
>>> 4907 5113
>>> 6465 5113
>>> 6707 5113
>>> 7089 5113
>>> 7172 5113
>>> 14310 5113
>>> 6252 5113
>>> 33855 5113
>>> 7976 5113
>>> 26284 5113 <262%20845%20113>
>>> 8056 5113
>>> 10371 5113
>>> 16785 5113
>>> 19801 5113
>>> 6715 5113
>>> 31724 5113
>>> 32443 5113
>>> 10370 5113
>>>
>>> I am not sure what I may be doing wrong, but is there perhaps some form
>>> of normalization lacking in my execution of:
>>>
>>> org.apache.flink.graph.library.clustering.undirected.LocalCl
>>> usteringCoefficient.Result;
>>> org.apache.flink.graph.library.clustering.undirected.LocalCl
>>> usteringCoefficient;
>>>
>>> Am I supposed to divide all scores by the greatest score obtained by the
>>> algorithm?
>>>
>>> Thank you very much!
>>>
>>> Miguel E. Coimbra
>>> Email: miguel.e.coim...@gmail.com <miguel.e.coim...@ist.utl.pt>
>>> Skype: miguel.e.coimbra
>>>
>>
>>
>
