date:20120510

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

2012-05-10 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/GIRAPH-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272418#comment-13272418
 ] 

Hudson commented on GIRAPH-20:
--

Integrated in Giraph-trunk-Commit #109 (See 
[https://builds.apache.org/job/Giraph-trunk-Commit/109/])
GIRAPH-20 Move temporary test files from the project directory (Revision 
1336743)

 Result = SUCCESS
ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336743
Files : 
* /incubator/giraph/trunk/CHANGELOG
* 
/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java
* 
/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/utils/FileUtils.java
* 
/incubator/giraph/trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java
* /incubator/giraph/trunk/src/test/java/org/apache/giraph/BspCase.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java
* /incubator/giraph/trunk/src/test/java/org/apache/giraph/TestBspBasic.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java
* /incubator/giraph/trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java


> Move temporary test files from the project directory
> 
>
> Key: GIRAPH-20
> URL: https://issues.apache.org/jira/browse/GIRAPH-20
> Project: Giraph
>  Issue Type: Improvement
>  Components: test
>Affects Versions: 0.2.0
>Reporter: Owen O'Malley
>Assignee: Sebastian Schelter
> Fix For: 0.2.0
>
> Attachments: GIRAPH-20.patch
>
>
> We shouldn't use the project directory as the location for temporary files 
> generated by the tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

2012-05-10 Thread jirapos...@reviews.apache.org (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272377#comment-13272377
 ] 

jirapos...@reviews.apache.org commented on GIRAPH-20:
-

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7772
---

Ship it!

+1, thanks Sebastian!  I'm assuming it also passes 'mvn verify', but please 
double check before you commit.

- Avery

On 2012-05-10 09:32:10, Sebastian Schelter wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5077/
bq.  ---
bq.  
bq.  (Updated 2012-05-10 09:32:10)
bq.  
bq.  
bq.  Review request for giraph.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  All temporary files that the tests generate are now written to 
/tmp/_giraphTests including zooKeeper files, checkpoints etc. 
bq.  
bq.  This behavior will be automatically configured whenever 
InternalVertexRunner.run() or BspCase.prepareJob() is used.
bq.  
bq.  Usually I can't stop myself once I have my refactoring hat on, therefore I 
also tidied up a lot of minor stuff, removed code duplications etc.
bq.  
bq.  
bq.  This addresses bug GIRAPH-20.
bq.  https://issues.apache.org/jira/browse/GIRAPH-20
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
1336504 
bq.trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1336504 
bq.trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
1336504 
bq.trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
bq.trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
1336504 
bq.trunk/src/test/java/org/apache/giraph/BspCase.java 1336504 
bq.trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1336506 
bq.trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1336504 
bq.trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1336504 
bq.trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1336504 
bq.trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1336506 
bq.trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1336504 
bq.trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1336504 
bq.trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1336504 
bq.trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 
1336504 
bq.  
bq.  Diff: https://reviews.apache.org/r/5077/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Sebastian
bq.  
bq.

> Move temporary test files from the project directory
> 
>
> Key: GIRAPH-20
> URL: https://issues.apache.org/jira/browse/GIRAPH-20
> Project: Giraph
>  Issue Type: Improvement
>  Components: test
>Affects Versions: 0.2.0
>Reporter: Owen O'Malley
>Assignee: Sebastian Schelter
> Attachments: GIRAPH-20.patch
>
>
> We shouldn't use the project directory as the location for temporary files 
> generated by the tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

2012-05-10 Thread Avery Ching


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7772
---

Ship it!


+1, thanks Sebastian!  I'm assuming it also passes 'mvn verify', but please 
double check before you commit.

- Avery


On 2012-05-10 09:32:10, Sebastian Schelter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5077/
> ---
> 
> (Updated 2012-05-10 09:32:10)
> 
> 
> Review request for giraph.
> 
> 
> Summary
> ---
> 
> All temporary files that the tests generate are now written to 
> /tmp/_giraphTests including zooKeeper files, checkpoints etc. 
> 
> This behavior will be automatically configured whenever 
> InternalVertexRunner.run() or BspCase.prepareJob() is used.
> 
> Usually I can't stop myself once I have my refactoring hat on, therefore I 
> also tidied up a lot of minor stuff, removed code duplications etc.
> 
> 
> This addresses bug GIRAPH-20.
> https://issues.apache.org/jira/browse/GIRAPH-20
> 
> 
> Diffs
> -
> 
>   trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
> 1336504 
>   trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1336504 
>   trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
> 1336504 
>   trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
>   trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
> 1336504 
>   trunk/src/test/java/org/apache/giraph/BspCase.java 1336504 
>   trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1336506 
>   trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1336504 
>   trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1336504 
>   trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1336504 
>   trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1336506 
>   trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1336504 
>   trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1336504 
>   trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1336504 
>   trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1336504 
> 
> Diff: https://reviews.apache.org/r/5077/diff
> 
> 
> Testing
> ---
> 
> successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
> 
> 
> Thanks,
> 
> Sebastian
> 
>

[jira] [Commented] (GIRAPH-153) HBase/Accumulo Input and Output formats

2012-05-10 Thread Brian Femiano (JIRA)


[ 
https://issues.apache.org/jira/browse/GIRAPH-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272309#comment-13272309
 ] 

Brian Femiano commented on GIRAPH-153:
--

I'll take down the confluence entry until this is approved. We don't want 
anyone getting the wrong idea. 

> HBase/Accumulo Input and Output formats
> ---
>
> Key: GIRAPH-153
> URL: https://issues.apache.org/jira/browse/GIRAPH-153
> Project: Giraph
>  Issue Type: New Feature
>  Components: bsp
>Affects Versions: 0.1.0
> Environment: Single host OSX 10.6.8 2.2Ghz Intel i7, 8GB
>Reporter: Brian Femiano
> Attachments: GIRAPH-153.1.patch, GIRAPH-153.2.patch, 
> GIRAPH-153.3.patch, GIRAPH-153.patch
>
>
> Four abstract classes that wrap their respective delegate input/output 
> formats for
> easy hooks into vertex input format subclasses. I've included some sample 
> programs that show two very simple graph
> algorithms. I have a graph generator that builds out a very simple directed 
> structure, starting with a few 'root' nodes.
> Root nodes are defined as nodes which are not listed as a child anywhere in 
> the graph. 
> Algorithm 1) AccumuloRootMarker.java  --> Accumulo as read/write source. 
> Every vertex starts thinking it's a root. At superstep 0, send a message down 
> to each
> child as a non-root notification. After superstep 1, only root nodes will 
> have never been messaged. 
> Algorithm 2) TableRootMarker --> HBase as read/write source. Expands on A1 by 
> bundling the notification logic followed by root node propagation. Once we've 
> marked the appropriate nodes as roots, tell every child which roots it can be 
> traced back to via one or more spanning trees. This will take N + 2 
> supersteps where N is the maximum number of hops from any root to any leaf, 
> plus 2 supersteps for the initial root flagging. 
> I've included all relevant code plus DistributedCacheHelper.java for 
> recursive cache file and archive searches. It is more hadoop centric than 
> giraph, but these jobs use it so I figured why not commit here. 
> These have been tested through local JobRunner, pseudo-distributed on the 
> aforementioned hardware, and full distributed on EC2. More details in the 
> comments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

2012-05-10 Thread jirapos...@reviews.apache.org (JIRA)


[ 
https://issues.apache.org/jira/browse/GIRAPH-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272217#comment-13272217
 ] 

jirapos...@reviews.apache.org commented on GIRAPH-20:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/
---

(Updated 2012-05-10 09:32:10.140325)


Review request for giraph.


Changes
---

Updated the patch to reflect Avery's comments.

The line FileUtils:87 has to be kept, we initially delete the new file (if it 
existed) as it will be recreated later.


Summary
---

All temporary files that the tests generate are now written to 
/tmp/_giraphTests including zooKeeper files, checkpoints etc. 

This behavior will be automatically configured whenever 
InternalVertexRunner.run() or BspCase.prepareJob() is used.

Usually I can't stop myself once I have my refactoring hat on, therefore I also 
tidied up a lot of minor stuff, removed code duplications etc.


This addresses bug GIRAPH-20.
https://issues.apache.org/jira/browse/GIRAPH-20


Diffs (updated)
-

  trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
1336504 
  trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1336504 
  trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 1336504 
  trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
  trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 1336504 
  trunk/src/test/java/org/apache/giraph/BspCase.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1336504 
  trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1336504 

Diff: https://reviews.apache.org/r/5077/diff


Testing
---

successfully passed local and pseudo-distributed tests with Hadoop 0.20.203


Thanks,

Sebastian



> Move temporary test files from the project directory
> 
>
> Key: GIRAPH-20
> URL: https://issues.apache.org/jira/browse/GIRAPH-20
> Project: Giraph
>  Issue Type: Improvement
>  Components: test
>Affects Versions: 0.2.0
>Reporter: Owen O'Malley
>Assignee: Sebastian Schelter
> Attachments: GIRAPH-20.patch
>
>
> We shouldn't use the project directory as the location for temporary files 
> generated by the tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

2012-05-10 Thread Sebastian Schelter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/
---

(Updated 2012-05-10 09:32:10.140325)


Review request for giraph.


Changes
---

Updated the patch to reflect Avery's comments.

The line FileUtils:87 has to be kept, we initially delete the new file (if it 
existed) as it will be recreated later.


Summary
---

All temporary files that the tests generate are now written to 
/tmp/_giraphTests including zooKeeper files, checkpoints etc. 

This behavior will be automatically configured whenever 
InternalVertexRunner.run() or BspCase.prepareJob() is used.

Usually I can't stop myself once I have my refactoring hat on, therefore I also 
tidied up a lot of minor stuff, removed code duplications etc.


This addresses bug GIRAPH-20.
https://issues.apache.org/jira/browse/GIRAPH-20


Diffs (updated)
-

  trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
1336504 
  trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1336504 
  trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 1336504 
  trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
  trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 1336504 
  trunk/src/test/java/org/apache/giraph/BspCase.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1336504 
  trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1336504 

Diff: https://reviews.apache.org/r/5077/diff


Testing
---

successfully passed local and pseudo-distributed tests with Hadoop 0.20.203


Thanks,

Sebastian

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

2012-05-10 Thread Sebastian Schelter (JIRA)


[ 
https://issues.apache.org/jira/browse/GIRAPH-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272209#comment-13272209
 ] 

Sebastian Schelter commented on GIRAPH-141:
---

No need to excuse.

Maybe we simply misunderstand each other. I just wanted to say that it might be 
a lot of effort to implement real multigraph support deep into the system as a 
lot of special cases might have to be kept in mind when doing this, e.g. when 
mutating or partitioning the graph. I just wanted to say that we should first 
search for simple ways to add multigraph support transparently on top of what 
we already have.

> mulitgraph support in giraph
> 
>
> Key: GIRAPH-141
> URL: https://issues.apache.org/jira/browse/GIRAPH-141
> Project: Giraph
>  Issue Type: Improvement
>  Components: graph
>Reporter: André Kelpe
>
> The current vertex API only supports simple graphs, meaning that there can 
> only ever be one edge between two vertices. Many graphs like the road network 
> are in fact multigraphs, where many edges can connect two vertices at the 
> same time.
> Support for this could be added by introducing an Iterator 
> getEdgeValue() or a similar construct. Maybe introducing a slim object like a 
> Connector between the edge and the vertex is also a good idea, so that you 
> could do something like:
> {code} 
> for (final Connector conn: getEdgeValues(){
>  final EdgeWritable edge = conn.getEdge();
>  final VertexWritable otherVertex = conn.getOther();
>  doInterestingStuff(otherVertex);
>  doMoreInterestingStuff(edge);
> }
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

2012-05-10 Thread Roman K (JIRA)


[ 
https://issues.apache.org/jira/browse/GIRAPH-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272201#comment-13272201
 ] 

Roman K commented on GIRAPH-141:


Sorry for taking your time, but I am not sure I understand you correctly.
Are you talking about Avery's idea, to sub-class the BasicVertex class with 
some BasicMultiVertexClass and etc. or about introducing some MultiValueEdge 
class which implements Writable and could be "hidden" inside the existing 
simple graph structure?

> mulitgraph support in giraph
> 
>
> Key: GIRAPH-141
> URL: https://issues.apache.org/jira/browse/GIRAPH-141
> Project: Giraph
>  Issue Type: Improvement
>  Components: graph
>Reporter: André Kelpe
>
> The current vertex API only supports simple graphs, meaning that there can 
> only ever be one edge between two vertices. Many graphs like the road network 
> are in fact multigraphs, where many edges can connect two vertices at the 
> same time.
> Support for this could be added by introducing an Iterator 
> getEdgeValue() or a similar construct. Maybe introducing a slim object like a 
> Connector between the edge and the vertex is also a good idea, so that you 
> could do something like:
> {code} 
> for (final Connector conn: getEdgeValues(){
>  final EdgeWritable edge = conn.getEdge();
>  final VertexWritable otherVertex = conn.getOther();
>  doInterestingStuff(otherVertex);
>  doMoreInterestingStuff(edge);
> }
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

2012-05-10 Thread Sebastian Schelter (JIRA)


[ 
https://issues.apache.org/jira/browse/GIRAPH-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272189#comment-13272189
 ] 

Sebastian Schelter commented on GIRAPH-141:
---

I don't think it would be a good idea to force all users to use multigraphs. 
Furthermore a good principle of software engineering is to always search for 
the simplest solution. I'm not sure whether it would really be so easy to 
implement real multigraph support in the whole system.

Why not have a base vertex class that transparently (!) wraps multigraphs into 
simple graphs?

> mulitgraph support in giraph
> 
>
> Key: GIRAPH-141
> URL: https://issues.apache.org/jira/browse/GIRAPH-141
> Project: Giraph
>  Issue Type: Improvement
>  Components: graph
>Reporter: André Kelpe
>
> The current vertex API only supports simple graphs, meaning that there can 
> only ever be one edge between two vertices. Many graphs like the road network 
> are in fact multigraphs, where many edges can connect two vertices at the 
> same time.
> Support for this could be added by introducing an Iterator 
> getEdgeValue() or a similar construct. Maybe introducing a slim object like a 
> Connector between the edge and the vertex is also a good idea, so that you 
> could do something like:
> {code} 
> for (final Connector conn: getEdgeValues(){
>  final EdgeWritable edge = conn.getEdge();
>  final VertexWritable otherVertex = conn.getOther();
>  doInterestingStuff(otherVertex);
>  doMoreInterestingStuff(edge);
> }
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

2012-05-10 Thread Roman K (JIRA)


[ 
https://issues.apache.org/jira/browse/GIRAPH-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272177#comment-13272177
 ] 

Roman K commented on GIRAPH-141:


I think the best thing to do is to implement multigraph separately from simple 
graph, make some benchmarks to see the performance and memory usage impacts on 
processing (simple graph as multigraph) vs (simple graph) . If the impact could 
be neglected, refactor the code and leave the multigraph implementation only.

Another, more specific question, is what should be done with methods like
{quote}
  /**
   * Request to remove a vertex from the graph
   * (applied just prior to the next superstep).
   *
   * @param vertexId Id of the vertex to be removed.
   */
public void removeEdgeRequest(I sourceVertexId, I destVertexId)
{quote}
For multigraphs, behavior for this method is quiet undefined.
May be 
{quote}
public void removeEdgeRequest(I sourceVertexId, I destVertexId, E edgeValue)
{quote}
should be introduced?

> mulitgraph support in giraph
> 
>
> Key: GIRAPH-141
> URL: https://issues.apache.org/jira/browse/GIRAPH-141
> Project: Giraph
>  Issue Type: Improvement
>  Components: graph
>Reporter: André Kelpe
>
> The current vertex API only supports simple graphs, meaning that there can 
> only ever be one edge between two vertices. Many graphs like the road network 
> are in fact multigraphs, where many edges can connect two vertices at the 
> same time.
> Support for this could be added by introducing an Iterator 
> getEdgeValue() or a similar construct. Maybe introducing a slim object like a 
> Connector between the edge and the vertex is also a good idea, so that you 
> could do something like:
> {code} 
> for (final Connector conn: getEdgeValues(){
>  final EdgeWritable edge = conn.getEdge();
>  final VertexWritable otherVertex = conn.getOther();
>  doInterestingStuff(otherVertex);
>  doMoreInterestingStuff(edge);
> }
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

2012-05-10 Thread jirapos...@reviews.apache.org (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272152#comment-13272152
 ] 

jirapos...@reviews.apache.org commented on GIRAPH-20:
-

bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.  > Overall, looks great.  Can you address the questions/comments and then 
I'll re-review?

Thanks for the quick review!

I'll address your comments, merge this with the current trunk and posted a new 
patch.

bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.  > trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java, 
lines 85-87
bq.  > 
bq.  >
bq.  > Just out of curiosity, why this change?

If one uses writeUTF() and then reads the resulting file with a buffered 
reader, each line starts with a broken char. Directly writing the bytes out 
solved this.

bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.  > trunk/src/main/java/org/apache/giraph/utils/FileUtils.java, line 87
bq.  > 
bq.  >
bq.  > Why delete it?

I'll remove this.

bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.  > trunk/src/test/java/org/apache/giraph/TestBspBasic.java, line 242
bq.  > 
bq.  >
bq.  > shouldn't it be 49 not 491?

its a small L not a 1 :)

- Sebastian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7756
---

On 2012-05-09 11:37:47, Sebastian Schelter wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5077/
bq.  ---
bq.  
bq.  (Updated 2012-05-09 11:37:47)
bq.  
bq.  
bq.  Review request for giraph.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  All temporary files that the tests generate are now written to 
/tmp/_giraphTests including zooKeeper files, checkpoints etc. 
bq.  
bq.  This behavior will be automatically configured whenever 
InternalVertexRunner.run() or BspCase.prepareJob() is used.
bq.  
bq.  Usually I can't stop myself once I have my refactoring hat on, therefore I 
also tidied up a lot of minor stuff, removed code duplications etc.
bq.  
bq.  
bq.  This addresses bug GIRAPH-20.
bq.  https://issues.apache.org/jira/browse/GIRAPH-20
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 
1332106 
bq.trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/BspCase.java 1332106 
bq.trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
1332106 
bq.trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
1332106 
bq.trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1332106 
bq.trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
1332106 
bq.trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/5077/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Sebastian
bq.  
bq.

> Move temporary test files from the project directory
> 
>
> Key: GIRAPH-20
> URL: https://issues.apache.org/jira/browse/GIRAPH-20
> Project: Giraph
>  Issue Type: Improvement
>  Components: test
>Affects Versions: 0.2.0
>Reporter: Owen O'Malley
>Assignee: Sebastian Schelter
> Attachments: GIRAPH-20.patch
>
>
> We shouldn't use the project directory as the location for temporary files 
> generated by the tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JI

Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

2012-05-10 Thread Sebastian Schelter



> On 2012-05-10 06:57:01, Avery Ching wrote:
> > Overall, looks great.  Can you address the questions/comments and then I'll 
> > re-review?

Thanks for the quick review!

I'll address your comments, merge this with the current trunk and posted a new 
patch.


> On 2012-05-10 06:57:01, Avery Ching wrote:
> > trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java, 
> > lines 85-87
> > 
> >
> > Just out of curiosity, why this change?

If one uses writeUTF() and then reads the resulting file with a buffered 
reader, each line starts with a broken char. Directly writing the bytes out 
solved this.


> On 2012-05-10 06:57:01, Avery Ching wrote:
> > trunk/src/main/java/org/apache/giraph/utils/FileUtils.java, line 87
> > 
> >
> > Why delete it?

I'll remove this.


> On 2012-05-10 06:57:01, Avery Ching wrote:
> > trunk/src/test/java/org/apache/giraph/TestBspBasic.java, line 242
> > 
> >
> > shouldn't it be 49 not 491?

its a small L not a 1 :)


- Sebastian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7756
---


On 2012-05-09 11:37:47, Sebastian Schelter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5077/
> ---
> 
> (Updated 2012-05-09 11:37:47)
> 
> 
> Review request for giraph.
> 
> 
> Summary
> ---
> 
> All temporary files that the tests generate are now written to 
> /tmp/_giraphTests including zooKeeper files, checkpoints etc. 
> 
> This behavior will be automatically configured whenever 
> InternalVertexRunner.run() or BspCase.prepareJob() is used.
> 
> Usually I can't stop myself once I have my refactoring hat on, therefore I 
> also tidied up a lot of minor stuff, removed code duplications etc.
> 
> 
> This addresses bug GIRAPH-20.
> https://issues.apache.org/jira/browse/GIRAPH-20
> 
> 
> Diffs
> -
> 
>   trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1332106 
>   trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1332106 
>   trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1332106 
>   trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1332106 
>   trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1332106 
>   trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1332106 
>   trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1332106 
>   trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1332106 
>   trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1332106 
>   trunk/src/test/java/org/apache/giraph/BspCase.java 1332106 
>   trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
> 1332106 
>   trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
> 1332106 
>   trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1332106 
>   trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
> 1332106 
>   trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/5077/diff
> 
> 
> Testing
> ---
> 
> successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
> 
> 
> Thanks,
> 
> Sebastian
> 
>

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

[jira] [Commented] (GIRAPH-153) HBase/Accumulo Input and Output formats

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

12 matches

Site Navigation

Mail list logo

Footer information