[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139962#comment-13139962 ] Jake Mannix commented on GIRAPH-36: --- Regarding 1) - I guess I can see how VertexReader could be an abstract base class which hangs onto the GraphState, and instantiates the BasicVertex instances for its subclasses, using the private GraphState reference, and passing it to an abstract initializeVertex(BasicVertex v) method to be populated via the (newly added in this patch) BasicVertex#initialize() method. Thinking further on 2) - am I just not seeing how we could easily make VertexReader act like an Iterator / factory for BasicVertex instances without typing it by the type of the thing it's to instantiate? > Ensure that subclassing BasicVertex is possible by user apps > > > Key: GIRAPH-36 > URL: https://issues.apache.org/jira/browse/GIRAPH-36 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.70.0 >Reporter: Jake Mannix >Assignee: Jake Mannix >Priority: Blocker > Fix For: 0.70.0 > > Attachments: GIRAPH-36.diff > > > Original assumptions in Giraph were that all users would subclass Vertex > (which extended MutableVertex extended BasicVertex). Classes which wish to > have application specific data structures (ie. not a TreeMap>) > may need to extend either MutableVertex or BasicVertex. Unfortunately > VertexRange extends ArrayList, and there are other places where the > assumption is that vertex classes are either Vertex, or at least > MutableVertex. > Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139923#comment-13139923 ] Jake Mannix commented on GIRAPH-36: --- Hey Avery, Glad you could get it to all work (unit-test-wise) for you - did you run the unit tests talking to a real cluster, or just localhost? I haven't tried the former yet... Regarding the comments: I'm fine keeping GraphState hidden to the implementation details, but will it always work in practice? I mean, all of the state in there used to be global static state in Vertex, and all subclasses had full read/write access to it. We've segregated it in a non-static object which we're treating as a singleton (but aren't forcing it to be one programmaticly), but can we really hide it? For example, BasicVertex#sendMsg() requires the WorkerCommunications, which it gets from the GraphMapper, which it gets from the GraphState. Do you think we should implement all the things we think the user will need from the GraphMapper directly *in* BasicVertex (or MutableVertex), and remove the getGraphState() method entirely, or just keep it package private for now. In general, I like keeping impl details away from developers too, but this early in the project, we also shouldn't hamstring ourselves by thinking there is a difference from a "casual user" of the system, and someone willing to hack around a bit. :) 2) If VertexReader is going to return a BasicVertex from getCurrentVertex() to callers who really need it to be fully specified nice and type-safely, what choice do we have? From a pratical, rather than API standpoint, is it possible we'd someday want to be able to read up a collection of vertices with initial messages ready to be processed? Not sure if that matters. 3) I tried, and can clean up as possible, but I really rely on my IDE settings for that. Not being able to set autoformat on means I'm pretty doomed, but if you see places where I can clean up manually, I can do that, sure. > Ensure that subclassing BasicVertex is possible by user apps > > > Key: GIRAPH-36 > URL: https://issues.apache.org/jira/browse/GIRAPH-36 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.70.0 >Reporter: Jake Mannix >Assignee: Jake Mannix >Priority: Blocker > Fix For: 0.70.0 > > Attachments: GIRAPH-36.diff > > > Original assumptions in Giraph were that all users would subclass Vertex > (which extended MutableVertex extended BasicVertex). Classes which wish to > have application specific data structures (ie. not a TreeMap>) > may need to extend either MutableVertex or BasicVertex. Unfortunately > VertexRange extends ArrayList, and there are other places where the > assumption is that vertex classes are either Vertex, or at least > MutableVertex. > Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-50) Require Maven 3 in order to work with munging plugin
[ https://issues.apache.org/jira/browse/GIRAPH-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139898#comment-13139898 ] Hudson commented on GIRAPH-50: -- Integrated in Giraph-trunk-Commit #21 (See [https://builds.apache.org/job/Giraph-trunk-Commit/21/]) GIRAPH-50. Require Maven 3 in order to work with munging plugin. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195322 Files : * /incubator/giraph/trunk/CHANGELOG * /incubator/giraph/trunk/README * /incubator/giraph/trunk/pom.xml * /incubator/giraph/trunk/src/site/xdoc/index.xml > Require Maven 3 in order to work with munging plugin > > > Key: GIRAPH-50 > URL: https://issues.apache.org/jira/browse/GIRAPH-50 > Project: Giraph > Issue Type: Bug > Environment: maven 2.2.1 >Reporter: Mitul Tiwari >Assignee: Jakob Homan >Priority: Minor > Fix For: 0.70.0 > > Attachments: GIRAPH-50.patch > > > package/test works with maven 3.0.3, but package/test does not work with > maven 2.2.1. For example, the following command results in an error: > $ /usr/bin/mvn221 -Dhadoop=non_secure package > Last few lines of error: > Results : > Failed tests: > testSuperstepBalancer(org.apache.giraph.TestVertexRangeBalancer) > testBspCheckpoint(org.apache.giraph.TestManualCheckpoint) > testBspSuperStep(org.apache.giraph.TestBspBasic) > testBspMsg(org.apache.giraph.TestBspBasic) > testBspCombiner(org.apache.giraph.TestBspBasic) > testBspPageRank(org.apache.giraph.TestBspBasic) > testBspShortestPaths(org.apache.giraph.TestBspBasic) > testMutateGraph(org.apache.giraph.TestMutateGraphVertex) > testContinue(org.apache.giraph.TestJsonBase64Format) > Tests run: 27, Failures: 9, Errors: 0, Skipped: 0 > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > 11/10/10 11:23:37 WARN zk.ZooKeeperManager: onlineZooKeeperServers: Forced > a shutdown hook kill of the ZooKeeper process. > [INFO] > > [ERROR] BUILD FAILURE > [INFO] > > [INFO] There are test failures. > Please refer to > /Users/mtiwari/pkgs/giraph/Apache/giraph/target/surefire-reports for the > individual test results. > [INFO] > > [INFO] For more information, run Maven with the -e switch > [INFO] > > [INFO] Total time: 23 seconds > [INFO] Finished at: Mon Oct 10 11:23:37 PDT 2011 > [INFO] Final Memory: 51M/106M > [INFO] > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-36) Ensure that subclassing BasicVertex is possible by user apps
[ https://issues.apache.org/jira/browse/GIRAPH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139871#comment-13139871 ] Avery Ching commented on GIRAPH-36: --- Tried out your stuff Jake. Unittests passed for me. I took a quick look at it and was happy to see the new input format style interfaces (nextVertex() and getCurrentVertex()). Also very cool to see LongDoubleFloatDoubleVertex in action as a proof-of-concept non-Vertex implementation. Couple of questions/comments. 1) I would personally prefer that GraphState not be exposed to developers/users from VertexReader. We can always set the GraphState from the infrastructure after the vertices have been read. 2) Does VertexReader really need the message type M? There aren't any messages at that point and complicates things a bit. 3) I know it's not ready for primetime as you mentioned, but while we currently have two styles of formatting, hopefully we can at least keep files consistent until one of us fixes everything to the new convention. > Ensure that subclassing BasicVertex is possible by user apps > > > Key: GIRAPH-36 > URL: https://issues.apache.org/jira/browse/GIRAPH-36 > Project: Giraph > Issue Type: Improvement > Components: graph >Affects Versions: 0.70.0 >Reporter: Jake Mannix >Assignee: Jake Mannix >Priority: Blocker > Fix For: 0.70.0 > > Attachments: GIRAPH-36.diff > > > Original assumptions in Giraph were that all users would subclass Vertex > (which extended MutableVertex extended BasicVertex). Classes which wish to > have application specific data structures (ie. not a TreeMap>) > may need to extend either MutableVertex or BasicVertex. Unfortunately > VertexRange extends ArrayList, and there are other places where the > assumption is that vertex classes are either Vertex, or at least > MutableVertex. > Let's make sure the internal APIs allow for BasicVertex to be the base class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
