I saw in the documentation that InternalVertexRunner has a configuration field (LOCAL_ZOOKEEPER_PORT) does that mean I have to have Zookeper installed and configured? Also what else would I need to have installed to test that way?
I couldn't find any good guides how to do this online so I'm thankful for every hint. So thanks everyone for the help so far. On Wed, Feb 19, 2014 at 10:44 AM, Lukas Nalezenec < lukas.naleze...@firma.seznam.cz> wrote: > Hi, > Its simple, just pass the TestGraph to InternalVertexRunner. > > If i had lot of logic in input format, I would test as much as i could in > pure isolated InputFormat unit test without any framework. I would make > POJO mock for Record Reader and then add some Mockito mocks. > (InternalVertexRunner is little bit slow - it starts giraph in every unit > test). Then i would test the input format together with computations in > InternalVertexRunner in separate unit test classes. > I have never used InternalVertexRunner for testing input/output formats - > I am not sure if it possible. > > Lukas > > > > > On 18.2.2014 17:38, Martin Neumann wrote: > > @ Lukas: Do you have some links where I can read how to use it to debug > programs? > > My current program has a lot of logic in the EdgeInputFormat as well > that was why I was looking at MapReduce tools since loading the data uses > the same classes (as far as I can see). But I agree its a rather desperate > try. > > I need a simple way to test the input format, output format and the > vertex program. > For the vertex program I currently develop it on a white bord to get the > interactions right then some unit test to make sure it works on a micro > level. I have not found a good way to test the input format (e.g. if the > Graph was build correctly) or how to do a good end to end test. > > cheers Martin > > On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec < > lukas.naleze...@firma.seznam.cz> wrote: > >> Hi, >> How about org.apache.giraph.utils.TestGraph? >> I would not try using mrunit for testing Giraph. >> >> cheers >> Lukas >> >> >> >> On 18.2.2014 17:01, Mirko Kämpf wrote: >> >> Hi Martin, >> >> it depends on the details of your implementation. In principle, you do >> not write Mapper and Reducer >> classes, which are tested by MR-Unit. You usually have to implement the >> application / algorithm specific >> logic, in e.g. the Vertex class. MR-Unit might not help that much here. >> >> This leads to the question, if there is a comparable approach / or tool >> like MR Unit for Giraph? >> >> Best wishes >> Mirko >> >> >> >> >> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mneum...@spotify.com>wrote: >> >>> How much of the Giraph 1.1.0 version is still based on MapReduce? >>> Would something like mrunit (http://mrunit.apache.org/) work with it? >>> >>> >>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann >>> <mneum...@spotify.com>wrote: >>> >>>> I'm using the Yarn version (1.1.0 trunk) and I was not able to get >>>> it to work that way. But I'm no expert on that, if anyone knows a way to >>>> get it done It would be great. >>>> >>>> >>>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <ro...@shaposhnik.org >>>> > wrote: >>>> >>>>> I find running Giraph apps via hadoop's local execution mode >>>>> quite useful for debugging. >>>>> >>>>> Thanks, >>>>> Roman. >>>>> >>>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mneum...@spotify.com> >>>>> wrote: >>>>> > Hej, >>>>> > >>>>> > I find Giraph programs quite difficult to debug (all I have is log >>>>> outputs). >>>>> > Anyone has some hints for debugging tools or coding practices to >>>>> make it >>>>> > simpler? >>>>> > >>>>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce >>>>> debugging >>>>> > aids (at least I think so, I haven't tried any) >>>>> > >>>>> > cheers >>>>> > Martin >>>>> >>>> >>>> >>> >> >> > >
