Hi Martin,

There is no need to install ZooKeeper to use InternalVertexRunner. It will also use the input and output formats specified.

InternalVertexRunner writes your input data to disk and starts ZooKeeper and Giraph locally in a single VM.

It is very easy to use, I suggest to look at the following class for an example:

org.apache.giraph.examples.ConnectedComponentsComputationTest#testToyData

Best,
Sebastian

On 02/19/2014 12:55 PM, Martin Neumann wrote:
I saw in the documentation that InternalVertexRunner has a configuration
field (LOCAL_ZOOKEEPER_PORT) does that mean I have to have Zookeper
installed and configured? Also what else would I need to have installed to
test that way?

I couldn't find any good guides how to do this online so I'm thankful for
every hint. So thanks everyone for the help so far.



On Wed, Feb 19, 2014 at 10:44 AM, Lukas Nalezenec <
lukas.naleze...@firma.seznam.cz> wrote:

  Hi,
Its simple, just pass the TestGraph to InternalVertexRunner.

If i had  lot of logic in input format, I would test as much as i could in
pure isolated InputFormat unit test without any framework. I would make
POJO mock for Record Reader and then add some Mockito mocks.
(InternalVertexRunner is little bit slow - it starts giraph in every unit
test). Then i would test the input format together with computations in
InternalVertexRunner in separate unit test classes.
I have never used InternalVertexRunner for testing input/output formats -
I am not sure if it possible.

Lukas




On 18.2.2014 17:38, Martin Neumann wrote:

  @ Lukas: Do you have some links where I can read how to use it to debug
programs?

  My current program has a lot of logic in the EdgeInputFormat as well
that was why I was looking at MapReduce tools since loading the data uses
the same classes (as far as I can see). But I agree its a rather desperate
try.

  I need a simple way to test the input format, output format and the
vertex program.
For the vertex program I currently develop it on a white bord to get the
interactions right then some unit test to make sure it works on a micro
level. I have not found a good way to test the input format (e.g. if the
Graph was build correctly) or how to do a good end to end test.

  cheers Martin

  On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec <
lukas.naleze...@firma.seznam.cz> wrote:

  Hi,
How about org.apache.giraph.utils.TestGraph?
I would not try using mrunit for testing Giraph.

cheers
Lukas



On 18.2.2014 17:01, Mirko Kämpf wrote:

   Hi Martin,

  it depends on the details of your implementation. In principle, you do
not write Mapper and Reducer
  classes, which are tested by MR-Unit. You usually have to implement the
application / algorithm specific
logic, in e.g. the Vertex class. MR-Unit might not help that much here.

  This leads to the question, if there is a comparable approach / or tool
like MR Unit for Giraph?

  Best wishes
Mirko




On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mneum...@spotify.com>wrote:

  How much of the Giraph 1.1.0 version is still based on MapReduce?
Would something like mrunit (http://mrunit.apache.org/) work with it?


On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mneum...@spotify.com>wrote:

  I'm using the Yarn version (1.1.0 trunk)  and I was not able to get
it to work that way. But I'm no expert on that, if anyone knows a way to
get it done It would be great.


On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <ro...@shaposhnik.org
wrote:

I find running Giraph apps via hadoop's local execution mode
quite useful for debugging.

Thanks,
Roman.

On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mneum...@spotify.com>
wrote:
Hej,

I find Giraph programs quite difficult to debug (all I have is log
outputs).
Anyone has some hints for debugging tools or coding practices to
make it
simpler?

I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
debugging
aids (at least I think so, I haven't tried any)

cheers
Martin










Reply via email to