Re: using partition class

Thu, 03 Apr 2014 01:39:24 -0700 

Hi
It depends on what you want to do with Giraph.
You don't necessary need to write custom PartitionerFactory. You would need to turn on PartitionBalancer but its broken. I think I have already found the bug, i will commit it soon. I you are really interested in balancing I am writing advanced PartitionBalancer with solver inside. I would appreciate if somebody would help me. 

Regards,
Lukas


On 2.4.2014 18:24, Liannet Reyes wrote:

Thanks again  for your help Lukas,

It works!

Hereby the full command line for future reference.


    hadoop jar
    
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar
    
org.apache.giraph.GiraphRunner-Dgiraph.graphPartitionerFactoryClass=org.apache.giraph.partition.SimpleLongRangePartitionerFactory
    
-Dgiraph.vertexKeySpaceSize=5org.apache.giraph.examples.SimpleShortestPathsComputation
    -vif
    org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
    -vip /user/hduser/input/tiny_graph.txt -vof
    org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
    /user/hduser/output/shortestpaths-w 2




I guest the next step is to implement my custom PartitionerFactoryClass.

Best Regards,

Liannet


2014-04-02 17:32 GMT+02:00 Lukas Nalezenec 
<lukas.naleze...@firma.seznam.cz 
<mailto:lukas.naleze...@firma.seznam.cz>>:


    Hi,

    1. It have to work if you set giraph.vertexKeySpaceSize just after
    or before the other property. But it can interfere with the other
    command line options :(.
    2. Its Number of Vertexes in your graph.

    Regards,
    Lukas



    On 2.4.2014 17:21, Liannet Reyes wrote:

    Thank you very much Lukas,

    I've notice that using SimpleLongRangePartitionerFactory require
    the GiraphConstant giraph.vertexKeySpaceSize to be set,
    but I have 2 questions:

    1- Where should I set the giraph constant values? I've tried
    using -Dgiraph.vertexKeySpaceSize=256 after
    -Dgiraph.graphPartitionerFactoryClass but it didn't work
    2- Which unit is this PARTITION_VERTEX_KEY_SPACE_SIZE? Bytes??

    Regards,

    Liannet




    2014-04-01 20:55 GMT+02:00 Lukas Nalezenec
    <lukas.naleze...@firma.seznam.cz
    <mailto:lukas.naleze...@firma.seznam.cz>>:

        Hi,
        Partition is for storing vertexes, Partitioner is for
        distributing vertexes between Partitions.

        Try this:
        
-Dgiraph.graphPartitionerFactoryClass=org.apache.giraph.partition.SimpleLongRangePartitionerFactory

        Its good idea to switch Partition to ByteArrayPartition (or
        better).

        Lukas



        On 1.4.2014 19:59, Liannet Reyes wrote:

        Hi all,

        I´m trying to use the giraph partition class to improve my
        graph distribution, however I hardly found info on how to
        use it.
        I´m testing the SimpleShortestPathsComputation example with
        the tiny_graph.txt input file and the
        SimpleLongRangePartitionerFactory out-of-the-box in
        org.apache.giraph.partition but without success. Apparentely
        I need a Partition class and the
        SimpleLongRangePartitionerFactory is not.

        Can you kindly please let me know the proper way to do it?
        See below my command and console output

        hadoop jar
        
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar
        org.apache.giraph.GiraphRunner
        org.apache.giraph.examples.SimpleShortestPathsComputation
        -vif
        org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
        -vip /user/hduser/input/tiny_graph.txt -vof
        org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
        /user/hduser/output/shortestpaths-pc
        org.apache.giraph.partition.SimpleLongRangePartitionerFactory -w
        2

        Exception in thread "main" java.lang.RuntimeException: class
        org.apache.giraph.partition.SimpleLongRangePartitionerFactory not
        org.apache.giraph.partition.Partition
        at
        org.apache.hadoop.conf.Configuration.setClass(Configuration.java:915)
        at
        org.apache.giraph.conf.ClassConfOption.set(ClassConfOption.java:182)
        at
        
org.apache.giraph.conf.GiraphConfiguration.setPartitionClass(GiraphConfiguration.java:594)
        at
        
org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:375)
        at
        
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:207)
        at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
        
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
        
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

        Thanks in advance

        Liannet






























					Reply via email to