Sebastian, thank you very much for your help!
On Tue, Feb 4, 2014 at 10:10 AM, Sebastian Schelter <s...@apache.org> wrote: > Hi Alexander, > > the only inputformat that we have which is compatible with > SimpleShortestPathsComputation is the > > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > > which is used in > > org.apache.giraph.examples.SimpleShortestPathsComputationTest#testToyData > > It's not as nice to use as the other inputformats we have, so I opened > https://issues.apache.org/jira/browse/GIRAPH-844 to ensure that we have a > nice easy to use TextInputFormat for the shortest paths example. > > --sebastian > > > On 02/03/2014 06:52 PM, Alexander Frolov wrote: > >> It seems that I understood the problem. PageRank and >> SingleSourceShortestPath both need weighted graph to compute. I just >> missed it somehow. >> Thank you for help. >> >> >> On Mon, Feb 3, 2014 at 9:25 PM, Alexander Frolov >> <alexndr.fro...@gmail.com>wrote: >> >> >>> >>> >>> On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <s...@apache.org> >>> wrote: >>> >>> If you just wanna play with Giraph, I suggest you start with the >>>> PageRank >>>> example: >>>> >>>> https://giraph.apache.org/pagerank.html >>>> >>> >>> >>> I have seen it. I think there is no much difference in SSSP and PageRank >>> examples. By the way, I have not found LongDoubleNullTextInputFormat in >>> my >>> sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned >>> there. >>> The same issue with SimplePageRankComputation. >>> >>> [hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar >>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0- >>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar >>> org.apache.giraph.GiraphRunner >>> org.apache.giraph.examples.SimplePageRankComputation -vif >>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip >>> /user/hduser/input/test -vof >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>> /user/hduser/output/amazon-shortestpaths -w 1 >>> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format >>> specified. Ensure your InputFormat does not require one. >>> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format >>> specified. Ensure your OutputFormat does not require one. >>> Exception in thread "main" java.lang.IllegalArgumentException: >>> checkClassTypes: vertex value types not assignable, computation - class >>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class >>> org.apache.hadoop.io.LongWritable >>> at >>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable( >>> GiraphConfigurationValidator.java:381) >>> at >>> org.apache.giraph.job.GiraphConfigurationValidator. >>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator. >>> java:228) >>> at >>> org.apache.giraph.job.GiraphConfigurationValidator. >>> validateConfiguration(GiraphConfigurationValidator.java:141) >>> at >>> org.apache.giraph.utils.ConfigurationUtils.parseArgs( >>> ConfigurationUtils.java:214) >>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke( >>> NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >>> DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>> >>> >>> >>>> --sebastian >>>> >>>> >>>> On 02/03/2014 05:49 PM, Alexander Frolov wrote: >>>> >>>> I've tried .LongLongNullTextInputFormat and got this: >>>>> >>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar >>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0- >>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar >>>>> org.apache.giraph.GiraphRunner >>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip >>>>> /user/hduser/input/test -vof >>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>> /user/hduser/output/amazon-shortestpaths -w 1 >>>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format >>>>> specified. Ensure your InputFormat does not require one. >>>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format >>>>> specified. Ensure your OutputFormat does not require one. >>>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>>> checkClassTypes: vertex value types not assignable, computation - class >>>>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class >>>>> org.apache.hadoop.io.LongWritable >>>>> at >>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable( >>>>> GiraphConfigurationValidator.java:381) >>>>> at >>>>> org.apache.giraph.job.GiraphConfigurationValidator. >>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator. >>>>> java:228) >>>>> at >>>>> org.apache.giraph.job.GiraphConfigurationValidator. >>>>> validateConfiguration(GiraphConfigurationValidator.java:141) >>>>> >>>>> at >>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs( >>>>> ConfigurationUtils.java:214) >>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at >>>>> sun.reflect.NativeMethodAccessorImpl.invoke( >>>>> NativeMethodAccessorImpl.java:57) >>>>> at >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>>> DelegatingMethodAccessorImpl.java:43) >>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>>> >>>>> >>>>> >>>>> >>>>> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov >>>>> <alexndr.fro...@gmail.com>wrote: >>>>> >>>>> >>>>> >>>>>> >>>>>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <s...@apache.org> >>>>>> wrote >>>>>> >>>>>> What types does the vertex class that you use in your computation >>>>>> >>>>>>> require? >>>>>>> >>>>>>> >>>>>> >>>>>> I see. >>>>>> >>>>>> public class SimpleShortestPathsComputation extends BasicComputation< >>>>>> LongWritable, DoubleWritable, FloatWritable, DoubleWritable> >>>>>> {..} >>>>>> >>>>>> So that is a mistake. >>>>>> Thank you very much. >>>>>> >>>>>> >>>>>> Is there any means to import data from file like this >>>>> (amazon0601.txt >>>>> >>>>>> from >>>>>> SNAP data collection): >>>>>> >>>>>> 0 1 >>>>>> 0 2 >>>>>> 0 3 >>>>>> 0 4 >>>>>> 0 5 >>>>>> 0 6 >>>>>> 0 7 >>>>>> 0 8 >>>>>> 0 9 >>>>>> 0 10 >>>>>> ... >>>>>> >>>>>> and to perform basic tests from giraph-examples? Most of the tests are >>>>>> using LongWritable for vertex id. >>>>>> >>>>>> >>>>>> >>>>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote: >>>>>>> >>>>>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <s...@apache.org >>>>>>> > >>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Alexander, >>>>>>>> >>>>>>>> >>>>>>>>> what do you use as type for your vertex ids? It looks that you are >>>>>>>>> trying >>>>>>>>> to use longs, while IntIntNullTextInputFormat only provides ints, >>>>>>>>> that >>>>>>>>> could be the error. >>>>>>>>> >>>>>>>>> --sebastian >>>>>>>>> >>>>>>>>> >>>>>>>>> I also thought so, but I could not understand why it regard it >>>>>>>>> as >>>>>>>>> long. >>>>>>>>> >>>>>>>>> I just use -vif org.apache.giraph.io.formats. >>>>>>>> IntIntNullTextInputFormat >>>>>>>> -vip /user/hduser/input/test -vof >>>>>>>> >>>>>>>> and file consists only of two lines: >>>>>>>> >>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat >>>>>>>> /user/hduser/input/test >>>>>>>> 0 1 >>>>>>>> 1 0 >>>>>>>> >>>>>>>> I dont know any other means to specify type of vertex ids. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> >>>>>>>>>> I am trying to load data in from file in >>>>>>>>>> IntIntNullTextInputFormat. >>>>>>>>>> But I >>>>>>>>>> get is the following message: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar >>>>>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0- >>>>>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar >>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip >>>>>>>>>> /user/hduser/input/test -vof >>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>> /user/hduser/output/amazon-shortestpaths -w 1 >>>>>>>>>> >>>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input >>>>>>>>>> format >>>>>>>>>> specified. Ensure your InputFormat does not require one. >>>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output >>>>>>>>>> format >>>>>>>>>> specified. Ensure your OutputFormat does not require one. >>>>>>>>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>>>>>>>> checkClassTypes: vertex index types not assignable, computation - >>>>>>>>>> class >>>>>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class >>>>>>>>>> org.apache.hadoop.io.IntWritable >>>>>>>>>> at >>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator. >>>>>>>>>> checkAssignable( >>>>>>>>>> GiraphConfigurationValidator.java:381) >>>>>>>>>> at >>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator. >>>>>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator. >>>>>>>>>> java:226) >>>>>>>>>> at >>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator. >>>>>>>>>> validateConfiguration( >>>>>>>>>> GiraphConfigurationValidator.java:141) >>>>>>>>>> at >>>>>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs( >>>>>>>>>> ConfigurationUtils.java:214) >>>>>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >>>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>>>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>>>>>> at >>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke( >>>>>>>>>> NativeMethodAccessorImpl.java:57) >>>>>>>>>> at >>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>>>>>>>> DelegatingMethodAccessorImpl.java:43) >>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The input file is here (just for testing): >>>>>>>>>> >>>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat >>>>>>>>>> /user/hduser/input/test >>>>>>>>>> 0 1 >>>>>>>>>> 1 0 >>>>>>>>>> >>>>>>>>>> Could you please help with this issue? >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Alex >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
