Hi Alexander,
the only inputformat that we have which is compatible with
SimpleShortestPathsComputation is the
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
which is used in
org.apache.giraph.examples.SimpleShortestPathsComputationTest#testToyData
It's not as nice to use as the other inputformats we have, so I opened
https://issues.apache.org/jira/browse/GIRAPH-844 to ensure that we have
a nice easy to use TextInputFormat for the shortest paths example.
--sebastian
On 02/03/2014 06:52 PM, Alexander Frolov wrote:
It seems that I understood the problem. PageRank and
SingleSourceShortestPath both need weighted graph to compute. I just
missed it somehow.
Thank you for help.
On Mon, Feb 3, 2014 at 9:25 PM, Alexander Frolov
<alexndr.fro...@gmail.com>wrote:
On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <s...@apache.org> wrote:
If you just wanna play with Giraph, I suggest you start with the PageRank
example:
https://giraph.apache.org/pagerank.html
I have seen it. I think there is no much difference in SSSP and PageRank
examples. By the way, I have not found LongDoubleNullTextInputFormat in my
sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned
there.
The same issue with SimplePageRankComputation.
[hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimplePageRankComputation -vif
org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
/user/hduser/input/test -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/amazon-shortestpaths -w 1
14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: vertex value types not assignable, computation - class
org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
org.apache.hadoop.io.LongWritable
at
org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at
org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at
org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
--sebastian
On 02/03/2014 05:49 PM, Alexander Frolov wrote:
I've tried .LongLongNullTextInputFormat and got this:
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
/user/hduser/input/test -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/amazon-shortestpaths -w 1
14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: vertex value types not assignable, computation - class
org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
org.apache.hadoop.io.LongWritable
at
org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
GiraphConfigurationValidator.java:381)
at
org.apache.giraph.job.GiraphConfigurationValidator.
verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
java:228)
at
org.apache.giraph.job.GiraphConfigurationValidator.
validateConfiguration(GiraphConfigurationValidator.java:141)
at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(
ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
<alexndr.fro...@gmail.com>wrote:
On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <s...@apache.org>
wrote
What types does the vertex class that you use in your computation
require?
I see.
public class SimpleShortestPathsComputation extends BasicComputation<
LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
So that is a mistake.
Thank you very much.
Is there any means to import data from file like this (amazon0601.txt
from
SNAP data collection):
0 1
0 2
0 3
0 4
0 5
0 6
0 7
0 8
0 9
0 10
...
and to perform basic tests from giraph-examples? Most of the tests are
using LongWritable for vertex id.
On 02/03/2014 05:28 PM, Alexander Frolov wrote:
On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <s...@apache.org>
wrote:
Hi Alexander,
what do you use as type for your vertex ids? It looks that you are
trying
to use longs, while IntIntNullTextInputFormat only provides ints,
that
could be the error.
--sebastian
I also thought so, but I could not understand why it regard it as
long.
I just use -vif org.apache.giraph.io.formats.
IntIntNullTextInputFormat
-vip /user/hduser/input/test -vof
and file consists only of two lines:
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
/user/hduser/input/test
0 1
1 0
I dont know any other means to specify type of vertex ids.
On 02/03/2014 05:06 PM, Alexander Frolov wrote:
Hello,
I am trying to load data in from file in IntIntNullTextInputFormat.
But I
get is the following message:
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
/user/hduser/input/test -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/amazon-shortestpaths -w 1
14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input
format
specified. Ensure your InputFormat does not require one.
14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output
format
specified. Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: vertex index types not assignable, computation -
class
org.apache.hadoop.io.LongWritable, VertexInputFormat - class
org.apache.hadoop.io.IntWritable
at
org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
GiraphConfigurationValidator.java:381)
at
org.apache.giraph.job.GiraphConfigurationValidator.
verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
java:226)
at
org.apache.giraph.job.GiraphConfigurationValidator.
validateConfiguration(
GiraphConfigurationValidator.java:141)
at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(
ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
The input file is here (just for testing):
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
/user/hduser/input/test
0 1
1 0
Could you please help with this issue?
Best,
Alex
