And finally, success is at hand! This is a bit quirky, but here's what
fixed it:
My command line originally looked like this:
$> hadoop jar
/home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.ap
OK, this was easy enough to fix, once I understood what
was actually happening. Since I'm running on EC2 nodes on
AWS, it is not the case that any give node can talk to any other
node on any port (at least not by default). I had tried to
cherry-pick which ports to whitelist in the security group,
Gang:
I am getting further with my attempt to get Giraph running on a YARN cluster,
but now I'm stuck at this error:
Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster
I've tried everything I can find in previous messages on this topic,
to no avail. My command line
This seems like the known problem with MapReduce counters. Try adding the
following to your hadoop-*/conf/mapred-site.xml:
mapreduce.job.counters.max
100
mapreduce.job.counters.limit
100
This does the trick for me on Hadoop 1.0.4, and should work for 0.20 as
we
I was able to increase the counters limit with: Counters.MAX_COUNTER_LIMIT
= 2024 (works for hadoop_1 and hadoop 1.2.1).
Then it turned out that whatever limit I set, it is always exceeded.
It turned out that for some reason IntOverwriteAggregator that
SccPhaseMasterCompute uses to propagate algo
Hi,
I believe the answer to your question is yes, though I've never done it. If
you use only the edge reader, only the vertices in your graph that have at
least one edge attached to them will be present in your graph. So, if you
have vertices that are entirely disconnected that you want included,
Hi Phillip,
I am using Hadoop 2.5.2 with Giraph 1.1.0 and it runs fine with
-Phadoop2 (from scratch) and -Phadoop_yarn (after removing
STATIC_SASL_SYMBOL from munge.symbols in pom.xml).
Maybe you can also try the stable Giraph
version and report your problem as an issue?
Cheers,
Martin
On 11.03
Hi Ralph,
you can set a vertex or edge input format when running a Giraph job.
In the example, you used the vertex input format (vif)
"-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat"
Your wikitalk input format is an edge list and Giraph offers, e.g.,
"org.apache.gi
