Thanks Mirko ! that makes sense. The only reason I didn’t use Yarn is because I
was worried about version compatibility issues between giraph and Hadoop, so I
used the older version of hadoop.
I will try to go through the log4j/flume/hdfs route.
Thanks,
Tamer
From: Mirko Kämpf [mailto:mirko.kae
Hi Tamer,
if you run Giraph on YARN you can use the log aggregation feature. If you
try to write to HDFS you should consider the HDFS API, but many mappers
would have to write into individual files. Why not writing all logs via
Log4j into Flume and from here to HDFS?
There is a Log4J appender for
To follow-up on my question, I have found the messages only in the Task logs
(in Browser from the hadoop task logs).
How do you write these same messages to hdfs output file?
Thanks,
-Tamer
From: Tamer Yousef
Sent: Monday, October 06, 2014 4:25 PM
To: user@giraph.apache.org
Subject: RE: How to
Thanks Charith, but my main question still remains, even with the examples that
comes with Giraph, such as simple shortest path computation example, the
System.out.println or the Log.Debug (or I also tried Log.Info) they all do not
print out customer messages that I write in the compute method.
I want to access the aggregated value of an aggregator, from my custom
VertexOutputFormat class.
Is it possible ?
RegardsPuneetIIT Delhi
Dear Mathew,
Pardon the late reply.
In my work, every vertex will have a score, I want to find the top-K scoring
vertexes.I planned to use an aggregator for this, here, I store my aggregated
value as Text, which stores the top-K vertex-id and their scores.
By the way, I have solved my problem in
Theoretically, Giraph on YARN would be much better (actual resource
request rather than mapper hack). That being said, Eli is the best
person to talk about that. We haven't tried YARN.
Avery
On 10/6/14, 8:51 AM, Matthew Cornell wrote:
Hi Folks. I don't think I paid enough attention to YARN v
Hi Folks. I don't think I paid enough attention to YARN vs. MR1 when I
built Giraph 1.0.0 for our system. How much better is Giraph on YARN?
Thank you.
--
Matthew Cornell | m...@matthewcornell.org
