Re: How to ensure that only one worker runs per node

-- Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34 Dickinson Street, Amherst MA 01002 | matthewcornell.org

Missing chosen worker ERROR drills down to end of stream exception (likely client has closed socket). help!

was the problem. But why did *that* happen? I could really use your help here! Thank you, matt -- Matthew Cornell | m...@matthewcornell.org

Re: YARN vs. MR1: is YARN a good idea ? Out-of-Core Graph ?

) I'm still figuring it out and can get back once it's resolved. I have patched a few things and will share them for people who might face similar issue. If u have a fix for scalability, do let me know Thanks, Tripti Sent from my iPhone On 06-Oct-2014, at 9:22 pm, Matthew Cornell m

YARN vs. MR1: is YARN a good idea?

Hi Folks. I don't think I paid enough attention to YARN vs. MR1 when I built Giraph 1.0.0 for our system. How much better is Giraph on YARN? Thank you. -- Matthew Cornell | m...@matthewcornell.org

Re: Giraph 1.0 | Computation stuck at map 100% - reduce 0% for my algorithm only, at multi-node cluster

Is there anything I'm overlooking? I have Googled the obvious stack overflow solutions for two days now. Has anyone encountered anything similar? Regards, Panagiotis Eustratiadis. -- Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34 Dickinson Street, Amherst MA 01002

Re: receiving messages that I didn't send

it this time. In general, does this anomalous behavior indicate a Giraph/Hadoop gotcha'? It's really confounding! Thank very much -- matt -- Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34 Dickinson Street, Amherst MA 01002 | matthewcornell.org -- Matthew Cornell | m

receiving messages that I didn't send

of re-using it, but I haven't been able to solve it this time. In general, does this anomalous behavior indicate a Giraph/Hadoop gotcha'? It's really confounding! Thank very much -- matt -- Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34 Dickinson Street, Amherst MA 01002

understanding failing my job, Giraph/Hadoop memory usage, under-utilized nodes, and moving forward

)=8284 INFO mapred.JobClient: Total (milliseconds)=20322 EOF -- Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34 Dickinson Street, Amherst MA 01002 | matthewcornell.org

Re: how do I maintain a cached List across supersteps?

wrote: I would use a workercontext, it is shared and persistent during computation by all vertices in a worker. If it's readonly, you won't have to manage concurrency. On Tue, Sep 16, 2014 at 9:42 PM, Matthew Cornell m...@matthewcornell.org wrote: Hi Folks. I have a custom argument that's

how do I maintain a cached List across supersteps?

by getConf(), but it supports only String properties. I looked at using my custom MasterCompute to manage it, but I couldn't find how to access the master compute instance from the vertex. My last idea is to use (abuse?) an aggregator to do this. I'd appreciate your thoughts! -- matt -- Matthew

How do I validate customArguments?

formatted. Where do I do that? The only thing I thought of is to specify a DefaultMasterCompute subclass whose initialize() does the check, but all the initialize() examples do is register aggregators; none of them check args or do anything else. Thanks in advance! -- matt -- Matthew Cornell | m

Which is better to use to manage Vertex state: POJO instance variables or Giraph values?

vertex value (IntArrayListWritable). I'm a little surprised that using an instance variable is legit due to possibly screwing up serialization (?) My question: Is either valid? If so, how do I choose one over the other? Thanks very much. -- matt -- Matthew Cornell | m...@matthewcornell.org | 413-626

How do I look up a Vertex using its ID?

partition. Thanks in advance. -- Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34 Dickinson Street, Amherst MA 01002 | matthewcornell.org