Re: failing GraphX application ('GC overhead limit exceeded', 'Lost executor', 'Connection refused', etc.)

2015-02-14 Thread Matthew Cornell
, not just the 0-3 one, so it's not node-specific. Any ideas about how to fix this? Thanks again, matt On Feb 12, 2015, at 10:46 AM, Matthew Cornell wrote: Hi Folks, I'm running a five-step path following-algorithm on a movie graph with 120K verticies and 400K edges. The graph has vertices

failing GraphX application ('GC overhead limit exceeded', 'Lost executor', 'Connection refused', etc.)

2015-02-12 Thread Matthew Cornell
is running out of memory, though I think it has adequate aggregate ram to handle this app. Thanks very much -- matt Matthew Cornell, Research Fellow, Computer Science Department, Umass Amherst - To unsubscribe, e-mail: user

GraphX pregel: getting the current iteration number

2015-02-03 Thread Matthew Cornell
Hi Folks, I'm new to GraphX and Scala and my sendMsg function needs to index into an input list to my algorithm based on the pregel()() iteration number, but I don't see a way to access that. I see in

getting started writing unit tests for my app

2015-01-20 Thread Matthew Cornell
Hi Folks, I'm writing a GraphX app and I need to do some test-driven development. I've got Spark running on our little cluster and have built and run some hello world apps, so that's all good. I've looked through the test source and found lots of helpful examples that use SharedSparkContext, and