OutOfMemory Error

2014-08-17 Thread Ghousia Taj
Hi, I am trying to implement machine learning algorithms on Spark. I am working on a 3 node cluster, with each node having 5GB of memory. Whenever I am working with slightly more number of records, I end up with OutOfMemory Error. Problem is, even if number of records is slightly high, the

Query on Merge Message (Graph: pregel operator)

2014-06-19 Thread Ghousia Taj
Hi, Can someone please clarify a small query on Graph.pregel operator. As per the documentation on merge Message function, only two inbound messages can be merged to a single value. Is it the actual case, if so how can one merge n inbound messages . Any help is truly appreciated. Many Thanks,