Hi, I am trying to do a the connected component example. I have modified it, the Writables a bit to fit my example. Some of the vertex indexes are negative (long hashed of strings), I tried with a sample example with around 10 vertices and it works properly. But when I load the whole set around 1.5B vertices , all of them do not get loaded and the final output misses a lot of the pairs. Any idea what might be the reason ? is it required to have the vertex indexes as positive only, I tried with -ve with a sample set and it works, any other place I should look into or put debug statements ?
I am running with 118 workers. Cluster Config : Running Map TasksRunning Reduce TasksTotal SubmissionsNodesOccupied Map SlotsOccupied Reduce SlotsReserved Map SlotsReserved Reduce SlotsMap Task CapacityReduce Task CapacityAvg. Tasks/NodeBlacklisted NodesExcluded Nodes 0 0387 <http://had24.rsk.admobius.com:50030/machines.jsp?type=active>000016884 36.000 <http://had24.rsk.admobius.com:50030/machines.jsp?type=blacklisted>0 -- Best Regards, Jyotirmoy Sundi Data Engineer, Admobius San Francisco, CA 94158