Hi, I am able to successfully build hadoop_yarn profile for running Giraph 1.1. I am also able to test run Connected Components on a small dataset. However, I am seeing 2 issues while running on a bigger dataset with 400 mappers:
1. I am unable to use out of Core Graph option. It errors out saying that it cannot read INIT partition. (Sorry I don’t have the log currently but I will share after I run that again). I am expecting that if the out of Core option is fixed, I should be able to run the workflow with less mappers. 2. In order to run the workflow anyhow, I removed the out of Core option and adjusted the heap size. This also runs with smaller dataset but fails with huge dataset. Worker logs are mostly empty. Non-empty logs end like this: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition [STATUS: task-374] setup: Beginning worker setup. setup: Log level remains at info [STATUS: task-374] setup: Initializing Zookeeper services. mapred.job.id is deprecated. Instead, use mapreduce.job.id job.local.dir is deprecated. Instead, use mapreduce.job.local.dir [STATUS: task-374] setup: Setting up Zookeeper manager. createCandidateStamp: Made the directory _bsp/_defaultZkManagerDir/giraph_yarn_application_1407992474095_708614 createCandidateStamp: Made the directory _bsp/_defaultZkManagerDir/giraph_yarn_application_1407992474095_708614/_zkServer createCandidateStamp: Creating my filestamp _bsp/_defaultZkManagerDir/giraph_yarn_application_1407992474095_708614/_task/gsta33201.tan.ygrid.yahoo.com 374 getZooKeeperServerList: For task 374, got file 'null' (polling period is 3000) Master log has log statements for launching the container, opening proxy and processing event like this: Opening proxy : gsta31118.tan.ygrid.yahoo.com:8041 Processing Event EventType: QUERY_CONTAINER for Container container_1407992474095_708614_01_000314 …… I am not using SASL authentication. Any idea what might be wrong? Thanks, Tripti.
