Hello,
Still rather new to Giraph and Hadoop but trying to research and learn the
software.
Am trying to get the SimpleShortestPathsVertex in the examples directory to
run from within Eclipse. I would swear that it ran and produced results once
last week, but have been unable to get the results output from the code since.
If I am reading this output correctly it appears that the application is not
(currently) reading the input data, although I have no idea why that would be
the case.
Input args:
/home/hduser/Downloads/shortestPathsInputGraph/
/home/hduser/Downloads/shortestPathsOutputGraph
1
1
The three files in the input directory are as follows:
part-m-00001 part-m-00002 part-m-00003
and contain:
[10,4500,[[11,2000]]]
[11,5500,[[12,1100]]]
[12,6600,[[13,1200]]]
[13,7800,[[14,1300]]]
[14,9100,[[0,1400]]]
[5,1000,[[6,500]]]
[6,1500,[[7,600]]]
[7,2100,[[8,700]]]
[8,2800,[[9,800]]]
[9,3600,[[10,900]]]
[0,0,[[1,0]]]
[1,0,[[2,100]]]
[2,100,[[3,200]]]
[3,300,[[4,300]]]
[4,600,[[5,400]]]
Running with the above results in creation of the output directory having an
empty file named _success within. No accompanying results.
Log messages from the run are as follows:
: 13/01/14 16:44:45 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
13/01/14 16:44:45 INFO mapred.JobClient: Running job: job_local_0001
13/01/14 16:44:45 INFO util.ProcessTree: setsid exited with exit code 0
13/01/14 16:44:45 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@676c6370
13/01/14 16:44:45 INFO graph.GraphMapper: Distributed cache is empty. Assuming
fatjar.
13/01/14 16:44:45 INFO graph.GraphMapper: setup: classpath @
/tmp/hadoop-hduser/mapred/staging/hduser-418008403/.staging/job_local_0001/job.jar
13/01/14 16:44:45 INFO zk.ZooKeeperManager: createCandidateStamp: Made the
directory _bsp/_defaultZkManagerDir/job_local_0001
13/01/14 16:44:45 INFO zk.ZooKeeperManager: createCandidateStamp: Creating my
filestamp _bsp/_defaultZkManagerDir/job_local_0001/_task/bitskammer 0
13/01/14 16:44:45 INFO zk.ZooKeeperManager: getZooKeeperServerList: For task 0,
got file 'zkServerList_bitskammer 0 ' (polling period is 3000)
13/01/14 16:44:45 INFO zk.ZooKeeperManager: getZooKeeperServerList: Found
[bitskammer, 0] 2 hosts in filename 'zkServerList_bitskammer 0 '
13/01/14 16:44:45 INFO graph.GraphMapper: cleanup: Starting for UNKNOWN
13/01/14 16:44:45 INFO mapred.Task: Task:attempt_local_0001_m_000000_0 is done.
And is in the process of commiting
13/01/14 16:44:46 INFO mapred.JobClient: map 0% reduce 0%
13/01/14 16:44:48 INFO mapred.LocalJobRunner: setup: Setting up Zookeeper
manager.
13/01/14 16:44:48 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0' done.
13/01/14 16:44:49 INFO mapred.JobClient: map 100% reduce 0%
13/01/14 16:44:49 INFO mapred.JobClient: Job complete: job_local_0001
13/01/14 16:44:49 INFO mapred.JobClient: Counters: 12
13/01/14 16:44:49 INFO mapred.JobClient: File Output Format Counters
13/01/14 16:44:49 INFO mapred.JobClient: Bytes Written=0
13/01/14 16:44:49 INFO mapred.JobClient: File Input Format Counters
13/01/14 16:44:49 INFO mapred.JobClient: Bytes Read=0
13/01/14 16:44:49 INFO mapred.JobClient: FileSystemCounters
13/01/14 16:44:49 INFO mapred.JobClient: FILE_BYTES_READ=5138685
13/01/14 16:44:49 INFO mapred.JobClient: FILE_BYTES_WRITTEN=5212576
13/01/14 16:44:49 INFO mapred.JobClient: Map-Reduce Framework
13/01/14 16:44:49 INFO mapred.JobClient: Map input records=1
13/01/14 16:44:49 INFO mapred.JobClient: Physical memory (bytes) snapshot=0
13/01/14 16:44:49 INFO mapred.JobClient: Spilled Records=0
13/01/14 16:44:49 INFO mapred.JobClient: Total committed heap usage
(bytes)=60162048
13/01/14 16:44:49 INFO mapred.JobClient: CPU time spent (ms)=0
13/01/14 16:44:49 INFO mapred.JobClient: Virtual memory (bytes) snapshot=0
13/01/14 16:44:49 INFO mapred.JobClient: SPLIT_RAW_BYTES=44
13/01/14 16:44:49 INFO mapred.JobClient: Map output records=0
Any help would be appreciated.
Is there by any chance a tutorial anywhere on how to build your own Giraph
application? I've been reading through source code but feel like I've started
somewhere in the middle and can never find the starting place - just going
around in circles. I could really use a "start-here" reference.
In any event, thanks for any and all help!
[https://mail.google.com/mail/images/cleardot.gif]
