Re: TestJsonBase64Format failure on 1.0.0
Here's an easy way to calculate the number of map slots required.
workers + 1. So for instance if you only have 2 map slots, you can only
run at most one worker. If you're running on your laptop, feel free to
change the number of map slots to have you need.
Hope that helps,
Avery
On 5/7/13 11:40 AM, Kiru Pakkirisamy wrote:
Avery et al,
Please let me know if I should use Giraph only on a multi-node cluster
(what is the min # of nodes ?)
Regards,
- kiru
Kiru Pakkirisamy | webcloudtech.wordpress.com
--- On *Mon, 5/6/13, Kiru Pakkirisamy //*
wrote:
From: Kiru Pakkirisamy
Subject: TestJsonBase64Format failure on 1.0.0
To: user@giraph.apache.org
Date: Monday, May 6, 2013, 9:40 AM
I got over my compilation issues (thanks - @Avery, @Roman).
Now, I am trying to run the test and one pearticular test is failing.
I want to get to the bottom of this, because I am unable to run
the PageRank example. Maybe, it is because I have only on
tasktracker (?) (Apache pseudo-cluster on my ubuntu laptop).
Regards,
- kiru
2013-05-06 09:12:32,197 INFO org.apache.hadoop.mapred.JobTracker:
Adding task (MAP) 'attempt_201305052325_0013_m_03_0' to tip
task_201305052325_0013_m_03, for tracker
'tracker_kiru-N53SV:localhost/127.0.0.1:42265'
2013-05-06 09:12:35,198 INFO
org.apache.hadoop.mapred.TaskInProgress: Error from
attempt_201305052325_0013_m_02_0:
java.lang.IllegalStateException: run: Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@482d59a3
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.IllegalStateException: waitFor:
ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@482d59a3
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:151)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:111)
at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:73)
at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:192)
at
org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:276)
at
org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:323)
at
org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:506)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:230)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
... 7 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: call: IOException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
at java.util.concurrent.FutureTask.get(FutureTask.java:91)
at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:271)
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:143)
... 15 more
Caused by: java.lang.IllegalStateException: call: IOException
at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:172)
at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:58)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException:
/tmp/_giraphTests/testContinue/_logs (Is a directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:120)
at
org.apache.hadoop.fs.RawLocalFileSystem$TrackingFileInputStream.(RawLocalFileSystem.java:71)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.(RawLocalFileSystem.java:107)
at
org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:177)
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecke
Re: TestJsonBase64Format failure on 1.0.0
Avery et al,Please let me know if I should use Giraph only on a multi-node cluster (what is the min # of nodes ?) Regards, - kiru Kiru Pakkirisamy | webcloudtech.wordpress.com --- On Mon, 5/6/13, Kiru Pakkirisamy wrote: From: Kiru Pakkirisamy Subject: TestJsonBase64Format failure on 1.0.0 To: user@giraph.apache.org Date: Monday, May 6, 2013, 9:40 AM I got over my compilation issues (thanks - @Avery, @Roman).Now, I am trying to run the test and one pearticular test is failing.I want to get to the bottom of this, because I am unable to run the PageRank example. Maybe, it is because I have only on tasktracker (?) (Apache pseudo-cluster on my ubuntu laptop). Regards,- kiru 2013-05-06 09:12:32,197 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP) 'attempt_201305052325_0013_m_03_0' to tip task_201305052325_0013_m_03, for tracker 'tracker_kiru-N53SV:localhost/127.0.0.1:42265'2013-05-06 09:12:35,198 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201305052325_0013_m_02_0: java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@482d59a3 at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249)Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@482d59a3 at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:151) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:111) at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:73) at org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:192) at org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:276) at org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:323) at org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:506) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:230) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92) ... 7 moreCaused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: call: IOExceptionat java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) at java.util.concurrent.FutureTask.get(FutureTask.java:91) at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:271) at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:143) ... 15 moreCaused by: java.lang.IllegalStateException: call: IOException at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:172) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:58) at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)Caused by: java.io.FileNotFoundException: /tmp/_giraphTests/testContinue/_logs (Is a directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(FileInputStream.java:120) at org.apache.hadoop.fs.RawLocalFileSystem$TrackingFileInputStream.(RawLocalFileSystem.java:71) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.(RawLocalFileSystem.java:107) at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:177) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:126) at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427) at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:67) at org.apache.giraph.io.formats.TextVertexInputFormat$TextVertexReader.initialize(TextVertexInputFormat.java:96) at org.apache.giraph.io.formats.JsonBase64VertexInputFormat$JsonBase64VertexReader.initialize(JsonBase64VertexInputFormat
TestJsonBase64Format failure on 1.0.0
I got over my compilation issues (thanks - @Avery, @Roman).Now, I am trying to run the test and one pearticular test is failing.I want to get to the bottom of this, because I am unable to run the PageRank example. Maybe, it is because I have only on tasktracker (?) (Apache pseudo-cluster on my ubuntu laptop). Regards,- kiru 2013-05-06 09:12:32,197 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP) 'attempt_201305052325_0013_m_03_0' to tip task_201305052325_0013_m_03, for tracker 'tracker_kiru-N53SV:localhost/127.0.0.1:42265'2013-05-06 09:12:35,198 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201305052325_0013_m_02_0: java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@482d59a3 at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249)Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@482d59a3 at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:151) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:111) at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:73) at org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:192) at org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:276) at org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:323) at org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:506) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:230) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92) ... 7 moreCaused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: call: IOExceptionat java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) at java.util.concurrent.FutureTask.get(FutureTask.java:91) at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:271) at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:143) ... 15 moreCaused by: java.lang.IllegalStateException: call: IOException at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:172) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:58) at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)Caused by: java.io.FileNotFoundException: /tmp/_giraphTests/testContinue/_logs (Is a directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(FileInputStream.java:120) at org.apache.hadoop.fs.RawLocalFileSystem$TrackingFileInputStream.(RawLocalFileSystem.java:71) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.(RawLocalFileSystem.java:107) at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:177) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:126) at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427) at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:67) at org.apache.giraph.io.formats.TextVertexInputFormat$TextVertexReader.initialize(TextVertexInputFormat.java:96) at org.apache.giraph.io.formats.JsonBase64VertexInputFormat$JsonBase64VertexReader.initialize(JsonBase64VertexInputFormat.java:71) at org.apache.giraph.worker.VertexInputSplitsCallable.readInputSplit(VertexInputSplitsCallable.java:120) at org.apache.giraph.worker.InputSplitsCallable.loadInputSplit(InputSplitsCallable.java:220) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:161) ... 7 more 2013-05-06 09:22:44,485 INFO org.apache.hadoop.mapred.TaskInProgress:
