Hey Everione.
I need to read vertices in the format [Id, Val1, Val2, [[taget1, weight1], [target2, weight2]..]], and I want to get out simply the sum of its vertex values... For example, for the tiny_graph.txt file: [0,1,2,[[1,1],[3,3]]] [1,3,4,[[0,1],[2,2],[3,1]]] [2,5,5,[[1,2],[4,4]]] [3,6,7,[[0,3],[1,1],[4,4]]] [4,8,9,[[3,4],[2,4]]] So, the output would be like: 0,3 1,7 2,10 3,13 4,17 What must I do to achieve this goal? What classes Do I have to define? How Do I link these classes? I followed adives of http://stackoverflow.com/questions/24800957/vertices-with-complex-values-in-apache-giraph, so I created a few new classes TripletVertexInputFormat.java and TripletVertexWritable.java in the Giraph examples folder. 1.tar.gz presents a code to show each Vertex Id and its value. This code functions to read the tiny_graph.txt file. In Eclipse, I use as an Run As - Run Configuration - Argument "org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /usr/local/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /usr/local/output/shortestpaths -w 1 -ca giraph.SplitMasterWorker=false". Nertheless, in 2.tar.gz, when I define new classes to read a 2nd argument in the tiny_graph2.txt file, the used Run As - Run Configuration - Argument is "org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.examples.TripletVertexInputFormat -vip /usr/local/tiny_graph2.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /usr/local/output/shortestpaths -w 1 -ca giraph.SplitMasterWorker=false", shows me the following error messages: - 2015-06-03 00:36:19,985 ERROR [load-0] utils.LogStacktraceCallable (LogStacktraceCallable.java:call(57)) - Execution of callable failed java.lang.IllegalStateException: newInstance: Illegal access org.apache.giraph.examples.TripletVertexWritable at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86) at org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84) at org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717) at org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69) at org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60) at org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60) at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalAccessException: Class org.apache.giraph.utils.ReflectionUtils can not access a member of class org.apache.giraph.examples.TripletVertexWritable with modifiers "public" at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109) at java.lang.Class.newInstance(Class.java:373) at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81) ... 15 more 2015-06-03 00:36:19,998 ERROR [LocalJobRunner Map Task Executor #0] graph.GraphMapper (GraphMapper.java:run(99)) - Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3 java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3 at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136) at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99) at org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233) at org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:323) at org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:416) at org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:636) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:296) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: newInstance: Illegal access org.apache.giraph.examples.TripletVertexWritable at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:202) at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312) at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185) ... 17 more Caused by: java.lang.IllegalStateException: newInstance: Illegal access org.apache.giraph.examples.TripletVertexWritable at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86) at org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84) at org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717) at org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69) at org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60) at org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60) at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51) ... 4 more Caused by: java.lang.IllegalAccessException: Class org.apache.giraph.utils.ReflectionUtils can not access a member of class org.apache.giraph.examples.TripletVertexWritable with modifiers "public" at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109) at java.lang.Class.newInstance(Class.java:373) at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81) ... 15 more 2015-06-03 00:36:20,000 ERROR [LocalJobRunner Map Task Executor #0] worker.BspServiceWorker (BspServiceWorker.java:unregisterHealth(784)) - unregisterHealth: Got failure, unregistering health on /_hadoopBsp/job_local869455398_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir/ubuntu_0 on superstep -1 2015-06-03 00:36:20,011 INFO [Thread-13] mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete. 2015-06-03 00:36:20,016 WARN [Thread-13] mapred.LocalJobRunner (LocalJobRunner.java:run(560)) - job_local869455398_0001 java.lang.Exception: java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3 at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) Caused by: java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3 at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3 at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136) at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99) at org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233) at org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:323) at org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:416) at org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:636) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:296) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91) ... 8 more Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: newInstance: Illegal access org.apache.giraph.examples.TripletVertexWritable at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:202) at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312) at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185) ... 17 more Caused by: java.lang.IllegalStateException: newInstance: Illegal access org.apache.giraph.examples.TripletVertexWritable at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86) at org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84) at org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717) at org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69) at org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60) at org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60) at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51) ... 4 more Caused by: java.lang.IllegalAccessException: Class org.apache.giraph.utils.ReflectionUtils can not access a member of class org.apache.giraph.examples.TripletVertexWritable with modifiers "public" at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109) at java.lang.Class.newInstance(Class.java:373) at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81) ... 15 more Thanks for your assistance, and I wait for your comments. -- Cristian Vidal Silva Computer Engineer Master in Computer Science, University of Concepcion - Chile Master of Science in Computer Science, Michigan State University, MI - USA Ph.D(c) in Software Engineering and Technology, University of Seville - Spain Chilean Fulbright Scholar
2.tar.gz
Description: application/gzip
1.tar.gz
Description: application/gzip
