Hi, 步?云 Please try to use TRUNK version. and you can refer the https://github.com/apache/hama/blob/master/examples/src/test/java/org/apache/hama/examples/CustomVertexReadWriteStateTest.java and https://github.com/apache/hama/blob/master/ml/src/main/java/org/apache/hama/ml/kcore/KCoreVertex.java. Both code contains custom read/writeState use case.
-- Best Regards, Edward J. Yoon -----Original Message----- From: Edward J. Yoon [mailto:[email protected]] Sent: Thursday, January 28, 2016 8:33 PM To: [email protected] Subject: Re: RE: RE: RE: RE: Do Hama support member member variable? I filed on https://issues.apache.org/jira/browse/HAMA-982 Once it's fixed, I'll let you know. On Thu, Jan 28, 2016 at 8:11 PM, Edward J. Yoon <[email protected]> wrote: > Wow, you find the bug. Thanks ;) > >> at >> org.apache.hama.graph.GraphJobRunner$Parser.run(GraphJobRunner.java:557) > > When the framework assign vertices to the proper machine at initial > phase, vertex objects are transferred in serialized form. At this step > user defined code won't work correctly. I'll fix soon. > > Anyway, you should able to manage an array of TextPair objects like below: > > private TextPair[] test = new TextPair[1]; > > public void readState(DataInput in) throws IOException { > int size = in.readInt(); > test = new TextPair[size]; > for(int i = 0; i < size; i++) { > test[i].readFields(in); > } > } > > public void writeState(DataOutput out) throws IOException { > out.writeInt(test.length); > for(int i = 0; i < test.length; i++) { > test[i].write(out); > } > } > > Thanks. > > > On Thu, Jan 28, 2016 at 6:45 PM, 步?云 <[email protected]> wrote: >> Thanks for your reply. You do help me a lot. >> I have tried to use two methods. Some problem are still bothering me. >> When I use the first method of using Hadoop built-in writable classes, I >> get a NullPointerException meaning the parents is null， even though I have >> initialize parents. The code is like this. >> static ArrayWritable parents= new ArrayWritable(TextPair.class); >> >> >> public void writeState(DataOutput out) throws IOException { >> out.writeBoolean(match); >> parents.write(out); >> } >> >> >> public void readState(DataInput in) throws IOException { >> match = in.readBoolean(); >> parents.readFields(in); >> } >> >> >> >> And the error message is as follow: >> Exception in thread "pool-6-thread-2" java.lang.RuntimeException: >> java.lang.NullPointerException >> at >> org.apache.hama.graph.GraphJobRunner$Parser.run(GraphJobRunner.java:562) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.lang.NullPointerException >> at org.apache.hadoop.io.ArrayWritable.write(ArrayWritable.java:103) >> at ProbMatch$ProbMatchVertex.writeState(ProbMatch.java:154) >> at org.apache.hama.graph.Vertex.write(Vertex.java:311) >> at >> org.apache.hama.util.WritableUtils.unsafeSerialize(WritableUtils.java:55) >> at >> org.apache.hama.graph.MapVerticesInfo.put(MapVerticesInfo.java:64) >> at >> org.apache.hama.graph.GraphJobRunner.addVertex(GraphJobRunner.java:577) >> at >> org.apache.hama.graph.GraphJobRunner.access$300(GraphJobRunner.java:64) >> at >> org.apache.hama.graph.GraphJobRunner$Parser.run(GraphJobRunner.java:557) >> >> >> >> When I'm trying use the second method, I don't know how to read and write >> an object array. I can use out.writeInt() method to write int. But when I >> need to write an object. How can I do this? I'm so sorry that I'm not good >> at Java. Here is the some code I'm trying to write. >> >> >> static TextPair[] arr = new TextPair[1]; >> >> >> public void writeState(DataOutput out) throws IOException { >> out.writeBoolean(match); >> out.writeInt(arr.length); >> for(int i = 0; i < arr.length; i++) { >> out.writeInt(arr[i].toString()); // Is this right? >> }} >> >> >> public void readState(DataInput in) throws IOException { >> match = in.readBoolean(); >> int length = in.readInt(); >> for(int i=0;i<length;i++){ >> arr[i] = in.read // How can I read the textpair here? >> } >> } >> >> >> >> I'm very grateful to your help. Thanks again. >> Best Regards, Ping Liu. >> >> >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";<[email protected]>; >> Date: Thu, Jan 28, 2016 11:51 AM >> To: "user"<[email protected]>; >> >> Subject: RE: RE: RE: RE: Do Hama support member member variable? >> >> >> >> Hi, >> >> You can use Hadoop built-in writable classes or own custom Writable. >> >> static ArrayWritable arr = new ArrayWritable(DoubleWritable.class); >> >> public void writeState(DataOutput out) throws IOException { >> arr.write(out); >> } >> >> Or, >> >> static int[] arr2 = new int[3]; >> >> public void writeState(DataOutput out) throws IOException { >> out.writeInt(arr2.length); >> for(int i = 0; i < arr2.length; i++) { >> out.writeInt(arr2[i]); >> } >> } >> >> public void readState(DataInput in) throws IOException { >> int size = in.readInt(); >> for(int i = 0; i < size; i++) { >> arr2[i] = in.readInt(); >> } >> } >> >> -- >> Best Regards, Edward J. Yoon >> >> >> -----Original Message----- >> From: 步?云 [mailto:[email protected]] >> Sent: Thursday, January 28, 2016 11:16 AM >> To: user >> Subject: Re: RE: RE: RE: Do Hama support member member variable? >> >> Hi, >> I still don't know how to use member variable. What's the input >> parameter >> DataOutput out in writeState() method? Could you please give me a example >> to >> use this method? Thanks a lot. >> By the way, how can I deal with object member variable? For example, >> the >> member variable "private List parents" store the parents of the vertex. How >> can I read and write this variable? >> If you can take time to reply me, I will be very grateful to you. >> Best Regard, Ping Liu. >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";<[email protected]>; >> Date: Wed, Jan 27, 2016 03:12 PM >> To: "user"<[email protected]>; >> >> Subject: RE: RE: RE: Do Hama support member member variable? >> >> >> >> Just FYI, Hama 0.7.1 release candidate is now available. Please feel free >> to >> use this, and it'd nice if you can let me know whether it works well with >> you. >> >> Release tarball: http://people.apache.org/~edwardyoon/dist/0.7.1-RC1 >> >> -- >> Best Regards, Edward J. Yoon >> >> >> -----Original Message----- >> From: 步?云 [mailto:[email protected]] >> Sent: Wednesday, January 27, 2016 11:01 AM >> To: user >> Subject: Re: RE: RE: Do Hama support member member variable? >> >> I know. If I use the static variable like "private static boolean match", I >> could get the right value of match too. Could I use static variable? >> Thanks a lot. >> Best regards. Ping Liu. >> >> >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";<[email protected]>; >> Date: Wed, Jan 27, 2016 09:39 AM >> To: "user"<[email protected]>; >> >> Subject: RE: RE: Do Hama support member member variable? >> >> >> >> Basically Vertex object is writable, and we store the vertex objects in >> serialized form. There are two purposes: 1) to reduce memory usage 2) to >> write >> on file system (checkpoint and recovery). >> >> So, you should use readState() and writeState() methods to save object >> member >> variables. >> >> Thanks! >> >> -- >> Best Regards, Edward J. Yoon >> >> >> -----Original Message----- >> From: 步?云 [mailto:[email protected]] >> Sent: Wednesday, January 27, 2016 10:27 AM >> To: user >> Subject: Re: RE: Do Hama support member member variable? >> >> Hi, >> Thank you very much. You helped me a lot. But I still don't know how to use >> readState() and writeState() methods. What is the input parameter DataInput >> in >> and DataOutput out? And as far as I know, I can assign the value of member >> variable directly in hama-0.6.3, such as match = true. Why I can't do this >> in >> Hama-0.7.0? >> Waiting for your reply. Thanks. >> Best Regards, Ping Liu. >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";<[email protected]>; >> Date: Wed, Jan 27, 2016 07:15 AM >> To: "user"<[email protected]>; >> >> Subject: RE: Do Hama support member member variable? >> >> >> >> Hi, >> >> You should use readState() and writeState() methods like below: >> >> public static class ProbMatchVertex extends Vertex<Text, NullWritable, >> TriTextPair> { >> private boolean match = false; >> >> public void readState(DataInput in) throws IOException { >> match = in.readBoolean(); >> } >> >> public void writeState(DataOutput out) throws IOException { >> out.writeBoolean(match); >> } >> >> .. >> >> } >> >> -- >> Best Regards, Edward J. Yoon >> >> -----Original Message----- >> From: 꼍행暾 [mailto:[email protected]] >> Sent: Tuesday, January 26, 2016 10:16 PM >> To: user >> Subject: Do Hama support member member variable? >> >> Hello, >> I'm trying to run a graph job. But i have got some problems. >> I want to use member variable in vertex class, the code is as follow. >> I have changed the value of the member variable in one superstep. But when >> I >> use this member variable in next superstep, the value of this member >> variable is still the defalult value. For example, "match" is the member >> variable. I have changed the value of "match" to be ture in superstep 0, >> but >> when I print "match" in superstep 1, the result was "match: false". >> Could anyone tell me why the value of member variable is changed? >> Thanks very much. >> Best wishes. >> >> >> public static class ProbMatchVertex extends Vertex<Text, >> NullWritable, >> TriTextPair> { >> private boolean match = false; >> >> >> @Override >> public void compute(Iterable<TriTextPair> messages) throws >> IOException { >> if (getSuperstepCount() == 0) { >> match = true; >> sendMessageToNeighbors(new >> TriTextPair(getVertexID(), >> getVertexLabel(), new >> Text(""))); >> } else if(getSuperstepCount() == 1){ >> System.out.println("match:" + match); >> parents = new ArrayList<TriTextPair>(); >> for(TriTextPair msg : messages){ >> parents.add(msg); >> sendMessage(msg.getFirst(), new >> TriTextPair(getVertexID(), getVertexLabel(), new Text(""))); >> } >> } >> } >> >> >> >> >> Ping Liu > > > > -- > Best Regards, Edward J. Yoon -- Best Regards, Edward J. Yoon
