Re: Problem occured when running job with 1 worker.
Hi Jyoti; I assume this is the log of master vertex. It seems like master can not reach a worker for some reason. Did you also check the worker vertex's log? Maybe you can share it too. Sertug On 20-01-2014 09:22, Jyoti Yadav wrote: *h.master.MasterThread: masterThread: Master algorithm failed with ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException: -1*
Re: About LineRank algo ..
do you plan to share it when you're done? :) On Mon, Jan 20, 2014 at 9:15 AM, Sebastian Schelter s...@apache.org wrote: I have a student working on an implementation, do you have questions? On 01/20/2014 08:11 AM, Jyoti Yadav wrote: Hi.. Is there anyone who is working with linerank algorithm?? Thanks Jyoti -- Claudio Martella claudio.marte...@gmail.com
Re: About LineRank algo ..
Sure :) On 01/20/2014 09:39 AM, Claudio Martella wrote: do you plan to share it when you're done? :) On Mon, Jan 20, 2014 at 9:15 AM, Sebastian Schelter s...@apache.org wrote: I have a student working on an implementation, do you have questions? On 01/20/2014 08:11 AM, Jyoti Yadav wrote: Hi.. Is there anyone who is working with linerank algorithm?? Thanks Jyoti
Re: About LineRank algo ..
Hi Sebastian.. I code this algorithm,but while running,it is not converging.. One more question,for power iteration.is it necessary to column normalize the matrix or we can work with row normalized matrix? Thanks Jyoti On Mon, Jan 20, 2014 at 1:45 PM, Sebastian Schelter s...@apache.org wrote: I have a student working on an implementation, do you have questions? On 01/20/2014 08:11 AM, Jyoti Yadav wrote: Hi.. Is there anyone who is working with linerank algorithm?? Thanks Jyoti
Re: About LineRank algo ..
Hi Sebastian... while referring the paper,paper talks about the normalization of L(G) matrix..Below is the few lines from the paper which talks about it.. Computing Normalization Factors. The ith element of the diagonal matrix D contains the sum of ith column of L(G). D is used to column-normalize L(G) so that the resulting matrix can be used for the power iteration. The ’./’ in line 5 represents the element-wise inverse operation. One more question... Is LineRank algo is applicable to undirected and weighted graph? Thanks On Mon, Jan 20, 2014 at 2:40 PM, Sebastian Schelter s...@apache.org wrote: Jyoti, We started with a Matlab implementation on a small example graph and saw the algorithm converge. I don't think that the paper mentions that you have to normalize the matrix in a certain way. In the standard power iteration, the vector that estimates the principal eigenvector has to be rescaled to unit length. IIRC this is also done in the LineRank algorithm in the paper. --sebastian On 01/20/2014 10:04 AM, Jyoti Yadav wrote: Hi Sebastian.. I code this algorithm,but while running,it is not converging.. One more question,for power iteration.is it necessary to column normalize the matrix or we can work with row normalized matrix? Thanks Jyoti On Mon, Jan 20, 2014 at 1:45 PM, Sebastian Schelter s...@apache.org wrote: I have a student working on an implementation, do you have questions? On 01/20/2014 08:11 AM, Jyoti Yadav wrote: Hi.. Is there anyone who is working with linerank algorithm?? Thanks Jyoti
Re: Problem occured when running job with 1 worker.
Hi Kaya.. Below is the worker's log.. WARN org.apache.giraph.comm.netty.handler.ResponseClientHandler: exceptionCaught: Channel failed with remote address kanha-Vostro-1014/ 127.0.1.1:30002 java.nio.channels.ClosedChannelException at org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:674) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:642) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:98) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:385) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:256) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 2014-01-20 12:29:40,161 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server kanha-Vostro-1014/127.0.1.1:22181 2014-01-20 12:29:40,106 WARN org.apache.giraph.comm.netty.handler.ResponseClientHandler: exceptionCaught: Channel failed with remote address null java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:366) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:282) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 2014-01-20 12:29:40,297 WARN org.apache.zookeeper.ClientCnxn: Session 0x143ae2e202a0001 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) 2014-01-20 12:29:40,044 WARN org.apache.giraph.comm.netty.handler.RequestServerHandler: exceptionCaught: Channel failed with remote address /127.0.0.1:43641 java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) at sun.nio.ch.IOUtil.read(IOUtil.java:193) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:63) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:385) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:256) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 2014-01-20 12:29:40,074 WARN org.apache.giraph.comm.netty.handler.ResponseClientHandler: exceptionCaught: Channel failed with remote address kanha-Vostro-1014/ 127.0.1.1:30002 java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) at sun.nio.ch.IOUtil.read(IOUtil.java:193) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:63) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:385) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:256) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) 2014-01-20 12:29:40,044 WARN org.apache.giraph.comm.netty.NettyClient: getNextChannel: Failed to reconnect to kanha-Vostro-1014/127.0.1.1:30002 on attempt 1 out of 1000 max attempts, sleeping for 5 secs java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at
Re: About LineRank algo ..
On 01/20/2014 11:48 AM, Jyoti Yadav wrote: Hi Sebastian... while referring the paper,paper talks about the normalization of L(G) matrix..Below is the few lines from the paper which talks about it.. Computing Normalization Factors. The ith element of the diagonal matrix D contains the sum of ith column of L(G). D is used to column-normalize L(G) so that the resulting matrix can be used for the power iteration. The ’./’ in line 5 represents the element-wise inverse operation. Ah I see. You're right conceptually this is the same as normalizing L(G), although this is not explicitly done in Algorithm 2 shown in the paper. One more question... Is LineRank algo is applicable to undirected and weighted graph? The paper explicitly mentions that LineRank is applicable to weighted graphs. Furthermore, you can transform any undirected to a directed graph by substituting an undirected edge by two directed ones. Regarding your problems with convergence, I can give you access to my matlab code and some toy data that it converges on, so that you can test your implementation. --sebastian Thanks On Mon, Jan 20, 2014 at 2:40 PM, Sebastian Schelter s...@apache.org wrote: Jyoti, We started with a Matlab implementation on a small example graph and saw the algorithm converge. I don't think that the paper mentions that you have to normalize the matrix in a certain way. In the standard power iteration, the vector that estimates the principal eigenvector has to be rescaled to unit length. IIRC this is also done in the LineRank algorithm in the paper. --sebastian On 01/20/2014 10:04 AM, Jyoti Yadav wrote: Hi Sebastian.. I code this algorithm,but while running,it is not converging.. One more question,for power iteration.is it necessary to column normalize the matrix or we can work with row normalized matrix? Thanks Jyoti On Mon, Jan 20, 2014 at 1:45 PM, Sebastian Schelter s...@apache.org wrote: I have a student working on an implementation, do you have questions? On 01/20/2014 08:11 AM, Jyoti Yadav wrote: Hi.. Is there anyone who is working with linerank algorithm?? Thanks Jyoti
Re: About LineRank algo ..
Thanks Sebastian.. You pls send your code,I will also check where i went wrong.. On Mon, Jan 20, 2014 at 8:51 PM, Sebastian Schelter s...@apache.org wrote: On 01/20/2014 11:48 AM, Jyoti Yadav wrote: Hi Sebastian... while referring the paper,paper talks about the normalization of L(G) matrix..Below is the few lines from the paper which talks about it.. Computing Normalization Factors. The ith element of the diagonal matrix D contains the sum of ith column of L(G). D is used to column-normalize L(G) so that the resulting matrix can be used for the power iteration. The ’./’ in line 5 represents the element-wise inverse operation. Ah I see. You're right conceptually this is the same as normalizing L(G), although this is not explicitly done in Algorithm 2 shown in the paper. One more question... Is LineRank algo is applicable to undirected and weighted graph? The paper explicitly mentions that LineRank is applicable to weighted graphs. Furthermore, you can transform any undirected to a directed graph by substituting an undirected edge by two directed ones. Regarding your problems with convergence, I can give you access to my matlab code and some toy data that it converges on, so that you can test your implementation. --sebastian Thanks On Mon, Jan 20, 2014 at 2:40 PM, Sebastian Schelter s...@apache.org wrote: Jyoti, We started with a Matlab implementation on a small example graph and saw the algorithm converge. I don't think that the paper mentions that you have to normalize the matrix in a certain way. In the standard power iteration, the vector that estimates the principal eigenvector has to be rescaled to unit length. IIRC this is also done in the LineRank algorithm in the paper. --sebastian On 01/20/2014 10:04 AM, Jyoti Yadav wrote: Hi Sebastian.. I code this algorithm,but while running,it is not converging.. One more question,for power iteration.is it necessary to column normalize the matrix or we can work with row normalized matrix? Thanks Jyoti On Mon, Jan 20, 2014 at 1:45 PM, Sebastian Schelter s...@apache.org wrote: I have a student working on an implementation, do you have questions? On 01/20/2014 08:11 AM, Jyoti Yadav wrote: Hi.. Is there anyone who is working with linerank algorithm?? Thanks Jyoti
Re: About LineRank algo ..
Hi Jyoti, I'll put the files inline for simplicity. Let me know if you have anymore questions. --sebastian -- FILE: linerank.m -- load source_incidence.csv load target_incidence.csv S = spconvert(source_incidence); T = spconvert(target_incidence); n = 10; m = 19; d1 = S' * ones(m, 1); d2 = T * d1; d = 1 ./ d2; v = rand(m, 1); r = ones(m, 1) / m; diff = 1; while diff 0.1 v1 = d .* v; v2 = S' * v1; v3 = T * v2; v_next = .85 * v3 + .15 * r; diff = norm(v - v_next, 2); v = v_next; end lineranks = (S + T)' * v; lineranks -- -- FILE: source_incidence.csv -- 1 1 1 2 1 1 3 1 1 4 2 1 5 3 1 6 3 1 7 3 1 8 4 1 9 4 1 10 4 1 11 5 1 12 6 1 13 6 1 14 7 1 15 8 1 16 8 1 17 9 1 18 10 1 19 10 1 -- -- FILE: target_incidence.csv -- 1 1 1 2 3 1 3 4 1 4 2 1 5 1 1 6 3 1 7 4 1 8 3 1 9 4 1 10 7 1 11 5 1 12 2 1 13 6 1 14 7 1 15 4 1 16 8 1 17 9 1 18 4 1 19 10 1 -- On 01/20/2014 05:07 PM, Jyoti Yadav wrote: Thanks Sebastian.. You pls send your code,I will also check where i went wrong.. On Mon, Jan 20, 2014 at 8:51 PM, Sebastian Schelter s...@apache.org wrote: On 01/20/2014 11:48 AM, Jyoti Yadav wrote: Hi Sebastian... while referring the paper,paper talks about the normalization of L(G) matrix..Below is the few lines from the paper which talks about it.. Computing Normalization Factors. The ith element of the diagonal matrix D contains the sum of ith column of L(G). D is used to column-normalize L(G) so that the resulting matrix can be used for the power iteration. The ’./’ in line 5 represents the element-wise inverse operation. Ah I see. You're right conceptually this is the same as normalizing L(G), although this is not explicitly done in Algorithm 2 shown in the paper. One more question... Is LineRank algo is applicable to undirected and weighted graph? The paper explicitly mentions that LineRank is applicable to weighted graphs. Furthermore, you can transform any undirected to a directed graph by substituting an undirected edge by two directed ones. Regarding your problems with convergence, I can give you access to my matlab code and some toy data that it converges on, so that you can test your implementation. --sebastian Thanks On Mon, Jan 20, 2014 at 2:40 PM, Sebastian Schelter s...@apache.org wrote: Jyoti, We started with a Matlab implementation on a small example graph and saw the algorithm converge. I don't think that the paper mentions that you have to normalize the matrix in a certain way. In the standard power iteration, the vector that estimates the principal eigenvector has to be rescaled to unit length. IIRC this is also done in the LineRank algorithm in the paper. --sebastian On 01/20/2014 10:04 AM, Jyoti Yadav wrote: Hi Sebastian.. I code this algorithm,but while running,it is not converging.. One more question,for power iteration.is it necessary to column normalize the matrix or we can work with row normalized matrix? Thanks Jyoti On Mon, Jan 20, 2014 at 1:45 PM, Sebastian Schelter s...@apache.org wrote: I have a student working on an implementation, do you have questions? On 01/20/2014 08:11 AM, Jyoti Yadav wrote: Hi.. Is there anyone who is working with linerank algorithm?? Thanks Jyoti
