[GitHub] coldsheephot commented on issue #11299: row_sparse_pull, push row_sparse gradient is too slow, it has 10+ times difference
coldsheephot commented on issue #11299: row_sparse_pull,push row_sparse gradient is too slow,it has 10+ times difference URL: https://github.com/apache/incubator-mxnet/issues/11299#issuecomment-398595855 How long does it take to solve those problems？？？I am very anxious This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coldsheephot commented on issue #11299: row_sparse_pull, push row_sparse gradient is too slow, it has 10+ times difference
coldsheephot commented on issue #11299: row_sparse_pull,push row_sparse gradient is too slow,it has 10+ times difference URL: https://github.com/apache/incubator-mxnet/issues/11299#issuecomment-398591011 @eric-haibin-lin yes. I want to use the feature for multi-device and multi-machine case. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coldsheephot commented on issue #11299: row_sparse_pull, push row_sparse gradient is too slow, it has 10+ times difference
coldsheephot commented on issue #11299: row_sparse_pull,push row_sparse gradient is too slow,it has 10+ times difference URL: https://github.com/apache/incubator-mxnet/issues/11299#issuecomment-398237645 In distributed training, may I get the real indices which have values? can you add the method @eric-haibin-lin @kalyc @sandeep-krishnamurthy This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coldsheephot commented on issue #11299: row_sparse_pull, push row_sparse gradient is too slow, it has 10+ times difference
coldsheephot commented on issue #11299: row_sparse_pull,push row_sparse gradient is too slow,it has 10+ times difference URL: https://github.com/apache/incubator-mxnet/issues/11299#issuecomment-397530492 @eric-haibin-lin when I push the sparse gradient, after merge,update ,I can get the weight; but I can not get the which row has values and which row doesn't has values. how can I know the row_ids in single machine,I can know the indices from the sparse gradient,but in distributed training,different worker has different indices,when I pull from the ps , I also can not know the final row_ids This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coldsheephot commented on issue #11299: row_sparse_pull, push row_sparse gradient is too slow, it has 10+ times difference
coldsheephot commented on issue #11299: row_sparse_pull,push row_sparse gradient is too slow,it has 10+ times difference URL: https://github.com/apache/incubator-mxnet/issues/11299#issuecomment-397530492 @eric-haibin-lin when I push the sparse gradient, after merge,update ,I can get the weight;but I can not get the which row has values and which row doesn't has values. how can I know the row_ids This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coldsheephot commented on issue #11299: row_sparse_pull, push row_sparse gradient is too slow, it has 10+ times difference
coldsheephot commented on issue #11299: row_sparse_pull,push row_sparse gradient is too slow,it has 10+ times difference URL: https://github.com/apache/incubator-mxnet/issues/11299#issuecomment-397504089 and there has another problem: when I `delete kv.push(i,a)` in test_row_sparse_pull() function,it needs more time than use push,it is very strange！！ row_sparse push and pull time: 8.14685201645 def test_row_sparse_pull(): out = a for i in range(100): # kv.push(i, a) kv.row_sparse_pull(i, out=out, priority=i, row_ids=all_row_ids) # out.wait_to_read() # mx.base._LIB.MXNDArrayWaitToWrite(a.handle) mx.nd.waitall() This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] coldsheephot commented on issue #11299: row_sparse_pull, push row_sparse gradient is too slow, it has 10+ times difference
coldsheephot commented on issue #11299: row_sparse_pull,push row_sparse gradient is too slow,it has 10+ times difference URL: https://github.com/apache/incubator-mxnet/issues/11299#issuecomment-397504089 and there has another problem: when I `delete kv.push(i,a)` is test_row_sparse_pull() function,it needs more time than use push,it is very strange！！ row_sparse push and pull time: 8.14685201645 def test_row_sparse_pull(): out = a for i in range(100): # kv.push(i, a) kv.row_sparse_pull(i, out=out, priority=i, row_ids=all_row_ids) # out.wait_to_read() # mx.base._LIB.MXNDArrayWaitToWrite(a.handle) mx.nd.waitall() This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
