[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-384427810 Anybody here? This is an automated message from the Apache Git Ser

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-383099913 I removed .asscalar() call from for loop, because it takes too much time for execution (again question about performance!) and agai

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-383069867 The same situation with memory consumption persists when I'm adding (using +=) NDArray from 1 element in a loop and after loop call

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-383068084 So, when I replaced algorithm for evaluation accuracy for my own, memory consumption became stable. WAS: def evaluate_acc

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-383046496 Also the topic starting problem still persists: File "SOM.py", line 429, in File "SOM.py", line 328, in evaluate_accuracyML

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-382822803 So I think I was wrong and 'weights' do not have actual data if not wait_to_read() called. Because after testing my code with wait_

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-382801189 So as I concluded I can call wait_to_read() not each time I'm updating weights.set_data(weights.data() + output), but for example w

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-382778869 Addition: I've measured time again with .wait_to_read() and calculaton time was 330 seconds but copying time became 2400 seconds!

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-382761026 Ok, for the first question and topic question I have the answer: Need to use weightsData.wait_to_read() after weights.set_data(we

[GitHub] KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory

KaiserSozo commented on issue #10532: NDArray failed to allocate CPU memory URL: https://github.com/apache/incubator-mxnet/issues/10532#issuecomment-382621382 As I understand now memory consumes here: weights.set_data(weights.data() + output). Where weights is the Parameter with grad_req {