[julia-users] Re: select! with range creating a memory allocation
I checked it a bit more carefully, the reason for the high memory usage is that I create a new instance of an anonymous function every time I call select! I will try work around it. On Monday, February 2, 2015 at 5:03:12 PM UTC+1, Kristoffer Carlsson wrote: > > I did some benchmarkling and I noticed that select! uses the majority of > memory(27 mb out of ~30) when I create a kd tree. I tried to remove the > return value from the select! function but it didn't make a big difference. > > The way I use select! is like this: > > select!(perm, mid_idx, by = i -> data[split_dim, i]) > > Is it because I am using an anonymous function that I get heavy memory > usage? > > > > > > On Sunday, February 1, 2015 at 11:53:02 AM UTC+1, julia.e...@gmail.com > wrote: >> >> I realize this is a more developer issue, but I'm wondering if anyone >> else thinks it would be a good idea for select! to be changed to not return >> A[r] (which allocates memory). >> >> >> I think it could return (A[r.start], A[r.stop]), sub(A, r) or even >> nothing. As it stands it seems like a performance penalty that can be >> avoided since the rest of the function should operate on memory in place. >> >> >> In my own project I've made a local copy of select for this reason and >> taken out the return value. >> >
[julia-users] Re: select! with range creating a memory allocation
I did some benchmarkling and I noticed that select! uses the majority of memory(27 mb out of ~30) when I create a kd tree. I tried to remove the return value from the select! function but it didn't make a big difference. The way I use select! is like this: select!(perm, mid_idx, by = i -> data[split_dim, i]) Is it because I am using an anonymous function that I get heavy memory usage? On Sunday, February 1, 2015 at 11:53:02 AM UTC+1, julia.e...@gmail.com wrote: > > I realize this is a more developer issue, but I'm wondering if anyone else > thinks it would be a good idea for select! to be changed to not return A[r] > (which allocates memory). > > > I think it could return (A[r.start], A[r.stop]), sub(A, r) or even > nothing. As it stands it seems like a performance penalty that can be > avoided since the rest of the function should operate on memory in place. > > > In my own project I've made a local copy of select for this reason and > taken out the return value. >
[julia-users] Re: select! with range creating a memory allocation
Yes, if you are on windows, the nightly installer is the easiest way. Be sure to keep the old downloads, and remember what versions you use so that you can revert if a new change causes trouble with your setup. (How careful you must be, depends on how acceptable it is for you to have a broken Julia install until you adapt your code to a breaking change)
[julia-users] Re: select! with range creating a memory allocation
Thanks for the reply. To start using 0.4 will I just have to gamble on a nightly windows installer? On Sunday, February 1, 2015 at 1:11:40 PM UTC-6, Ivar Nesje wrote: > > This issue will fix itself when `a[r]` returns a subarray with 0.4 (this > hasn't landed in master yet, and reminds me that the pending change will > require a huge documentation effort that change affect APIs). > > The improved garbage collection that was merged in master recently will > likely greatly reduce the performance penalty for such short lived > allocations. > > Ivar > > søndag 1. februar 2015 11.53.02 UTC+1 skrev julia.e...@gmail.com følgende: >> >> I realize this is a more developer issue, but I'm wondering if anyone >> else thinks it would be a good idea for select! to be changed to not return >> A[r] (which allocates memory). >> >> >> I think it could return (A[r.start], A[r.stop]), sub(A, r) or even >> nothing. As it stands it seems like a performance penalty that can be >> avoided since the rest of the function should operate on memory in place. >> >> >> In my own project I've made a local copy of select for this reason and >> taken out the return value. >> >
[julia-users] Re: select! with range creating a memory allocation
This issue will fix itself when `a[r]` returns a subarray with 0.4 (this hasn't landed in master yet, and reminds me that the pending change will require a huge documentation effort that change affect APIs). The improved garbage collection that was merged in master recently will likely greatly reduce the performance penalty for such short lived allocations. Ivar søndag 1. februar 2015 11.53.02 UTC+1 skrev julia.e...@gmail.com følgende: > > I realize this is a more developer issue, but I'm wondering if anyone else > thinks it would be a good idea for select! to be changed to not return A[r] > (which allocates memory). > > > I think it could return (A[r.start], A[r.stop]), sub(A, r) or even > nothing. As it stands it seems like a performance penalty that can be > avoided since the rest of the function should operate on memory in place. > > > In my own project I've made a local copy of select for this reason and > taken out the return value. >
