Re: [julia-users] Help needed with excessive memory allocation for Tuples

I created [1] to make specific note of this issue. Bill. [1] https://github.com/JuliaLang/julia/issues/18084

Re: [julia-users] Help needed with excessive memory allocation for Tuples

On Wednesday, 17 August 2016 20:22:56 UTC+2, Kristoffer Carlsson wrote: > > See pr 8134. I dont think you have any worse cache locality than with your > original tuple. You are right. I just have more memory usage. The heap will in fact have better cache locality. I'll just have exp

Re: [julia-users] Help needed with excessive memory allocation for Tuples

See pr 8134. I dont think you have any worse cache locality than with your original tuple.

Re: [julia-users] Help needed with excessive memory allocation for Tuples

I basically extended the heap_s struct to also include the exp field. This uses twice the memory, but gets rid of the massive number of heap allocations. Unfortunately I lose the cache efficiency of the original solution, so it's still about 20-50% slower than it should be. It'll have to do for

Re: [julia-users] Help needed with excessive memory allocation for Tuples

Just make a new type instead of the tuple and directly set the fields of it?

Re: [julia-users] Help needed with excessive memory allocation for Tuples

Wowsers! That's one hell of a serious issue. On 17 August 2016 at 18:33, Kristoffer Carlsson wrote: > Immutables and tuples that contain references to heap allocated objects > are currently themselves allocated on the heap. There is an issue about it. > >

Re: [julia-users] Help needed with excessive memory allocation for Tuples

Immutables and tuples that contain references to heap allocated objects are currently themselves allocated on the heap. There is an issue about it.

Re: [julia-users] Help needed with excessive memory allocation for Tuples

Wait, I understand why heap_s would be allocated on the heap. But why would a tuple involving a heap_s need to be allocated on the heap assuming the heap_s already exists? The tuple should just have the NTuple and one pointer (to a heap_s) at the machine level. This is clearly the problem, but I

Re: [julia-users] Help needed with excessive memory allocation for Tuples

Unless heap_s is a bitstype then the tuple will be allocated on the heap. But you are saying it is getting allocated twice?

Re: [julia-users] Help needed with excessive memory allocation for Tuples

It's very, very hard to produce working code that illustrates a problem when working on a very large system like ours. There's nearly 1000 lines of code just to set up the data! If you want to take a look, it's here, though I'm not sure how enlightening it will be:

[julia-users] Help needed with excessive memory allocation for Tuples

As always, it is very helpful if there is some code that is actually runnable in the post that contains the problem. It is easier to run Julia code in Julia than in your brain :)

Re: [julia-users] Help needed with excessive memory allocation for Tuples

N = 1 in the case I'm profiling/timing. It's typically < 10 in applications, usually much less. Bil. On 17 August 2016 at 17:50, Kristoffer Carlsson wrote: > What is N typically?

[julia-users] Help needed with excessive memory allocation for Tuples

What is N typically?