Re: [julia-users] Julia Memory Management

[Ford Ox]

>
> This is the wrong question to ask
>
I wanted to write 'good enough' but then I discarded it since its too broad 
term. It was a mistake.

I was wondering, whether there will ever be so well rounded GC, that ... 
"reference counting, manual allocation / freeing and all that c++ stuff 
will be obsolete." 
 

Dne čtvrtek 26. května 2016 17:46:00 UTC+2 Yichao Yu napsal(a):
>
> On Thu, May 26, 2016 at 11:41 AM, Ford Ox  
> wrote: 
> > I wonder there will be ever be so fast Garbage Collector, that reference 
> > counting, manual allocation / freeing and all that c++ stuff will be 
> > obsolete... 
>
> . As I mentioned already, a tracing 
> GC is usually much faster (in throughput) than any other GC techniques 
> that I know of (except not freeing memory). Even the simple GC julia 
> has outperform the best malloc implementation I can find by a large 
> margin. 
>
> The right GC techniques to use is usually a trade off between 
> throughput and latency. 
>

Re: [julia-users] Julia Memory Management

[Jeff Bezanson]

I love this topic; can't resist chiming in. There is indeed a lot of
misinformation out there about "manual memory management". I believe
all the gains (if any) to be had from manual memory management come
from (1) requesting stack allocation (it can be hard for a compiler to
prove that the lifetime of an object is limited enough for this to
work), (2) reusing space efficiently by overwriting it.

(1) mostly applies to small objects. We handle it by providing
immutable objects, which are much easier to automatically stack
allocate. We can't yet stack allocate all immutable objects, but work
has been done on it and we'll get there eventually. Also due to
stack-bounded lifetimes, these objects are likely to become young
garbage that GC is especially good at handling.

(2) is possible in any language with mutation. Overwriting data is a
form of manual memory management (you're saying you don't need the old
version of the data any more). In Julia folks get speedups from this
all the time. While I'd love it if we could get that level of
performance automatically, in the meantime this is a highly effective
form of manual memory management that's fully compatible with GC. So I
get annoyed when somebody says languages should have manual memory
management *instead of* GC (which fortunately nobody argued here). If
you're willing to use manual memory management techniques, a GC isn't
going to slow you down! And Yichao is right that for a given heap
allocation workload, GC handily beats malloc/free (since the gains
come from reusing space and avoiding heap allocation, not from
malloc/free being fast). I suspect a lot of the confusion is due to
languages with GC tending not to offer in-place algorithms and other
kinds of support for mutation, even though they could.


On Thu, May 26, 2016 at 12:13 PM, Chris Rackauckas  wrote:
> Ahh, that makes sense. Thanks for clearing that up.
>
> On Thursday, May 26, 2016 at 8:59:14 AM UTC-7, Yichao Yu wrote:
>>
>> On Thu, May 26, 2016 at 11:44 AM, Chris Rackauckas 
>> wrote:
>> > Thanks, that's good to know. Can you explain a little bit about why it
>> > would
>> > hurt performance?
>>
>> See my response on an old thread.
>>
>> https://groups.google.com/forum/?fromgroups=#!searchin/julia-users/Suspending$20Garbage$20Collection$20for$20Performance...good$20idea$20or$20bad$20idea$3F/julia-users/6_XvoLBzN60/nkB30SwmdHQJ
>>
>> In short, the key points are that
>>
>> 1. Freeing memory is not free.
>>
>> You need to do work so that you can reclaim it later, (why would
>> you free it otherwise)
>>
>> 2. Doing the work incrementally is typically more expensive than doing
>> the work in batches.
>>
>> This is where the trade off between latency and throughput comes in.
>>
>> These being said, there are of course cases that you can construct
>> with manually tweaked memory management that beats the best GC which
>> has to guess the best strategy (and finding such example should be
>> easier for a less optimized GC). However, I think currently the
>> unsafeness, the likelihood of misuse and the impact on other part of
>> the runtime doesn't really justify introducing manual memory
>> management of julia objects.
>>
>> >
>> > On Thursday, May 26, 2016 at 8:23:34 AM UTC-7, Yichao Yu wrote:
>> >>
>> >> On Thu, May 26, 2016 at 11:06 AM, Chris Rackauckas 
>> >> wrote:
>> >> > I see mentions like this one every once in awhile:
>> >> >
>> >> > "D language is a special case, as it has GC, but it's also optional
>> >> > (as
>> >> > with
>> >> > Julia)"
>> >> >
>> >> > Is GC optional?
>> >>
>> >> No, Not for julia objects.
>> >>
>> >> > I thought the only way to discard something from memory was
>> >> > to set it to zero and call garbage control (which then runs the whole
>> >> > garbage control). Is there a more targeted way to delete things?
>> >>
>> >> No, Not for julia objects.
>> >>
>> >> > If it's not
>> >> > already available, it seems like it would be useful for code focusing
>> >> > performance as an option.
>> >>
>> >> This will actually **not** improve performance most of the time and
>> >> can actually hurt performance a lot.
>> >> The only case I can think of that can have better performance is
>> >> manually managing an object pool.
>> >> This might improve latency or memory usage. All of the advantage can
>> >> be obtained by improving the GC itself, which is the preferred way.
>> >> This feature will also be extremely unsafe and can break many
>> >> assumptions made in the runtime.

Re: [julia-users] Julia Memory Management

[Chris Rackauckas]

Ahh, that makes sense. Thanks for clearing that up. 

On Thursday, May 26, 2016 at 8:59:14 AM UTC-7, Yichao Yu wrote:
>
> On Thu, May 26, 2016 at 11:44 AM, Chris Rackauckas  > wrote: 
> > Thanks, that's good to know. Can you explain a little bit about why it 
> would 
> > hurt performance? 
>
> See my response on an old thread. 
>
> https://groups.google.com/forum/?fromgroups=#!searchin/julia-users/Suspending$20Garbage$20Collection$20for$20Performance...good$20idea$20or$20bad$20idea$3F/julia-users/6_XvoLBzN60/nkB30SwmdHQJ
>  
>
> In short, the key points are that 
>
> 1. Freeing memory is not free. 
>
> You need to do work so that you can reclaim it later, (why would 
> you free it otherwise) 
>
> 2. Doing the work incrementally is typically more expensive than doing 
> the work in batches. 
>
> This is where the trade off between latency and throughput comes in. 
>
> These being said, there are of course cases that you can construct 
> with manually tweaked memory management that beats the best GC which 
> has to guess the best strategy (and finding such example should be 
> easier for a less optimized GC). However, I think currently the 
> unsafeness, the likelihood of misuse and the impact on other part of 
> the runtime doesn't really justify introducing manual memory 
> management of julia objects. 
>
> > 
> > On Thursday, May 26, 2016 at 8:23:34 AM UTC-7, Yichao Yu wrote: 
> >> 
> >> On Thu, May 26, 2016 at 11:06 AM, Chris Rackauckas  
> >> wrote: 
> >> > I see mentions like this one every once in awhile: 
> >> > 
> >> > "D language is a special case, as it has GC, but it's also optional 
> (as 
> >> > with 
> >> > Julia)" 
> >> > 
> >> > Is GC optional? 
> >> 
> >> No, Not for julia objects. 
> >> 
> >> > I thought the only way to discard something from memory was 
> >> > to set it to zero and call garbage control (which then runs the whole 
> >> > garbage control). Is there a more targeted way to delete things? 
> >> 
> >> No, Not for julia objects. 
> >> 
> >> > If it's not 
> >> > already available, it seems like it would be useful for code focusing 
> >> > performance as an option. 
> >> 
> >> This will actually **not** improve performance most of the time and 
> >> can actually hurt performance a lot. 
> >> The only case I can think of that can have better performance is 
> >> manually managing an object pool. 
> >> This might improve latency or memory usage. All of the advantage can 
> >> be obtained by improving the GC itself, which is the preferred way. 
> >> This feature will also be extremely unsafe and can break many 
> >> assumptions made in the runtime. 
>

Re: [julia-users] Julia Memory Management

[Yichao Yu]

On Thu, May 26, 2016 at 11:44 AM, Chris Rackauckas  wrote:
> Thanks, that's good to know. Can you explain a little bit about why it would
> hurt performance?

See my response on an old thread.
https://groups.google.com/forum/?fromgroups=#!searchin/julia-users/Suspending$20Garbage$20Collection$20for$20Performance...good$20idea$20or$20bad$20idea$3F/julia-users/6_XvoLBzN60/nkB30SwmdHQJ

In short, the key points are that

1. Freeing memory is not free.

You need to do work so that you can reclaim it later, (why would
you free it otherwise)

2. Doing the work incrementally is typically more expensive than doing
the work in batches.

This is where the trade off between latency and throughput comes in.

These being said, there are of course cases that you can construct
with manually tweaked memory management that beats the best GC which
has to guess the best strategy (and finding such example should be
easier for a less optimized GC). However, I think currently the
unsafeness, the likelihood of misuse and the impact on other part of
the runtime doesn't really justify introducing manual memory
management of julia objects.

>
> On Thursday, May 26, 2016 at 8:23:34 AM UTC-7, Yichao Yu wrote:
>>
>> On Thu, May 26, 2016 at 11:06 AM, Chris Rackauckas 
>> wrote:
>> > I see mentions like this one every once in awhile:
>> >
>> > "D language is a special case, as it has GC, but it's also optional (as
>> > with
>> > Julia)"
>> >
>> > Is GC optional?
>>
>> No, Not for julia objects.
>>
>> > I thought the only way to discard something from memory was
>> > to set it to zero and call garbage control (which then runs the whole
>> > garbage control). Is there a more targeted way to delete things?
>>
>> No, Not for julia objects.
>>
>> > If it's not
>> > already available, it seems like it would be useful for code focusing
>> > performance as an option.
>>
>> This will actually **not** improve performance most of the time and
>> can actually hurt performance a lot.
>> The only case I can think of that can have better performance is
>> manually managing an object pool.
>> This might improve latency or memory usage. All of the advantage can
>> be obtained by improving the GC itself, which is the preferred way.
>> This feature will also be extremely unsafe and can break many
>> assumptions made in the runtime.

Re: [julia-users] Julia Memory Management

[Ford Ox]

I wonder there will be ever be so fast Garbage Collector, that reference 
counting, manual allocation / freeing and all that c++ stuff will be 
obsolete...

Re: [julia-users] Julia Memory Management

[Yichao Yu]

On Thu, May 26, 2016 at 11:41 AM, Ford Ox  wrote:
> I wonder there will be ever be so fast Garbage Collector, that reference
> counting, manual allocation / freeing and all that c++ stuff will be
> obsolete...

This is the wrong question to ask. As I mentioned already, a tracing
GC is usually much faster (in throughput) than any other GC techniques
that I know of (except not freeing memory). Even the simple GC julia
has outperform the best malloc implementation I can find by a large
margin.

The right GC techniques to use is usually a trade off between
throughput and latency.

Re: [julia-users] Julia Memory Management

[Chris Rackauckas]

Thanks, that's good to know. Can you explain a little bit about why it 
would hurt performance? 

On Thursday, May 26, 2016 at 8:23:34 AM UTC-7, Yichao Yu wrote:
>
> On Thu, May 26, 2016 at 11:06 AM, Chris Rackauckas  > wrote: 
> > I see mentions like this one every once in awhile: 
> > 
> > "D language is a special case, as it has GC, but it's also optional (as 
> with 
> > Julia)" 
> > 
> > Is GC optional? 
>
> No, Not for julia objects. 
>
> > I thought the only way to discard something from memory was 
> > to set it to zero and call garbage control (which then runs the whole 
> > garbage control). Is there a more targeted way to delete things? 
>
> No, Not for julia objects. 
>
> > If it's not 
> > already available, it seems like it would be useful for code focusing 
> > performance as an option. 
>
> This will actually **not** improve performance most of the time and 
> can actually hurt performance a lot. 
> The only case I can think of that can have better performance is 
> manually managing an object pool. 
> This might improve latency or memory usage. All of the advantage can 
> be obtained by improving the GC itself, which is the preferred way. 
> This feature will also be extremely unsafe and can break many 
> assumptions made in the runtime. 
>

Re: [julia-users] Julia Memory Management

[Yichao Yu]

On Thu, May 26, 2016 at 11:06 AM, Chris Rackauckas  wrote:
> I see mentions like this one every once in awhile:
>
> "D language is a special case, as it has GC, but it's also optional (as with
> Julia)"
>
> Is GC optional?

No, Not for julia objects.

> I thought the only way to discard something from memory was
> to set it to zero and call garbage control (which then runs the whole
> garbage control). Is there a more targeted way to delete things?

No, Not for julia objects.

> If it's not
> already available, it seems like it would be useful for code focusing
> performance as an option.

This will actually **not** improve performance most of the time and
can actually hurt performance a lot.
The only case I can think of that can have better performance is
manually managing an object pool.
This might improve latency or memory usage. All of the advantage can
be obtained by improving the GC itself, which is the preferred way.
This feature will also be extremely unsafe and can break many
assumptions made in the runtime.