Re: Eliminate class allocators and deallocators?

[Robert M. Münch]

Picking up this old topic & state.

What's the solution at the moment? I'm a bit lost. Is there now a way 
to use / make custom alloators with D2 or not?


If yes, how to do it?


On 2009-10-07 21:16:37 +0200, Andrei Alexandrescu said:


So for placement construction of a class, I guess it would look something like:

auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?


Yes, I think so, but I haven't checked all the details. For example I'm 
not sure whether __ctor copies .init over the memory before running the 
user-defined constructor, or expects that to have been done already.


My understanding from Walter is that __ctor(x, y, z) are simply the 
functions this(x, y, z) as written by the user, so you'd need to memcpy 
the .init by hand before calling __ctor.


Aw hell I got curious so let me check.

class MyClass {
 int x = 42;
 this() {}
}

void main() {
 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
 x.__ctor();
 writeln(x.x);
 writeln(x.toString);
}

That prints 0 and then crashes on my machine. Looks like you need to 
memcpy the .init before calling __ctor.


I'm very glad we're starting to look into this. There are very nice 
opportunities for adding custom allocation support in the stdlib.



--
Robert M. Münch
http://www.robertmuench.de

Re: Eliminate class allocators and deallocators?

[Yigal Chripun]

On 09/10/2009 04:54, Michel Fortin wrote:

On 2009-10-08 10:35:01 -0400, Andrei Alexandrescu
 said:


I think you'd find this article interesting:

http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/



That's



quite interesting inded. At the end, the author would like a
Smalltalk-like approach, but believe it's not really possible in a
static language.

But that's exactly what we can have in D by remaking 'new' as a function
template. :-)

Just as the author wants, with a template 'new' function it seems quite
possible to change 'new' into a factory function instanciating the best
class for the given arguments:

void newGC(T, A...)(A args); // create a garbage-collected instance

String new(T: String)(immutable(char)[] utf8Str) {
return newGC!UTF8ImmutableString(utf8str);
}

String new(T: String)(immutable(ubyte)[] strData, int encoding) {
// instanciate the best string type depending on encoding.
if (encoding == UTF8)
return newGC!UTF8ImmutableString(cast(string)strData); else if (encoding
== ISO_LATIN_1)
return newGC!OneBytePerCharImmutableString(cast(string)strData, encoding);
else
...
}




I like the ruby style syntax option.
I'll also would like the option to encapsulate all the machinery in an 
allocator hierarchy conveniently provided in the stdlib.


my templating skills are weak but is something like this could be possible?

class Foo {
   this(args) {};

   static Foo new(Allocator, ARGS...) (Allocator al, ARGS args) {
  alloc = al;
  return alloc.allocate!Foo(args); // alloc will also call the ctor
   }

   static Foo new(ARGS... args) {
  return GC.allocate!Foo(args);
   }
   Allocator alloc;
}

how does this affect ctors? they could become regular functions.
how does this work with abstract classes?

Re: Eliminate class allocators and deallocators?

[Kagamin]

Andrei Alexandrescu Wrote:

> My point is, languages are never modular. To be even marginally
> effective in a language, you must have some understanding of it all.
> That definitely isn't the case for libraries.

XHTML is modular. And various C++ style recomendations show that you can use 
quite a subset of a language. So as I little know C++ templates, I'm not even 
margianlly effective in boost, but I can peek into scintilla.

Re: Eliminate class allocators and deallocators?

[Michel Fortin]

On 2009-10-08 10:35:01 -0400, Andrei Alexandrescu 
 said:



I think you'd find this article interesting:

http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/


That's 


quite interesting inded. At the end, the author would like a 
Smalltalk-like approach, but believe it's not really possible in a 
static language.


But that's exactly what we can have in D by remaking 'new' as a 
function template. :-)


Just as the author wants, with a template 'new' function it seems quite 
possible to change 'new' into a factory function instanciating the best 
class for the given arguments:


void newGC(T, A...)(A args); // create a garbage-collected instance

String new(T: String)(immutable(char)[] utf8Str) {
return newGC!UTF8ImmutableString(utf8str);
}

String new(T: String)(immutable(ubyte)[] strData, int encoding) {
// instanciate the best string type depending on encoding.
if (encoding == UTF8)
			return newGC!UTF8ImmutableString(cast(string)strData);		else if 
(encoding == ISO_LATIN_1)

return 
newGC!OneBytePerCharImmutableString(cast(string)strData, encoding);
else
...
}


--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: Eliminate class allocators and deallocators?

[Craig Black]

Jeremie Pelletier Wrote:

> Andrei Alexandrescu wrote:
> > Jeremie Pelletier wrote:
> >> Andrei Alexandrescu wrote:
> >>> Jeremie Pelletier wrote:
>  Yeah I agree now after reading most of this thread, I know that 
>  these keywords just map to functions.
> 
>  I've seen a proposal of a global new template somewhere, I don't 
>  like that since at the global scope there are also structs, arrays 
>  and whatnot that can be allocated by 'new'.
> >>>
> >>> Well it's easy to handle all of those with conditional templates.
> >>>
>  I don't like the static new either since it prevents subclasses from 
>  overriding their new/delete operations.
> 
>  What would then be a good way to replace new/delete operators to 
>  still have them overridable? Isn't that the convenience that first 
>  got them to be used in the first place? Other than global new/delete 
>  overrides which is plain silly in D.
> 
>  I've pretty much found alternatives to all my other points against 
>  taking out new/delete except for the override feature, find me an 
>  alternative for that too and I'll be voting for new/delete to be 
>  runtime function instead of language keywords, cause I can't think 
>  of anything right now.
> >>>
> >>> I think you'd find this article interesting:
> >>>
> >>> http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/
> >>>  
> >>>
> >>>
> >>>
> >>> Andrei
> >>
> >> That was a long read, but a most interesting one! I already was 
> >> familiar of these design pattens but only used them where new didn't 
> >> make sense, this article really was an eye opener on that level, 
> >> thanks a lot!
> >>
> >> You have my vote to remove new/delete now :)
> > 
> > Someone convinced someone else of something on the Internets. What's 
> > this world coming to???
> > 
> > Andrei
> 
> World peace, open-minded societies and money-free economies where love 
> and sharing has won over fear and competition.
> 
> Maybe not, but I can dream.

Perhaps not in our lifetime, but eventually. What you envision is inevitable. 
Kudos for not being emotionally attached to your view point.  You are ahead of 
your time.

-Craig

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

Andrei Alexandrescu wrote:

Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Jeremie Pelletier wrote:
Yeah I agree now after reading most of this thread, I know that 
these keywords just map to functions.


I've seen a proposal of a global new template somewhere, I don't 
like that since at the global scope there are also structs, arrays 
and whatnot that can be allocated by 'new'.


Well it's easy to handle all of those with conditional templates.

I don't like the static new either since it prevents subclasses from 
overriding their new/delete operations.


What would then be a good way to replace new/delete operators to 
still have them overridable? Isn't that the convenience that first 
got them to be used in the first place? Other than global new/delete 
overrides which is plain silly in D.


I've pretty much found alternatives to all my other points against 
taking out new/delete except for the override feature, find me an 
alternative for that too and I'll be voting for new/delete to be 
runtime function instead of language keywords, cause I can't think 
of anything right now.


I think you'd find this article interesting:

http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/ 




Andrei


That was a long read, but a most interesting one! I already was 
familiar of these design pattens but only used them where new didn't 
make sense, this article really was an eye opener on that level, 
thanks a lot!


You have my vote to remove new/delete now :)


Someone convinced someone else of something on the Internets. What's 
this world coming to???


Andrei


World peace, open-minded societies and money-free economies where love 
and sharing has won over fear and competition.


Maybe not, but I can dream.

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Jeremie Pelletier wrote:
Yeah I agree now after reading most of this thread, I know that these 
keywords just map to functions.


I've seen a proposal of a global new template somewhere, I don't like 
that since at the global scope there are also structs, arrays and 
whatnot that can be allocated by 'new'.


Well it's easy to handle all of those with conditional templates.

I don't like the static new either since it prevents subclasses from 
overriding their new/delete operations.


What would then be a good way to replace new/delete operators to 
still have them overridable? Isn't that the convenience that first 
got them to be used in the first place? Other than global new/delete 
overrides which is plain silly in D.


I've pretty much found alternatives to all my other points against 
taking out new/delete except for the override feature, find me an 
alternative for that too and I'll be voting for new/delete to be 
runtime function instead of language keywords, cause I can't think of 
anything right now.


I think you'd find this article interesting:

http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/ 




Andrei


That was a long read, but a most interesting one! I already was familiar 
of these design pattens but only used them where new didn't make sense, 
this article really was an eye opener on that level, thanks a lot!


You have my vote to remove new/delete now :)


Someone convinced someone else of something on the Internets. What's 
this world coming to???


Andrei

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  7 de octubre a las 16:03 me escribiste:
> Leandro Lucarella wrote:
> >Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:
> >>You seem to be asserting that without additional built-in language
> >>support, manual memory management is unduly difficult. Why so?
> >
> >Because of this:
> >
> >auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
> >x.__ctor( a, b, c ); // construct
> >...
> >x.__dtor();
> >free( cast(void*) x );
> >
> >:)
> >
> >You even forgot to register your object as a root in the GC, so if your
> >MyClass has any pointers to the GC your program will blow in your face.
> >
> >If you plan to library support to ease this and avoid repetitive and
> >bug-prone work, you can ignore my complains...
> 
> I too think it would be great to add the necessary support to the
> stdlib. In fact, since you have a great deal of expertise in the matter,
> feel free to suggest API functions! They'd need to be approved by Sean
> too because probably they belong to druntime.

I think the only API change should be adding a function to call the
destructors but not GC.free() (as David suggested). Then, the other
changes are only moving operators to library code. So nothing changes much
there.

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
No existe nada más intenso que un reloj, ni nada más flaco que una
bicicleta. No intenso como el café, ni flaco como escopeta.
-- Ricardo Vaporeso

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

Andrei Alexandrescu wrote:

Jeremie Pelletier wrote:
Yeah I agree now after reading most of this thread, I know that these 
keywords just map to functions.


I've seen a proposal of a global new template somewhere, I don't like 
that since at the global scope there are also structs, arrays and 
whatnot that can be allocated by 'new'.


Well it's easy to handle all of those with conditional templates.

I don't like the static new either since it prevents subclasses from 
overriding their new/delete operations.


What would then be a good way to replace new/delete operators to still 
have them overridable? Isn't that the convenience that first got them 
to be used in the first place? Other than global new/delete overrides 
which is plain silly in D.


I've pretty much found alternatives to all my other points against 
taking out new/delete except for the override feature, find me an 
alternative for that too and I'll be voting for new/delete to be 
runtime function instead of language keywords, cause I can't think of 
anything right now.


I think you'd find this article interesting:

http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/ 




Andrei


That was a long read, but a most interesting one! I already was familiar 
of these design pattens but only used them where new didn't make sense, 
this article really was an eye opener on that level, thanks a lot!


You have my vote to remove new/delete now :)

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

Adam D. Ruppe wrote:

On Thu, Oct 08, 2009 at 04:00:03AM -0500, Chris Nicholson-Sauls wrote:
One consideration is that new(), perhaps, ought not be a static member of 
its class at all, but rather a global written along similar lines to tools 
such as "to".


Agreed. One benefit here is we can convert old code to it just by find/
replacing new -> new!. We can provide easy freestanding functions
for manual management too.


sed is even better in this case :)


new! - gc
manualNew! - malloc() wrapper

Maybe even convenience structs too:

RAII! - a struct that uses the malloc() wrapper


Well if new is a template, its dead easy to use static ifs to detect if 
you're allocating an array, a struct or an object and initialize the 
memory appropriately.


The only thing that bugs me is that it makes it very hard to implement 
overridable new/delete methods for classes, if not impossible.

Re: Eliminate class allocators and deallocators?

[Nick Sabalausky]

Andrei Alexandrescu wrote:

> To be even marginally effective in a language, you must have some
> understanding of it all. That definitely isn't the case for libraries.

I disagree because assuming a point of view, that is high (or low) enough, 
all programming languages only have a very small indispensable core.

This is because turing-completenes suffices to show the effectiveness of a 
programming language. Only the elements of the language needed for that 
proof build the indispensable core of the language. Every other elements 
can at least be classified into
- helpers for humans to write and read, or
- hints for the target machines on how to perform efficiently, or
- (syntactic) sugar for (informal) proofs of correctness 
and therefore can be put into libraries.

-manfred

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Jeremie Pelletier wrote:
Yeah I agree now after reading most of this thread, I know that these 
keywords just map to functions.


I've seen a proposal of a global new template somewhere, I don't like 
that since at the global scope there are also structs, arrays and 
whatnot that can be allocated by 'new'.


Well it's easy to handle all of those with conditional templates.

I don't like the static new either since it prevents subclasses from 
overriding their new/delete operations.


What would then be a good way to replace new/delete operators to still 
have them overridable? Isn't that the convenience that first got them to 
be used in the first place? Other than global new/delete overrides which 
is plain silly in D.


I've pretty much found alternatives to all my other points against 
taking out new/delete except for the override feature, find me an 
alternative for that too and I'll be voting for new/delete to be runtime 
function instead of language keywords, cause I can't think of anything 
right now.


I think you'd find this article interesting:

http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/


Andrei

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

Don wrote:

Andrei Alexandrescu wrote:

Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They 
have a number of problems that make them unsuitable for D 2.0. The 
most obvious issue is that D 2.0 will _not_ conflate destruction 
with deallocation anymore: invoking delete against an object will 
call ~this() against it but will not recycle its memory. In 
contrast, class deallocators are designed around the idea that 
invoking delete calls the destructor and also deallocates memory.


So I'm thinking of removing at least class deallocators from the 
language. Class allocators may be marginally and occasionally useful 
if the user takes the matter of deallocation in her own hands.


A much better way to handle custom allocation of classes would be in 
the standard library.


What do you think?


Andrei


I wouldn't like delete to go away at all, I use it for all my non-gc 
objects like this watered down example:


class ManualObject : Object {
new(size_t size) { return malloc(size); }
delete(void* mem) { free(mem); }
}

And then I can easily subclass it for any objects that doesn't need 
the GC. I've got similar constructs for arrays and structs.


Clearly you use those objects in a very different manner than GC 
objects. So by using new/delete with them you're fooling yourself.


// untested
class ManualObject {
static T create(T : ManualObject)() {
auto p = malloc(__traits(classInstanceSize, T));
memcpy(p, T.classinfo.init.ptr, __traits(classInstanceSize, T));
auto result = cast(T) p;
result.__ctor();
return result;
}
static void yank(ManualObject obj) {
free(cast(void*) obj);
}
}

Looks like a fair amount of work? At some level it actually should, 
but we can put that kind of stuff in the standard library.


malloc/free are nice, but they don't allow for elegant abstractions 
like new/delete does (for example if you want to use a specialized 
non-gc allocator you can just replace a few calls instead of every 
allocation).


They do if you're willing to write just a bit of scaffolding.

I also use delete when I no longer need large blocks of memory, I 
don't want them to just become uninitialized and sitting on the GC. 
When I want to do that I just nullify my references.


If you're afraid of deleting an object that may still have valid 
references, use smart pointers, or don't delete it at all if it sits 
on the gc and just call a .destroy() method.


Also in my runtime the delete implementations do free the memory, 
they don't just call the finalizer.


In any ways, just don't remove new/delete overrides from the language 
please, just call it a low-level technique or something to scare the 
beginners away and let people who want it have it :)


I strongly believe custom new/delete must go.


Andrei


Yes. The only reason you want them in C++ is because C++ makes 
constructors magical, by always glueing a memory allocation in front of 
them, and pretending they're not a function.
Then you need to introduce placement new to avoid the memory allocation 
bit.


Let's call a spade a spade: a constructor is just a function that 
establishes the invariant on a piece of memory which it recieves as a 
parameter. If you stop the pretense, you don't need the language machinery.




Yeah I agree now after reading most of this thread, I know that these 
keywords just map to functions.


I've seen a proposal of a global new template somewhere, I don't like 
that since at the global scope there are also structs, arrays and 
whatnot that can be allocated by 'new'.


I don't like the static new either since it prevents subclasses from 
overriding their new/delete operations.


What would then be a good way to replace new/delete operators to still 
have them overridable? Isn't that the convenience that first got them to 
be used in the first place? Other than global new/delete overrides which 
is plain silly in D.


I've pretty much found alternatives to all my other points against 
taking out new/delete except for the override feature, find me an 
alternative for that too and I'll be voting for new/delete to be runtime 
function instead of language keywords, cause I can't think of anything 
right now.

Re: Eliminate class allocators and deallocators?

[Adam D. Ruppe]

On Thu, Oct 08, 2009 at 04:00:03AM -0500, Chris Nicholson-Sauls wrote:
> One consideration is that new(), perhaps, ought not be a static member of 
> its class at all, but rather a global written along similar lines to tools 
> such as "to".

Agreed. One benefit here is we can convert old code to it just by find/
replacing new -> new!. We can provide easy freestanding functions
for manual management too.

new! - gc
manualNew! - malloc() wrapper

Maybe even convenience structs too:

RAII! - a struct that uses the malloc() wrapper


-- 
Adam D. Ruppe
http://arsdnet.net

Re: Eliminate class allocators and deallocators?

[Max Samukha]

On Thu, 08 Oct 2009 16:31:00 +0400, "Denis Koroskin"
<2kor...@gmail.com> wrote:
>
>But you are right, casting void* to Object does a reinterpret cast instead
>of dynamic cast. I'm not sure if that's a good design decision, though.
>

It is needed in situations where the cast is known to be safe. There
is also the undocumented _d_toObject(void*). What does it do exactly?

Re: Eliminate class allocators and deallocators?

[Don]

Steven Schveighoffer wrote:
On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2kor...@gmail.com> 
wrote:


On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer 
 wrote:


On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin 
<2kor...@gmail.com> wrote:


On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly 
 wrote:



auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);


I would expect a dynamic cast to occur at this line. Which will 
either result in an access violation (since you are trying to cast a 
garbage to an object) or result in a null being returned.


malloc returns void *, so no dynamic cast.

-Steve


I know malloc returns void*. I didn't know you can hijack type system 
that easily.


But then, if no dynamic cast takes place why cast(Object)cast(void*)0 
cannot be evaluated at compile time?


Your message made me test it :)

import std.stdio;

void *foo()
{
return cast(void*)0;
}

void main()
{
auto o = cast(Object)foo();
writefln("here!");
o.opEquals(o);
}

outputs:

here!
Segmentation fault

So, no dynamic cast (dynamic cast would have looked at the classinfo of 
null, segfaulting before the output).


So I would say, the fact that compile time evaluation doesn't work is a 
bug maybe?


-Steve


CTFE is not yet supported for classes.

Re: Eliminate class allocators and deallocators?

[Steven Schveighoffer]

On Thu, 08 Oct 2009 08:31:00 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Thu, 08 Oct 2009 15:48:56 +0400, Steven Schveighoffer  
 wrote:


On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
 wrote:


On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin  
<2kor...@gmail.com> wrote:


On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly  
 wrote:



auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);


I would expect a dynamic cast to occur at this line. Which will  
either result in an access violation (since you are trying to cast a  
garbage to an object) or result in a null being returned.


malloc returns void *, so no dynamic cast.

-Steve


I know malloc returns void*. I didn't know you can hijack type system  
that easily.


But then, if no dynamic cast takes place why cast(Object)cast(void*)0  
cannot be evaluated at compile time?


Your message made me test it :)

import std.stdio;

void *foo()
{
 return cast(void*)0;
}

void main()
{
 auto o = cast(Object)foo();
 writefln("here!");
 o.opEquals(o);
}

outputs:

here!
Segmentation fault

So, no dynamic cast (dynamic cast would have looked at the classinfo of  
null, segfaulting before the output).




No, IIRC, casting null to Object is perfectly valid and returns null.


Oh yeah :)  I forgot.  Should have returned 1 instead of 0.

I did also verify via obj2asm that dynamic cast was not being called :)



But you are right, casting void* to Object does a reinterpret cast  
instead

of dynamic cast. I'm not sure if that's a good design decision, though.


It's the only sane decision.  void * is really the developer's way of  
saying "I'll handle the typing from here, thanks" to the compiler.  So the  
compiler can't expect to interpret void * as anything special.  There's no  
deterministic way to detect an object anyways, so the compiler can't make  
any assumptions without the typesystem.




So I would say, the fact that compile time evaluation doesn't work is a  
bug maybe?


-Steve


Probably. Not only it doesn't work at compile time, it doesn't work at  
all!


void main()
{
 auto o = cast(Object)cast(void*)0; // Error: cannot cast void* to  
object.Object

}


Hm.. strange that my example compiles and yours does not.  I'd think it to  
be the same thing.  That definitely should be flagged as a bug.


I used dmd 2.033.

-Steve

Re: Eliminate class allocators and deallocators?

[Denis Koroskin]

On Thu, 08 Oct 2009 15:48:56 +0400, Steven Schveighoffer  
 wrote:


On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
 wrote:


On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly  
 wrote:



auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);


I would expect a dynamic cast to occur at this line. Which will  
either result in an access violation (since you are trying to cast a  
garbage to an object) or result in a null being returned.


malloc returns void *, so no dynamic cast.

-Steve


I know malloc returns void*. I didn't know you can hijack type system  
that easily.


But then, if no dynamic cast takes place why cast(Object)cast(void*)0  
cannot be evaluated at compile time?


Your message made me test it :)

import std.stdio;

void *foo()
{
 return cast(void*)0;
}

void main()
{
 auto o = cast(Object)foo();
 writefln("here!");
 o.opEquals(o);
}

outputs:

here!
Segmentation fault

So, no dynamic cast (dynamic cast would have looked at the classinfo of  
null, segfaulting before the output).




No, IIRC, casting null to Object is perfectly valid and returns null.

But you are right, casting void* to Object does a reinterpret cast instead
of dynamic cast. I'm not sure if that's a good design decision, though.

So I would say, the fact that compile time evaluation doesn't work is a  
bug maybe?


-Steve


Probably. Not only it doesn't work at compile time, it doesn't work at all!

void main()
{
auto o = cast(Object)cast(void*)0; // Error: cannot cast void* to  
object.Object

}

Re: Eliminate class allocators and deallocators?

[Michel Fortin]

On 2009-10-08 05:00:03 -0400, Chris Nicholson-Sauls 
 said:



class C {...}

C new (T:C, A...) (A a) {
 auto c = GC.alloc!T();
 c.__ctor(a);
 return c;
}

auto somevar = new! C (1, 2, 3);


Nice idea, and it can already work... as long as your constructor is 
public (or you have private access from the module the template is 
defined in).




// free-listed
class F {...}

F new (T:F, A...) (A a) {
 return F.List.length != 0
 ? F.List.pop
 : defaultNew! F (a)
 ;
}

The latter examples shows my thinking: that the stdlib/druntime could 
easily provide a default new() that does what the current new operator 
does. Class designers could then overload this default new() as needed. 
 Provide a reasonable alias for the standard new() (I used "defaultNew" 
above, but its probably not the best) and it can still be used as 
backup in custom functions, such as in the free-list example.


What about 'newGarbageCollected!F' (or 'newGC!F' for short)?


Incidentally... does anyone else notice that, in the static-new 
proposal, we've once again recreated Ruby?


Proposed D2:
auto foo = Foo.new;

Ruby:
foo = Foo.new


Ah! I knew I had seen this pattern somewhere. Personally, I had more in 
mind the object instanciation pattern in Objective-C:


NSString *s = [[NSString alloc] init];

and decided to combine that alloc & init pair (GC.alloc & __ctor in D) 
into 'new'.




At least mine looks more like current syntax:
auto foo = new! Foo;


I'd call this a marginal gain, but a gain nonetheless. A bigger gain of 
'new!Foo' over 'Foo.new' is that it lets users invent their own 
allocation method without having to change any class or struct.


But it'd require some changes to how protection attributes are handled 
in templates, because right now it just won't work with a non-public 
contructor.



--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: Eliminate class allocators and deallocators?

[Steven Schveighoffer]

On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
 wrote:


On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly  
 wrote:



auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);


I would expect a dynamic cast to occur at this line. Which will either  
result in an access violation (since you are trying to cast a garbage  
to an object) or result in a null being returned.


malloc returns void *, so no dynamic cast.

-Steve


I know malloc returns void*. I didn't know you can hijack type system  
that easily.


But then, if no dynamic cast takes place why cast(Object)cast(void*)0  
cannot be evaluated at compile time?


Your message made me test it :)

import std.stdio;

void *foo()
{
return cast(void*)0;
}

void main()
{
auto o = cast(Object)foo();
writefln("here!");
o.opEquals(o);
}

outputs:

here!
Segmentation fault

So, no dynamic cast (dynamic cast would have looked at the classinfo of  
null, segfaulting before the output).


So I would say, the fact that compile time evaluation doesn't work is a  
bug maybe?


-Steve

Re: Eliminate class allocators and deallocators?

[Denis Koroskin]

On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
 wrote:


On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly   
wrote:



auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);


I would expect a dynamic cast to occur at this line. Which will either  
result in an access violation (since you are trying to cast a garbage  
to an object) or result in a null being returned.


malloc returns void *, so no dynamic cast.

-Steve


I know malloc returns void*. I didn't know you can hijack type system that  
easily.


But then, if no dynamic cast takes place why cast(Object)cast(void*)0  
cannot be evaluated at compile time?

Re: Eliminate class allocators and deallocators?

[Steven Schveighoffer]

On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2kor...@gmail.com>  
wrote:


On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly   
wrote:



auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);


I would expect a dynamic cast to occur at this line. Which will either  
result in an access violation (since you are trying to cast a garbage to  
an object) or result in a null being returned.


malloc returns void *, so no dynamic cast.

-Steve

Re: Eliminate class allocators and deallocators?

[Don]

Andrei Alexandrescu wrote:

Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They 
have a number of problems that make them unsuitable for D 2.0. The 
most obvious issue is that D 2.0 will _not_ conflate destruction with 
deallocation anymore: invoking delete against an object will call 
~this() against it but will not recycle its memory. In contrast, 
class deallocators are designed around the idea that invoking delete 
calls the destructor and also deallocates memory.


So I'm thinking of removing at least class deallocators from the 
language. Class allocators may be marginally and occasionally useful 
if the user takes the matter of deallocation in her own hands.


A much better way to handle custom allocation of classes would be in 
the standard library.


What do you think?


Andrei


I wouldn't like delete to go away at all, I use it for all my non-gc 
objects like this watered down example:


class ManualObject : Object {
new(size_t size) { return malloc(size); }
delete(void* mem) { free(mem); }
}

And then I can easily subclass it for any objects that doesn't need 
the GC. I've got similar constructs for arrays and structs.


Clearly you use those objects in a very different manner than GC 
objects. So by using new/delete with them you're fooling yourself.


// untested
class ManualObject {
static T create(T : ManualObject)() {
auto p = malloc(__traits(classInstanceSize, T));
memcpy(p, T.classinfo.init.ptr, __traits(classInstanceSize, T));
auto result = cast(T) p;
result.__ctor();
return result;
}
static void yank(ManualObject obj) {
free(cast(void*) obj);
}
}

Looks like a fair amount of work? At some level it actually should, but 
we can put that kind of stuff in the standard library.


malloc/free are nice, but they don't allow for elegant abstractions 
like new/delete does (for example if you want to use a specialized 
non-gc allocator you can just replace a few calls instead of every 
allocation).


They do if you're willing to write just a bit of scaffolding.

I also use delete when I no longer need large blocks of memory, I 
don't want them to just become uninitialized and sitting on the GC. 
When I want to do that I just nullify my references.


If you're afraid of deleting an object that may still have valid 
references, use smart pointers, or don't delete it at all if it sits 
on the gc and just call a .destroy() method.


Also in my runtime the delete implementations do free the memory, they 
don't just call the finalizer.


In any ways, just don't remove new/delete overrides from the language 
please, just call it a low-level technique or something to scare the 
beginners away and let people who want it have it :)


I strongly believe custom new/delete must go.


Andrei


Yes. The only reason you want them in C++ is because C++ makes 
constructors magical, by always glueing a memory allocation in front of 
them, and pretending they're not a function.

Then you need to introduce placement new to avoid the memory allocation bit.

Let's call a spade a spade: a constructor is just a function that 
establishes the invariant on a piece of memory which it recieves as a 
parameter. If you stop the pretense, you don't need the language machinery.

Re: Eliminate class allocators and deallocators?

[Chris Nicholson-Sauls]

Michel Fortin wrote:

On 2009-10-07 17:53:21 -0400, Craig Black  said:


Yes, recycling is best and I'm considering it. I'm only worried about
the extra cost.

Andrei


No this is a bad idea.  Removing the possibility to delete data will 
cause serious problems with heap fragmentation in some programs.


Hum, perhaps we need to review more thoroughly how memory allocation 
works. As Andrei said himself, we now have all the necessary parts in 
the language to reimplement 'new' as a library function.


So let's say we ditch 'new' and 'delete' as keywords. Let's first 
replace the keyword 'new' with a static function of the same name in a 
class or a struct. It could be implemented this way:


static T new(A...)(A a) {
T t = GC.alloc!T(); // GC.alloc sets the T.init bits.
t.__ctor(a);
return t;
}

Usage:

Foo foo = Foo.new();

That's a static function template that needs to be reimplemented for 
every subclass (Andrei already proposed such kind of mixins) and that 
returns a garbage-collected object reference. Now, if you want manual 
allocation:


static T new(A...)(A a) {
T t = GC.allocNoCollect!T(); // GC won't collect this bit.
t.__ctor(a);
return t;
}

void dispose() {
this.__dtor();
GC.free(this);
}

Usage:

Foo foo = Foo.new();
...
foo.dispose();

But then you could do much better: 'new' could return a different type: 
a smart reference-counted pointer struct for instance. The possibilities 
are endless.




Prior to this post I'd been on the side of retaining "good ole" delete, owing somewhat to 
my own tendency to do Evil Things with overloaded new/delete, such as transparent 
free-lists.


I've become neutral in light of the above proposed technique, because it really doesn't 
break that kind of usage.  In fact, it technically makes it more reliable and more 
flexible since the behavior of these is more predictable (not subject to compiler 
quality/method-of-implementation, and guaranteed to be "just another function").


That said, the stdlib (or probably druntime) needs to provide good general-case support 
for this, which should include some sort of IDisposable interface (as mentioned repeatedly 
by others) otherwise we're jumping into the abyss (of massive repetitive coding) rather 
than over it (into the Elysian fields).


One consideration is that new(), perhaps, ought not be a static member of its class at 
all, but rather a global written along similar lines to tools such as "to".  Given that, 
one could write something like:


##
class C {...}

C new (T:C, A...) (A a) {
auto c = GC.alloc!T();
c.__ctor(a);
return c;
}

auto somevar = new! C (1, 2, 3);

// free-listed
class F {...}

F new (T:F, A...) (A a) {
return F.List.length != 0
? F.List.pop
: defaultNew! F (a)
;
}
##

The latter examples shows my thinking: that the stdlib/druntime could easily provide a 
default new() that does what the current new operator does.  Class designers could then 
overload this default new() as needed.  Provide a reasonable alias for the standard new() 
(I used "defaultNew" above, but its probably not the best) and it can still be used as 
backup in custom functions, such as in the free-list example.


Incidentally... does anyone else notice that, in the static-new proposal, we've once again 
recreated Ruby?


Proposed D2:
auto foo = Foo.new;

Ruby:
foo = Foo.new

At least mine looks more like current syntax:
auto foo = new! Foo;

-- Christopher Nicholson-Sauls

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Ary Borenszweig wrote:

Andrei Alexandrescu wrote:

grauzone wrote:

Andrei Alexandrescu wrote:

Sean Kelly wrote:
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
article

dsimcha wrote:
== Quote from Andrei Alexandrescu 
(seewebsiteforem...@erdani.org)'s article
It is a bad idea because distinguishing between release of 
(expensive)
resources from dangerous memory recycling is the correct way to 
obtain

deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts 
of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes 
it's a worthwhile
tradeoff to add a few manual delete statements to code and 
sacrifice some safety

for making the GC run less often.)

malloc.


So for placement construction of a class, I guess it would look 
something like:


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?


Yes, I think so, but I haven't checked all the details. For example 
I'm not sure whether __ctor copies .init over the memory before 
running the user-defined constructor, or expects that to have been 
done already.


Apparently it doesn't: 
http://www.digitalmars.com/techtips/class_objects.html


See, it's even documented.

Anyway, does your statement mean that _ctor is officially supported 
(by all conform D compilers)?


Because, quoting from the page above:
"This technique goes "under the hood" of how D works, and as such it 
is not guaranteed to work with every D compiler. In particular, how 
the constructors and destructors are called is not necessarilly 
portable."


That technique will be used by a library function.


So... the library will be related somehow to the implementing compiler?


I'd believe so!

Andrei

Re: Eliminate class allocators and deallocators?

[Ary Borenszweig]

Andrei Alexandrescu wrote:

grauzone wrote:

Andrei Alexandrescu wrote:

Sean Kelly wrote:
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
article

dsimcha wrote:
== Quote from Andrei Alexandrescu 
(seewebsiteforem...@erdani.org)'s article
It is a bad idea because distinguishing between release of 
(expensive)
resources from dangerous memory recycling is the correct way to 
obtain

deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts 
of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes 
it's a worthwhile
tradeoff to add a few manual delete statements to code and 
sacrifice some safety

for making the GC run less often.)

malloc.


So for placement construction of a class, I guess it would look 
something like:


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?


Yes, I think so, but I haven't checked all the details. For example 
I'm not sure whether __ctor copies .init over the memory before 
running the user-defined constructor, or expects that to have been 
done already.


Apparently it doesn't: 
http://www.digitalmars.com/techtips/class_objects.html


See, it's even documented.

Anyway, does your statement mean that _ctor is officially supported 
(by all conform D compilers)?


Because, quoting from the page above:
"This technique goes "under the hood" of how D works, and as such it 
is not guaranteed to work with every D compiler. In particular, how 
the constructors and destructors are called is not necessarilly 
portable."


That technique will be used by a library function.


So... the library will be related somehow to the implementing compiler?

Re: Eliminate class allocators and deallocators?

[Sean Kelly]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> >> dsimcha wrote:
> >>> == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
> >>> article
> >>> 2.  Maintaining two separate heaps (the manually memory managed C heap 
> >>> and the
> >>> GC'd D heap) is a massive and completely unacceptable kludge because:
> >> Coding in a way that requires the GC to offer manual deletion is a
> >> completely unacceptable kludge. Most GCs could NOT offer a primitive to
> >> manually release memory. Designing D around a requirement that manual
> >> deletions work on the GC is crippling pressure on GC designers.
> >
> > Ok, fine, you got me on one point:  Manual freeing of objects only makes 
> > sense in
> > certain GC implementations.  So what?  GC.free() can be defined by the 
> > runtime
> > implementation.  If you're using something like pointer bump allocation with
> > generational, moving GC, the implementation is free to do nothing.  If 
> > you're
> > using conservative mark/sweep, it should actually free memory.
> I think there is convergence! My larger point is that we can leave
> GC.free() with loose semantics (e.g. may or may not act on it), and that
> we need to remove class-level allocators and probably the delete keyword
> too.

The docs for GC.free() should already state that what actually happens is
implementation-defined.  If they don't it's an oversight on my part.  I do
agree that the presence of "delete" in D is a bit weird, and would be happy
to see it replaced by a library routine.  new as well.

Re: Eliminate class allocators and deallocators?

[dsimcha]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> >> dsimcha wrote:
> >>> == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
> >>> article
> >>> 2.  Maintaining two separate heaps (the manually memory managed C heap 
> >>> and the
> >>> GC'd D heap) is a massive and completely unacceptable kludge because:
> >> Coding in a way that requires the GC to offer manual deletion is a
> >> completely unacceptable kludge. Most GCs could NOT offer a primitive to
> >> manually release memory. Designing D around a requirement that manual
> >> deletions work on the GC is crippling pressure on GC designers.
> >
> > Ok, fine, you got me on one point:  Manual freeing of objects only makes 
> > sense in
> > certain GC implementations.  So what?  GC.free() can be defined by the 
> > runtime
> > implementation.  If you're using something like pointer bump allocation with
> > generational, moving GC, the implementation is free to do nothing.  If 
> > you're
> > using conservative mark/sweep, it should actually free memory.
> I think there is convergence! My larger point is that we can leave
> GC.free() with loose semantics (e.g. may or may not act on it), and that
> we need to remove class-level allocators and probably the delete keyword
> too.
> Andrei

Perfect.  I'd be happy with this proposal as long as noone makes it harder to
manually free GC-allocated memory while the GC implementation is still
conservative mark-sweep or something similar.  I had been under the impression
that you wanted to flat-out get rid of GC.free().  Making it implementation
defined but requiring that it at least exist even if it does nothing makes 
perfect
sense.  If the implementation changes to some better algorithm (not likely in 
the
short term, but fairly likely in the long run), then my whole rationale for
wanting to free stuff manually in the first place may change.

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
2.  Maintaining two separate heaps (the manually memory managed C heap and the
GC'd D heap) is a massive and completely unacceptable kludge because:

Coding in a way that requires the GC to offer manual deletion is a
completely unacceptable kludge. Most GCs could NOT offer a primitive to
manually release memory. Designing D around a requirement that manual
deletions work on the GC is crippling pressure on GC designers.


Ok, fine, you got me on one point:  Manual freeing of objects only makes sense 
in
certain GC implementations.  So what?  GC.free() can be defined by the runtime
implementation.  If you're using something like pointer bump allocation with
generational, moving GC, the implementation is free to do nothing.  If you're
using conservative mark/sweep, it should actually free memory.


I think there is convergence! My larger point is that we can leave 
GC.free() with loose semantics (e.g. may or may not act on it), and that 
we need to remove class-level allocators and probably the delete keyword 
too.



Andrei

Re: Eliminate class allocators and deallocators?

[dsimcha]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> > 2.  Maintaining two separate heaps (the manually memory managed C heap and 
> > the
> > GC'd D heap) is a massive and completely unacceptable kludge because:
> Coding in a way that requires the GC to offer manual deletion is a
> completely unacceptable kludge. Most GCs could NOT offer a primitive to
> manually release memory. Designing D around a requirement that manual
> deletions work on the GC is crippling pressure on GC designers.

Ok, fine, you got me on one point:  Manual freeing of objects only makes sense 
in
certain GC implementations.  So what?  GC.free() can be defined by the runtime
implementation.  If you're using something like pointer bump allocation with
generational, moving GC, the implementation is free to do nothing.  If you're
using conservative mark/sweep, it should actually free memory.

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Denis Koroskin wrote:
On Thu, 08 Oct 2009 04:39:20 +0400, Andrei Alexandrescu 
 wrote:



Denis Koroskin wrote:
On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu 
 wrote:



Denis Koroskin wrote:
I'm not sure you will convince people to use foo.recycle() instead 
of foo.delete(). Not only it's slower, I believe recycling an 
object works for hiding bugs: accessing a recycled object - 
obviously a bug - will no longer be detected.


Is anyone under the illusion that today there's any detection going on?

Andrei

 There is none, but it's possible. It's just not implemented.


It's not possible if you allow actual memory reuse! Now I'm not sure I 
understand what you want.


Andrei


In our custom memory management system, deallocated memory gets filled 
with a debug data, which is checked for consistency when memory gets 
allocated again. Any write to that memory we be noticed. Not 
immediately, but still, it's better than nothing. Microsoft C++ debug 
runtime does the same.


Under Windows (2000 and later) you can also mark a range of memory as 
not accessible (by calling VirtualProtect on that memory with a 
PAGE_NOACCESS flag). Any read/write attempt with cause an immediate 
access violation exception. This is not widely used, probably because 
it's slow, but when you have a memory damage (caused by modifying some 
memory via a dangling pointer) performance is of lesser importance.


I believe similar mechanisms exist for nixes, too.


There are (anyway, page-level marking is not the right level of 
granularity).


My overall point is twofold:

1. new and delete were symmetric in C++. In D they aren't and aren't 
supposed to be symmetric. The delete keyword should be deprecated and 
the functionality of delete should be relegated to a function.


2. Mostly as a consequence of (1), class-level operators new and delete 
are misdesigned and should be eliminated. Object 
factories/pools/regions/etc. should be the way to go for custom class 
allocation.


Heck, others are shunning new and we're clinging on to it?


Andrei

Re: Eliminate class allocators and deallocators?

[Michel Fortin]

On 2009-10-07 20:11:31 -0400, Andrei Alexandrescu 
 said:


That's just awesome. Incidentally it would dovetail nicely with the 
code injection feature that I recently discussed here.


Indeed. That's what gave me the idea. :-)



But then that increases the size of the language...


Really? Remove new and delete; add code injection. Seems like a tie to 
me, except the later is much less limited and will solve problems well 
beyond memory allocation.



--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: Eliminate class allocators and deallocators?

[Denis Koroskin]

On Thu, 08 Oct 2009 04:39:20 +0400, Andrei Alexandrescu  
 wrote:



Denis Koroskin wrote:
On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu  
 wrote:



Denis Koroskin wrote:
I'm not sure you will convince people to use foo.recycle() instead of  
foo.delete(). Not only it's slower, I believe recycling an object  
works for hiding bugs: accessing a recycled object - obviously a bug  
- will no longer be detected.


Is anyone under the illusion that today there's any detection going on?

Andrei

 There is none, but it's possible. It's just not implemented.


It's not possible if you allow actual memory reuse! Now I'm not sure I  
understand what you want.


Andrei


In our custom memory management system, deallocated memory gets filled  
with a debug data, which is checked for consistency when memory gets  
allocated again. Any write to that memory we be noticed. Not immediately,  
but still, it's better than nothing. Microsoft C++ debug runtime does the  
same.


Under Windows (2000 and later) you can also mark a range of memory as not  
accessible (by calling VirtualProtect on that memory with a PAGE_NOACCESS  
flag). Any read/write attempt with cause an immediate access violation  
exception. This is not widely used, probably because it's slow, but when  
you have a memory damage (caused by modifying some memory via a dangling  
pointer) performance is of lesser importance.


I believe similar mechanisms exist for nixes, too.

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Denis Koroskin wrote:
On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu 
 wrote:



Denis Koroskin wrote:
I'm not sure you will convince people to use foo.recycle() instead of 
foo.delete(). Not only it's slower, I believe recycling an object 
works for hiding bugs: accessing a recycled object - obviously a bug 
- will no longer be detected.


Is anyone under the illusion that today there's any detection going on?

Andrei


There is none, but it's possible. It's just not implemented.


It's not possible if you allow actual memory reuse! Now I'm not sure I 
understand what you want.


Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Michel Fortin wrote:

On 2009-10-07 17:53:21 -0400, Craig Black  said:


Yes, recycling is best and I'm considering it. I'm only worried about
the extra cost.

Andrei


No this is a bad idea.  Removing the possibility to delete data will 
cause serious problems with heap fragmentation in some programs.


Hum, perhaps we need to review more thoroughly how memory allocation 
works. As Andrei said himself, we now have all the necessary parts in 
the language to reimplement 'new' as a library function.


So let's say we ditch 'new' and 'delete' as keywords. Let's first 
replace the keyword 'new' with a static function of the same name in a 
class or a struct. It could be implemented this way:


static T new(A...)(A a) {
T t = GC.alloc!T(); // GC.alloc sets the T.init bits.
t.__ctor(a);
return t;
}

Usage:

Foo foo = Foo.new();

That's a static function template that needs to be reimplemented for 
every subclass (Andrei already proposed such kind of mixins) and that 
returns a garbage-collected object reference. Now, if you want manual 
allocation:


static T new(A...)(A a) {
T t = GC.allocNoCollect!T(); // GC won't collect this bit.
t.__ctor(a);
return t;
}

void dispose() {
this.__dtor();
GC.free(this);
}

Usage:

Foo foo = Foo.new();
...
foo.dispose();

But then you could do much better: 'new' could return a different type: 
a smart reference-counted pointer struct for instance. The possibilities 
are endless.




That's just awesome. Incidentally it would dovetail nicely with the code 
injection feature that I recently discussed here. But then that 
increases the size of the language...


Andrei

Re: Eliminate class allocators and deallocators?

[Denis Koroskin]

On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu  
 wrote:



Denis Koroskin wrote:
I'm not sure you will convince people to use foo.recycle() instead of  
foo.delete(). Not only it's slower, I believe recycling an object works  
for hiding bugs: accessing a recycled object - obviously a bug - will  
no longer be detected.


Is anyone under the illusion that today there's any detection going on?

Andrei


There is none, but it's possible. It's just not implemented.

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Denis Koroskin wrote:
I'm not sure you will convince people to use foo.recycle() instead of 
foo.delete(). Not only it's slower, I believe recycling an object works 
for hiding bugs: accessing a recycled object - obviously a bug - will no 
longer be detected.


Is anyone under the illusion that today there's any detection going on?

Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

grauzone wrote:

Andrei Alexandrescu wrote:

Sean Kelly wrote:
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
article

dsimcha wrote:
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
article
It is a bad idea because distinguishing between release of 
(expensive)
resources from dangerous memory recycling is the correct way to 
obtain

deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts 
of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes 
it's a worthwhile
tradeoff to add a few manual delete statements to code and 
sacrifice some safety

for making the GC run less often.)

malloc.


So for placement construction of a class, I guess it would look 
something like:


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?


Yes, I think so, but I haven't checked all the details. For example 
I'm not sure whether __ctor copies .init over the memory before 
running the user-defined constructor, or expects that to have been 
done already.


Apparently it doesn't: 
http://www.digitalmars.com/techtips/class_objects.html


See, it's even documented.

Anyway, does your statement mean that _ctor is officially supported (by 
all conform D compilers)?


Because, quoting from the page above:
"This technique goes "under the hood" of how D works, and as such it is 
not guaranteed to work with every D compiler. In particular, how the 
constructors and destructors are called is not necessarilly portable."


That technique will be used by a library function.

Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Jeremie Pelletier wrote:
[snip]

I vote to keep delete in D.


But your argument builds evidence for a deallocation function, not a 
keyword plus a smorgasbord of language support.


Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
2.  Maintaining two separate heaps (the manually memory managed C heap and the
GC'd D heap) is a massive and completely unacceptable kludge because:


Coding in a way that requires the GC to offer manual deletion is a 
completely unacceptable kludge. Most GCs could NOT offer a primitive to 
manually release memory. Designing D around a requirement that manual 
deletions work on the GC is crippling pressure on GC designers.



1.  If you just want to delete a few objects to make the GC run less often, you
can just add delete statements for the common code paths, or paths where the end
of an object's lifetime is obvious.  You then just let the GC handle the less
common code paths or cases where object lifetimes are non-trivial and gain tons 
of
simplicity for only a small performance loss.  If you have to handle all the odd
code paths manually too, this is when bugs really start to seep in.


Many people's famous last void were "the end of an object's lifetime is 
obvious".



2.  Heaps have overhead.  Two heaps have twice the overhead.


Where did that come from?


3.  addroot(), etc. is a PITA *and* adds yet another place where you have to 
lock
on the GC mutex.  Half the need for manual memory management in D is because the
GC sometimes scales poorly to large numbers of threads.  This would definitely 
not
help the situation.


So right now do you have it for free? I don't understand. What are you 
comparing against what?



4.  Using the C heap whenever you want the ability to manually free something
doesn't play nicely w/ builtin language features such as classes, arrays,
associative arrays, etc., or objects returned from library functions.


It shouldn't too.


Because of these 4 issues, I feel that only being allowed to do manual memory
management if you use the C heap is such an unacceptably bad kludge that it is 
for
many practical purposes akin to not being allowed to do manual memory management
at all.  This is unacceptable in a systems/performance language.


I completely disagree. I believe that "unifying" safe and unsafe styles 
under the same umbrella is an unacceptably bad kludge that is for many 
practical purposes akin to not being allowed to provide the slightest 
guarantee about any piece of D code. That's not where D should be going.



Remember, performance/systems languages can't place excessive emphasis on safety
and absolutely MUST assume the programmer knows what he/she is doing.  If you 
want
Java, you know where to find it.


We agree on D being able to provide every bit of performance control 
needed. But my understanding you foster a programming style that's a 
hodge-podge of safe and unsafe coding under the same syntactic pretense. 
I don't think that's a good way to go.



Andrei

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

Leandro Lucarella wrote:

Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:

Leandro Lucarella wrote:

Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:

Sean Kelly wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
worthwhile
tradeoff to add a few manual delete statements to code and sacrifice some safety
for making the GC run less often.)

malloc.

So for placement construction of a class, I guess it would look something like:

auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?

Yes, I think so, but I haven't checked all the details. For example
I'm not sure whether __ctor copies .init over the memory before
running the user-defined constructor, or expects that to have been
done already.

My understanding from Walter is that __ctor(x, y, z) are simply the
functions this(x, y, z) as written by the user, so you'd need to
memcpy the .init by hand before calling __ctor.

What I don't understand is why you're willing to make that hard to do
manual memory management in D. Do you see that you're making the
programmer's job deliberately for no reason? D needs conservative GC,
which means slow GC; by definition. D is a system programming language, so
it's expected to be fast, but because of the GC there will be often
situations where you have to do manual MM. Why are you making that much
harder?

You know that in the search for safety you'll be making much more unsafe
(or bug-prone) to do manual MM?

You seem to be asserting that without additional built-in language
support, manual memory management is unduly difficult. Why so?


Because of this:


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );


:)

You even forgot to register your object as a root in the GC, so if your
MyClass has any pointers to the GC your program will blow in your face.

If you plan to library support to ease this and avoid repetitive and
bug-prone work, you can ignore my complains...



I agree, that manual allocation code looks plain ugly. Why should we 
remove delete from the language anyways? If its unsafe then don't use it 
and your memory will get collected someday. If you need to reclaim 
memory right away, or you're doing manual memory management then its 
there, just like the original spec said.


We can't always just let the GC collect everything, its most useful for 
memory that travels around a lot like strings and whatnot, but it's 
definitely too slow and too memory hungry for performance code.


From the testing I did the D garbage collector is very fast to allocate 
memory, but *very* slow to reclaim it by mark&sweep, yet reclaim by 
delete is as fast as allocating it.


The argument that it may be safer doesn't count, cause you can just not 
call delete and have safe code.


I vote to keep delete in D.

Re: Eliminate class allocators and deallocators?

[Denis Koroskin]

On Wed, 07 Oct 2009 21:55:42 +0400, Andrei Alexandrescu  
 wrote:



Michel Fortin wrote:
On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu  
 said:


The matter has been discussed quite a bit around here and in other  
places. I'm not having as much time as I'd want to explain things. In  
short, destroying without freeing memory avoids dangling references  
and preserves memory safety without impacting on other resources.


It's a safety hack, not a performance hack.
 In my opinion, it's mostly an illusion of safety. If you call the  
destructor on an object, the object state after the call doesn't  
necessarily respects the object invariants and doing anything with it  
could result in, well, anything, from returning wrong results to  
falling into an infinite loop (basically undefined behaviour). What you  
gain is that no object will be allocated on top of the old one, and  
thus new objects can't get corrupted. But it's still undefined  
behaviour, only with less side effects and more memory consumption.
 I don't think it's a so bad idea on the whole, but it'd be more  
valuable if accessing an invalidated object could be made an error  
instead of undefined behaviour. If this can't be done, then we should  
encourage "destructors" to put the object in a clean state and not  
leave any dirt behind. But should that still be called a "destructor"?
 Perhaps we could change the paradigm a little and replace "deletion"  
with "recycling". Recycling an object would call the destructor and  
immeditately call the default constructor, so the object is never left  
in an invalid state. Objects with no default constructor cannot be  
recycled. This way you know memory is always left in a clean state, and  
you encourage programmers to safely reuse the memory blocks from  
objects they have already allocated when possible.


Yes, recycling is best and I'm considering it. I'm only worried about  
the extra cost.


Andrei


I rarely use delete these days (certainly not as often as in my early D  
days, which is a good sign IMO), and I'm afraid I'll drop its use entirely  
if delete will be replaced with a recycle.


I mostly manage memory manually as part of performance optimization. The  
change you are talking about contradicts with my goals of manual object  
destruction.


You don't even need to change a language to support your semantics:

template Recyclable()
{
final void recycle()
{
this.__dtor();
memcpy(this, classinfo.init.ptr, classinfo.init.length);

// ctors are not virtual
auto defaultCtor = (void delegate(Object))classinfo.defaultConstructor;
ctor(this);
}
}

class Foo
{
mixin Recyclable!();
int i = 42;
this() { i = -1; }
}

Foo foo = new Foo();
foo.i = 0;
foo.recycle();
writeln(foo.i); // -1

And even if the proposed change will occur, old behavior will still be  
accessible:


template Deletable()
{
final void delete()
{
this.__dtor();
GC.free(this);
}
}

I'm not sure you will convince people to use foo.recycle() instead of  
foo.delete(). Not only it's slower, I believe recycling an object works  
for hiding bugs: accessing a recycled object - obviously a bug - will no  
longer be detected.

Re: Eliminate class allocators and deallocators?

[Michel Fortin]

On 2009-10-07 17:53:21 -0400, Craig Black  said:


Yes, recycling is best and I'm considering it. I'm only worried about
the extra cost.

Andrei


No this is a bad idea.  Removing the possibility to delete data will 
cause serious problems with heap fragmentation in some programs.


Hum, perhaps we need to review more thoroughly how memory allocation 
works. As Andrei said himself, we now have all the necessary parts in 
the language to reimplement 'new' as a library function.


So let's say we ditch 'new' and 'delete' as keywords. Let's first 
replace the keyword 'new' with a static function of the same name in a 
class or a struct. It could be implemented this way:


static T new(A...)(A a) {
T t = GC.alloc!T(); // GC.alloc sets the T.init bits.
t.__ctor(a);
return t;
}

Usage:

Foo foo = Foo.new();

That's a static function template that needs to be reimplemented for 
every subclass (Andrei already proposed such kind of mixins) and that 
returns a garbage-collected object reference. Now, if you want manual 
allocation:


static T new(A...)(A a) {
T t = GC.allocNoCollect!T(); // GC won't collect this bit.
t.__ctor(a);
return t;
}

void dispose() {
this.__dtor();
GC.free(this);
}

Usage:

Foo foo = Foo.new();
...
foo.dispose();

But then you could do much better: 'new' could return a different type: 
a smart reference-counted pointer struct for instance. The 
possibilities are endless.


--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: Eliminate class allocators and deallocators?

[grauzone]

Andrei Alexandrescu wrote:

Sean Kelly wrote:
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
article

dsimcha wrote:
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
article

It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts of 
data).
2.  Garbage collection is never a major bottleneck.  (Sometimes it's 
a worthwhile
tradeoff to add a few manual delete statements to code and sacrifice 
some safety

for making the GC run less often.)

malloc.


So for placement construction of a class, I guess it would look 
something like:


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?


Yes, I think so, but I haven't checked all the details. For example I'm 
not sure whether __ctor copies .init over the memory before running the 
user-defined constructor, or expects that to have been done already.


Apparently it doesn't: 
http://www.digitalmars.com/techtips/class_objects.html


See, it's even documented.

Anyway, does your statement mean that _ctor is officially supported (by 
all conform D compilers)?


Because, quoting from the page above:
"This technique goes "under the hood" of how D works, and as such it is 
not guaranteed to work with every D compiler. In particular, how the 
constructors and destructors are called is not necessarilly portable."

Re: Eliminate class allocators and deallocators?

[dsimcha]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> >> dsimcha wrote:
> >>> == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
> >>> article
>  It is a bad idea because distinguishing between release of (expensive)
>  resources from dangerous memory recycling is the correct way to obtain
>  deterministic resource management within the confines of safety.
> >>> This is based on two faulty assumptions:
> >>>
> >>> 1.  Memory is cheap.  (Not if you are working with absurd amounts of 
> >>> data).
> >>> 2.  Garbage collection is never a major bottleneck.  (Sometimes it's a
worthwhile
> >>> tradeoff to add a few manual delete statements to code and sacrifice some 
> >>> safety
> >>> for making the GC run less often.)
> >> malloc.
> >> Andrei
> >
> > Kludge.  Requires using two separate heaps (inefficient) and worrying about
> > whether your stuff is manually freed on all code paths, not just the ones 
> > that are
> > executed often enough for performance to matter.
> Au contraire, once the GC heap becomes safe, I have less to worry about.
> Andrei

If you're that concerned about making the GC heap safe, here's a less 
destructive
(to other people's programming styles) way to do it:

1.  Make delete only call the d'tor and not release memory.  (I'm fine with this
provided the stuff below is done.)

2.  Add a std. lib convenience function to core.memory that does what delete 
does
now (calls d'tor AND frees memory).  For the purposes of this discussion, we'll
call it deleteFree().  There's already a std. lib. function that just frees
memory, GC.free().  Keep it.

3.  If you really insist on absolute heap safety even at the expense of
performance, grep your code and get rid of all deleteFree() and GC.free() calls.

Frankly, I consider the ability to manually free GC allocated memory to be a 
HUGE
asset for the following reasons, which I've mentioned before but would like to
distill:

1.  GC is usually the best way to program, but can be a huge bottleneck in some
corner cases.

2.  Maintaining two separate heaps (the manually memory managed C heap and the
GC'd D heap) is a massive and completely unacceptable kludge because:

1.  If you just want to delete a few objects to make the GC run less often, you
can just add delete statements for the common code paths, or paths where the end
of an object's lifetime is obvious.  You then just let the GC handle the less
common code paths or cases where object lifetimes are non-trivial and gain tons 
of
simplicity for only a small performance loss.  If you have to handle all the odd
code paths manually too, this is when bugs really start to seep in.

2.  Heaps have overhead.  Two heaps have twice the overhead.

3.  addroot(), etc. is a PITA *and* adds yet another place where you have to 
lock
on the GC mutex.  Half the need for manual memory management in D is because the
GC sometimes scales poorly to large numbers of threads.  This would definitely 
not
help the situation.

4.  Using the C heap whenever you want the ability to manually free something
doesn't play nicely w/ builtin language features such as classes, arrays,
associative arrays, etc., or objects returned from library functions.

Because of these 4 issues, I feel that only being allowed to do manual memory
management if you use the C heap is such an unacceptably bad kludge that it is 
for
many practical purposes akin to not being allowed to do manual memory management
at all.  This is unacceptable in a systems/performance language.

Remember, performance/systems languages can't place excessive emphasis on safety
and absolutely MUST assume the programmer knows what he/she is doing.  If you 
want
Java, you know where to find it.

Re: Eliminate class allocators and deallocators?

[Denis Koroskin]

On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly   
wrote:



auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);


I would expect a dynamic cast to occur at this line. Which will either  
result in an access violation (since you are trying to cast a garbage to  
an object) or result in a null being returned.

Re: Eliminate class allocators and deallocators?

[Craig Black]

Andrei Alexandrescu Wrote:

> Michel Fortin wrote:
> > On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu 
> >  said:
> > 
> >> The matter has been discussed quite a bit around here and in other 
> >> places. I'm not having as much time as I'd want to explain things. In 
> >> short, destroying without freeing memory avoids dangling references 
> >> and preserves memory safety without impacting on other resources.
> >>
> >> It's a safety hack, not a performance hack.
> > 
> > In my opinion, it's mostly an illusion of safety. If you call the 
> > destructor on an object, the object state after the call doesn't 
> > necessarily respects the object invariants and doing anything with it 
> > could result in, well, anything, from returning wrong results to falling 
> > into an infinite loop (basically undefined behaviour). What you gain is 
> > that no object will be allocated on top of the old one, and thus new 
> > objects can't get corrupted. But it's still undefined behaviour, only 
> > with less side effects and more memory consumption.
> > 
> > I don't think it's a so bad idea on the whole, but it'd be more valuable 
> > if accessing an invalidated object could be made an error instead of 
> > undefined behaviour. If this can't be done, then we should encourage 
> > "destructors" to put the object in a clean state and not leave any dirt 
> > behind. But should that still be called a "destructor"?
> > 
> > Perhaps we could change the paradigm a little and replace "deletion" 
> > with "recycling". Recycling an object would call the destructor and 
> > immeditately call the default constructor, so the object is never left 
> > in an invalid state. Objects with no default constructor cannot be 
> > recycled. This way you know memory is always left in a clean state, and 
> > you encourage programmers to safely reuse the memory blocks from objects 
> > they have already allocated when possible.
> 
> Yes, recycling is best and I'm considering it. I'm only worried about 
> the extra cost.
> 
> Andrei

No this is a bad idea.  Removing the possibility to delete data will cause 
serious problems with heap fragmentation in some programs.

-Craig

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
worthwhile
tradeoff to add a few manual delete statements to code and sacrifice some safety
for making the GC run less often.)

malloc.
Andrei


Kludge.  Requires using two separate heaps (inefficient) and worrying about
whether your stuff is manually freed on all code paths, not just the ones that 
are
executed often enough for performance to matter.


Au contraire, once the GC heap becomes safe, I have less to worry about.

Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Leandro Lucarella wrote:

Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:

You seem to be asserting that without additional built-in language
support, manual memory management is unduly difficult. Why so?


Because of this:


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );


:)

You even forgot to register your object as a root in the GC, so if your
MyClass has any pointers to the GC your program will blow in your face.

If you plan to library support to ease this and avoid repetitive and
bug-prone work, you can ignore my complains...


I too think it would be great to add the necessary support to the
stdlib. In fact, since you have a great deal of expertise in the matter,
feel free to suggest API functions! They'd need to be approved by Sean
too because probably they belong to druntime.

Andrei

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:
> Leandro Lucarella wrote:
> >Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:
> >>Sean Kelly wrote:
> >>>== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> dsimcha wrote:
> >== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
> >article
> >>It is a bad idea because distinguishing between release of (expensive)
> >>resources from dangerous memory recycling is the correct way to obtain
> >>deterministic resource management within the confines of safety.
> >This is based on two faulty assumptions:
> >
> >1.  Memory is cheap.  (Not if you are working with absurd amounts of 
> >data).
> >2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
> >worthwhile
> >tradeoff to add a few manual delete statements to code and sacrifice 
> >some safety
> >for making the GC run less often.)
> malloc.
> >>>So for placement construction of a class, I guess it would look something 
> >>>like:
> >>>
> >>>auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
> >>>x.__ctor( a, b, c ); // construct
> >>>...
> >>>x.__dtor();
> >>>free( cast(void*) x );
> >>>
> >>>Is that right?
> >>Yes, I think so, but I haven't checked all the details. For example
> >>I'm not sure whether __ctor copies .init over the memory before
> >>running the user-defined constructor, or expects that to have been
> >>done already.
> >>
> >>My understanding from Walter is that __ctor(x, y, z) are simply the
> >>functions this(x, y, z) as written by the user, so you'd need to
> >>memcpy the .init by hand before calling __ctor.
> >
> >What I don't understand is why you're willing to make that hard to do
> >manual memory management in D. Do you see that you're making the
> >programmer's job deliberately for no reason? D needs conservative GC,
> >which means slow GC; by definition. D is a system programming language, so
> >it's expected to be fast, but because of the GC there will be often
> >situations where you have to do manual MM. Why are you making that much
> >harder?
> >
> >You know that in the search for safety you'll be making much more unsafe
> >(or bug-prone) to do manual MM?
> 
> You seem to be asserting that without additional built-in language
> support, manual memory management is unduly difficult. Why so?

Because of this:

> >>>auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
> >>>x.__ctor( a, b, c ); // construct
> >>>...
> >>>x.__dtor();
> >>>free( cast(void*) x );

:)

You even forgot to register your object as a root in the GC, so if your
MyClass has any pointers to the GC your program will blow in your face.

If you plan to library support to ease this and avoid repetitive and
bug-prone work, you can ignore my complains...

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
Y tuve amores, que fue uno sólo
El que me dejó de a pie y me enseñó todo...

Re: Eliminate class allocators and deallocators?

[dsimcha]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> >> It is a bad idea because distinguishing between release of (expensive)
> >> resources from dangerous memory recycling is the correct way to obtain
> >> deterministic resource management within the confines of safety.
> >
> > This is based on two faulty assumptions:
> >
> > 1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
> > 2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
> > worthwhile
> > tradeoff to add a few manual delete statements to code and sacrifice some 
> > safety
> > for making the GC run less often.)
> malloc.
> Andrei

Kludge.  Requires using two separate heaps (inefficient) and worrying about
whether your stuff is manually freed on all code paths, not just the ones that 
are
executed often enough for performance to matter.

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Leandro Lucarella wrote:

Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:

Sean Kelly wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
worthwhile
tradeoff to add a few manual delete statements to code and sacrifice some safety
for making the GC run less often.)

malloc.

So for placement construction of a class, I guess it would look something like:

auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?

Yes, I think so, but I haven't checked all the details. For example
I'm not sure whether __ctor copies .init over the memory before
running the user-defined constructor, or expects that to have been
done already.

My understanding from Walter is that __ctor(x, y, z) are simply the
functions this(x, y, z) as written by the user, so you'd need to
memcpy the .init by hand before calling __ctor.


What I don't understand is why you're willing to make that hard to do
manual memory management in D. Do you see that you're making the
programmer's job deliberately for no reason? D needs conservative GC,
which means slow GC; by definition. D is a system programming language, so
it's expected to be fast, but because of the GC there will be often
situations where you have to do manual MM. Why are you making that much
harder?

You know that in the search for safety you'll be making much more unsafe
(or bug-prone) to do manual MM?


You seem to be asserting that without additional built-in language 
support, manual memory management is unduly difficult. Why so?


Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Leandro Lucarella wrote:

Andrei Alexandrescu, el  7 de octubre a las 14:18 me escribiste:

Manfred_Nowak wrote:

Andrei Alexandrescu wrote:


if I saw a highlighted keyword that I had no idea what it does I'd
get quite worried

Why wouldn't you try to look at the documentation of the
language---as you do with the documentation of a library?

-manfred


I didn't say I wouldn't. I just said I'd be much more worried.

My point is, languages are never modular. To be even marginally
effective in a language, you must have some understanding of it all.
That definitely isn't the case for libraries.


Languages are modular when they let you define new syntax, but that's
another topic ;)


A topic at which no language succeeded.

Andrei

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  7 de octubre a las 14:18 me escribiste:
> Manfred_Nowak wrote:
> >Andrei Alexandrescu wrote:
> >
> >> if I saw a highlighted keyword that I had no idea what it does I'd
> >> get quite worried
> >
> >Why wouldn't you try to look at the documentation of the
> >language---as you do with the documentation of a library?
> >
> >-manfred
> >
> 
> I didn't say I wouldn't. I just said I'd be much more worried.
> 
> My point is, languages are never modular. To be even marginally
> effective in a language, you must have some understanding of it all.
> That definitely isn't the case for libraries.

Languages are modular when they let you define new syntax, but that's
another topic ;)

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
You look so tired-unhappy,
bring down the government,
they don't, they don't speak for us.

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:
> Sean Kelly wrote:
> >== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> >>dsimcha wrote:
> >>>== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> It is a bad idea because distinguishing between release of (expensive)
> resources from dangerous memory recycling is the correct way to obtain
> deterministic resource management within the confines of safety.
> >>>This is based on two faulty assumptions:
> >>>
> >>>1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
> >>>2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
> >>>worthwhile
> >>>tradeoff to add a few manual delete statements to code and sacrifice some 
> >>>safety
> >>>for making the GC run less often.)
> >>malloc.
> >
> >So for placement construction of a class, I guess it would look something 
> >like:
> >
> >auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
> >x.__ctor( a, b, c ); // construct
> >...
> >x.__dtor();
> >free( cast(void*) x );
> >
> >Is that right?
> 
> Yes, I think so, but I haven't checked all the details. For example
> I'm not sure whether __ctor copies .init over the memory before
> running the user-defined constructor, or expects that to have been
> done already.
> 
> My understanding from Walter is that __ctor(x, y, z) are simply the
> functions this(x, y, z) as written by the user, so you'd need to
> memcpy the .init by hand before calling __ctor.

What I don't understand is why you're willing to make that hard to do
manual memory management in D. Do you see that you're making the
programmer's job deliberately for no reason? D needs conservative GC,
which means slow GC; by definition. D is a system programming language, so
it's expected to be fast, but because of the GC there will be often
situations where you have to do manual MM. Why are you making that much
harder?

You know that in the search for safety you'll be making much more unsafe
(or bug-prone) to do manual MM?

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
Ya ni el cielo me quiere, ya ni la muerte me visita
Ya ni el sol me calienta, ya ni el viento me acaricia

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  7 de octubre a las 13:06 me escribiste:
> >I think is a good idea not to force the GC to free the memory immediately
> >with a delete, but it should if it's easy. Other protection methods as
> >using mprotect to protect the objects pages it's very desirable too,
> >because you can spot an access to a inconsistent (destroyed) object as
> >soon as it first happen.
> 
> (mprotect is much too coarse to be useful.) With the dispose()
> function the state of the object will be restored to default
> construction:
> 
> void dispose(T)(T obj) if (is(T == class) || is(typeof(*T.init))) {
>... call destructor if any ...
>... obliterate object with .init ...
>... invoke default ctor if any ...
> }

Ok, if you're going to name that dispose, is fine with me. End of
discussion. With the addition of calling a constructor after destroying
the object, make a little more sense too (I still find it too bug prone,
you can end up with corruption if you dispose an object that other part of
the program think it's not disposed yet, i.e., in a state different than
the recently constructed object).

> >If you want to introduce a new semantic, I think you should provide a new
> >method, not change the semantic of an existent one.
> 
> Agreed. I hereby vote for deprecating delete with extreme prejudice.
> 
> >And BTW, is there any reason why this can't be implemented in the library
> >instead of using an operator? Why don't you provide a "destroy()" function
> >for that in Phobos?
> 
> That sounds great.
> 
> >Really, I can't see any advantages on changing the delete operator
> >semantics, only problems.
> 
> I agree.

I'm glad to see that.

> >Why? Using malloc and free is a lot more trouble, you have to register the
> >roots yourself for example. It's not like you do malloc() and free() and
> >everything works magically. You have to have more knowledge of the GC to
> >use them. Being able to manually manage the *GC* heap (if the GC support
> >that, if not it can make it a NOP) is good IMHO.
> 
> We can make things a tad better with library functions, but we do
> need to have a garbage collected heap that guarantees safety.

I don't think I understand this very well. What kind of safety? If the
user disposed/freed an object before it should, it's an user bug, with
unavoidable bad side effects. The best you can do is make the program
blow in the user face ASAP.

I don't understand what all this have to do with GC safety.

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
It's not a lie, if you believe it.
-- George Constanza

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Manfred_Nowak wrote:

Andrei Alexandrescu wrote:


 if I saw a highlighted keyword that I had no idea what it does I'd
 get quite worried 


Why wouldn't you try to look at the documentation of the language---as you 
do with the documentation of a library?


-manfred



I didn't say I wouldn't. I just said I'd be much more worried.

My point is, languages are never modular. To be even marginally
effective in a language, you must have some understanding of it all.
That definitely isn't the case for libraries.


Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Sean Kelly wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
worthwhile
tradeoff to add a few manual delete statements to code and sacrifice some safety
for making the GC run less often.)

malloc.


So for placement construction of a class, I guess it would look something like:

auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?


Yes, I think so, but I haven't checked all the details. For example I'm 
not sure whether __ctor copies .init over the memory before running the 
user-defined constructor, or expects that to have been done already.


My understanding from Walter is that __ctor(x, y, z) are simply the 
functions this(x, y, z) as written by the user, so you'd need to memcpy 
the .init by hand before calling __ctor.


Aw hell I got curious so let me check.

class MyClass {
int x = 42;
this() {}
}

void main() {
auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor();
writeln(x.x);
writeln(x.toString);
}

That prints 0 and then crashes on my machine. Looks like you need to 
memcpy the .init before calling __ctor.


I'm very glad we're starting to look into this. There are very nice 
opportunities for adding custom allocation support in the stdlib.



Andrei

Re: Eliminate class allocators and deallocators?

[Manfred_Nowak]

Andrei Alexandrescu wrote:

>  if I saw a highlighted keyword that I had no idea what it does I'd
>  get quite worried 

Why wouldn't you try to look at the documentation of the language---as you 
do with the documentation of a library?

-manfred

Re: Eliminate class allocators and deallocators?

[Sean Kelly]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> >> It is a bad idea because distinguishing between release of (expensive)
> >> resources from dangerous memory recycling is the correct way to obtain
> >> deterministic resource management within the confines of safety.
> >
> > This is based on two faulty assumptions:
> >
> > 1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
> > 2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
> > worthwhile
> > tradeoff to add a few manual delete statements to code and sacrifice some 
> > safety
> > for making the GC run less often.)
> malloc.

So for placement construction of a class, I guess it would look something like:

auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );

Is that right?

Re: Eliminate class allocators and deallocators?

[downs]

Jeremie Pelletier wrote:
> downs wrote:
>> Don wrote:
>>> downs wrote:
 Andrei Alexandrescu wrote:
> downs wrote:
>> Andrei Alexandrescu wrote:
>>> downs wrote:
 Andrei Alexandrescu wrote:
> Hello,
>
>
> D currently allows defining class allocators and deallocators.
> They
> have
> a number of problems that make them unsuitable for D 2.0. The most
> obvious issue is that D 2.0 will _not_ conflate destruction with
> deallocation anymore: invoking delete against an object will call
> ~this() against it but will not recycle its memory. In contrast,
> class
> deallocators are designed around the idea that invoking delete
> calls the
> destructor and also deallocates memory.
>
> So I'm thinking of removing at least class deallocators from the
> language. Class allocators may be marginally and occasionally
> useful if
> the user takes the matter of deallocation in her own hands.
>
> A much better way to handle custom allocation of classes would be
> in the
> standard library.
>
> What do you think?
>
>
> Andrei
 Do you trust the D GC to be good enough to always free everything
 you've allocated, without error?

 If your answer was 'ye- maaybe ... no actually', please rethink
 this.
>>> People will always be able to call functions in the garbage
>>> collector
>>> manually. The discussion on class allocators and deallocators has
>>> nothing to do with that.
>>>
>>> Andrei
>> So you can still deallocate a class by hand, only it's not called
>> delete anymore?
> That is correct.
>
> Andrei
 Isn't that a pretty big violation of Least Surprise?

 http://en.wikipedia.org/wiki/Principle_of_least_astonishment :
 "In user interface design, programming language design, and
 ergonomics, the principle (or rule or law) of least astonishment (or
 surprise) states that, when two elements of an interface conflict, or
 are ambiguous, the behaviour should be that which will *least
 surprise* the human user or programmer at the time the conflict
 arises."
>>> I think the basic rule being introduced is:
>>> that every object can be managed by the gc, or manually managed. But not
>>> both. That seems reasonable to me. But if delete no longer deletes, it
>>> needs a name change.
>>
>>
>> Oh, that makes more sense.
>>
>> Do manually managed objects still count under Mark&Sweep?
> 
> You have to register the memory range they cover to the GC if they
> contain pointers to GC memory. Otherwise the GC don't know they exist at
> all.

Well I certainly wouldn't expect that! :p

This sounds like something that might trip people up. I believe at least 
scanning objects by GC should always be the default for any object, if only 
because the association "D heap => GC managed" is I think a fairly core part of 
the language.

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.


This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
worthwhile
tradeoff to add a few manual delete statements to code and sacrifice some safety
for making the GC run less often.)


malloc.

Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Jeremie Pelletier wrote:

Sean Kelly wrote:
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s 
article

downs wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They 
have

a number of problems that make them unsuitable for D 2.0. The most
obvious issue is that D 2.0 will _not_ conflate destruction with
deallocation anymore: invoking delete against an object will call
~this() against it but will not recycle its memory. In contrast, class
deallocators are designed around the idea that invoking delete 
calls the

destructor and also deallocates memory.

So I'm thinking of removing at least class deallocators from the
language. Class allocators may be marginally and occasionally 
useful if

the user takes the matter of deallocation in her own hands.

A much better way to handle custom allocation of classes would be 
in the

standard library.

What do you think?


Andrei
Do you trust the D GC to be good enough to always free everything 
you've allocated, without error?


If your answer was 'ye- maaybe ... no actually', please rethink this.

People will always be able to call functions in the garbage collector
manually. The discussion on class allocators and deallocators has
nothing to do with that.


Right.  There's no plan to eliminate GC.free().


But that's runtime dependent, for example on my runtime its 
Memory.Free(). Removing 'delete' would therefore bind the code to a 
certain runtime, that's not a very portable solution, and far from being 
as elegant as delete.


There's nothing elegant about delete.

Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They 
have a number of problems that make them unsuitable for D 2.0. The 
most obvious issue is that D 2.0 will _not_ conflate destruction with 
deallocation anymore: invoking delete against an object will call 
~this() against it but will not recycle its memory. In contrast, class 
deallocators are designed around the idea that invoking delete calls 
the destructor and also deallocates memory.


So I'm thinking of removing at least class deallocators from the 
language. Class allocators may be marginally and occasionally useful 
if the user takes the matter of deallocation in her own hands.


A much better way to handle custom allocation of classes would be in 
the standard library.


What do you think?


Andrei


I wouldn't like delete to go away at all, I use it for all my non-gc 
objects like this watered down example:


class ManualObject : Object {
new(size_t size) { return malloc(size); }
delete(void* mem) { free(mem); }
}

And then I can easily subclass it for any objects that doesn't need the 
GC. I've got similar constructs for arrays and structs.


Clearly you use those objects in a very different manner than GC 
objects. So by using new/delete with them you're fooling yourself.


// untested
class ManualObject {
static T create(T : ManualObject)() {
auto p = malloc(__traits(classInstanceSize, T));
memcpy(p, T.classinfo.init.ptr, __traits(classInstanceSize, T));
auto result = cast(T) p;
result.__ctor();
return result;
}
static void yank(ManualObject obj) {
free(cast(void*) obj);
}
}

Looks like a fair amount of work? At some level it actually should, but 
we can put that kind of stuff in the standard library.


malloc/free are nice, but they don't allow for elegant abstractions like 
new/delete does (for example if you want to use a specialized non-gc 
allocator you can just replace a few calls instead of every allocation).


They do if you're willing to write just a bit of scaffolding.

I also use delete when I no longer need large blocks of memory, I don't 
want them to just become uninitialized and sitting on the GC. When I 
want to do that I just nullify my references.


If you're afraid of deleting an object that may still have valid 
references, use smart pointers, or don't delete it at all if it sits on 
the gc and just call a .destroy() method.


Also in my runtime the delete implementations do free the memory, they 
don't just call the finalizer.


In any ways, just don't remove new/delete overrides from the language 
please, just call it a low-level technique or something to scare the 
beginners away and let people who want it have it :)


I strongly believe custom new/delete must go.


Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Michel Fortin wrote:
On 2009-10-07 08:46:06 -0400, Andrei Alexandrescu 
 said:


You're right. It would be great to dispose of the delete keyword and 
define a member function and/or a free function that invokes the 
destructor and obliterates the object with its .init bits.


I guess I should have read this before posting mine. :-)

You're suggesting obiterating with the .init bits, but I believe this is 
insufficient: you need to call a constructor if you want to be sure 
object invariants holds. If you can't make the invariants hold, you're 
in undefined behaviour territory.


That is correct. The default constructor must be called for classes. For 
structs, copying .init over will do.


At any rate: deletion + memory reclamation must go. If you want to do 
manual memory management, malloc/free are yours. D's native GC heap is 
not the right place.


Well, yes you're entirely right saying that. But I fail to see how this 
is linked to class allocators and deallocators.


Discussion took a turn.

Class allocators and 
deallocators are just a way to tell the runtime (including the GC) how 
to allocate and deallocate a specific class of objects. There is no need 
to manually call delete for the allocator and deallocator to be useful.


The way it is currently, if you want objects of a certain class to be 
allocated in one big object pool, you can encapsulate that detail in the 
class so clients don't have to bother about it. I've done that in C++ to 
speed up things without having to touch the rest of the code base and 
it's quite handy.


At other times the client of the class that wants to manage memory, and 
that should be allowed too, bypassing the class's allocator and 
deallocator and calling directly the constructor and destructor.


I agree that some would want to manage their own allocation, and see no 
fault with a pool that exposes factory methods a la create() and 
recycle() or whatever.


The language has become larger and more powerful. Now we're in an odd 
situation: the language has become powerful enough to render obsolete 
some things that previously were in the language because they couldn't 
be expressed. Consider a factory method create(). In the olden days, 
there was no way to properly forward variadic arguments to an object's 
constructor. So repeating C++'s awful hack seemed like a reasonable 
thing to do. Now even the new keyword isn't that justified because a 
simple function could do everything new does, plus custom allocation and 
whatever if we so want.


Walter, Don and myself are looking into ways of making the language 
smaller and moving some of built-in functionality to the standard 
library. Tomasz' post on making an in-situ class instance was a 
watershed point for me. I thought about it some more and realized that 
language size and library size aren't the same thing. (I had a feeling 
before that, but no good argument.)


Language is not modular and doesn't have well-defined boundaries that 
carve subunits. Libraries do. I can always say "I will/won't use this 
module/package/library" but the language just comes at you in parallel. 
Conversely, if you see something you don't know in some code and it's in 
a library, you can always decide to look at that library's code and/or 
documentation and figure out what's what. In contrast, if I saw a 
highlighted keyword that I had no idea what it does I'd get quite worried.



Andrei

Re: Eliminate class allocators and deallocators?

[dsimcha]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> It is a bad idea because distinguishing between release of (expensive)
> resources from dangerous memory recycling is the correct way to obtain
> deterministic resource management within the confines of safety.

This is based on two faulty assumptions:

1.  Memory is cheap.  (Not if you are working with absurd amounts of data).
2.  Garbage collection is never a major bottleneck.  (Sometimes it's a 
worthwhile
tradeoff to add a few manual delete statements to code and sacrifice some safety
for making the GC run less often.)

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Leandro Lucarella wrote:

Andrei Alexandrescu, el  6 de octubre a las 21:42 me escribiste:

should call a destructor but not call delete() or notify the GC
that the memory is free.

That is correct. In particular, an object remains usable after delete.

Why would you do that? What is the rationale to not notify the GC?

Because there may be other live references to the object.


But when using delete that's exactly what it should happen. You are hiding
a bug if you let that happen on purpose.


That is not hiding a bug. That's even worse than Walter's crappy 
argument :o).



You're saying that there is a problem, but you're not telling us
what's wrong. Why the hell do you want to destroy an object
without recycling its memory? Why does the inability to do so
cause a problem?

The matter has been discussed quite a bit around here and in other
places. I'm not having as much time as I'd want to explain things.
In short, destroying without freeing memory avoids dangling
references and preserves memory safety without impacting on other
resources.

But D is a system programming language.

Well it is but there are quite a few more things at stake. First, it
is a reality that it is often desirable to distinguish between
calling the destructor and reclaiming memory. D's current delete
continues the bad tradition started by C++ of conflating the two.


Why is a bad idea? If you are destroying an object, the object will be in
an inconsistent state. What's the point of keeping it alive. Again, you're
just hiding a bug; letting the bug live longer. The language should try to
expose bugs ASAP, not delay the detection.


It is a bad idea because distinguishing between release of (expensive) 
resources from dangerous memory recycling is the correct way to obtain 
deterministic resource management within the confines of safety.



I think is a good idea not to force the GC to free the memory immediately
with a delete, but it should if it's easy. Other protection methods as
using mprotect to protect the objects pages it's very desirable too,
because you can spot an access to a inconsistent (destroyed) object as
soon as it first happen.


(mprotect is much too coarse to be useful.) With the dispose() function 
the state of the object will be restored to default construction:


void dispose(T)(T obj) if (is(T == class) || is(typeof(*T.init))) {
   ... call destructor if any ...
   ... obliterate object with .init ...
   ... invoke default ctor if any ...
}


If you wrote delete x; the
language should assume you know what you're doing.

I think delete should be present in SafeD and if you want manual
memory management you should build on malloc and free.


If you want to introduce a new semantic, I think you should provide a new
method, not change the semantic of an existent one.


Agreed. I hereby vote for deprecating delete with extreme prejudice.


And BTW, is there any reason why this can't be implemented in the library
instead of using an operator? Why don't you provide a "destroy()" function
for that in Phobos?


That sounds great.


Really, I can't see any advantages on changing the delete operator
semantics, only problems.


I agree.


If you only want to
"deinitialize" an object, you can write a .destroy() method for example,
and call that. I think delete have a strong established semantic to change
it now, and without any gain.

It has a thoroughly broken and undesired semantics. It would be a step
forward to divorce it of that.


Why it's broken? Why it's undesired?


(See above in this message.)


Why? Using malloc and free is a lot more trouble, you have to register the
roots yourself for example. It's not like you do malloc() and free() and
everything works magically. You have to have more knowledge of the GC to
use them. Being able to manually manage the *GC* heap (if the GC support
that, if not it can make it a NOP) is good IMHO.


We can make things a tad better with library functions, but we do need 
to have a garbage collected heap that guarantees safety.



Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Michel Fortin wrote:
On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu 
 said:


The matter has been discussed quite a bit around here and in other 
places. I'm not having as much time as I'd want to explain things. In 
short, destroying without freeing memory avoids dangling references 
and preserves memory safety without impacting on other resources.


It's a safety hack, not a performance hack.


In my opinion, it's mostly an illusion of safety. If you call the 
destructor on an object, the object state after the call doesn't 
necessarily respects the object invariants and doing anything with it 
could result in, well, anything, from returning wrong results to falling 
into an infinite loop (basically undefined behaviour). What you gain is 
that no object will be allocated on top of the old one, and thus new 
objects can't get corrupted. But it's still undefined behaviour, only 
with less side effects and more memory consumption.


I don't think it's a so bad idea on the whole, but it'd be more valuable 
if accessing an invalidated object could be made an error instead of 
undefined behaviour. If this can't be done, then we should encourage 
"destructors" to put the object in a clean state and not leave any dirt 
behind. But should that still be called a "destructor"?


Perhaps we could change the paradigm a little and replace "deletion" 
with "recycling". Recycling an object would call the destructor and 
immeditately call the default constructor, so the object is never left 
in an invalid state. Objects with no default constructor cannot be 
recycled. This way you know memory is always left in a clean state, and 
you encourage programmers to safely reuse the memory blocks from objects 
they have already allocated when possible.


Yes, recycling is best and I'm considering it. I'm only worried about 
the extra cost.


Andrei

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

downs wrote:

Don wrote:

downs wrote:

Andrei Alexandrescu wrote:

downs wrote:

Andrei Alexandrescu wrote:

downs wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They
have
a number of problems that make them unsuitable for D 2.0. The most
obvious issue is that D 2.0 will _not_ conflate destruction with
deallocation anymore: invoking delete against an object will call
~this() against it but will not recycle its memory. In contrast,
class
deallocators are designed around the idea that invoking delete
calls the
destructor and also deallocates memory.

So I'm thinking of removing at least class deallocators from the
language. Class allocators may be marginally and occasionally
useful if
the user takes the matter of deallocation in her own hands.

A much better way to handle custom allocation of classes would be
in the
standard library.

What do you think?


Andrei

Do you trust the D GC to be good enough to always free everything
you've allocated, without error?

If your answer was 'ye- maaybe ... no actually', please rethink this.

People will always be able to call functions in the garbage collector
manually. The discussion on class allocators and deallocators has
nothing to do with that.

Andrei

So you can still deallocate a class by hand, only it's not called
delete anymore?

That is correct.

Andrei

Isn't that a pretty big violation of Least Surprise?

http://en.wikipedia.org/wiki/Principle_of_least_astonishment :
"In user interface design, programming language design, and
ergonomics, the principle (or rule or law) of least astonishment (or
surprise) states that, when two elements of an interface conflict, or
are ambiguous, the behaviour should be that which will *least
surprise* the human user or programmer at the time the conflict arises."

I think the basic rule being introduced is:
that every object can be managed by the gc, or manually managed. But not
both. That seems reasonable to me. But if delete no longer deletes, it
needs a name change.



Oh, that makes more sense.

Do manually managed objects still count under Mark&Sweep?


You have to register the memory range they cover to the GC if they 
contain pointers to GC memory. Otherwise the GC don't know they exist at 
all.

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

Sean Kelly wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

downs wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They have
a number of problems that make them unsuitable for D 2.0. The most
obvious issue is that D 2.0 will _not_ conflate destruction with
deallocation anymore: invoking delete against an object will call
~this() against it but will not recycle its memory. In contrast, class
deallocators are designed around the idea that invoking delete calls the
destructor and also deallocates memory.

So I'm thinking of removing at least class deallocators from the
language. Class allocators may be marginally and occasionally useful if
the user takes the matter of deallocation in her own hands.

A much better way to handle custom allocation of classes would be in the
standard library.

What do you think?


Andrei

Do you trust the D GC to be good enough to always free everything you've 
allocated, without error?

If your answer was 'ye- maaybe ... no actually', please rethink this.

People will always be able to call functions in the garbage collector
manually. The discussion on class allocators and deallocators has
nothing to do with that.


Right.  There's no plan to eliminate GC.free().


But that's runtime dependent, for example on my runtime its 
Memory.Free(). Removing 'delete' would therefore bind the code to a 
certain runtime, that's not a very portable solution, and far from being 
as elegant as delete.

Re: Eliminate class allocators and deallocators?

[Jeremie Pelletier]

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They have 
a number of problems that make them unsuitable for D 2.0. The most 
obvious issue is that D 2.0 will _not_ conflate destruction with 
deallocation anymore: invoking delete against an object will call 
~this() against it but will not recycle its memory. In contrast, class 
deallocators are designed around the idea that invoking delete calls the 
destructor and also deallocates memory.


So I'm thinking of removing at least class deallocators from the 
language. Class allocators may be marginally and occasionally useful if 
the user takes the matter of deallocation in her own hands.


A much better way to handle custom allocation of classes would be in the 
standard library.


What do you think?


Andrei


I wouldn't like delete to go away at all, I use it for all my non-gc 
objects like this watered down example:


class ManualObject : Object {
new(size_t size) { return malloc(size); }
delete(void* mem) { free(mem); }
}

And then I can easily subclass it for any objects that doesn't need the 
GC. I've got similar constructs for arrays and structs.


malloc/free are nice, but they don't allow for elegant abstractions like 
new/delete does (for example if you want to use a specialized non-gc 
allocator you can just replace a few calls instead of every allocation).


I also use delete when I no longer need large blocks of memory, I don't 
want them to just become uninitialized and sitting on the GC. When I 
want to do that I just nullify my references.


If you're afraid of deleting an object that may still have valid 
references, use smart pointers, or don't delete it at all if it sits on 
the gc and just call a .destroy() method.


Also in my runtime the delete implementations do free the memory, they 
don't just call the finalizer.


In any ways, just don't remove new/delete overrides from the language 
please, just call it a low-level technique or something to scare the 
beginners away and let people who want it have it :)


Jeremie

Re: Eliminate class allocators and deallocators?

[Sean Kelly]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> downs wrote:
> > Andrei Alexandrescu wrote:
> >> Hello,
> >>
> >>
> >> D currently allows defining class allocators and deallocators. They have
> >> a number of problems that make them unsuitable for D 2.0. The most
> >> obvious issue is that D 2.0 will _not_ conflate destruction with
> >> deallocation anymore: invoking delete against an object will call
> >> ~this() against it but will not recycle its memory. In contrast, class
> >> deallocators are designed around the idea that invoking delete calls the
> >> destructor and also deallocates memory.
> >>
> >> So I'm thinking of removing at least class deallocators from the
> >> language. Class allocators may be marginally and occasionally useful if
> >> the user takes the matter of deallocation in her own hands.
> >>
> >> A much better way to handle custom allocation of classes would be in the
> >> standard library.
> >>
> >> What do you think?
> >>
> >>
> >> Andrei
> >
> > Do you trust the D GC to be good enough to always free everything you've 
> > allocated, without error?
> >
> > If your answer was 'ye- maaybe ... no actually', please rethink this.
> People will always be able to call functions in the garbage collector
> manually. The discussion on class allocators and deallocators has
> nothing to do with that.

Right.  There's no plan to eliminate GC.free().

Re: Eliminate class allocators and deallocators?

[Michel Fortin]

On 2009-10-07 08:46:06 -0400, Andrei Alexandrescu 
 said:


You're right. It would be great to dispose of the delete keyword and 
define a member function and/or a free function that invokes the 
destructor and obliterates the object with its .init bits.


I guess I should have read this before posting mine. :-)

You're suggesting obiterating with the .init bits, but I believe this 
is insufficient: you need to call a constructor if you want to be sure 
object invariants holds. If you can't make the invariants hold, you're 
in undefined behaviour territory.



At any rate: deletion + memory reclamation must go. If you want to do 
manual memory management, malloc/free are yours. D's native GC heap is 
not the right place.


Well, yes you're entirely right saying that. But I fail to see how this 
is linked to class allocators and deallocators. Class allocators and 
deallocators are just a way to tell the runtime (including the GC) how 
to allocate and deallocate a specific class of objects. There is no 
need to manually call delete for the allocator and deallocator to be 
useful.


The way it is currently, if you want objects of a certain class to be 
allocated in one big object pool, you can encapsulate that detail in 
the class so clients don't have to bother about it. I've done that in 
C++ to speed up things without having to touch the rest of the code 
base and it's quite handy.


At other times the client of the class that wants to manage memory, and 
that should be allowed too, bypassing the class's allocator and 
deallocator and calling directly the constructor and destructor.


--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  6 de octubre a las 21:42 me escribiste:
> >>>should call a destructor but not call delete() or notify the GC
> >>>that the memory is free.
> >>That is correct. In particular, an object remains usable after delete.
> >
> >Why would you do that? What is the rationale to not notify the GC?
> 
> Because there may be other live references to the object.

But when using delete that's exactly what it should happen. You are hiding
a bug if you let that happen on purpose.

> >>>You're saying that there is a problem, but you're not telling us
> >>>what's wrong. Why the hell do you want to destroy an object
> >>>without recycling its memory? Why does the inability to do so
> >>>cause a problem?
> >>The matter has been discussed quite a bit around here and in other
> >>places. I'm not having as much time as I'd want to explain things.
> >>In short, destroying without freeing memory avoids dangling
> >>references and preserves memory safety without impacting on other
> >>resources.
> >
> >But D is a system programming language.
> 
> Well it is but there are quite a few more things at stake. First, it
> is a reality that it is often desirable to distinguish between
> calling the destructor and reclaiming memory. D's current delete
> continues the bad tradition started by C++ of conflating the two.

Why is a bad idea? If you are destroying an object, the object will be in
an inconsistent state. What's the point of keeping it alive. Again, you're
just hiding a bug; letting the bug live longer. The language should try to
expose bugs ASAP, not delay the detection.

I think is a good idea not to force the GC to free the memory immediately
with a delete, but it should if it's easy. Other protection methods as
using mprotect to protect the objects pages it's very desirable too,
because you can spot an access to a inconsistent (destroyed) object as
soon as it first happen.

> >If you wrote delete x; the
> >language should assume you know what you're doing.
> 
> I think delete should be present in SafeD and if you want manual
> memory management you should build on malloc and free.

If you want to introduce a new semantic, I think you should provide a new
method, not change the semantic of an existent one.

And BTW, is there any reason why this can't be implemented in the library
instead of using an operator? Why don't you provide a "destroy()" function
for that in Phobos?

Really, I can't see any advantages on changing the delete operator
semantics, only problems.

> >If you only want to
> >"deinitialize" an object, you can write a .destroy() method for example,
> >and call that. I think delete have a strong established semantic to change
> >it now, and without any gain.
> 
> It has a thoroughly broken and undesired semantics. It would be a step
> forward to divorce it of that.

Why it's broken? Why it's undesired?

> In fact i'd love to simply make delete disappear as a keyword and make
> it a function.

I agree on this one, no need for an operator (AFAIK). But again, I don't
see how letting the user to use a destroyed object is any safer. It's
really bad in fact.

> >>>It seems like a performance hack to me -- you've got an object
> >>>that isn't valid anymore, but you want to hang on to the memory
> >>>for some other purpose. And you could override new() and delete(),
> >>>but you don't want to incur the performance penalty of calling the
> >>>runtime to fetch the deallocator.
> >>It's a safety hack, not a performance hack.
> >
> >But you shouldn't provide safety where the programmer is not expecting it.
> >delete is for *manual* memory management. It makes no sense to guarantee
> >that the memory is *not* freed. It makes sense not guaranteeing that it
> >will actually be freed either. I think that's a good idea actually,
> >because it gives more flexibility to the GC implementation.
> 
> I think we should move away from the idea that delete is for manual
> memory management. We should leave that to the likes of malloc and
> free alone.

Why? Using malloc and free is a lot more trouble, you have to register the
roots yourself for example. It's not like you do malloc() and free() and
everything works magically. You have to have more knowledge of the GC to
use them. Being able to manually manage the *GC* heap (if the GC support
that, if not it can make it a NOP) is good IMHO.

> >>>The only remaining use that I see is a way to reset a shared
> >>>object without explicitly passing around a reference to the new
> >>>version of the object. This seems potentially dangerous, and
> >>>nothing I want for default behavior.
> >>Well incidentally at least as of now "delete obj" puts null in obj...
> >
> >That's nice :)
> 
> I think it's a false sense of security.

Why it's bad for D? (I don't care that much about C++ reasons :)

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 

Re: Eliminate class allocators and deallocators?

[Michel Fortin]

On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu 
 said:


The matter has been discussed quite a bit around here and in other 
places. I'm not having as much time as I'd want to explain things. In 
short, destroying without freeing memory avoids dangling references and 
preserves memory safety without impacting on other resources.


It's a safety hack, not a performance hack.


In my opinion, it's mostly an illusion of safety. If you call the 
destructor on an object, the object state after the call doesn't 
necessarily respects the object invariants and doing anything with it 
could result in, well, anything, from returning wrong results to 
falling into an infinite loop (basically undefined behaviour). What you 
gain is that no object will be allocated on top of the old one, and 
thus new objects can't get corrupted. But it's still undefined 
behaviour, only with less side effects and more memory consumption.


I don't think it's a so bad idea on the whole, but it'd be more 
valuable if accessing an invalidated object could be made an error 
instead of undefined behaviour. If this can't be done, then we should 
encourage "destructors" to put the object in a clean state and not 
leave any dirt behind. But should that still be called a "destructor"?


Perhaps we could change the paradigm a little and replace "deletion" 
with "recycling". Recycling an object would call the destructor and 
immeditately call the default constructor, so the object is never left 
in an invalid state. Objects with no default constructor cannot be 
recycled. This way you know memory is always left in a clean state, and 
you encourage programmers to safely reuse the memory blocks from 
objects they have already allocated when possible.


--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Kagamin wrote:

I don't see any problem with dispose() method (except that it doesn't
nullifies the pointer, which can be a performance issue for some GC
implementations). If you plan to go C# way, it's reasonable to adopt
its techniques of destruction. Moreover C# and C++ approaches are
compatible. If the programmer doesn't guarantee ownership of the
object, it's just unreasonable to call delete, here adding the
dispose() method to the Object and using it for destruction will
help.

Your proposal is indeed better than the scheme above and it's not a
pain to implement and use destruct+free function, but delete and
dispose are already well-known idioms, as you were already told
about.


You're right. It would be great to dispose of the delete keyword and 
define a member function and/or a free function that invokes the 
destructor and obliterates the object with its .init bits.


At any rate: deletion + memory reclamation must go. If you want to do 
manual memory management, malloc/free are yours. D's native GC heap is 
not the right place.



Andrei

Re: Eliminate class allocators and deallocators?

[Christopher Wright]

Andrei Alexandrescu wrote:

Christopher Wright wrote:

What exactly is your suggestion?

It seems that you mean that:
delete obj;

should call a destructor but not call delete() or notify the GC that 
the memory is free.


That is correct. In particular, an object remains usable after delete.

You're saying that there is a problem, but you're not telling us 
what's wrong. Why the hell do you want to destroy an object without 
recycling its memory? Why does the inability to do so cause a problem?


The matter has been discussed quite a bit around here and in other 
places. I'm not having as much time as I'd want to explain things. In 
short, destroying without freeing memory avoids dangling references and 
preserves memory safety without impacting on other resources.


Memory safety, sure, but you're deleting the object. It is no longer 
valid. You need to add a flag to the object indicating it's invalid, and 
everything that uses it needs to check that flag. Instead of a probable 
segfault in the current system, you'll get strange errors.


It sounds like a complicated way of supporting a rare use case. Why not 
use a library solution? Make an IDisposable interface with methods "void 
dispose()" and "bool disposed()"?


If you don't have enough time to explain the reasoning, could you post a 
link to a more detailed explanation?

Re: Eliminate class allocators and deallocators?

[downs]

Don wrote:
> downs wrote:
>> Andrei Alexandrescu wrote:
>>> downs wrote:
 Andrei Alexandrescu wrote:
> downs wrote:
>> Andrei Alexandrescu wrote:
>>> Hello,
>>>
>>>
>>> D currently allows defining class allocators and deallocators. They
>>> have
>>> a number of problems that make them unsuitable for D 2.0. The most
>>> obvious issue is that D 2.0 will _not_ conflate destruction with
>>> deallocation anymore: invoking delete against an object will call
>>> ~this() against it but will not recycle its memory. In contrast,
>>> class
>>> deallocators are designed around the idea that invoking delete
>>> calls the
>>> destructor and also deallocates memory.
>>>
>>> So I'm thinking of removing at least class deallocators from the
>>> language. Class allocators may be marginally and occasionally
>>> useful if
>>> the user takes the matter of deallocation in her own hands.
>>>
>>> A much better way to handle custom allocation of classes would be
>>> in the
>>> standard library.
>>>
>>> What do you think?
>>>
>>>
>>> Andrei
>> Do you trust the D GC to be good enough to always free everything
>> you've allocated, without error?
>>
>> If your answer was 'ye- maaybe ... no actually', please rethink this.
> People will always be able to call functions in the garbage collector
> manually. The discussion on class allocators and deallocators has
> nothing to do with that.
>
> Andrei
 So you can still deallocate a class by hand, only it's not called
 delete anymore?
>>> That is correct.
>>>
>>> Andrei
>>
>> Isn't that a pretty big violation of Least Surprise?
>>
>> http://en.wikipedia.org/wiki/Principle_of_least_astonishment :
>> "In user interface design, programming language design, and
>> ergonomics, the principle (or rule or law) of least astonishment (or
>> surprise) states that, when two elements of an interface conflict, or
>> are ambiguous, the behaviour should be that which will *least
>> surprise* the human user or programmer at the time the conflict arises."
> 
> I think the basic rule being introduced is:
> that every object can be managed by the gc, or manually managed. But not
> both. That seems reasonable to me. But if delete no longer deletes, it
> needs a name change.


Oh, that makes more sense.

Do manually managed objects still count under Mark&Sweep?

Re: Eliminate class allocators and deallocators?

[Don]

downs wrote:

Andrei Alexandrescu wrote:

downs wrote:

Andrei Alexandrescu wrote:

downs wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They
have
a number of problems that make them unsuitable for D 2.0. The most
obvious issue is that D 2.0 will _not_ conflate destruction with
deallocation anymore: invoking delete against an object will call
~this() against it but will not recycle its memory. In contrast, class
deallocators are designed around the idea that invoking delete
calls the
destructor and also deallocates memory.

So I'm thinking of removing at least class deallocators from the
language. Class allocators may be marginally and occasionally
useful if
the user takes the matter of deallocation in her own hands.

A much better way to handle custom allocation of classes would be
in the
standard library.

What do you think?


Andrei

Do you trust the D GC to be good enough to always free everything
you've allocated, without error?

If your answer was 'ye- maaybe ... no actually', please rethink this.

People will always be able to call functions in the garbage collector
manually. The discussion on class allocators and deallocators has
nothing to do with that.

Andrei

So you can still deallocate a class by hand, only it's not called
delete anymore?

That is correct.

Andrei


Isn't that a pretty big violation of Least Surprise?

http://en.wikipedia.org/wiki/Principle_of_least_astonishment :
"In user interface design, programming language design, and ergonomics, the 
principle (or rule or law) of least astonishment (or surprise) states that, when two 
elements of an interface conflict, or are ambiguous, the behaviour should be that which 
will *least surprise* the human user or programmer at the time the conflict arises."


I think the basic rule being introduced is:
that every object can be managed by the gc, or manually managed. But not 
both. That seems reasonable to me. But if delete no longer deletes, it 
needs a name change.

Re: Eliminate class allocators and deallocators?

[Kagamin]

I don't see any problem with dispose() method (except that it doesn't nullifies 
the pointer, which can be a performance issue for some GC implementations). If 
you plan to go C# way, it's reasonable to adopt its techniques of destruction. 
Moreover C# and C++ approaches are compatible. If the programmer doesn't 
guarantee ownership of the object, it's just unreasonable to call delete, here 
adding the dispose() method to the Object and using it for destruction will 
help.

Your proposal is indeed better than the scheme above and it's not a pain to 
implement and use destruct+free function, but delete and dispose are already 
well-known idioms, as you were already told about.

Re: Eliminate class allocators and deallocators?

[downs]

Andrei Alexandrescu wrote:
> downs wrote:
>> Andrei Alexandrescu wrote:
>>> downs wrote:
 Andrei Alexandrescu wrote:
> Hello,
>
>
> D currently allows defining class allocators and deallocators. They
> have
> a number of problems that make them unsuitable for D 2.0. The most
> obvious issue is that D 2.0 will _not_ conflate destruction with
> deallocation anymore: invoking delete against an object will call
> ~this() against it but will not recycle its memory. In contrast, class
> deallocators are designed around the idea that invoking delete
> calls the
> destructor and also deallocates memory.
>
> So I'm thinking of removing at least class deallocators from the
> language. Class allocators may be marginally and occasionally
> useful if
> the user takes the matter of deallocation in her own hands.
>
> A much better way to handle custom allocation of classes would be
> in the
> standard library.
>
> What do you think?
>
>
> Andrei
 Do you trust the D GC to be good enough to always free everything
 you've allocated, without error?

 If your answer was 'ye- maaybe ... no actually', please rethink this.
>>> People will always be able to call functions in the garbage collector
>>> manually. The discussion on class allocators and deallocators has
>>> nothing to do with that.
>>>
>>> Andrei
>>
>> So you can still deallocate a class by hand, only it's not called
>> delete anymore?
> 
> That is correct.
> 
> Andrei

Isn't that a pretty big violation of Least Surprise?

http://en.wikipedia.org/wiki/Principle_of_least_astonishment :
"In user interface design, programming language design, and ergonomics, the 
principle (or rule or law) of least astonishment (or surprise) states that, 
when two elements of an interface conflict, or are ambiguous, the behaviour 
should be that which will *least surprise* the human user or programmer at the 
time the conflict arises."

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

downs wrote:

Andrei Alexandrescu wrote:

downs wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They have
a number of problems that make them unsuitable for D 2.0. The most
obvious issue is that D 2.0 will _not_ conflate destruction with
deallocation anymore: invoking delete against an object will call
~this() against it but will not recycle its memory. In contrast, class
deallocators are designed around the idea that invoking delete calls the
destructor and also deallocates memory.

So I'm thinking of removing at least class deallocators from the
language. Class allocators may be marginally and occasionally useful if
the user takes the matter of deallocation in her own hands.

A much better way to handle custom allocation of classes would be in the
standard library.

What do you think?


Andrei

Do you trust the D GC to be good enough to always free everything
you've allocated, without error?

If your answer was 'ye- maaybe ... no actually', please rethink this.

People will always be able to call functions in the garbage collector
manually. The discussion on class allocators and deallocators has
nothing to do with that.

Andrei


So you can still deallocate a class by hand, only it's not called delete 
anymore?


That is correct.

Andrei

Re: Eliminate class allocators and deallocators?

[downs]

Andrei Alexandrescu wrote:
> downs wrote:
>> Andrei Alexandrescu wrote:
>>> Hello,
>>>
>>>
>>> D currently allows defining class allocators and deallocators. They have
>>> a number of problems that make them unsuitable for D 2.0. The most
>>> obvious issue is that D 2.0 will _not_ conflate destruction with
>>> deallocation anymore: invoking delete against an object will call
>>> ~this() against it but will not recycle its memory. In contrast, class
>>> deallocators are designed around the idea that invoking delete calls the
>>> destructor and also deallocates memory.
>>>
>>> So I'm thinking of removing at least class deallocators from the
>>> language. Class allocators may be marginally and occasionally useful if
>>> the user takes the matter of deallocation in her own hands.
>>>
>>> A much better way to handle custom allocation of classes would be in the
>>> standard library.
>>>
>>> What do you think?
>>>
>>>
>>> Andrei
>>
>> Do you trust the D GC to be good enough to always free everything
>> you've allocated, without error?
>>
>> If your answer was 'ye- maaybe ... no actually', please rethink this.
> 
> People will always be able to call functions in the garbage collector
> manually. The discussion on class allocators and deallocators has
> nothing to do with that.
> 
> Andrei

So you can still deallocate a class by hand, only it's not called delete 
anymore?

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

downs wrote:

Andrei Alexandrescu wrote:

Hello,


D currently allows defining class allocators and deallocators. They have
a number of problems that make them unsuitable for D 2.0. The most
obvious issue is that D 2.0 will _not_ conflate destruction with
deallocation anymore: invoking delete against an object will call
~this() against it but will not recycle its memory. In contrast, class
deallocators are designed around the idea that invoking delete calls the
destructor and also deallocates memory.

So I'm thinking of removing at least class deallocators from the
language. Class allocators may be marginally and occasionally useful if
the user takes the matter of deallocation in her own hands.

A much better way to handle custom allocation of classes would be in the
standard library.

What do you think?


Andrei


Do you trust the D GC to be good enough to always free everything you've 
allocated, without error?

If your answer was 'ye- maaybe ... no actually', please rethink this.


People will always be able to call functions in the garbage collector 
manually. The discussion on class allocators and deallocators has 
nothing to do with that.


Andrei

Re: Eliminate class allocators and deallocators?

[downs]

Andrei Alexandrescu wrote:
> Hello,
> 
> 
> D currently allows defining class allocators and deallocators. They have
> a number of problems that make them unsuitable for D 2.0. The most
> obvious issue is that D 2.0 will _not_ conflate destruction with
> deallocation anymore: invoking delete against an object will call
> ~this() against it but will not recycle its memory. In contrast, class
> deallocators are designed around the idea that invoking delete calls the
> destructor and also deallocates memory.
> 
> So I'm thinking of removing at least class deallocators from the
> language. Class allocators may be marginally and occasionally useful if
> the user takes the matter of deallocation in her own hands.
> 
> A much better way to handle custom allocation of classes would be in the
> standard library.
> 
> What do you think?
> 
> 
> Andrei

Do you trust the D GC to be good enough to always free everything you've 
allocated, without error?

If your answer was 'ye- maaybe ... no actually', please rethink this.

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  6 de octubre a las 21:36 me escribiste:
> >I don't think it is a good idea (GC-wise) to say that in the specs.
> >I think the GC implementor should be free to decide if a delete really
> >free the memory or not. Some collectors can do this very naturally (like
> >the current one) and some others don't (like allocators that uses
> >pointer-bump allocation). I think the language should divide destruction
> >and deallocation, but I don't think is a good idea not to notify the GC at
> >all when delete is used. I think the GC should be able to do whatever it
> >feels is good for him (so the user should not rely either on the memory
> >being actually freed or otherwise).
> >
> 
> I agree insofar as a GC could be tipped by the compiler that no live
> reference of the object exists after delete.

Great! For example, this would let me protect the object pages (if it's
a large object that uses one or more full pages) when they are freed so
the program segfaults as soon as a deleted object is used when it
shouldn't. That could be a nice debugging feature :)

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
Hey you, with you ear against the wall
Waiting for someone to call out
Would you touch me?

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Leandro Lucarella wrote:

Andrei Alexandrescu, el  6 de octubre a las 19:26 me escribiste:

Christopher Wright wrote:

What exactly is your suggestion?

It seems that you mean that:
delete obj;

should call a destructor but not call delete() or notify the GC
that the memory is free.

That is correct. In particular, an object remains usable after delete.


Why would you do that? What is the rationale to not notify the GC?


Because there may be other live references to the object.


You're saying that there is a problem, but you're not telling us
what's wrong. Why the hell do you want to destroy an object
without recycling its memory? Why does the inability to do so
cause a problem?

The matter has been discussed quite a bit around here and in other
places. I'm not having as much time as I'd want to explain things.
In short, destroying without freeing memory avoids dangling
references and preserves memory safety without impacting on other
resources.


But D is a system programming language.


Well it is but there are quite a few more things at stake. First, it is 
a reality that it is often desirable to distinguish between calling the 
destructor and reclaiming memory. D's current delete continues the bad 
tradition started by C++ of conflating the two.



If you wrote delete x; the
language should assume you know what you're doing.


I think delete should be present in SafeD and if you want manual memory 
management you should build on malloc and free.



If you only want to
"deinitialize" an object, you can write a .destroy() method for example,
and call that. I think delete have a strong established semantic to change
it now, and without any gain.


It has a thoroughly broken and undesired semantics. It would be a step 
forward to divorce it of that. In fact i'd love to simply make delete 
disappear as a keyword and make it a function.



It seems like a performance hack to me -- you've got an object
that isn't valid anymore, but you want to hang on to the memory
for some other purpose. And you could override new() and delete(),
but you don't want to incur the performance penalty of calling the
runtime to fetch the deallocator.

It's a safety hack, not a performance hack.


But you shouldn't provide safety where the programmer is not expecting it.
delete is for *manual* memory management. It makes no sense to guarantee
that the memory is *not* freed. It makes sense not guaranteeing that it
will actually be freed either. I think that's a good idea actually,
because it gives more flexibility to the GC implementation.


I think we should move away from the idea that delete is for manual 
memory management. We should leave that to the likes of malloc and free 
alone.



The only remaining use that I see is a way to reset a shared
object without explicitly passing around a reference to the new
version of the object. This seems potentially dangerous, and
nothing I want for default behavior.

Well incidentally at least as of now "delete obj" puts null in obj...


That's nice :)


I think it's a false sense of security. C++ beginners keep on suggesting 
that feature and C++ pundits keep on explaining them that it's ungainful.



Andrei

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Leandro Lucarella wrote:

Andrei Alexandrescu, el  6 de octubre a las 11:01 me escribiste:

Hello,


D currently allows defining class allocators and deallocators. They
have a number of problems that make them unsuitable for D 2.0. The
most obvious issue is that D 2.0 will _not_ conflate destruction
with deallocation anymore: invoking delete against an object will
call ~this() against it but will not recycle its memory.


I don't think it is a good idea (GC-wise) to say that in the specs.
I think the GC implementor should be free to decide if a delete really
free the memory or not. Some collectors can do this very naturally (like
the current one) and some others don't (like allocators that uses
pointer-bump allocation). I think the language should divide destruction
and deallocation, but I don't think is a good idea not to notify the GC at
all when delete is used. I think the GC should be able to do whatever it
feels is good for him (so the user should not rely either on the memory
being actually freed or otherwise).



I agree insofar as a GC could be tipped by the compiler that no live 
reference of the object exists after delete.


Andrei

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  6 de octubre a las 19:26 me escribiste:
> Christopher Wright wrote:
> >What exactly is your suggestion?
> >
> >It seems that you mean that:
> >delete obj;
> >
> >should call a destructor but not call delete() or notify the GC
> >that the memory is free.
> 
> That is correct. In particular, an object remains usable after delete.

Why would you do that? What is the rationale to not notify the GC?

> >You're saying that there is a problem, but you're not telling us
> >what's wrong. Why the hell do you want to destroy an object
> >without recycling its memory? Why does the inability to do so
> >cause a problem?
> 
> The matter has been discussed quite a bit around here and in other
> places. I'm not having as much time as I'd want to explain things.
> In short, destroying without freeing memory avoids dangling
> references and preserves memory safety without impacting on other
> resources.

But D is a system programming language. If you wrote delete x; the
language should assume you know what you're doing. If you only want to
"deinitialize" an object, you can write a .destroy() method for example,
and call that. I think delete have a strong established semantic to change
it now, and without any gain.

> >It seems like a performance hack to me -- you've got an object
> >that isn't valid anymore, but you want to hang on to the memory
> >for some other purpose. And you could override new() and delete(),
> >but you don't want to incur the performance penalty of calling the
> >runtime to fetch the deallocator.
> 
> It's a safety hack, not a performance hack.

But you shouldn't provide safety where the programmer is not expecting it.
delete is for *manual* memory management. It makes no sense to guarantee
that the memory is *not* freed. It makes sense not guaranteeing that it
will actually be freed either. I think that's a good idea actually,
because it gives more flexibility to the GC implementation.

> >The only remaining use that I see is a way to reset a shared
> >object without explicitly passing around a reference to the new
> >version of the object. This seems potentially dangerous, and
> >nothing I want for default behavior.
> 
> Well incidentally at least as of now "delete obj" puts null in obj...

That's nice :)

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
Never let a fool kiss you, or let a kiss fool you

Re: Eliminate class allocators and deallocators?

[Leandro Lucarella]

Andrei Alexandrescu, el  6 de octubre a las 11:01 me escribiste:
> Hello,
> 
> 
> D currently allows defining class allocators and deallocators. They
> have a number of problems that make them unsuitable for D 2.0. The
> most obvious issue is that D 2.0 will _not_ conflate destruction
> with deallocation anymore: invoking delete against an object will
> call ~this() against it but will not recycle its memory.

I don't think it is a good idea (GC-wise) to say that in the specs.
I think the GC implementor should be free to decide if a delete really
free the memory or not. Some collectors can do this very naturally (like
the current one) and some others don't (like allocators that uses
pointer-bump allocation). I think the language should divide destruction
and deallocation, but I don't think is a good idea not to notify the GC at
all when delete is used. I think the GC should be able to do whatever it
feels is good for him (so the user should not rely either on the memory
being actually freed or otherwise).

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
If you don't know what direction you should take
You don't know where you are

Re: Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Christopher Wright wrote:

What exactly is your suggestion?

It seems that you mean that:
delete obj;

should call a destructor but not call delete() or notify the GC that the 
memory is free.


That is correct. In particular, an object remains usable after delete.

You're saying that there is a problem, but you're not telling us what's 
wrong. Why the hell do you want to destroy an object without recycling 
its memory? Why does the inability to do so cause a problem?


The matter has been discussed quite a bit around here and in other 
places. I'm not having as much time as I'd want to explain things. In 
short, destroying without freeing memory avoids dangling references and 
preserves memory safety without impacting on other resources.


It seems like a performance hack to me -- you've got an object that 
isn't valid anymore, but you want to hang on to the memory for some 
other purpose. And you could override new() and delete(), but you don't 
want to incur the performance penalty of calling the runtime to fetch 
the deallocator.


It's a safety hack, not a performance hack.

The only remaining use that I see is a way to reset a shared object 
without explicitly passing around a reference to the new version of the 
object. This seems potentially dangerous, and nothing I want for default 
behavior.


Well incidentally at least as of now "delete obj" puts null in obj...


Andrei

Re: Eliminate class allocators and deallocators?

[Christopher Wright]

What exactly is your suggestion?

It seems that you mean that:
delete obj;

should call a destructor but not call delete() or notify the GC that the 
memory is free.


You're saying that there is a problem, but you're not telling us what's 
wrong. Why the hell do you want to destroy an object without recycling 
its memory? Why does the inability to do so cause a problem?


It seems like a performance hack to me -- you've got an object that 
isn't valid anymore, but you want to hang on to the memory for some 
other purpose. And you could override new() and delete(), but you don't 
want to incur the performance penalty of calling the runtime to fetch 
the deallocator.


The only remaining use that I see is a way to reset a shared object 
without explicitly passing around a reference to the new version of the 
object. This seems potentially dangerous, and nothing I want for default 
behavior.

Re: Eliminate class allocators and deallocators?

[dsimcha]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
> Hello,
> D currently allows defining class allocators and deallocators. They have
> a number of problems that make them unsuitable for D 2.0. The most
> obvious issue is that D 2.0 will _not_ conflate destruction with
> deallocation anymore: invoking delete against an object will call
> ~this() against it but will not recycle its memory. In contrast, class
> deallocators are designed around the idea that invoking delete calls the
> destructor and also deallocates memory.
> So I'm thinking of removing at least class deallocators from the
> language. Class allocators may be marginally and occasionally useful if
> the user takes the matter of deallocation in her own hands.
> A much better way to handle custom allocation of classes would be in the
> standard library.
> What do you think?
> Andrei

Please give at least a little more detail.  I think it's an absolute must that
advanced users who really want to do manual memory management in D (even if they
want to only do it selectively and without kludges like using two completely
separate heaps) be able to do so.  I can't really comment until I know at least
roughly what a standard lib solution might look like.

Re: Eliminate class allocators and deallocators?

[Denis Koroskin]

On Tue, 06 Oct 2009 20:01:01 +0400, Andrei Alexandrescu  
 wrote:



Hello,


D currently allows defining class allocators and deallocators. They have  
a number of problems that make them unsuitable for D 2.0. The most  
obvious issue is that D 2.0 will _not_ conflate destruction with  
deallocation anymore: invoking delete against an object will call  
~this() against it but will not recycle its memory. In contrast, class  
deallocators are designed around the idea that invoking delete calls the  
destructor and also deallocates memory.


So I'm thinking of removing at least class deallocators from the  
language. Class allocators may be marginally and occasionally useful if  
the user takes the matter of deallocation in her own hands.


A much better way to handle custom allocation of classes would be in the  
standard library.


What do you think?


Andrei


I think it's okay, but I have a few points to discuss:

- Is it worth a keyword (delete) just as a shortcut for obj.__dtor();?

- I believe __ctor and __dtor (and __traits, too) are ugly reserved  
identifiers.


- It's often useful to distinguish between dtor being called by user and a  
dtor being called by a GC.
   In the latter case you can't dereference any reference since they might  
be invalid.


- How about object.d provides an IDisposable (or something like this)  
interface which defines a void dispose(bool finalizing); and we get rid of  
~this/__dtor entirely?


- obj.__ctor() is mostly used as a placement new. A new placement new  
mechanism would allow drop that identifier, too.


- D has 2 placement new mechanisms for a struct, that are absolutely the  
same (but different in syntax):



Foo foo1 = void;
Foo foo2 = void;



foo1 = Foo();  // indirect
foo2.__ctor();// direct


The latter one could also be used for class initialization, but not the  
former one.
You once announced an idea of dropping the 'new' keyword altogether and  
make the former case also valid for classes. What's about it now?


- Could you elaborate on "a much better way to handle custom allocation of  
classes ... in the standard library"? An example would be very appreciated.

Re: Eliminate class allocators and deallocators?

[Jarrett Billingsley]

On Tue, Oct 6, 2009 at 12:01 PM, Andrei Alexandrescu
 wrote:
> Hello,
>
>
> D currently allows defining class allocators and deallocators. They have a
> number of problems that make them unsuitable for D 2.0. The most obvious
> issue is that D 2.0 will _not_ conflate destruction with deallocation
> anymore: invoking delete against an object will call ~this() against it but
> will not recycle its memory. In contrast, class deallocators are designed
> around the idea that invoking delete calls the destructor and also
> deallocates memory.
>
> So I'm thinking of removing at least class deallocators from the language.
> Class allocators may be marginally and occasionally useful if the user takes
> the matter of deallocation in her own hands.
>
> A much better way to handle custom allocation of classes would be in the
> standard library.

..How? Without a custom allocator to do even a simple placement new,
how would you handle such a thing in the library?

Eliminate class allocators and deallocators?

[Andrei Alexandrescu]

Hello,


D currently allows defining class allocators and deallocators. They have 
a number of problems that make them unsuitable for D 2.0. The most 
obvious issue is that D 2.0 will _not_ conflate destruction with 
deallocation anymore: invoking delete against an object will call 
~this() against it but will not recycle its memory. In contrast, class 
deallocators are designed around the idea that invoking delete calls the 
destructor and also deallocates memory.


So I'm thinking of removing at least class deallocators from the 
language. Class allocators may be marginally and occasionally useful if 
the user takes the matter of deallocation in her own hands.


A much better way to handle custom allocation of classes would be in the 
standard library.


What do you think?


Andrei