Re: A Refcounted Array Type

2015-03-05 Thread Andrei Alexandrescu via Digitalmars-d

On 3/5/15 8:50 AM, monarch_dodra wrote:

Still, you shouldn't need end, and bounds checking would just
work.


Correct. -- Andrei


Re: A Refcounted Array Type

2015-03-05 Thread via Digitalmars-d

On Thursday, 5 March 2015 at 15:20:47 UTC, monarch_dodra wrote:
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright 
wrote:

 private:
   E[] array;
   size_t start, end;
   int* count;


What is the point of keeping start/end? Aren't those baked into 
the array slice? Not storing start end means not having to do 
index arithmetic (minor), reducing struct size (always nice). 
But more importantly, it allows implicit (and conditional) 
bounds checking (awesome), which actually runs regardless 
anyways.


Or did I miss something?


`GC.free()` needs a pointer to the start of the allocated block; 
it will not release memory if it gets an interior pointer. But as 
far as I can see, one pointer that stores the original address 
should be enough.


Re: A Refcounted Array Type

2015-03-05 Thread monarch_dodra via Digitalmars-d

On Thursday, 5 March 2015 at 16:19:09 UTC, Marc Schütz wrote:

On Thursday, 5 March 2015 at 15:20:47 UTC, monarch_dodra wrote:
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright 
wrote:

private:
  E[] array;
  size_t start, end;
  int* count;


What is the point of keeping start/end? Aren't those baked 
into the array slice? Not storing start end means not having 
to do index arithmetic (minor), reducing struct size (always 
nice). But more importantly, it allows implicit (and 
conditional) bounds checking (awesome), which actually runs 
regardless anyways.


Or did I miss something?


`GC.free()` needs a pointer to the start of the allocated 
block; it will not release memory if it gets an interior 
pointer. But as far as I can see, one pointer that stores the 
original address should be enough.


Still, you shouldn't need end, and bounds checking would just
work.


Re: A Refcounted Array Type

2015-03-05 Thread monarch_dodra via Digitalmars-d

On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

  private:
E[] array;
size_t start, end;
int* count;


What is the point of keeping start/end? Aren't those baked into 
the array slice? Not storing start end means not having to do 
index arithmetic (minor), reducing struct size (always nice). But 
more importantly, it allows implicit (and conditional) bounds 
checking (awesome), which actually runs regardless anyways.


Or did I miss something?


Re: A Refcounted Array Type

2015-03-05 Thread Kagamin via Digitalmars-d
On Wednesday, 4 March 2015 at 17:12:38 UTC, Steven Schveighoffer 
wrote:
No tiring like it is to have to explain obvious details to a 
child.


Safety system is only partially proactive in a sense it doesn't 
cover all possible scenarios, of one finds a bug, he provides an 
example, illustrating the bug (did int myglobal do it? if it was 
a child thing, why it was got wrong?). And scenarios tend to be 
very non-obvious, especially if one speaks about yet unseen bug. 
How an unknown bug can be obvious? And you believe, that 
expressing yourself in Nostradamus style greatly helps make 
things obvious? What things in safety system are on a child level 
at all? You should have a longer way to go down to a child 
argument.


Re: A Refcounted Array Type

2015-03-04 Thread Kagamin via Digitalmars-d
On Wednesday, 4 March 2015 at 15:39:43 UTC, Steven Schveighoffer 
wrote:

Wow, you really aren't getting it. I give up, it's tiring.


Tiring? Did it take that much effort to not say what you mean?


Re: A Refcounted Array Type

2015-03-04 Thread Steven Schveighoffer via Digitalmars-d

On 3/4/15 2:26 AM, Kagamin wrote:

On Tuesday, 3 March 2015 at 18:38:11 UTC, Steven Schveighoffer wrote:

int *myglobal;

repeat same code, but access what myglobal points to instead of
setting to 42. Memory corruption is what @safe code is trying to
prevent. This would lead to memory corruption.


You mean, the class destructor accesses a GC-allocated memory? That's
indeed unsafe, but unrelated to globals.


Wow, you really aren't getting it. I give up, it's tiring.

-Steve


Re: A Refcounted Array Type

2015-03-04 Thread Steven Schveighoffer via Digitalmars-d

On 3/4/15 11:02 AM, Kagamin wrote:

On Wednesday, 4 March 2015 at 15:39:43 UTC, Steven Schveighoffer wrote:

Wow, you really aren't getting it. I give up, it's tiring.


Tiring? Did it take that much effort to not say what you mean?


No tiring like it is to have to explain obvious details to a child.

In any case, either you are being intentionally obtuse or really 
ignorant. Either way, I don't have any desire to continue with this 
conversation.


-Steve


Re: A Refcounted Array Type

2015-03-03 Thread Kagamin via Digitalmars-d
On Tuesday, 3 March 2015 at 14:42:40 UTC, Steven Schveighoffer 
wrote:

   // trigger assert in A.dtor.


You mean assert breaks something?


Re: A Refcounted Array Type

2015-03-03 Thread Steven Schveighoffer via Digitalmars-d

On 3/3/15 9:57 AM, Kagamin wrote:

On Tuesday, 3 March 2015 at 14:42:40 UTC, Steven Schveighoffer wrote:

   // trigger assert in A.dtor.


You mean assert breaks something?


OK, if you want to be pedantic:

int *myglobal;

repeat same code, but access what myglobal points to instead of setting 
to 42. Memory corruption is what @safe code is trying to prevent. This 
would lead to memory corruption.


-Steve


Re: A Refcounted Array Type

2015-03-03 Thread Kagamin via Digitalmars-d
On Tuesday, 3 March 2015 at 18:38:11 UTC, Steven Schveighoffer 
wrote:

int *myglobal;

repeat same code, but access what myglobal points to instead of 
setting to 42. Memory corruption is what @safe code is trying 
to prevent. This would lead to memory corruption.


You mean, the class destructor accesses a GC-allocated memory? 
That's indeed unsafe, but unrelated to globals.


Re: A Refcounted Array Type

2015-03-03 Thread ketmar via Digitalmars-d
On Tue, 03 Mar 2015 08:53:03 -0500, Steven Schveighoffer wrote:

 Classes are expected to be GC allocated.
O_O

classes are expected to be heap-allocated, methinks, and heap is not 
necessary garbage collected.

 it's quite possible to write a GC that behaves correctly here.

it's *almost* possible, i'd say. or, in another words, it's not possible 
without turning threads to garbage-collected resource (at least it's not 
possible without havy and dirty hackery and/or support from compiler 
itself).

signature.asc
Description: PGP signature


Re: A Refcounted Array Type

2015-03-03 Thread ketmar via Digitalmars-d
On Tue, 03 Mar 2015 14:22:15 +, Kagamin wrote:

 On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:
 His case is not @safe, so we can ignore that problem.

 the compiler tends to disagree:

 === test.d ===
 int myglobal;

 class A {
   ~this () @safe { if (myglobal == 42) assert(0); }
 }

 void main () {
   auto a = new A;
 }
 ==

 dmd -w -c -o- test.d

 wow! no warnings, no errors!
 
 And what's the problem?

the answer is in your quotation.

signature.asc
Description: PGP signature


Re: A Refcounted Array Type

2015-03-03 Thread Steven Schveighoffer via Digitalmars-d

On 3/2/15 10:38 PM, ketmar wrote:

On Mon, 02 Mar 2015 15:43:42 -0500, Steven Schveighoffer wrote:


I think in this case, the compiler can probably statically reject the
code.


it can't, 'cause it doesn't know how the given object is going to be
allocated. i may completely skip GC and use my own allocator, which has
no troubles with threads, for example.



Classes are expected to be GC allocated. But the point is moot, because 
statically rejecting this does not guarantee abuse will not happen. And 
I don't think this restriction should be enshrined in the language 
anyway -- it's quite possible to write a GC that behaves correctly here.


-Steve


Re: A Refcounted Array Type

2015-03-03 Thread Kagamin via Digitalmars-d

On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:

His case is not @safe, so we can ignore that problem.


the compiler tends to disagree:

=== test.d ===
int myglobal;

class A {
  ~this () @safe { if (myglobal == 42) assert(0); }
}

void main () {
  auto a = new A;
}
==

dmd -w -c -o- test.d

wow! no warnings, no errors!


And what's the problem?


Re: A Refcounted Array Type

2015-03-03 Thread Steven Schveighoffer via Digitalmars-d

On 3/3/15 9:22 AM, Kagamin wrote:

On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:

His case is not @safe, so we can ignore that problem.


the compiler tends to disagree:

=== test.d ===
int myglobal;

class A {
  ~this () @safe { if (myglobal == 42) assert(0); }
}

void main () {
  auto a = new A;
}
==

dmd -w -c -o- test.d

wow! no warnings, no errors!


And what's the problem?


This:

void main()
{
   myglobal = 42;
   spawnThread((){myglobal = 0; auto a = new A; a = null; while(1){}});

   // trigger assert in A.dtor.
   int[] mem;
   while(1)
   {
  mem.length += 1;
   }
}

-Steve


Re: A Refcounted Array Type

2015-03-02 Thread ketmar via Digitalmars-d
On Mon, 02 Mar 2015 15:43:42 -0500, Steven Schveighoffer wrote:

 On 2/27/15 11:18 PM, ketmar wrote:
 On Fri, 27 Feb 2015 20:51:54 +, deadalnix wrote:

 On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer
 wrote:
 In that case, you shouldn't be subject to any kind of race
 conditions.
 But we can't make the library/language based on this assumption. Your
 case is the exceptional case.


 His case is not @safe, so we can ignore that problem.

 the compiler tends to disagree:

 === test.d ===
 int myglobal;

 class A {
~this () @safe { if (myglobal == 42) assert(0); }
 }

 void main () {
auto a = new A;
 }
 ==

 dmd -w -c -o- test.d

 wow! no warnings, no errors!


 I think in this case, the compiler can probably statically reject the
 code.

it can't, 'cause it doesn't know how the given object is going to be 
allocated. i may completely skip GC and use my own allocator, which has 
no troubles with threads, for example.

signature.asc
Description: PGP signature


Re: A Refcounted Array Type

2015-03-02 Thread Steven Schveighoffer via Digitalmars-d

On 2/27/15 11:18 PM, ketmar wrote:

On Fri, 27 Feb 2015 20:51:54 +, deadalnix wrote:


On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:

In that case, you shouldn't be subject to any kind of race conditions.
But we can't make the library/language based on this assumption. Your
case is the exceptional case.



His case is not @safe, so we can ignore that problem.


the compiler tends to disagree:

=== test.d ===
int myglobal;

class A {
   ~this () @safe { if (myglobal == 42) assert(0); }
}

void main () {
   auto a = new A;
}
==

dmd -w -c -o- test.d

wow! no warnings, no errors!



I think in this case, the compiler can probably statically reject the 
code. But there are definitely ways around that, so I don't think full 
compliance is possible. I also think it's wrong-headed to have the 
compiler reject such code based on a GC deficiency.


The only solution is to have GC-allocated data be destroyed in the same 
thread it's allocated in. This brings the multi-threaded program into 
the same realm as yours -- looks like one thread for unshared data :)


-Steve


Re: A Refcounted Array Type

2015-02-28 Thread via Digitalmars-d

On Saturday, 28 February 2015 at 04:27:36 UTC, weaselcat wrote:

On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:

On Fri, 27 Feb 2015 20:51:54 +, deadalnix wrote:

On Friday, 27 February 2015 at 04:13:03 UTC, Steven 
Schveighoffer wrote:
In that case, you shouldn't be subject to any kind of race 
conditions.
But we can't make the library/language based on this 
assumption. Your

case is the exceptional case.



His case is not @safe, so we can ignore that problem.


the compiler tends to disagree:

=== test.d ===
int myglobal;

class A {
 ~this () @safe { if (myglobal == 42) assert(0); }
}

void main () {
 auto a = new A;
}
==

dmd -w -c -o- test.d

wow! no warnings, no errors!


Slighty off-topic,
If DIP74 is approved it would be nice if destructors were 
redesigned(and possibly dropped altogether for 
non-deterministic lifetimes.)


Yeah, we should have distinct finalizers (with or without DIP74).


Re: A Refcounted Array Type

2015-02-27 Thread weaselcat via Digitalmars-d

On Saturday, 28 February 2015 at 04:18:38 UTC, ketmar wrote:

On Fri, 27 Feb 2015 20:51:54 +, deadalnix wrote:

On Friday, 27 February 2015 at 04:13:03 UTC, Steven 
Schveighoffer wrote:
In that case, you shouldn't be subject to any kind of race 
conditions.
But we can't make the library/language based on this 
assumption. Your

case is the exceptional case.



His case is not @safe, so we can ignore that problem.


the compiler tends to disagree:

=== test.d ===
int myglobal;

class A {
  ~this () @safe { if (myglobal == 42) assert(0); }
}

void main () {
  auto a = new A;
}
==

dmd -w -c -o- test.d

wow! no warnings, no errors!


Slighty off-topic,
If DIP74 is approved it would be nice if destructors were 
redesigned(and possibly dropped altogether for non-deterministic 
lifetimes.)


Re: A Refcounted Array Type

2015-02-27 Thread ketmar via Digitalmars-d
On Fri, 27 Feb 2015 20:51:54 +, deadalnix wrote:

 On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:
 In that case, you shouldn't be subject to any kind of race conditions.
 But we can't make the library/language based on this assumption. Your
 case is the exceptional case.


 His case is not @safe, so we can ignore that problem.

the compiler tends to disagree:

=== test.d ===
int myglobal;

class A {
  ~this () @safe { if (myglobal == 42) assert(0); }
}

void main () {
  auto a = new A;
}
==

dmd -w -c -o- test.d

wow! no warnings, no errors!

signature.asc
Description: PGP signature


Re: A Refcounted Array Type

2015-02-27 Thread deadalnix via Digitalmars-d
On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer 
wrote:
In that case, you shouldn't be subject to any kind of race 
conditions. But we can't make the library/language based on 
this assumption. Your case is the exceptional case.




His case is not @safe, so we can ignore that problem. However, 
this do not change the fact that the type qualifier do not 
provide the guarantee they are supposed to.


Re: A Refcounted Array Type

2015-02-27 Thread via Digitalmars-d
On Thursday, 26 February 2015 at 23:04:28 UTC, Steven 
Schveighoffer wrote:
On 2/26/15 4:40 PM, Marc =?UTF-8?B?U2Now7x0eiI=?= 
schue...@gmx.net wrote:
On Thursday, 26 February 2015 at 18:08:28 UTC, Steven 
Schveighoffer wrote:

On 2/26/15 12:56 PM, Marc =?UTF-8?B?U2Now7x0eiI=?=
schue...@gmx.net wrote:
On Thursday, 26 February 2015 at 16:51:30 UTC, Steven 
Schveighoffer

wrote:
As talked about before, running dtors in the originating 
thread can

solve this problem.


This probably implies forcibly destroying objects whose 
creating thread

terminated.


I don't think so, those objects can just be destroyed by the
GC-collection running thread. If the thread is no longer 
present,

there can't be a race.


That's true. However, what if the destructors access global
(thread-local) variables? Is that already disallowed?


Hm... I don't know. Is it disallowed? I don't think so (a 
simple test would suffice), but if any code exists today that 
does it, it's very very wrong :) I would suspect that such code 
should be banned.


By disallowed, I indeed mean banned. Unfortunately, something 
that's just banned will sooner or later be used by someone, 
unless it's also statically prevented by the compiler.


Re: A Refcounted Array Type

2015-02-26 Thread ketmar via Digitalmars-d
On Thu, 26 Feb 2015 23:13:01 -0500, Steven Schveighoffer wrote:

 I think it's going to be difficult to impossible to prevent people from
 using TLS in destructors anyway. It can probably be a guideline more
 than a compiler error.

and a warning, please! a warning that i can silence, of course.

signature.asc
Description: PGP signature


Re: A Refcounted Array Type

2015-02-26 Thread Michel Fortin via Digitalmars-d

On 2015-02-26 21:07:26 +, Andrei Alexandrescu said:


On 2/26/15 12:54 PM, deadalnix wrote:

On Thursday, 26 February 2015 at 16:57:51 UTC, Andrei Alexandrescu wrote:

On 2/26/15 8:51 AM, Steven Schveighoffer wrote:

As talked about before, running dtors in the originating thread can
solve this problem.


Yah, that will solve the nonatomic reference counting. What do you
think about http://forum.dlang.org/thread/mcllre$1abs$1...@digitalmars.com?

Andrei


class BazingaException : Exception {

RefCount!Stuff reallyImportantStuff;

// ...

}

void main() {
auto t = new Thread({
RefCount!Stuff s = ...;
throw new BazingaException(s);
});

t.start();
t.join();
}


Could you please walk me through what the matter is here. Thanks. -- Andrei


The exception is thrown by t.join() in another thread, after the 
originating thread died. Thus, obviously, it cannot be destructed in 
the originating thread as stated above. But everyone already know that.


But the example doesn't make one problem quite clear: Unless the GC 
destroys the exception in the thread join() was called, there can be a 
race. That's because join() moves the exception to another thread, and 
the thread that now owns the exception could make copies of that 
reallyImportantStuff and access the counter beyond the exception's 
lifetime. So it turns out that the GC heap needs call the exception's 
destructor in the thread calling join() to avoid races.


Additionally, if the old thread has leftover objects still not yet 
collected, they'll need to be destroyed in the thread calling join() 
too. Otherwise you might get races when the exception is destroyed.


So you could solve all that by changing of ownership for things 
originating from the worker thread to the thread that is calling 
join(). Or if no one calls join(), then you can destroy objects 
originating from the dead thread in any thread, as long as they are all 
destroyed *in the same thread* (because objects originating from the 
same thread might all points to the same thread-local reference counts).


--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/



Re: A Refcounted Array Type

2015-02-26 Thread Andrei Alexandrescu via Digitalmars-d

On 2/26/15 8:51 AM, Steven Schveighoffer wrote:

As talked about before, running dtors in the originating thread can
solve this problem.


Yah, that will solve the nonatomic reference counting. What do you think 
about http://forum.dlang.org/thread/mcllre$1abs$1...@digitalmars.com?


Andrei



Re: A Refcounted Array Type

2015-02-26 Thread Steven Schveighoffer via Digitalmars-d

On 2/24/15 3:34 PM, Walter Bright wrote:

On 2/24/2015 6:56 AM, Steven Schveighoffer wrote:

Actually, RCArray can never be allocated on GC, or you may corrupt
memory. count
may be non-null, and point at invalid memory when the dtor is called.


Just set count to null after the delete.


No, the GC may have already deallocated the memory count points at, and 
count is still non-null.


This is why you can never refer to GC-allocated memory in dtors, if they 
can be called from the GC.



Only safe way to do this is to C malloc/free the count. And yes, at
that point,
you need atomics.


No, RCArray is not intended for shared access between threads.


GC is what shares the access. For instance, if you have an RCArray in 
one object that is being collected, it will attempt to decrement count. 
However, that doesn't mean that another RCArray reference in the 
originating thread won't also be trying to adjust count. In reality, 
it's only shared between the GC-collection running thread and the 
originating thread. I was not talking at all about making RCArray 
shared-usable.


As talked about before, running dtors in the originating thread can 
solve this problem.


-Steve


Re: A Refcounted Array Type

2015-02-26 Thread via Digitalmars-d
On Thursday, 26 February 2015 at 16:51:30 UTC, Steven 
Schveighoffer wrote:
As talked about before, running dtors in the originating thread 
can solve this problem.


This probably implies forcibly destroying objects whose creating 
thread terminated.


Re: A Refcounted Array Type

2015-02-26 Thread Steven Schveighoffer via Digitalmars-d

On 2/26/15 11:57 AM, Andrei Alexandrescu wrote:

On 2/26/15 8:51 AM, Steven Schveighoffer wrote:

As talked about before, running dtors in the originating thread can
solve this problem.


Yah, that will solve the nonatomic reference counting. What do you think
about http://forum.dlang.org/thread/mcllre$1abs$1...@digitalmars.com?


I saw that, it sounds reasonable. I have to mull over what it means.

I think possibly a better solution is to have a finalize function, 
similar to how tango does it, so the dtor is only called from 
destroy/delete, and the finalize method is called from the GC.


-Steve



Re: A Refcounted Array Type

2015-02-26 Thread Steven Schveighoffer via Digitalmars-d
On 2/26/15 12:56 PM, Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net 
wrote:

On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:

As talked about before, running dtors in the originating thread can
solve this problem.


This probably implies forcibly destroying objects whose creating thread
terminated.


However, this has to be done carefully. For instance, you have to run 
dtors on next allocation, or when thread terminates, because if you do 
it inside stop-the-world signal handler, you still potentially have a race.


-Steve


Re: A Refcounted Array Type

2015-02-26 Thread Steven Schveighoffer via Digitalmars-d
On 2/26/15 12:56 PM, Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net 
wrote:

On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer wrote:

As talked about before, running dtors in the originating thread can
solve this problem.


This probably implies forcibly destroying objects whose creating thread
terminated.


I don't think so, those objects can just be destroyed by the 
GC-collection running thread. If the thread is no longer present, there 
can't be a race.


-Steve


Re: A Refcounted Array Type

2015-02-26 Thread Andrei Alexandrescu via Digitalmars-d

On 2/26/15 12:54 PM, deadalnix wrote:

On Thursday, 26 February 2015 at 16:57:51 UTC, Andrei Alexandrescu wrote:

On 2/26/15 8:51 AM, Steven Schveighoffer wrote:

As talked about before, running dtors in the originating thread can
solve this problem.


Yah, that will solve the nonatomic reference counting. What do you
think about http://forum.dlang.org/thread/mcllre$1abs$1...@digitalmars.com?

Andrei


class BazingaException : Exception {

 RefCount!Stuff reallyImportantStuff;

 // ...

}

void main() {
 auto t = new Thread({
 RefCount!Stuff s = ...;
 throw new BazingaException(s);
 });

 t.start();
 t.join();
}


Could you please walk me through what the matter is here. Thanks. -- Andrei



Re: A Refcounted Array Type

2015-02-26 Thread deadalnix via Digitalmars-d
On Thursday, 26 February 2015 at 16:57:51 UTC, Andrei 
Alexandrescu wrote:

On 2/26/15 8:51 AM, Steven Schveighoffer wrote:
As talked about before, running dtors in the originating 
thread can

solve this problem.


Yah, that will solve the nonatomic reference counting. What do 
you think about 
http://forum.dlang.org/thread/mcllre$1abs$1...@digitalmars.com?


Andrei


class BazingaException : Exception {

RefCount!Stuff reallyImportantStuff;

// ...

}

void main() {
auto t = new Thread({
RefCount!Stuff s = ...;
throw new BazingaException(s);
});

t.start();
t.join();
}


Re: A Refcounted Array Type

2015-02-26 Thread via Digitalmars-d
On Thursday, 26 February 2015 at 18:04:11 UTC, Steven 
Schveighoffer wrote:
I think possibly a better solution is to have a finalize 
function, similar to how tango does it, so the dtor is only 
called from destroy/delete, and the finalize method is called 
from the GC.


Ugh, so you still have to identify objects before releasing the 
memory?


Put all objects that need cleanup on a separate GC heap then, to 
reduce impact.


Re: A Refcounted Array Type

2015-02-26 Thread via Digitalmars-d
On Thursday, 26 February 2015 at 18:08:28 UTC, Steven 
Schveighoffer wrote:
On 2/26/15 12:56 PM, Marc =?UTF-8?B?U2Now7x0eiI=?= 
schue...@gmx.net wrote:
On Thursday, 26 February 2015 at 16:51:30 UTC, Steven 
Schveighoffer wrote:
As talked about before, running dtors in the originating 
thread can

solve this problem.


This probably implies forcibly destroying objects whose 
creating thread

terminated.


I don't think so, those objects can just be destroyed by the 
GC-collection running thread. If the thread is no longer 
present, there can't be a race.


That's true. However, what if the destructors access global 
(thread-local) variables? Is that already disallowed?


Re: A Refcounted Array Type

2015-02-26 Thread Steven Schveighoffer via Digitalmars-d
On 2/26/15 4:40 PM, Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net 
wrote:

On Thursday, 26 February 2015 at 18:08:28 UTC, Steven Schveighoffer wrote:

On 2/26/15 12:56 PM, Marc =?UTF-8?B?U2Now7x0eiI=?=
schue...@gmx.net wrote:

On Thursday, 26 February 2015 at 16:51:30 UTC, Steven Schveighoffer
wrote:

As talked about before, running dtors in the originating thread can
solve this problem.


This probably implies forcibly destroying objects whose creating thread
terminated.


I don't think so, those objects can just be destroyed by the
GC-collection running thread. If the thread is no longer present,
there can't be a race.


That's true. However, what if the destructors access global
(thread-local) variables? Is that already disallowed?


Hm... I don't know. Is it disallowed? I don't think so (a simple test 
would suffice), but if any code exists today that does it, it's very 
very wrong :) I would suspect that such code should be banned.


-Steve


Re: A Refcounted Array Type

2015-02-26 Thread deadalnix via Digitalmars-d
On Thursday, 26 February 2015 at 21:07:26 UTC, Andrei 
Alexandrescu wrote:
Could you please walk me through what the matter is here. 
Thanks. -- Andrei


1/ The originating thread is likely to be dead by the time we 
collect.
2/ The RefCounted Stuff crossed thread boundaries so even if we 
kept the thread alive somehow to be able to destruct as 
mentioned, this is still not thread safe.


Re: A Refcounted Array Type

2015-02-26 Thread ketmar via Digitalmars-d
On Thu, 26 Feb 2015 18:04:28 -0500, Steven Schveighoffer wrote:

 That's true. However, what if the destructors access global
 (thread-local) variables? Is that already disallowed?
 
 Hm... I don't know. Is it disallowed? I don't think so (a simple test
 would suffice), but if any code exists today that does it, it's very
 very wrong :) I would suspect that such code should be banned.

but what if i'm doing memory managing myself and i *know* that it's ok to 
use thread-locals in my destructor?

signature.asc
Description: PGP signature


Re: A Refcounted Array Type

2015-02-26 Thread ketmar via Digitalmars-d
On Thu, 26 Feb 2015 21:56:30 -0500, Steven Schveighoffer wrote:

 On 2/26/15 7:34 PM, ketmar wrote:
 On Thu, 26 Feb 2015 18:04:28 -0500, Steven Schveighoffer wrote:

 That's true. However, what if the destructors access global
 (thread-local) variables? Is that already disallowed?

 Hm... I don't know. Is it disallowed? I don't think so (a simple
 test would suffice), but if any code exists today that does it, it's
 very very wrong :) I would suspect that such code should be banned.

 but what if i'm doing memory managing myself and i *know* that it's ok
 to use thread-locals in my destructor?


 Accessing thread-local data in the GC-run destructor means you are
 accessing in the GC collecting thread. In the case where we make dtors
 run in the originating thread, it is OK. But if the thread is gone, then
 any running of that dtor may incorrectly use the TL data in that thread,
 which may not be correct.
 
 In any case, accessing thread local data today is most certainly wrong,
 since there isn't even a likelihood the same thread will destroy the
 data.
 
 I can't see a situation where this would be what you wanted. Accessing
 global data should be fine, but thread-local data is not.

let's assume that i have a program that has only one thread. i know it 
for sure. and i don't like prepending ugly `__gshared` to my declarations.

sure, it all depends of GC code (it can be multithreaded itself, or it 
can deliberately run finalisers in another thread), but why i should not 
be allowed to do what i want? i can write that GC myself, and i *know* 
that there will be no other threads.

this is the same as allocating in finaliser: as finalisers aren't made 
implicitly @nogc, i can see no sense in restricting 'em in any other way, 
as the same arguments applies in both cases.

signature.asc
Description: PGP signature


Re: A Refcounted Array Type

2015-02-26 Thread Steven Schveighoffer via Digitalmars-d

On 2/26/15 10:56 PM, ketmar wrote:

On Thu, 26 Feb 2015 21:56:30 -0500, Steven Schveighoffer wrote:



I can't see a situation where this would be what you wanted. Accessing
global data should be fine, but thread-local data is not.


let's assume that i have a program that has only one thread. i know it
for sure. and i don't like prepending ugly `__gshared` to my declarations.


In that case, you shouldn't be subject to any kind of race conditions. 
But we can't make the library/language based on this assumption. Your 
case is the exceptional case.


I think it's going to be difficult to impossible to prevent people from 
using TLS in destructors anyway. It can probably be a guideline more 
than a compiler error.


-Steve


Re: A Refcounted Array Type

2015-02-26 Thread Steven Schveighoffer via Digitalmars-d

On 2/26/15 7:34 PM, ketmar wrote:

On Thu, 26 Feb 2015 18:04:28 -0500, Steven Schveighoffer wrote:


That's true. However, what if the destructors access global
(thread-local) variables? Is that already disallowed?


Hm... I don't know. Is it disallowed? I don't think so (a simple test
would suffice), but if any code exists today that does it, it's very
very wrong :) I would suspect that such code should be banned.


but what if i'm doing memory managing myself and i *know* that it's ok to
use thread-locals in my destructor?



Accessing thread-local data in the GC-run destructor means you are 
accessing in the GC collecting thread. In the case where we make dtors 
run in the originating thread, it is OK. But if the thread is gone, then 
any running of that dtor may incorrectly use the TL data in that thread, 
which may not be correct.


In any case, accessing thread local data today is most certainly wrong, 
since there isn't even a likelihood the same thread will destroy the data.


I can't see a situation where this would be what you wanted. Accessing 
global data should be fine, but thread-local data is not.


-Steve


Re: A Refcounted Array Type

2015-02-25 Thread via Digitalmars-d
On Wednesday, 25 February 2015 at 00:20:25 UTC, Walter Bright 
wrote:

On 2/24/2015 2:27 PM, deadalnix wrote:
I have to admit, that is pretty cool. But it is only gonna 
work with value types.


I don't see that limitation at all.


Just try to do the same with opSlice() and you will see what he 
means.


Re: A Refcounted Array Type

2015-02-25 Thread Jonathan M Davis via Digitalmars-d
On Tuesday, February 24, 2015 01:44:07 Walter Bright via Digitalmars-d wrote:
 On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:
  The issue is that delete is considered @safe by the compiler,

 I thought more people would be interested in how to do a memory safe reference
 counted container.

Oh, I think that that there's definitely something there and that we should
be interested. I don't think that either Andrei or I were trying to take
away from that. It's just that code showed a problem in that delete was
considered @safe when it should be @system, so Andrei pointed that out, and
then I agreed with him and pointed out that delete was really supposed to
have been deprecated by now anyway. But the concept is perfectly valid, and
delete can be replaced with destroy and core.memory.GC.free for the same
effect without actually needing delete, so it'll still work even if delete
is finally deprecated - though honestly, I would think that if you're going
to be doing manual memory management, it would just be better to use malloc
and free and avoid the GC altogether (though if the element type contains
any references or pointers, you probably need to tell the GC about the
memory so that it can scan it).

- Jonathan M Davis



Re: A Refcounted Array Type

2015-02-24 Thread Walter Bright via Digitalmars-d

On 2/24/2015 6:56 AM, Steven Schveighoffer wrote:

Actually, RCArray can never be allocated on GC, or you may corrupt memory. count
may be non-null, and point at invalid memory when the dtor is called.


Just set count to null after the delete.


Only safe way to do this is to C malloc/free the count. And yes, at that point,
you need atomics.


No, RCArray is not intended for shared access between threads.

Shared containers and local containers are different enough that they merit 
being different types with different implementations altogether. Trying to just 
slap 'shared' on a container isn't going to work.




Re: A Refcounted Array Type

2015-02-24 Thread Jonathan M Davis via Digitalmars-d
On Monday, February 23, 2015 23:02:03 Walter Bright via Digitalmars-d wrote:
 On 2/23/2015 9:59 PM, Jonathan M Davis via Digitalmars-d wrote:
  And delete is supposed to have been deprecated ages ago, but yeah, it
  _definitely_ shouldn't be considered @safe.

 Managing memory always is going to be unsafe. The idea is to encapsulate that,
 which RCArray shows how to do.

That's fine. The issue is that delete is considered @safe by the compiler,
not with the idea of RCArray. This code shouldn't compile:

void main() @safe
{
auto i = new int(5);
delete i;
}

and it does. Of course, delete is supposed to have been deprecated, but no
one has gotten around to doing it. So, maybe that's why no one made it so
that it was treated correctly with regards to @safe.

- Jonathan M Davis



Re: A Refcounted Array Type

2015-02-24 Thread matovitch via Digitalmars-d
On Tuesday, 24 February 2015 at 10:16:00 UTC, Ulrich Küttler 
wrote:

On Tuesday, 24 February 2015 at 10:13:36 UTC, matovitch wrote:

On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:

On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:
That's why:

this(this)
{
  if (count)
  ++*count;
}



Hmm, I don't see why that's why... :(


The counter is shared amount all copies of the array.


Ah yes, what an idiot ! Thanks !



Re: A Refcounted Array Type

2015-02-24 Thread Chris via Digitalmars-d

On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

This is pretty straightforward. More could be done:

1. small array optimization
2. support for ranges as constructor args
3. present a range interface
4. support for malloc/free instead of GC
5. bounds checking
6. the array[] and the count could be allocated together
7. array[] could be just a pointer

but the basic idea is there, I didn't want to hide it behind 
all the other flesh a professional type would have.


Note the return in opIndex(). This is DIP25 at work!

Compile:
dmd rcarray -unittest -main -dip25

===

struct RCArray(E) {

this(E[] a)
{
array = a.dup;
start = 0;
end = a.length;
count = new int;
*count = 1;
}

~this()
{
if (count  --*count == 0)
delete array;
}

this(this)
{
if (count)
++*count;
}

size_t length()
{
return end - start;
}

ref E opIndex(size_t i) return // here's the magic
{
return array[start + i];
}

RCArray opSlice(size_t lwr, size_t upr)
{
RCArray result = this;
result.start = start + lwr;
result.end = start + upr;
return result;
}

  private:
E[] array;
size_t start, end;
int* count;
}

unittest
{
static int[3] s = [7, 6, 4];
auto r = RCArray!int(s);
assert(r.length == 3);
assert(r[0] == 7);
assert(r[1] == 6);
assert(r[2] == 4);
assert(*r.count == 1);

{
auto r2 = r;
assert(r2[0] == 7);
assert(r2[1] == 6);
assert(r2[2] == 4);
assert(*r.count == 2);

r[1] = 3;
assert(r2[0] == 7);
assert(r2[1] == 3);
assert(r2[2] == 4);
}
assert(*r.count == 1);

auto r3 = r[1 .. 3];
r[2] = 9;
assert(r3[0] == 3);
assert(r3[1] == 9);

  /+
ref int test(ref RCArray!int rr)
{
return rr[1]; // this gives error
}
   +/
}


Thanks for the example.

I think we need more practical stuff like this compiled somewhere 
on the homepage, especially when a new feature is introduced, to 
answer questions like: What is it? When should I use it? And how 
do I implement it? Saves a lot of time and the frustration of 
finding out later that there _is_ such a feature, only I never 
heard of it until now.


Re: A Refcounted Array Type

2015-02-24 Thread matovitch via Digitalmars-d

On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright wrote:

On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:

The issue is that delete is considered @safe by the compiler,


I thought more people would be interested in how to do a memory 
safe reference counted container.


This is really interesting ! Thought as a beginner, I am 
wondering why the counter need to be a pointer ?


Re: A Refcounted Array Type

2015-02-24 Thread via Digitalmars-d

On Tuesday, 24 February 2015 at 10:13:36 UTC, matovitch wrote:

On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:

On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:
That's why:

this(this)
{
   if (count)
   ++*count;
}



Hmm, I don't see why that's why... :(


The counter is shared amount all copies of the array.


Re: A Refcounted Array Type

2015-02-24 Thread Namespace via Digitalmars-d

On Tuesday, 24 February 2015 at 10:13:36 UTC, matovitch wrote:

On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:

On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:
On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright 
wrote:
On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d 
wrote:
The issue is that delete is considered @safe by the 
compiler,


I thought more people would be interested in how to do a 
memory safe reference counted container.


This is really interesting ! Thought as a beginner, I am 
wondering why the counter need to be a pointer ?


That's why:

this(this)
{
   if (count)
   ++*count;
}



Hmm, I don't see why that's why... :(


It's more effective if you figure it out on your own.
But maybe this is a bit more clear?
https://github.com/Dgame/m3/blob/master/SmartPointer.d#L106


Re: A Refcounted Array Type

2015-02-24 Thread matovitch via Digitalmars-d

On Tuesday, 24 February 2015 at 10:11:02 UTC, Namespace wrote:

On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:
On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright 
wrote:
On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d 
wrote:

The issue is that delete is considered @safe by the compiler,


I thought more people would be interested in how to do a 
memory safe reference counted container.


This is really interesting ! Thought as a beginner, I am 
wondering why the counter need to be a pointer ?


That's why:

this(this)
{
if (count)
++*count;
}



Hmm, I don't see why that's why... :(


Re: A Refcounted Array Type

2015-02-24 Thread Namespace via Digitalmars-d

On Tuesday, 24 February 2015 at 10:08:23 UTC, matovitch wrote:
On Tuesday, 24 February 2015 at 09:44:13 UTC, Walter Bright 
wrote:

On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:

The issue is that delete is considered @safe by the compiler,


I thought more people would be interested in how to do a 
memory safe reference counted container.


This is really interesting ! Thought as a beginner, I am 
wondering why the counter need to be a pointer ?


That's why:

this(this)
{
if (count)
++*count;
}



Re: A Refcounted Array Type

2015-02-24 Thread Walter Bright via Digitalmars-d

On 2/24/2015 1:32 AM, Jonathan M Davis via Digitalmars-d wrote:

The issue is that delete is considered @safe by the compiler,


I thought more people would be interested in how to do a memory safe reference 
counted container.




Re: A Refcounted Array Type

2015-02-24 Thread ponce via Digitalmars-d
Does DIP25 means we can turn resources into RC types and be done 
with non-deterministic destructor calls by the GC, without using 
RefCounted or Unique?


On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

This is pretty straightforward. More could be done:

1. small array optimization
2. support for ranges as constructor args
3. present a range interface
4. support for malloc/free instead of GC
5. bounds checking
6. the array[] and the count could be allocated together
7. array[] could be just a pointer

but the basic idea is there, I didn't want to hide it behind 
all the other flesh a professional type would have.


Note the return in opIndex(). This is DIP25 at work!

Compile:
dmd rcarray -unittest -main -dip25

===

struct RCArray(E) {

this(E[] a)
{
array = a.dup;
start = 0;
end = a.length;
count = new int;
*count = 1;
}

~this()
{
if (count  --*count == 0)
delete array;
}

this(this)
{
if (count)
++*count;
}

size_t length()
{
return end - start;
}

ref E opIndex(size_t i) return // here's the magic
{
return array[start + i];
}

RCArray opSlice(size_t lwr, size_t upr)
{
RCArray result = this;
result.start = start + lwr;
result.end = start + upr;
return result;
}

  private:
E[] array;
size_t start, end;
int* count;
}

unittest
{
static int[3] s = [7, 6, 4];
auto r = RCArray!int(s);
assert(r.length == 3);
assert(r[0] == 7);
assert(r[1] == 6);
assert(r[2] == 4);
assert(*r.count == 1);

{
auto r2 = r;
assert(r2[0] == 7);
assert(r2[1] == 6);
assert(r2[2] == 4);
assert(*r.count == 2);

r[1] = 3;
assert(r2[0] == 7);
assert(r2[1] == 3);
assert(r2[2] == 4);
}
assert(*r.count == 1);

auto r3 = r[1 .. 3];
r[2] = 9;
assert(r3[0] == 3);
assert(r3[1] == 9);

  /+
ref int test(ref RCArray!int rr)
{
return rr[1]; // this gives error
}
   +/
}




Re: A Refcounted Array Type

2015-02-24 Thread Steven Schveighoffer via Digitalmars-d

On 2/24/15 8:17 AM, Michel Fortin wrote:

On 2015-02-23 22:15:46 +, Walter Bright said:


int* count;

[...] if (count  --*count == 0) [...]


Careful!

This isn't memory safe and you have to thank the GC for it. If you ever
use RCArray as a member variable in a class, the RCArray destructor is
going to be called from a random thread when the class destructor is
run. If some thread has a stack reference to the array you have a race.

You have to use an atomic counter unless you can prove the RCArray
struct will never be put in a GC-managed context. It is rather sad that
the language has no way to enforce such a restriction, and also that
@safe cannot detect that this is a problem here.



Actually, RCArray can never be allocated on GC, or you may corrupt 
memory. count may be non-null, and point at invalid memory when the dtor 
is called.


Only safe way to do this is to C malloc/free the count. And yes, at that 
point, you need atomics.


-Steve


Re: A Refcounted Array Type

2015-02-24 Thread Andrei Alexandrescu via Digitalmars-d

On 2/23/15 6:05 PM, Walter Bright wrote:

On 2/23/2015 5:41 PM, weaselcat wrote:

On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

ref E opIndex(size_t i) return // here's the magic


exactly what is this doing? I don't see this explained in DIP25 at all.


The 'this' is passed by reference. So the 'return' is really 'return ref
this', the rest is explained by DIP25.


We should amend DIP25 to explain that this is handled like a 
parameter. -- Andrei


Re: A Refcounted Array Type

2015-02-24 Thread Michel Fortin via Digitalmars-d

On 2015-02-23 22:15:46 +, Walter Bright said:


int* count;

[...] if (count  --*count == 0) [...]


Careful!

This isn't memory safe and you have to thank the GC for it. If you ever 
use RCArray as a member variable in a class, the RCArray destructor is 
going to be called from a random thread when the class destructor is 
run. If some thread has a stack reference to the array you have a race.


You have to use an atomic counter unless you can prove the RCArray 
struct will never be put in a GC-managed context. It is rather sad that 
the language has no way to enforce such a restriction, and also that 
@safe cannot detect that this is a problem here.


--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/



Re: A Refcounted Array Type

2015-02-24 Thread Andrei Alexandrescu via Digitalmars-d

On 2/24/15 3:14 AM, ponce wrote:

Does DIP25 means we can turn resources into RC types and be done with
non-deterministic destructor calls by the GC, without using RefCounted
or Unique?


That's what we're aiming for. -- Andrei



Re: A Refcounted Array Type

2015-02-24 Thread Andrei Alexandrescu via Digitalmars-d

On 2/24/15 5:17 AM, Michel Fortin wrote:

On 2015-02-23 22:15:46 +, Walter Bright said:


int* count;

[...] if (count  --*count == 0) [...]


Careful!

This isn't memory safe and you have to thank the GC for it. If you ever
use RCArray as a member variable in a class, the RCArray destructor is
going to be called from a random thread when the class destructor is
run. If some thread has a stack reference to the array you have a race.

You have to use an atomic counter unless you can prove the RCArray
struct will never be put in a GC-managed context. It is rather sad that
the language has no way to enforce such a restriction, and also that
@safe cannot detect that this is a problem here.


Good point. This is already a danger in existing code, e.g. File uses 
reference counting. I just submitted 
https://issues.dlang.org/show_bug.cgi?id=14221.


Andrei




Re: A Refcounted Array Type

2015-02-24 Thread Walter Bright via Digitalmars-d

On 2/24/2015 2:34 PM, deadalnix wrote:

Maybe we want to fix the GC, exceptions and delegates or disable them in @safe
code because they all cause implicit sharing.


Andrei already filed a bug report on the GC issue. If there aren't bug reports 
on the other two, file them.


Re: A Refcounted Array Type

2015-02-24 Thread deadalnix via Digitalmars-d
On Wednesday, 25 February 2015 at 00:13:03 UTC, Walter Bright 
wrote:

On 2/24/2015 2:34 PM, deadalnix wrote:
Maybe we want to fix the GC, exceptions and delegates or 
disable them in @safe

code because they all cause implicit sharing.


Andrei already filed a bug report on the GC issue. If there 
aren't bug reports on the other two, file them.


Well I filled a bug on the delegate one ~2 years ago, and the 2 
others are features, not bugs, as far as I know.


Re: A Refcounted Array Type

2015-02-24 Thread deadalnix via Digitalmars-d

On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

struct RCArray(E) {

this(E[] a)
{
array = a.dup;
start = 0;
end = a.length;
count = new int;
*count = 1;
}



This may not be @safe depending on the type of E. Also, this do 
not really solve the garbage problem



~this()
{
if (count  --*count == 0)
delete array;
}

this(this)
{
if (count)
++*count;
}



Here, we are going to run a bunch of null check because we can't 
enforce proper construction of struct. This is not per se a 
limitation of this piece of code, simply a long standing issue 
that should up.


Also, there are no way to ensure that the array is going to be 
bound to one thread, even without shared (exception, delegate, 
destructor, pure return). You need at least atomic increment 
and/or decrement.


But worse, as this can be assigned to static variables, you need 
to lock the whole damn struct unless you can update it 
atomically. The struct is 320 bits large and it is not updatable 
atomically in any arch I know of.


You need a mutex.


size_t length()
{
return end - start;
}

ref E opIndex(size_t i) return // here's the magic
{
return array[start + i];
}



I have to admit, that is pretty cool. But it is only gonna work 
with value types.



RCArray opSlice(size_t lwr, size_t upr)
{
RCArray result = this;
result.start = start + lwr;
result.end = start + upr;
return result;
}



You need bound check: end = start = array.length .


  private:
E[] array;
size_t start, end;
int* count;
}





Re: A Refcounted Array Type

2015-02-24 Thread deadalnix via Digitalmars-d

On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

struct RCArray(E) {

this(E[] a)
{
array = a.dup;
start = 0;
end = a.length;
count = new int;
*count = 1;
}



This may not be @safe depending on the type of E. Also, this do 
not really solve the garbage problem as you need a GCed slice to 
build the RCed slice. That mean that without a nicer way to 
construct this, this is not solving any problem.


Also, I guess that even if you use the GC in there, you want to 
make sure that the function is @nogc .



~this()
{
if (count  --*count == 0)
delete array;
}

this(this)
{
if (count)
++*count;
}



Here, we are going to run a bunch of null check because we can't 
enforce proper construction of struct. This is not per se a 
limitation of this piece of code, simply a long standing issue 
that should up.


Also, there are no way to ensure that the array is going to be 
bound to one thread, even without shared (exception, delegate, 
destructor, pure return). You need at least atomic increment 
and/or decrement.


But worse, as this can be assigned to static variables, you need 
to lock the whole damn struct unless you can update it 
atomically. The struct is 320 bits large and it is not updatable 
atomically in any arch I know of.


You need a mutex.


size_t length()
{
return end - start;
}

ref E opIndex(size_t i) return // here's the magic
{
return array[start + i];
}



I have to admit, that is pretty cool. But it is only gonna work 
with value types.



RCArray opSlice(size_t lwr, size_t upr)
{
RCArray result = this;
result.start = start + lwr;
result.end = start + upr;
return result;
}



You need bound check: end = start = array.length .


  private:
E[] array;
size_t start, end;
int* count;
}



Generally, this need support for other array operations, is not 
usable in @nogc, has threading issues, but this is a cool thing 
you can do with ref return.


Re: A Refcounted Array Type

2015-02-24 Thread deadalnix via Digitalmars-d

On Tuesday, 24 February 2015 at 20:35:04 UTC, Walter Bright wrote:
Only safe way to do this is to C malloc/free the count. And 
yes, at that point,

you need atomics.


No, RCArray is not intended for shared access between threads.

Shared containers and local containers are different enough 
that they merit being different types with different 
implementations altogether. Trying to just slap 'shared' on a 
container isn't going to work.


Maybe we want to fix the GC, exceptions and delegates or disable 
them in @safe code because they all cause implicit sharing.


Re: A Refcounted Array Type

2015-02-24 Thread deadalnix via Digitalmars-d
On Tuesday, 24 February 2015 at 15:51:42 UTC, Andrei Alexandrescu 
wrote:

On 2/23/15 6:05 PM, Walter Bright wrote:

On 2/23/2015 5:41 PM, weaselcat wrote:
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright 
wrote:

   ref E opIndex(size_t i) return // here's the magic


exactly what is this doing? I don't see this explained in 
DIP25 at all.


The 'this' is passed by reference. So the 'return' is really 
'return ref

this', the rest is explained by DIP25.


We should amend DIP25 to explain that this is handled like a 
parameter. -- Andrei


Note that I'm not surprised by that behavior (this is indeed ref 
for struct). But this is not ref for classes, which will defeat 
DIP25.


Re: A Refcounted Array Type

2015-02-24 Thread Walter Bright via Digitalmars-d

On 2/24/2015 2:27 PM, deadalnix wrote:

This may not be @safe depending on the type of E.


The code is a template, and so relies on inference.



Also, this do not really solve the garbage problem


Steven pointed out the correct fix for that.



Also, there are no way to ensure that the array is going to be bound to one
thread, even without shared (exception, delegate, destructor, pure return). You
need at least atomic increment and/or decrement.


No. The correct solution is to fix any cases of implicit sharing. If they don't 
have bugzilla issues, file them.




I have to admit, that is pretty cool. But it is only gonna work with value 
types.


I don't see that limitation at all.



You need bound check: end = start = array.length


See the antecedent:

More could be done:
[...]
5. bounds checking



Re: A Refcounted Array Type

2015-02-24 Thread Walter Bright via Digitalmars-d

On 2/24/2015 4:32 PM, deadalnix wrote:

On Wednesday, 25 February 2015 at 00:13:03 UTC, Walter Bright wrote:

On 2/24/2015 2:34 PM, deadalnix wrote:

Maybe we want to fix the GC, exceptions and delegates or disable them in @safe
code because they all cause implicit sharing.


Andrei already filed a bug report on the GC issue. If there aren't bug reports
on the other two, file them.


Well I filled a bug on the delegate one ~2 years ago, and the 2 others are
features, not bugs, as far as I know.


Without examples and bug reports, I don't really know what you're talking about. 
At least with a bug report, you have something easy to link to instead of trying 
to explain it over and over again.


Re: A Refcounted Array Type

2015-02-23 Thread Walter Bright via Digitalmars-d

On 2/23/2015 5:01 PM, Max Klyga wrote:

I thought that delete is deprecated, yet here Walter himself promotes a solution
that uses it.
Can we *PLEASE* finally deprecate things that are supposed to be deprecated and
remove things that are supposed to be removed?


That isn't the point. I could have well have used malloc/free, but I wanted the 
code example to be about how to make a memory safe ref counted container, not 
about the details of memory allocation. It demonstrates how 'return ref' is to 
be used.




Re: A Refcounted Array Type

2015-02-23 Thread Walter Bright via Digitalmars-d

On 2/23/2015 4:13 PM, bearophile wrote:

5. bounds checking


When you go past bounds of a built-in array you get an error located in the user
code, while if you put a pre-condition in your Array struct to detect the same
errors, you get a run-time error message located in that pre-condition instead.
I'd like some way to solve this small problem of giving more correctly located
error messages. In Contract programming lingo it's a problem of blame
management (I'd also like a way to detect some compile-time out-of-bound errors
for user-defined collections, but you said this is not worth the effort).


This is off-topic. The point of the array class as presented is to show how a 
memory safe reference counted container can be built.




Re: A Refcounted Array Type

2015-02-23 Thread Max Klyga via Digitalmars-d
I thought that delete is deprecated, yet here Walter himself promotes a 
solution that uses it.
Can we *PLEASE* finally deprecate things that are supposed to be 
deprecated and remove things that are supposed to be removed?


On 2015-02-23 22:15:46 +, Walter Bright said:


This is pretty straightforward. More could be done:

1. small array optimization
2. support for ranges as constructor args
3. present a range interface
4. support for malloc/free instead of GC
5. bounds checking
6. the array[] and the count could be allocated together
7. array[] could be just a pointer

but the basic idea is there, I didn't want to hide it behind all the 
other flesh a professional type would have.


Note the return in opIndex(). This is DIP25 at work!

Compile:
 dmd rcarray -unittest -main -dip25

===

struct RCArray(E) {

 this(E[] a)
 {
 array = a.dup;
 start = 0;
 end = a.length;
 count = new int;
 *count = 1;
 }

 ~this()
 {
 if (count  --*count == 0)
 delete array;
 }

 this(this)
 {
 if (count)
 ++*count;
 }

 size_t length()
 {
 return end - start;
 }

 ref E opIndex(size_t i) return // here's the magic
 {
 return array[start + i];
 }

 RCArray opSlice(size_t lwr, size_t upr)
 {
 RCArray result = this;
 result.start = start + lwr;
 result.end = start + upr;
 return result;
 }

   private:
 E[] array;
 size_t start, end;
 int* count;
}

unittest
{
 static int[3] s = [7, 6, 4];
 auto r = RCArray!int(s);
 assert(r.length == 3);
 assert(r[0] == 7);
 assert(r[1] == 6);
 assert(r[2] == 4);
 assert(*r.count == 1);

 {
 auto r2 = r;
 assert(r2[0] == 7);
 assert(r2[1] == 6);
 assert(r2[2] == 4);
 assert(*r.count == 2);

 r[1] = 3;
 assert(r2[0] == 7);
 assert(r2[1] == 3);
 assert(r2[2] == 4);
 }
 assert(*r.count == 1);

 auto r3 = r[1 .. 3];
 r[2] = 9;
 assert(r3[0] == 3);
 assert(r3[1] == 9);

   /+
 ref int test(ref RCArray!int rr)
 {
 return rr[1]; // this gives error
 }
+/
}





Re: A Refcounted Array Type

2015-02-23 Thread weaselcat via Digitalmars-d

On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

ref E opIndex(size_t i) return // here's the magic


exactly what is this doing? I don't see this explained in DIP25 
at all.


Re: A Refcounted Array Type

2015-02-23 Thread bearophile via Digitalmars-d

Walter Bright:


5. bounds checking


When you go past bounds of a built-in array you get an error 
located in the user code, while if you put a pre-condition in 
your Array struct to detect the same errors, you get a run-time 
error message located in that pre-condition instead. I'd like 
some way to solve this small problem of giving more correctly 
located error messages. In Contract programming lingo it's a 
problem of blame management (I'd also like a way to detect some 
compile-time out-of-bound errors for user-defined collections, 
but you said this is not worth the effort).


Bye,
bearophile


Re: A Refcounted Array Type

2015-02-23 Thread Walter Bright via Digitalmars-d

On 2/23/2015 5:41 PM, weaselcat wrote:

On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright wrote:

ref E opIndex(size_t i) return // here's the magic


exactly what is this doing? I don't see this explained in DIP25 at all.


The 'this' is passed by reference. So the 'return' is really 'return ref this', 
the rest is explained by DIP25.


Re: A Refcounted Array Type

2015-02-23 Thread Walter Bright via Digitalmars-d

On 2/23/2015 9:59 PM, Jonathan M Davis via Digitalmars-d wrote:

And delete is supposed to have been deprecated ages ago, but yeah, it
_definitely_ shouldn't be considered @safe.


Managing memory always is going to be unsafe. The idea is to encapsulate that, 
which RCArray shows how to do.




Re: A Refcounted Array Type

2015-02-23 Thread weaselcat via Digitalmars-d

On Tuesday, 24 February 2015 at 02:06:07 UTC, Walter Bright wrote:

On 2/23/2015 5:41 PM, weaselcat wrote:
On Monday, 23 February 2015 at 22:15:54 UTC, Walter Bright 
wrote:

   ref E opIndex(size_t i) return // here's the magic


exactly what is this doing? I don't see this explained in 
DIP25 at all.


The 'this' is passed by reference. So the 'return' is really 
'return ref this', the rest is explained by DIP25.


Oh, I see. That makes a lot more sense.


Re: A Refcounted Array Type

2015-02-23 Thread Jonathan M Davis via Digitalmars-d
On Monday, February 23, 2015 15:28:21 Andrei Alexandrescu via Digitalmars-d 
wrote:
 On 2/23/15 2:15 PM, Walter Bright wrote:
  This is pretty straightforward.
 [snip]

 The code builds if you slap a @safe: at the top. There is one bug in
 the compiler: delete must not be allowed in @safe code. The destructor
 must be @trusted.

And delete is supposed to have been deprecated ages ago, but yeah, it
_definitely_ shouldn't be considered @safe.

- Jonathan M Davis



Re: A Refcounted Array Type

2015-02-23 Thread Andrei Alexandrescu via Digitalmars-d

On 2/23/15 2:15 PM, Walter Bright wrote:

This is pretty straightforward.

[snip]

The code builds if you slap a @safe: at the top. There is one bug in 
the compiler: delete must not be allowed in @safe code. The destructor 
must be @trusted.


Understanding that this code (sans delete) is @safe (and the 
contribution of DIP25 to that) is of paramount importance.


Making it possible to define @safe structs that are still able to return 
reference to their internals is crucial. It paves the way for truly safe 
reference counted classes.



Andrei