Re: Would Lcl be better if it was in D?
On Monday, 2 March 2015 at 03:51:45 UTC, Taylor Hillegeist wrote:
On Monday, 2 March 2015 at 01:22:58 UTC, ketmar wrote:
On Sun, 01 Mar 2015 22:40:28 +, Taylor Hillegeist wrote:
But still the question was about smaller executable when
compiling d
code. The linker needs to know which .o files to include, the
pascal
notation is basically:
uses
thisBigoleThing, ThisOtherBigOleThing, AndMeToo;
I assume the linker just auto-magically includes the entire
thing even
if your only using a single function or value from each. Then
again
perhaps I am wrong.
FreePascal learnt the "smart linking" trick years ago, so only
actually
used functions ends in linked binary. but LCL is very big
library, and FPC
can't drop out unused virtual methods, so resulting binaries
are big.
with D we have the same situation, maybe even worse due to
template
instantiation. compiler is able to merge identical template
instanses,
but... empty `void main () {}` is ~200 KB in D (GNU/Linux,
x86). adding
simple `import std.stdio : writeln;` increases binary size to
~300 KB.
and adding `writeln("hello!");` increases binary size to ~350
KB.
D binaries are big. ;-)
That seems like alot of KB for just a little bit of code. I
wasn't aware that void main(){} was anything but entry
pointer...
;; pseudo-assembly-language
;; main(argc, argv, envp); call
push envp ;; rightmost argument
push argv ;;
push argc ;; leftmost argument ends up on top of stack
call main
I guess I'm confused about what is in there and why?
Even C startup code is more than just that.
You need to set up the runtime, initialize global variables, run
functions marked to execute before main.
--
Paulo
Re: Would Lcl be better if it was in D?
On Mon, 02 Mar 2015 03:51:43 +, Taylor Hillegeist wrote:
> That seems like alot of KB for just a little bit of code. I wasn't aware
> that void main(){} was anything but entry pointer...
>
> ;; pseudo-assembly-language ;; main(argc, argv, envp); call
>
> push envp ;; rightmost argument push argv ;;
> push argc ;; leftmost argument ends up on top of stack
>
> call main
>
> I guess I'm confused about what is in there and why?
you need runtime to support all D features. try to link static C code,
for example. typical C binary is cheating, using system libc as dynamic
library. we can do this for D too (at least for dmd), but there is no
sense, as there is no system that is bundled with libphobos2.so ;-)
signature.asc
Description: PGP signature
Re: Making RCSlice and DIP74 work with const and immutable
On Sunday, 1 March 2015 at 15:53:07 UTC, Marc Schütz wrote: On Sunday, 1 March 2015 at 15:28:56 UTC, Jakob Ovrum wrote: However, AFAICS, it comes with a serious problem. Immutable objects are freely passable between threads, so surely an immutable RC object would need atomic counting just like a shared RC object, but unlike shared, immutable does not necessarily express the intent of sharing between threads; the immutable RC object could easily be counting atomically for nothing. Argh! I didn't think about this. Any chance we can deprecate this behaviour? It's also an obstacle for the implementation of thread-local heaps. No way! Neither for joke, please! --- /P
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 00:37:05 UTC, Zach the Mystic wrote: I'm sure many inc/dec can still be removed. Do you agree or disagree with what I said? I can't tell. Yes, but I think this is overly conservative.
Re: Making RCSlice and DIP74 work with const and immutable
On Sunday, 1 March 2015 at 20:39:54 UTC, Michel Fortin wrote: There's a problem with reference counting immutable objects: they are implicitly shared. Any metadata attached to them thus needs to be shared. Accessing the metadata through a global shared hash table isn't going to be that much of a performance hit compared to whatever mechanism is used to synchronize access to that data. But there is no difference between having metadata as part of the object and using a hash table as long as the mutable and immutable data sits on separate cache lines. (The object address is basically a hash key, and memory is a big table).
Re: DIP74: Reference Counted Class Objects
On Thursday, 26 February 2015 at 21:50:56 UTC, Andrei Alexandrescu wrote: http://wiki.dlang.org/DIP74 got to reviewable form. Please destroy and discuss. Thanks, Andrei opAddRef/opRelease breaks SOLID's single responsibility principle.
Re: DIP74 updated with new protocol for function calls
On 2015-03-01 14:15, Daniel Murphy wrote: Because a compiler recognized UDA is an additional complication for something that is unlikely to be a problem for anyone. Do you have code with methods named opAddRef? No? Me neither. I don't agree. If those are the final names, no. But I do have method names like "retain" and "release". I don't really like how it's done with ranges either. -- /Jacob Carlborg
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 08:59:11 UTC, deadalnix wrote: On Monday, 2 March 2015 at 00:37:05 UTC, Zach the Mystic wrote: I'm sure many inc/dec can still be removed. Do you agree or disagree with what I said? I can't tell. Yes, but I think this is overly conservative. I'm arguing a rather liberal position: that only in a very exceptional case do you need to protect a variable for the duration of a function. For the most part, it's not necessary. What am I conserving?
Re: DIP74 updated with new protocol for function calls
On 1 March 2015 at 18:57, Matthias Bentrup via Digitalmars-d
wrote:
> On Sunday, 1 March 2015 at 07:04:09 UTC, Zach the Mystic wrote:
>>
>> On Saturday, 28 February 2015 at 21:12:54 UTC, Andrei Alexandrescu wrote:
>>>
>>> Defines a significantly better function call protocol:
>>>
>>> http://wiki.dlang.org/DIP74
>>>
>>> Andrei
>>
>>
>> This is obviously much better, Andrei.
>>
>> I think an alternative solution (I know -- another idea -- against my own
>> first idea!) is to keep track of this from the caller's side. The compiler,
>> in this case, when copying a ref-counted type (or derivative) into a
>> parameter, would actually check to see if it's splitting the variable in
>> two. Look at this:
>>
>> class RcType {...}
>>
>> void fun(RcType1 c, RcType1 d);
>>
>> auto x = new RcType;
>>
>> fun(x, x);
>>
>> If the compiler were smart, it would realize that by splitting parameters
>> this way, it's actually adding an additional reference to x. The function
>> should get one x for free, and then force an opAdd/opRelease, for every
>> additional x (or x derivative) it detects in the same call.
>>
>> This might be even better than the improved current proposal. The real key
>> is realizing that duplicating an lvalue into the same function call is
>> subtly adding a new reference to it.
>>
>> Eh??
>
>
> Note that you can get the same issue without duplicate parameters, if you
> pass an alias to a global variable.
>
>
> static A a;
>
> void fun(A x) {
> a = null; // Releases x
> x.func();
> }
>
> void main() {
> a = new A();
> fun(a);
> }
Thing here is that 'a' does not belong to main().
There must be a reference on the caller's stack for it to be passed to
a non-pure function.
If a function X passes a value it does not own to some other function,
then X needs to increment the refcount such that it holds one
reference for itself.
For the callee, it received via an argument, so it has confidence a
stack reference exists, and it doesn't need to bump anymore down the
call tree.
Re: RCArray is unsafe
On Monday, 2 March 2015 at 05:57:35 UTC, Walter Bright wrote: On 3/1/2015 12:51 PM, Michel Fortin wrote: That's actually not enough. You'll have to block access to global variables too: Hmm. That's not so easy to solve. But consider this. It's only an impure function which might alias a global. And since you already have access to the global in the impure function, there might be less incentive in general to pass it through a function. Other than that, you're stuck with a theoretical "@impure!varName" function attribute, for example, which tells the caller which globals are accessed.
[Bug?] Attributes for invariant()
class A
{
private bool isA() const pure @safe // OK
{
return true;
}
invariant() @safe pure const // Error, statement block expected
// @safe pure const invariant() // OK
{
assert(isA);
}
}
Does sufix-attribute style prohibited for invariant? Does it have
any sence to mark it with this attributes? As for me invariant
have to be const and pure by definition.
Also:
class A
{
invariant // Error: use 'immutable' instead of 'invariant'
{
assert(true);
}
}
http://dlang.org/contracts.html#Invariants
[compiled with http://dlang.org/]
Re: Last week for DConf 2015 submissions
On 2/22/15 7:03 PM, Andrei Alexandrescu wrote: Hi folks, At this point we don't have enough submissions to make DConf 2015 viable. Those we have are of good quality, but simply put more are needed. I'm sure you are poring over the proposals you received last week, but can you just let us know whether enough have been proposed to make sure the conference is still on? Thanks. -Steve
Re: RCArray is unsafe
On Sunday, 1 March 2015 at 20:51:35 UTC, Michel Fortin wrote:
On 2015-03-01 19:21:57 +, Walter Bright said:
The trouble seems to happen when there are two references to
the same object passed to a function. I.e. there can be only
one "borrowed" ref at a time.
I'm thinking this could be statically disallowed in @safe code.
That's actually not enough. You'll have to block access to
global variables too:
S s;
void main() {
s.array = RCArray!T([T()]); // s.array's refcount is now 1
foo(s.array[0]); // pass by ref
}
void foo(ref T t) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
What's the difference between that and this:
void fun() {
T[] ta = [T()].dup;
T* t = ta[0];
delete ta; // or however you do it
*t = ...;
}
Why is this a parameter passing issue and not a "you kept a
sub-reference to a deleted chunk" issue?
Re: RCArray is unsafe
On Monday, 2 March 2015 at 15:22:33 UTC, Zach the Mystic wrote:
void fun() {
T[] ta = [T()].dup;
T* t = ta[0];
I meant: T* t = &ta[0];
Cannot instantiate a std.container.Array of a class with a init() function member.
Taken from http://forum.dlang.org/thread/gjrbmskictrbcyedu...@forum.dlang.org trying to instantiate an Array!MyClass fails with a rather obscure error message if the MyClass has a member function "void init()": http://dpaste.dzfl.pl/16d202b7124d Is this a std library bug, or should this be considered a mistake on the library user side to name a member function "init"?
Re: Last week for DConf 2015 submissions
On 3/2/15 7:08 AM, Steven Schveighoffer wrote: On 2/22/15 7:03 PM, Andrei Alexandrescu wrote: Hi folks, At this point we don't have enough submissions to make DConf 2015 viable. Those we have are of good quality, but simply put more are needed. I'm sure you are poring over the proposals you received last week, but can you just let us know whether enough have been proposed to make sure the conference is still on? Thanks. DConf 2015 is definitely on. Thanks to the many last-minute submitters! -- Andrei
Re: Cannot instantiate a std.container.Array of a class with a init() function member.
On Monday, 2 March 2015 at 15:57:10 UTC, Tobias Pankrath wrote: Please file a bug report. Will do!
Re: Cannot instantiate a std.container.Array of a class with a init() function member.
On Monday, 2 March 2015 at 15:46:28 UTC, Francesco Cattoglio wrote: Taken from http://forum.dlang.org/thread/gjrbmskictrbcyedu...@forum.dlang.org trying to instantiate an Array!MyClass fails with a rather obscure error message if the MyClass has a member function "void init()": http://dpaste.dzfl.pl/16d202b7124d Is this a std library bug, or should this be considered a mistake on the library user side to name a member function "init"? I consider both your type and the library buggy. The type because .init is a property of every type with a special meaning, which is violated by your class. The library because it emits such a useful error message. Please file a bug report.
Re: Cannot instantiate a std.container.Array of a class with a init() function member.
On Monday, 2 March 2015 at 15:46:28 UTC, Francesco Cattoglio wrote: Taken from http://forum.dlang.org/thread/gjrbmskictrbcyedu...@forum.dlang.org trying to instantiate an Array!MyClass fails with a rather obscure error message if the MyClass has a member function "void init()": http://dpaste.dzfl.pl/16d202b7124d Is this a std library bug, or should this be considered a mistake on the library user side to name a member function "init"? It's kind of ridiculous that the compiler doesn't complain about this, considering that defining an init member causes an error that you would have no idea how to fix unless you already knew about this issue.
Re: Cannot instantiate a std.container.Array of a class with a init() function member.
On Mon, Mar 02, 2015 at 04:26:40PM +, Meta via Digitalmars-d wrote: > On Monday, 2 March 2015 at 15:46:28 UTC, Francesco Cattoglio wrote: > >Taken from > >http://forum.dlang.org/thread/gjrbmskictrbcyedu...@forum.dlang.org > > > >trying to instantiate an Array!MyClass fails with a rather obscure > >error message if the MyClass has a member function "void init()": > > > >http://dpaste.dzfl.pl/16d202b7124d > > > >Is this a std library bug, or should this be considered a mistake on > >the library user side to name a member function "init"? > > It's kind of ridiculous that the compiler doesn't complain about this, > considering that defining an init member causes an error that you > would have no idea how to fix unless you already knew about this > issue. I say file the bug against the compiler, not the library. The compiler should reject members named 'init' because .init already has a special meaning in the language. T -- Being able to learn is a great learning; being able to unlearn is a greater learning.
Re: [NEEDING HELP] Translation of Ali Cehreli's book in French
The chapter on function overloading is translated. If it is OK with you, I acquire the mutex on "const ref Parameters and const Member Functions", then :o)
Re: Mac Apps That Use Garbage Collection Must Move to ARC
On 02/26/2015 05:08 AM, Walter Bright wrote: > > A lot of benefit simply came from compacting all the remaining used > allocations together, essentially defragging the memory. Compacting is indeed easy once we have a precise GC, and can be done partially, i.e. objects pointed to by the stack/register are pinned.
Re: Mac Apps That Use Garbage Collection Must Move to ARC
On 02/25/2015 10:50 PM, deadalnix wrote: > > I don't think it make sens to completely discard the idea of barriers, > especially when it come to write barrier on the immutable heap. At least > that should certainly pay off. Before the argument gets lost. http://forum.dlang.org/post/mcqr3s$cmf$1...@digitalmars.com > Write barriers would cost a low single digit, e.g. 3-4%. > While searching for ways to avoid the cost I found an interesting alternative to generational GCs. > https://github.com/D-Programming-Language/druntime/pull/1081#issuecomment-69151660
Re: [NEEDING HELP] Translation of Ali Cehreli's book in French
Le 02/03/2015 19:27, Olivier Pisano a écrit : The chapter on function overloading is translated. If it is OK with you, I acquire the mutex on "const ref Parameters and const Member Functions", then :o) Ok for me. Mutex lock granted ;-)
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 13:30:39 UTC, Zach the Mystic wrote:
On Monday, 2 March 2015 at 08:59:11 UTC, deadalnix wrote:
On Monday, 2 March 2015 at 00:37:05 UTC, Zach the Mystic wrote:
I'm sure many inc/dec can still be removed.
Do you agree or disagree with what I said? I can't tell.
Yes, but I think this is overly conservative.
I'm arguing a rather liberal position: that only in a very
exceptional case do you need to protect a variable for the
duration of a function. For the most part, it's not necessary.
What am I conserving?
I let the night go over that one. Here is what I think is the
best road forward :
- triggering postblit and/or ref count bump/decrease is
prohibited on borrowed.
- Acquiring and releasing ownership does.
Now that we have this, let's get back to the exemple :
class C {
C c;
// Make ti refconted somehow, doesn't matter. Andrei's
proposal for instance.
}
void boom() {
C c = new C();
c.c = new C();
foo(c, c.c);
}
void foo(ref C c1, ref C c2) {
// Here is where things get different. c1 is borrowed, so you
can't
// do c1.c = null before acquiring c1.c beforehand. That
means the
// compiler needs to get a local copy of c1.c, bump the
refcount
// to get ownership before executing c1.c = null and decrease
// the refcount. The ownership expire when the function
returns
// so c2 is free when foo returns.
c1.c = null;
// c2 is dead.
}
The definition is a bit wonky ATM and most likely needs to be
refined, but I think this is the way forward with that issue. It
allow elision of a lot of ref increase/decrease by the compiler,
which is very important to get refcounting works fast.
Re: My Reference Safety System (DIP???)
On Sunday, 1 March 2015 at 19:35:57 UTC, deadalnix wrote: On Saturday, 28 February 2015 at 11:12:23 UTC, Marc Schütz wrote: Yes. Terminology is a problem here, I guess. When I talk about "the scope" of a variable, it means that only references to values can be stored there whose lifetimes are at least as large as the scope. Make sure you explicit that. The variable itself has a scope, and this scope is different from the scope of indirections stored in the variable. Additionally, this naturally bring the question of multiple indirection in a variable (for a struct for instance). Access to a struct member itself is not actually indirection, because the member is inside the structs memory. When we take the address of a member, we therefore know its lifetime statically: it's the lifetime of the struct variable. For accesses through a pointer (slice, etc.) one level deep, we also know some information about the destination's lifetime (that's what I'm calling scope) by looking at the return annotations (whether inferred or explicit), which the caller will enforce for us. Anything we store there needs to have a longer lifetime than the scope, and anything we read from there must not be stored where it can outlife the scope. For deeper levels of indirection, we have no such information, so we must assume the worst case: we may only store things there that we know will live indefinitely, and what we read from there could cease existing immediately after the reference to it disappears. You don't cover the lifetime of the address of operation, and I'm not how this is supposed to work in your proposal. It was in the examples, but it was wrong. I've corrected it: A dereference results in static lifetime. Will do a second pass on the damn thing :) I will also add examples how return and static annotations are handled. static annotation ? Seems like a bad idea and I'm sure we can do without. It's only necessary if parameters of `@safe` functions are automatically scoped; then we need a way to opt-out. This is actually optional and does not affect the consistency, but I thought it is a good idea, because it reduces the overall amount of annotations. And I assume that most @safe functions are already written in a way that conforms to this. We'd need to analyze some code bases to find out whether this is actually true. Ok I misunderstood what you meant by static anotation. Sounds good. Scope by default, and an optout. Problem is transition. We have a scope keyword what does it become ? It's not yet spelled out clearly enough, but by default, any parameters not marked as `scope` are treated as having infinite lifetime. This means that `scope` annotations would need to appear everywhere. However, they are inferred for template functions, together with `return` annotations, removing a large part of explicit annotations. (They are also always inferred for local variables.) For @safe functions, no inference is done, unless they are templates; instead, all references implicitly get a `scope` annotation, but no `return` annotations. The latter can be added manually, and we can opt-out from `scope` by using `static`. Ideally most code should be @safe, and there was even talk about @safe by default, therefore most code shouldn't need to be annotated manually. The transition can then be just like for DIP25. `scope`, `static` and `return` are no-ops, and can be enabled by a command-line switch. Later, they are enabled by default, and can be disabled by a switch. Finally, the switch is removed.
Re: My Reference Safety System (DIP???)
On Sunday, 1 March 2015 at 23:56:02 UTC, Zach the Mystic wrote:
On Sunday, 1 March 2015 at 14:40:54 UTC, Marc Schütz wrote:
I don't think a callee-based solution can work:
class T {
void doSomething() scope;
}
struct S {
RC!T t;
}
void main() {
auto s = S(RC!T()); // `s.t`'s refcount is 1
T t = s.t; // borrowing from the RC wrapper
foo(s);
t.doSomething();// oops, `t` is gone
}
void foo(ref S s) {
s.t = RC!T(); // drops the old `s.t`
}
I thought of this, and I disagree. The very fact of assigning
to `T t` adds the reference count you need to keep `s.t` from
disintegrating. As soon as you borrow, you increment the count.
Sorry, my mistake, should have explained what I have in mind.
`S.t` has type `RC!T`, but we're assigning it a variable of type
`T`. This is made possible because `RC!T` has an `alias this`
wrapper that returns `scope T`. The effect is that we're
implicitly borrowing the `T` reference, as if the variable were
declared `scope T`. The borrow checker (which I will specify
later, see the examples [1] for a foretaste) will prohibit any
unsafe use that would make the reference `t` outlive `s`.
Therefore, no postblit is called, and no reference count is
incremented.
[1] http://wiki.dlang.org/User_talk:Schuetzm/scope2
Re: RCArray is unsafe
On 3/1/2015 12:51 PM, Michel Fortin wrote:
That's actually not enough. You'll have to block access to global variables too:
S s;
void main() {
s.array = RCArray!T([T()]); // s.array's refcount is now 1
foo(s.array[0]); // pass by ref
}
void foo(ref T t) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
Thinking about it, there are many other ways this can happen. At the moment, I'm
stuck thinking of a solution other than requiring foo() to be pure. Anyone have
ideas?
Re: RCArray is unsafe
On 3/1/15 2:21 PM, Walter Bright wrote:
On 3/1/2015 7:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= "
wrote:
A weakness of the same kind affects DIP25, too. The core of the
problem is
borrowing (ref return as in DIP25), combined with manual (albeit
hidden) memory
management. An example to illustrate:
struct T {
void doSomething();
}
struct S {
RCArray!T array;
}
void main() {
auto s = S(RCArray!T([T()])); // s.array's refcount is now 1
foo(s, s.array[0]); // pass by ref
}
void foo(ref S s, ref T T) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
This is an odd example, how does one take a ref to an RCArray element
without the machinery to retain the array? I would think that RCArray[x]
would return something that isn't passable to a function. Or am I
missing something?
The trouble seems to happen when there are two references to the same
object passed to a function. I.e. there can be only one "borrowed" ref
at a time.
Not exactly. Note that we are taking by reference S, which is NOT
reference counted. So you are passing indirectly a reference to an RC
object. You aren't "borrowing" that reference.
-Steve
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 20:04:49 UTC, deadalnix wrote:
On Monday, 2 March 2015 at 13:30:39 UTC, Zach the Mystic wrote:
On Monday, 2 March 2015 at 08:59:11 UTC, deadalnix wrote:
On Monday, 2 March 2015 at 00:37:05 UTC, Zach the Mystic
wrote:
I'm sure many inc/dec can still be removed.
Do you agree or disagree with what I said? I can't tell.
Yes, but I think this is overly conservative.
I'm arguing a rather liberal position: that only in a very
exceptional case do you need to protect a variable for the
duration of a function. For the most part, it's not necessary.
What am I conserving?
I let the night go over that one. Here is what I think is the
best road forward :
- triggering postblit and/or ref count bump/decrease is
prohibited on borrowed.
- Acquiring and releasing ownership does.
Now that we have this, let's get back to the exemple :
class C {
C c;
// Make ti refconted somehow, doesn't matter. Andrei's
proposal for instance.
}
void boom() {
C c = new C();
c.c = new C();
foo(c, c.c);
}
void foo(ref C c1, ref C c2) {
// Here is where things get different. c1 is borrowed, so
you can't
// do c1.c = null before acquiring c1.c beforehand. That
means the
// compiler needs to get a local copy of c1.c, bump the
refcount
// to get ownership before executing c1.c = null and
decrease
// the refcount. The ownership expire when the function
returns
// so c2 is free when foo returns.
c1.c = null;
// c2 is dead.
}
The definition is a bit wonky ATM and most likely needs to be
refined, but I think this is the way forward with that issue.
It allow elision of a lot of ref increase/decrease by the
compiler, which is very important to get refcounting works fast.
Interesting approach. I will have to think about that. But I
think it does not really work. Your example hides the fact that
there are actually two types involved (or can be): an RC wrapper,
and the actual class. foo() would need to take at least `c1` as
the wrapper type `RC!C`, not `C` itself, otherwise it couldn't
copy it. But that defeats the purpose of borrowing, that it
neutralizes the actual memory management strategy; foo() should
know whether `c1` is reference counted or not.
Re: A Refcounted Array Type
On 2/27/15 11:18 PM, ketmar wrote:
On Fri, 27 Feb 2015 20:51:54 +, deadalnix wrote:
On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer wrote:
In that case, you shouldn't be subject to any kind of race conditions.
But we can't make the library/language based on this assumption. Your
case is the exceptional case.
His case is not @safe, so we can ignore that problem.
the compiler tends to disagree:
=== test.d ===
int myglobal;
class A {
~this () @safe { if (myglobal == 42) assert(0); }
}
void main () {
auto a = new A;
}
==
dmd -w -c -o- test.d
wow! no warnings, no errors!
I think in this case, the compiler can probably statically reject the
code. But there are definitely ways around that, so I don't think full
compliance is possible. I also think it's wrong-headed to have the
compiler reject such code based on a GC deficiency.
The only solution is to have GC-allocated data be destroyed in the same
thread it's allocated in. This brings the multi-threaded program into
the same realm as yours -- looks like one thread for unshared data :)
-Steve
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 20:36:53 UTC, Marc Schütz wrote: Interesting approach. I will have to think about that. But I think it does not really work. Your example hides the fact that there are actually two types involved (or can be): an RC wrapper, and the actual class. foo() would need to take at least `c1` as the wrapper type `RC!C`, not `C` itself, otherwise it couldn't copy it. But that defeats the purpose of borrowing, that it neutralizes the actual memory management strategy; foo() should know whether `c1` is reference counted or not. Please reread. I'm assuming a refcounting system like Andrei's proposal for objects. The result would be the same for a RefCounted wrapper (a solution that I would prefer) in the sense you'd have to copy the wrapper to get ownership of it before being able to assign to it.
Re: RCArray is unsafe
On 3/2/2015 12:37 PM, Steven Schveighoffer wrote: Not exactly. Note that we are taking by reference S, which is NOT reference counted. So you are passing indirectly a reference to an RC object. You aren't "borrowing" that reference. It's seems that to solve the problem, it has to be borrowed, in that a static check will be needed that what it points to cannot be modified/deleted through other references to the same data.
Re: RCArray is unsafe
On 3/2/2015 12:42 PM, Walter Bright wrote: On 3/2/2015 12:37 PM, Steven Schveighoffer wrote: Not exactly. Note that we are taking by reference S, which is NOT reference counted. So you are passing indirectly a reference to an RC object. You aren't "borrowing" that reference. It's seems that to solve the problem, it has to be borrowed, in that a static check will be needed that what it points to cannot be modified/deleted through other references to the same data. I looked at how Rust does it. I was hoping it was something clever, but it isn't. A copy is made of the object to which a borrowed reference is taken - in other words, the reference count is incremented. For D structs, that means, if there's a postblit, a copy must be made. For D ref counted classes, a ref count increment must be done. I was hoping to avoid that, but apparently there's no way. There are cases where this can be avoided, like calling pure functions. Another win for pure functions!
Re: RCArray is unsafe
On 3/2/15 12:53 PM, Walter Bright wrote: I was hoping to avoid that, but apparently there's no way. There is! There is! -- Andrei
Re: RCArray is unsafe
On Sunday, 1 March 2015 at 19:22:06 UTC, Walter Bright wrote:
On 3/1/2015 7:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?=
" wrote:
A weakness of the same kind affects DIP25, too. The core of
the problem is
borrowing (ref return as in DIP25), combined with manual
(albeit hidden) memory
management. An example to illustrate:
struct T {
void doSomething();
}
struct S {
RCArray!T array;
}
void main() {
auto s = S(RCArray!T([T()])); // s.array's refcount is
now 1
foo(s, s.array[0]); // pass by ref
}
void foo(ref S s, ref T T) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
The trouble seems to happen when there are two references to
the same object passed to a function. I.e. there can be only
one "borrowed" ref at a time.
I'm thinking this could be statically disallowed in @safe code.
Yes, it's a classical aliasing problem. Of course, if the
references can't possible alias because of their types, it's ok.
However, for non-pure functions, it's always (?) unsafe, because
they have access to all kinds of global variables. Too bad we
don't have pure by default :-(
Re: RCArray is unsafe
On 3/2/15 12:37 PM, Walter Bright wrote:
On 3/1/2015 12:51 PM, Michel Fortin wrote:
That's actually not enough. You'll have to block access to global
variables too:
S s;
void main() {
s.array = RCArray!T([T()]); // s.array's refcount is now 1
foo(s.array[0]); // pass by ref
}
void foo(ref T t) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
Thinking about it, there are many other ways this can happen. At the
moment, I'm stuck thinking of a solution other than requiring foo() to
be pure. Anyone have ideas?
I have a solution (as in working implementation), but also a deadline
that's staring me in the face so this will have to wait a couple more days.
I also have a bit of an attack to const/immutable support but that's
less good (allows too much freedom).
Andrei
Re: RCArray is unsafe
On Monday, 2 March 2015 at 21:00:54 UTC, Andrei Alexandrescu
wrote:
On 3/2/15 12:37 PM, Walter Bright wrote:
On 3/1/2015 12:51 PM, Michel Fortin wrote:
That's actually not enough. You'll have to block access to
global
variables too:
S s;
void main() {
s.array = RCArray!T([T()]); // s.array's refcount
is now 1
foo(s.array[0]); // pass by ref
}
void foo(ref T t) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
Thinking about it, there are many other ways this can happen.
At the
moment, I'm stuck thinking of a solution other than requiring
foo() to
be pure. Anyone have ideas?
I have a solution (as in working implementation), but also a
deadline that's staring me in the face so this will have to
wait a couple more days.
I also have a bit of an attack to const/immutable support but
that's less good (allows too much freedom).
"I have discovered a marvellous solution, but this post is too
short to describe it."
Re: RCArray is unsafe
On Monday, 2 March 2015 at 20:54:20 UTC, Walter Bright wrote: I looked at how Rust does it. I was hoping it was something clever, but it isn't. A copy is made of the object to which a borrowed reference is taken - in other words, the reference count is incremented. For D structs, that means, if there's a postblit, a copy must be made. For D ref counted classes, a ref count increment must be done. I was hoping to avoid that, but apparently there's no way. There are cases where this can be avoided, like calling pure functions. Another win for pure functions! I do think you are confusing how Rust does it. In Rust, borrowing makes the source and borrowed reference immutable by default. So by default the problem do not occurs. You can also borrow a mutable reference, in which case the source is disabled for the duration of the borrowing. That makes it impossible to borrow a mutable reference twice. The above mentioned scenario cannot happen at all in Rust, by design. As a result, there is solution for the problem, as the problem cannot occur in the first place.
Re: Making RCSlice and DIP74 work with const and immutable
On 3/2/15 4:10 PM, Steven Schveighoffer wrote: Just as an aside, I know that immutable is implicitly shared because it does not need to be synchronized with a mutex. This makes it ideal for sharing. But the reality is, there are other reasons to make something immutable. And it poisons const in this way. Note that the array runtime still treats const items as thread local, not shared. This could potentially cause problems if you shared a tail-immutable array, and tried appending. All this, except I meant the array runtime treats both const *and* immutable items as thread-local. -Steve
Re: Making RCSlice and DIP74 work with const and immutable
On 2/28/15 8:40 PM, Andrei Alexandrescu wrote: Tracing garbage collection can afford the luxury of e.g. mutating data that was immutable during its lifetime. Reference counting needs to make minute mutations to data while references to that data are created. In fact, it's not mutation of the "useful" data, the payload of a data structure; it's mutation of metadata, additional information about the data (i.e. a reference count integral). The RCOs described in DIP74 and also RCSlice discussed in this forum need to work properly with const and immutable. Therefore, they need a way to reliably define and access metadata for a data structure. One possible solution is to add a "@mutable" or "@metadata" attribute similar to C++'s keyword "mutable". Walter and I both dislike that solution because it's hamfisted and leaves too much opportunity for abuse - people can essentially create unbounded amounts of mutable payload for an object claimed to be immutable. That makes it impossible (or unsafe) to optimize code based on algebraic assumptions. We have a few candidates for solutions, but wanted to open with a good discussion first. So, how do you envision a way to define and access mutable metadata for objects (including immutable ones)? So, the largest problem (already pointed out by many here), is that immutable is implicitly shared. This means const must be treated as implicitly shared. But I have an idea, not sure if it's viable or not, to try and mitigate this. What if, at the point of passing in a mutable or immutable item to a const function, opAddRef is called. Then when the function call returns, opRelease is called. Then during the function, you never have to worry about const ref counting, and you never have to worry about trying to atomically ref count mutable items. This means opAddRef() const and opRelease() const would be illegal. Of course, for shared and immutable items, you will need to deal with an atomic/shared count. I haven't figured out that problem yet, but I think Michel Fortin has some good ideas. This leaves the only issue of creating "const" RC objects. We can just ban that (no sense in doing that, just create an immutable version). const then effectively becomes a borrowing type constructor. Just as an aside, I know that immutable is implicitly shared because it does not need to be synchronized with a mutex. This makes it ideal for sharing. But the reality is, there are other reasons to make something immutable. And it poisons const in this way. Note that the array runtime still treats const items as thread local, not shared. This could potentially cause problems if you shared a tail-immutable array, and tried appending. I would love to see immutable be by default thread local, and need to cast into shared(immutable) to be shareable. shared(immutable) is still pretty easy to use, since it would require no synchronization (except in these exceptional ways we are talking about). One very important thing to consider here, is the optimization of pure functions. An immutable pointer passed into a pure function means the compiler can elide identical calls. This means opAddRef() immutable CANNOT be pure, or at least it cannot be strong pure. This really hints towards some sort of global hash. But there may be a better way: What if the compiler marked a region of the data inside the object as "meta", and then when you called opAddRef() immutable, it's really calling opAddRef(Meta m) immutable. The way this works is: 1. The meta data is NOT EXTRACTABLE from the object any other way. 2. The meta data is ALWAYS mutable and (effectively) shared. Essentially, this is how synchronized works too. And you can logically place the metadata somewhere outside the object if that makes more sense. Just thinking out loud here... -Steve
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 20:04:49 UTC, deadalnix wrote:
I let the night go over that one. Here is what I think is the
best road forward :
- triggering postblit and/or ref count bump/decrease is
prohibited on borrowed.
- Acquiring and releasing ownership does.
Now that we have this, let's get back to the exemple :
class C {
C c;
// Make ti refconted somehow, doesn't matter. Andrei's
proposal for instance.
}
void boom() {
C c = new C();
c.c = new C();
foo(c, c.c);
}
void foo(ref C c1, ref C c2) {
// Here is where things get different. c1 is borrowed, so
you can't
// do c1.c = null before acquiring c1.c beforehand.
Right, I agree with this.
That means the
// compiler needs to get a local copy of c1.c, bump the
refcount
// to get ownership before executing c1.c = null and
decrease
// the refcount.
Yeah, but should it do this inside foo() or in bump() right
before it calls foo. I think in bump, and only for a parameter
which might be aliased by another parameter (an extremely rare
case). For any other case, the refcount has already been
preserved:
void boom() {
C c = new C(); // refcount(c) == 1
c.c = new C(); // refcount(c.c) == 1
auto d = c.c; // refcount(c.c) == 2 now
foo(c, d); // safe
}
The only problem is the rare case when the exact same identifier
is getting sent to two different parameters.
I'm sure there will be opportunities to elide a lot of refcount
calls, but in this case, I don't see much to left to elide.
Refactoring D as an MSc Project
Hello all! This is my first post on the forums. I'm interested in possibly creating a refactoring tool for D for my MSc dissertation (I notice there is currently no refactoring in D?). I wanted to ask if this is possible with D's current state? More specifically: - Can I easily access things such as the AST?
Re: Refactoring D as an MSc Project
On Monday, 2 March 2015 at 21:50:46 UTC, Jamie wrote: Hello all! This is my first post on the forums. I'm interested in possibly creating a refactoring tool for D for my MSc dissertation (I notice there is currently no refactoring in D?). I wanted to ask if this is possible with D's current state? More specifically: - Can I easily access things such as the AST? This should save you a lot of time: https://github.com/Hackerpilot/libdparse
Re: Refactoring D as an MSc Project
Thanks, I'll give it a look :)
Re: My Reference Safety System (DIP???)
You don't put the ownership acquire at the same place, but that is the same idea. It is probably even better to do it your way (or is it ?).
Re: RCArray is unsafe
On Monday, 2 March 2015 at 20:54:20 UTC, Walter Bright wrote: On 3/2/2015 12:42 PM, Walter Bright wrote: For D structs, that means, if there's a postblit, a copy must be made. For D ref counted classes, a ref count increment must be done. I was hoping to avoid that, but apparently there's no way. There are cases where this can be avoided, like calling pure functions. Another win for pure functions! It seems like the most common use case for passing a global to a parameter is to process that global in a pure way. You already have access to it in an impure function, so you could just access it directly. The good news is that from within the passed-to function, no further calls will treat it as global.
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 22:00:56 UTC, deadalnix wrote: You don't put the ownership acquire at the same place, but that is the same idea. It is probably even better to do it your way (or is it ?). Yes. Unless the compiler detects that you duplicate a variable in two parameters in the same call, you literally have *no* added cycles, anywhere: fun(c, c.c); This is the only time you pay any penalty (except for passing globals, as we now realize, since all globals can alias themselves as parameters -- nasty).
Re: RCArray is unsafe
On Monday, 2 March 2015 at 21:12:20 UTC, deadalnix wrote: On Monday, 2 March 2015 at 20:54:20 UTC, Walter Bright wrote: I looked at how Rust does it. I was hoping it was something clever, but it isn't. A copy is made of the object to which a borrowed reference is taken - in other words, the reference count is incremented. For D structs, that means, if there's a postblit, a copy must be made. For D ref counted classes, a ref count increment must be done. I was hoping to avoid that, but apparently there's no way. There are cases where this can be avoided, like calling pure functions. Another win for pure functions! I do think you are confusing how Rust does it. In Rust, borrowing makes the source and borrowed reference immutable by default. So by default the problem do not occurs. You can also borrow a mutable reference, in which case the source is disabled for the duration of the borrowing. That makes it impossible to borrow a mutable reference twice. The above mentioned scenario cannot happen at all in Rust, by design. As a result, there is solution for the problem, as the problem cannot occur in the first place. I don't think you were advocating for this but, +1 for a borrow system similar to Rust.
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 22:21:11 UTC, Zach the Mystic wrote: On Monday, 2 March 2015 at 22:00:56 UTC, deadalnix wrote: You don't put the ownership acquire at the same place, but that is the same idea. It is probably even better to do it your way (or is it ?). Yes. Unless the compiler detects that you duplicate a variable in two parameters in the same call, you literally have *no* added cycles, anywhere: fun(c, c.c); This is the only time you pay any penalty (except for passing globals, as we now realize, since all globals can alias themselves as parameters -- nasty). Global simply are parameter implicitly passed to all function from a theoretical perspective. There are no reason to thread them differently.
Re: RCArray is unsafe
On 3/2/2015 1:12 PM, deadalnix wrote: I do think you are confusing how Rust does it. In Rust, borrowing makes the source and borrowed reference immutable by default. So by default the problem do not occurs. May I refer you to: http://static.rust-lang.org/doc/0.6/tutorial-borrowed-ptr.html#borrowing-managed-boxes-and-rooting "Again the lifetime of y is L, the remainder of the function body. But there is a crucial difference: suppose x were to be reassigned during the lifetime L? If the compiler isn't careful, the managed box could become unrooted, and would therefore be subject to garbage collection. A heap box that is unrooted is one such that no pointer values in the heap point to it. It would violate memory safety for the box that was originally assigned to x to be garbage-collected, since a non-heap pointer---y---still points into it. [...] For this reason, whenever an & expression borrows the interior of a managed box stored in a mutable location, the compiler inserts a temporary that ensures that the managed box remains live for the entire lifetime. [...] This process is called rooting." It's possible that this is an old and obsolete document, but it's what I found.
Re: RCArray is unsafe
On 3/2/2015 1:09 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= " wrote: "I have discovered a marvellous solution, but this post is too short to describe it." Fortunately, Fermat (er, Andrei) was able to pass along his dazz idea to me this afternoon, before he expired to something he had to attend to. We were both in the dumps about this problem last night, but I think he solved it. His insight was that the deletion of the payload occurred before the end of the lifetime of the RC object, and that this was the source of the problem. If the deletion of the payload occurs during the destructor call, rather than the postblit, then although the ref count of the payload goes to zero, it doesn't actually get deleted. I.e. the postblit manipulates the ref count, but does NOT do payload deletions. The destructor checks the ref count, if it is zero, THEN it does the payload deletion. Pretty dazz idea, dontcha think? And DIP25 still stands unscathed :-) Unless, of course, we missed something obvious.
Re: Cannot instantiate a std.container.Array of a class with a init() function member.
On 03/02/2015 09:48 AM, H. S. Teoh via Digitalmars-d wrote: The compiler should reject members named 'init' because .init already has a special meaning in the language. Existing issue: https://issues.dlang.org/show_bug.cgi?id=12545 Ali
Re: RCArray is unsafe
On 3/2/2015 3:22 PM, Andrei Alexandrescu wrote: And since an RCArray may undergo several assignments during its lifetime (thus potentially needing to free several chunks of memory), the arrays to be destroyed will be kept in a freelist-style structure. Destructor walks the freelist and frees the chunks. Right, I had neglected to add that.
Re: RCArray is unsafe
On 3/2/15 2:57 PM, Walter Bright wrote: On 3/2/2015 1:09 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= " wrote: "I have discovered a marvellous solution, but this post is too short to describe it." Fortunately, Fermat (er, Andrei) was able to pass along his dazz idea to me this afternoon, before he expired to something he had to attend to. We were both in the dumps about this problem last night, but I think he solved it. His insight was that the deletion of the payload occurred before the end of the lifetime of the RC object, and that this was the source of the problem. If the deletion of the payload occurs during the destructor call, rather than the postblit, then although the ref count of the payload goes to zero, it doesn't actually get deleted. I.e. the postblit manipulates the ref count, but does NOT do payload deletions. The destructor checks the ref count, if it is zero, THEN it does the payload deletion. Pretty dazz idea, dontcha think? And DIP25 still stands unscathed :-) Unless, of course, we missed something obvious. And since an RCArray may undergo several assignments during its lifetime (thus potentially needing to free several chunks of memory), the arrays to be destroyed will be kept in a freelist-style structure. Destructor walks the freelist and frees the chunks. Andrei
Re: RCArray is unsafe
On 3/2/15 3:22 PM, Andrei Alexandrescu wrote: On 3/2/15 2:57 PM, Walter Bright wrote: On 3/2/2015 1:09 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= " wrote: "I have discovered a marvellous solution, but this post is too short to describe it." Fortunately, Fermat (er, Andrei) was able to pass along his dazz idea to me this afternoon, before he expired to something he had to attend to. We were both in the dumps about this problem last night, but I think he solved it. His insight was that the deletion of the payload occurred before the end of the lifetime of the RC object, and that this was the source of the problem. If the deletion of the payload occurs during the destructor call, rather than the postblit, then although the ref count of the payload goes to zero, it doesn't actually get deleted. I.e. the postblit manipulates the ref count, but does NOT do payload deletions. The destructor checks the ref count, if it is zero, THEN it does the payload deletion. Pretty dazz idea, dontcha think? And DIP25 still stands unscathed :-) Unless, of course, we missed something obvious. And since an RCArray may undergo several assignments during its lifetime (thus potentially needing to free several chunks of memory), the arrays to be destroyed will be kept in a freelist-style structure. Destructor walks the freelist and frees the chunks. We may apply the same principle (memory may only be freed during destruction) to DIP74. If we figure a cheap way to keep a freelist of objects to be deallocated, we can borrow multiple times without a reference count add. But I didn't think this through yet. -- Andrei
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 22:51:29 UTC, deadalnix wrote: On Monday, 2 March 2015 at 22:21:11 UTC, Zach the Mystic wrote: On Monday, 2 March 2015 at 22:00:56 UTC, deadalnix wrote: You don't put the ownership acquire at the same place, but that is the same idea. It is probably even better to do it your way (or is it ?). Yes. Unless the compiler detects that you duplicate a variable in two parameters in the same call, you literally have *no* added cycles, anywhere: fun(c, c.c); This is the only time you pay any penalty (except for passing globals, as we now realize, since all globals can alias themselves as parameters -- nasty). Global simply are parameter implicitly passed to all function from a theoretical perspective. There are no reason to thread them differently. Except for this: static Rctype t; // fun(t); Now you have that implicit parameter which screws things up. It's like calling: fun(@globals, t); ...where @globals is a namespace which can alias t. So you have two parameters which can alias each other. I think the only saving grace is that you probably don't really need to pass a global that often, since you already have it if you want it. Only if you want the global to "play the role" of a parameter. What do you think? How many times do you normally pass a global?
Re: RCArray is unsafe
On Monday, 2 March 2015 at 22:51:03 UTC, Walter Bright wrote: On 3/2/2015 1:12 PM, deadalnix wrote: I do think you are confusing how Rust does it. In Rust, borrowing makes the source and borrowed reference immutable by default. So by default the problem do not occurs. May I refer you to: http://static.rust-lang.org/doc/0.6/tutorial-borrowed-ptr.html#borrowing-managed-boxes-and-rooting "Again the lifetime of y is L, the remainder of the function body. But there is a crucial difference: suppose x were to be reassigned during the lifetime L? If the compiler isn't careful, the managed box could become unrooted, and would therefore be subject to garbage collection. A heap box that is unrooted is one such that no pointer values in the heap point to it. It would violate memory safety for the box that was originally assigned to x to be garbage-collected, since a non-heap pointer---y---still points into it. [...] For this reason, whenever an & expression borrows the interior of a managed box stored in a mutable location, the compiler inserts a temporary that ensures that the managed box remains live for the entire lifetime. [...] This process is called rooting." It's possible that this is an old and obsolete document, but it's what I found. That's actually very old(0.6 - nearly 2 years ago.) I believe rooting was dropped from the language completely. http://doc.rust-lang.org/book/ownership.html also, rust by example chapter 17-19 cover this http://rustbyexample.com/move.html
Re: RCArray is unsafe
On Monday, 2 March 2015 at 22:58:19 UTC, Walter Bright wrote: I.e. the postblit manipulates the ref count, but does NOT do payload deletions. The destructor checks the ref count, if it is zero, THEN it does the payload deletion. Pretty dazz idea, dontcha think? And DIP25 still stands unscathed :-) Unless, of course, we missed something obvious. Add me to "we". I'm dazzed! :-)
Re: RCArray is unsafe
On 3/2/15 4:01 PM, Zach the Mystic wrote: On Monday, 2 March 2015 at 22:58:19 UTC, Walter Bright wrote: His insight was that the deletion of the payload occurred before the end of the lifetime of the RC object, and that this was the source of the problem. If the deletion of the payload occurs during the destructor call, rather than the postblit, RcArray, a struct, already does this. You wouldn't delete in a postblit anyway would you? Deletion occurs in opAssign. Do you need opRelease and ~this to be separate for structs too? Probably not. Andrei
Re: RCArray is unsafe
On Monday, 2 March 2015 at 22:58:19 UTC, Walter Bright wrote: His insight was that the deletion of the payload occurred before the end of the lifetime of the RC object, and that this was the source of the problem. If the deletion of the payload occurs during the destructor call, rather than the postblit, RcArray, a struct, already does this. You wouldn't delete in a postblit anyway would you? Do you need opRelease and ~this to be separate for structs too?
Re: My Reference Safety System (DIP???)
On Monday, 2 March 2015 at 23:43:22 UTC, Zach the Mystic wrote: Except for this: static Rctype t; // fun(t); Now you have that implicit parameter which screws things up. It's like calling: fun(@globals, t); ...where @globals is a namespace which can alias t. So you have two parameters which can alias each other. I think the only saving grace is that you probably don't really need to pass a global that often, since you already have it if you want it. Only if you want the global to "play the role" of a parameter. What do you think? How many times do you normally pass a global? I fail too see how t being global vs t being a local that is doubly passed change anything.
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 00:05:50 UTC, Zach the Mystic wrote: I guess you also mean opAssigns -- they would manipulate refcounts too right? In fact, they would be the primary means of decrementing the refcount *apart* from the destructor, right? Nevermind. I was a minute too soon with my post!
Re: RCArray is unsafe
On Monday, 2 March 2015 at 22:58:19 UTC, Walter Bright wrote: His insight was that the deletion of the payload occurred before the end of the lifetime of the RC object, and that this was the source of the problem. If the deletion of the payload occurs during the destructor call, rather than the postblit, then although the ref count of the payload goes to zero, it doesn't actually get deleted. I.e. the postblit manipulates the ref count, but does NOT do payload deletions. The destructor checks the ref count, if it is zero, THEN it does the payload deletion. I guess you also mean opAssigns -- they would manipulate refcounts too right? In fact, they would be the primary means of decrementing the refcount *apart* from the destructor, right?
Re: RCArray is unsafe
On 3/2/2015 3:47 PM, weaselcat wrote: That's actually very old(0.6 - nearly 2 years ago.) I believe rooting was dropped from the language completely. http://doc.rust-lang.org/book/ownership.html also, rust by example chapter 17-19 cover this http://rustbyexample.com/move.html Thanks for the info. But I don't see how Marc's example is prevented by this.
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 00:08:10 UTC, Walter Bright wrote: On 3/2/2015 3:47 PM, weaselcat wrote: That's actually very old(0.6 - nearly 2 years ago.) I believe rooting was dropped from the language completely. http://doc.rust-lang.org/book/ownership.html also, rust by example chapter 17-19 cover this http://rustbyexample.com/move.html Thanks for the info. But I don't see how Marc's example is prevented by this. It wouldn't be compilable(it borrows the same object mutably twice) I think so, anyways. I'm not that big on rust. During a borrow, the owner may NOT (a) mutate the resource, or (b) mutably lend the resource. During a mutable borrow, the owner may NOT (a) access the resource, or (b) lend the resource.
Re: My Reference Safety System (DIP???)
On Tuesday, 3 March 2015 at 00:02:48 UTC, deadalnix wrote: What do you think? How many times do you normally pass a global? I fail too see how t being global vs t being a local that is doubly passed change anything. Within the function, the global passed as a parameter creates an alias to the global. Fortunately, Andrei Fermat may have just solved the issue: http://forum.dlang.org/post/md2pub$nqn$1...@digitalmars.com
Re: My Reference Safety System (DIP???)
On Tuesday, 3 March 2015 at 00:11:36 UTC, Zach the Mystic wrote: On Tuesday, 3 March 2015 at 00:02:48 UTC, deadalnix wrote: What do you think? How many times do you normally pass a global? I fail too see how t being global vs t being a local that is doubly passed change anything. Within the function, the global passed as a parameter creates an alias to the global. Fortunately, Andrei Fermat may have just solved the issue: http://forum.dlang.org/post/md2pub$nqn$1...@digitalmars.com This does not solve anything as postblit only increase refcount so it does not make any sense that it deletes the payload.
Re: RCArray is unsafe
On Monday, 2 March 2015 at 22:51:03 UTC, Walter Bright wrote: On 3/2/2015 1:12 PM, deadalnix wrote: I do think you are confusing how Rust does it. In Rust, borrowing makes the source and borrowed reference immutable by default. So by default the problem do not occurs. May I refer you to: http://static.rust-lang.org/doc/0.6/tutorial-borrowed-ptr.html#borrowing-managed-boxes-and-rooting "Again the lifetime of y is L, the remainder of the function body. But there is a crucial difference: suppose x were to be reassigned during the lifetime L? If the compiler isn't careful, the managed box could become unrooted, and would therefore be subject to garbage collection. A heap box that is unrooted is one such that no pointer values in the heap point to it. It would violate memory safety for the box that was originally assigned to x to be garbage-collected, since a non-heap pointer---y---still points into it. [...] For this reason, whenever an & expression borrows the interior of a managed box stored in a mutable location, the compiler inserts a temporary that ensures that the managed box remains live for the entire lifetime. [...] This process is called rooting." It's possible that this is an old and obsolete document, but it's what I found. Yes, this page is obsolete, but it is still very interesting as it is a viable solution for us.
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 00:08:10 UTC, Walter Bright wrote: On 3/2/2015 3:47 PM, weaselcat wrote: That's actually very old(0.6 - nearly 2 years ago.) I believe rooting was dropped from the language completely. http://doc.rust-lang.org/book/ownership.html also, rust by example chapter 17-19 cover this http://rustbyexample.com/move.html Thanks for the info. But I don't see how Marc's example is prevented by this. After moving resources, the previous owner can no longer be used. That means you cannot borrow twice, which mean you can't get yourself into the mentioned situation.
Re: My Reference Safety System (DIP???)
On 3/2/15 4:35 PM, deadalnix wrote: On Tuesday, 3 March 2015 at 00:11:36 UTC, Zach the Mystic wrote: On Tuesday, 3 March 2015 at 00:02:48 UTC, deadalnix wrote: What do you think? How many times do you normally pass a global? I fail too see how t being global vs t being a local that is doubly passed change anything. Within the function, the global passed as a parameter creates an alias to the global. Fortunately, Andrei Fermat may have just solved the issue: http://forum.dlang.org/post/md2pub$nqn$1...@digitalmars.com This does not solve anything as postblit only increase refcount so it does not make any sense that it deletes the payload. Yah, it's opAssign instead of postblit. -- Andrei
Re: Refactoring D as an MSc Project
On 3/03/2015 10:50 a.m., Jamie wrote: Hello all! This is my first post on the forums. I'm interested in possibly creating a refactoring tool for D for my MSc dissertation (I notice there is currently no refactoring in D?). I wanted to ask if this is possible with D's current state? More specifically: - Can I easily access things such as the AST? You may also be interested in SDC[0]. [0] https://github.com/deadalnix/SDC
Re: My Reference Safety System (DIP???)
On Tuesday, 3 March 2015 at 00:47:01 UTC, Andrei Alexandrescu wrote: This does not solve anything as postblit only increase refcount so it does not make any sense that it deletes the payload. Yah, it's opAssign instead of postblit. -- Andrei So it is an auto expanding arena, and when all refcount go to 0, the whole arena is blasted, is that right ? Sounds like it can work, but that means very few outside phobos will build upon this.
Re: Refactoring D as an MSc Project
On Monday, 2 March 2015 at 21:54:05 UTC, Brian Schott wrote: On Monday, 2 March 2015 at 21:50:46 UTC, Jamie wrote: Hello all! This is my first post on the forums. I'm interested in possibly creating a refactoring tool for D for my MSc dissertation (I notice there is currently no refactoring in D?). I wanted to ask if this is possible with D's current state? More specifically: - Can I easily access things such as the AST? This should save you a lot of time: https://github.com/Hackerpilot/libdparse I've had a look around and it seems promising, however are there any usage examples poking around somewhere I could look at? Also, does the produced AST have location information?
Re: RCArray is unsafe
On Monday, 2 March 2015 at 20:37:46 UTC, Walter Bright wrote:
On 3/1/2015 12:51 PM, Michel Fortin wrote:
That's actually not enough. You'll have to block access to
global variables too:
S s;
void main() {
s.array = RCArray!T([T()]); // s.array's refcount is
now 1
foo(s.array[0]); // pass by ref
}
void foo(ref T t) {
s.array = RCArray!T([]); // drop the old s.array
t.doSomething(); // oops, t is gone
}
So with Andrei's solution, will s.array ever get freed, since s
is a global? I guess it *should* never get freed, since s is a
global and it will always exist as a reference.
Which makes me think about a bigger problem... when you opAssign,
don't you redirect the variable to a different instance? Won't
the destructor then destroy *that* instance (or not destroy it,
since it just got a +1 count) instead of the one most recently
decremented? How does it hold onto the instance to be destroyed?
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 01:23:24 UTC, Zach the Mystic wrote: Which makes me think about a bigger problem... when you opAssign, don't you redirect the variable to a different instance? Won't the destructor then destroy *that* instance (or not destroy it, since it just got a +1 count) instead of the one most recently decremented? How does it hold onto the instance to be destroyed? auto y = new RcStruct; y = null; y's old RcStruct gets decremented to zero, but who owns it now? Whose destructor ever gets run on it?
Re: RCArray is unsafe
On 3/2/2015 4:40 PM, deadalnix wrote: After moving resources, the previous owner can no longer be used. How does that work with the example presented by Marc?
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote: On 3/2/2015 4:40 PM, deadalnix wrote: After moving resources, the previous owner can no longer be used. How does that work with the example presented by Marc? He couldn't pass s and a member of s because s is borrowed as mutable. He would have to pass both as immutable.
Re: Mac Apps That Use Garbage Collection Must Move to ARC
On 3/2/2015 11:38 AM, Martin Nowak wrote: Compacting is indeed easy once we have a precise GC, and can be done partially, i.e. objects pointed to by the stack/register are pinned. Also unions.
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 02:04:15 UTC, weaselcat wrote: On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote: On 3/2/2015 4:40 PM, deadalnix wrote: After moving resources, the previous owner can no longer be used. How does that work with the example presented by Marc? He couldn't pass s and a member of s because s is borrowed as mutable. He would have to pass both as immutable. Er, I was speaking in terms of Rust of course(it has no distinction between const and immutable like D afaik)
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 02:10:23 UTC, weaselcat wrote: On Tuesday, 3 March 2015 at 02:04:15 UTC, weaselcat wrote: On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote: On 3/2/2015 4:40 PM, deadalnix wrote: After moving resources, the previous owner can no longer be used. How does that work with the example presented by Marc? He couldn't pass s and a member of s because s is borrowed as mutable. He would have to pass both as immutable. Er, I was speaking in terms of Rust of course(it has no distinction between const and immutable like D afaik) http://dpaste.com/30WAMVQ I hope this helps explain it and I didn't butcher it too bad : ) Sorry for the triple reply.
Re: Cannot instantiate a std.container.Array of a class with a init() function member.
On Mon, 02 Mar 2015 16:26:40 +, Meta wrote: > On Monday, 2 March 2015 at 15:46:28 UTC, Francesco Cattoglio wrote: >> Taken from >> http://forum.dlang.org/thread/gjrbmskictrbcyedu...@forum.dlang.org >> >> trying to instantiate an Array!MyClass fails with a rather obscure >> error message if the MyClass has a member function "void init()": >> >> http://dpaste.dzfl.pl/16d202b7124d >> >> Is this a std library bug, or should this be considered a mistake on >> the library user side to name a member function "init"? > > It's kind of ridiculous that the compiler doesn't complain about this, > considering that defining an init member causes an error that you would > have no idea how to fix unless you already knew about this issue. standard policy: "D is only for smarts". signature.asc Description: PGP signature
Re: A Refcounted Array Type
On Mon, 02 Mar 2015 15:43:42 -0500, Steven Schveighoffer wrote:
> On 2/27/15 11:18 PM, ketmar wrote:
>> On Fri, 27 Feb 2015 20:51:54 +, deadalnix wrote:
>>
>>> On Friday, 27 February 2015 at 04:13:03 UTC, Steven Schveighoffer
>>> wrote:
In that case, you shouldn't be subject to any kind of race
conditions.
But we can't make the library/language based on this assumption. Your
case is the exceptional case.
>>> His case is not @safe, so we can ignore that problem.
>>
>> the compiler tends to disagree:
>>
>> === test.d ===
>> int myglobal;
>>
>> class A {
>>~this () @safe { if (myglobal == 42) assert(0); }
>> }
>>
>> void main () {
>>auto a = new A;
>> }
>> ==
>>
>> dmd -w -c -o- test.d
>>
>> wow! no warnings, no errors!
>>
>>
> I think in this case, the compiler can probably statically reject the
> code.
it can't, 'cause it doesn't know how the given object is going to be
allocated. i may completely skip GC and use my own allocator, which has
no troubles with threads, for example.
signature.asc
Description: PGP signature
Re: Refactoring D as an MSc Project
On 2015-03-02 16:50, Jamie wrote: Hello all! This is my first post on the forums. I'm interested in possibly creating a refactoring tool for D for my MSc dissertation (I notice there is currently no refactoring in D?). I wanted to ask if this is possible with D's current state? More specifically: - Can I easily access things such as the AST? If you can write C#, it would be interesting to add it as a feature in Mono-D https://github.com/aBothe/Mono-D/ It's an IDE engine, there is also a parser (written in C#) called D_Parser as a submodule. I use mono-d all the time to work on my libraries. Currently the refactoring feature in monodevelop is pretty much useless because nobody implemented it I guess.
Re: RCArray is unsafe
On 3/2/2015 6:04 PM, weaselcat wrote: On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote: On 3/2/2015 4:40 PM, deadalnix wrote: After moving resources, the previous owner can no longer be used. How does that work with the example presented by Marc? He couldn't pass s and a member of s because s is borrowed as mutable. He would have to pass both as immutable. A pointer to s could be obtained otherwise and passed.
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote:
On 3/2/2015 4:40 PM, deadalnix wrote:
After moving resources, the previous owner can no longer be
used.
How does that work with the example presented by Marc?
I'm not sure what you don't understand. The comment in the sample
code seems clear to me.
// "Move" `a` into `b`
// Here's what happens under the hood: the pointer `a` gets
copied (*not*
// the data on the heap, just its address) into `b`. Now both are
pointers
// to the *same* heap allocated data. But now, `b` *owns* the heap
// allocated data; `b` is now in charge of freeing the memory in
the heap.
let b = a;
Now you have a variable b that owns the data. a is not usable
anymore.
// After the previous move, `a` can no longer be used
// Error! `a` can no longer access the data, because it no longer
owns the
// heap memory
//println!("a contains: {}", a);
// TODO ^ Try uncommenting this line
As explained here, if a is used, this is an error.
In the example a is move to b. If you had let b = &a, then b
would borrow from a. The same way, a would not be usable anymore.
The difference with move is that, when b is destroyed, in the
first case memory is freed. in the second case, memory is not
freed and a is "reenabled".
It is either possible to borrow once and disable who you borrow
from, or borrow multiple time and everything become immutable for
the time you borrow.
This makes the problems mentioned in this thread effectively
impossible happen.
Re: RCArray is unsafe
On 3/2/2015 9:27 PM, deadalnix wrote:
On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote:
On 3/2/2015 4:40 PM, deadalnix wrote:
After moving resources, the previous owner can no longer be used.
How does that work with the example presented by Marc?
I'm not sure what you don't understand. The comment in the sample code seems
clear to me.
// "Move" `a` into `b`
// Here's what happens under the hood: the pointer `a` gets copied (*not*
// the data on the heap, just its address) into `b`. Now both are pointers
// to the *same* heap allocated data. But now, `b` *owns* the heap
// allocated data; `b` is now in charge of freeing the memory in the heap.
let b = a;
Now you have a variable b that owns the data. a is not usable anymore.
// After the previous move, `a` can no longer be used
// Error! `a` can no longer access the data, because it no longer owns the
// heap memory
//println!("a contains: {}", a);
// TODO ^ Try uncommenting this line
As explained here, if a is used, this is an error.
In the example a is move to b. If you had let b = &a, then b would borrow from
a. The same way, a would not be usable anymore. The difference with move is
that, when b is destroyed, in the first case memory is freed. in the second
case, memory is not freed and a is "reenabled".
It is either possible to borrow once and disable who you borrow from, or borrow
multiple time and everything become immutable for the time you borrow.
This makes the problems mentioned in this thread effectively impossible happen.
What if 'a' is a field of a struct in some non-trivial data structure? How is
the compiler going to statically keep track of which field instances in this
structure are not usable?
Re: RCArray is unsafe
On Tuesday, 3 March 2015 at 05:40:59 UTC, Walter Bright wrote:
On 3/2/2015 9:27 PM, deadalnix wrote:
On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote:
On 3/2/2015 4:40 PM, deadalnix wrote:
After moving resources, the previous owner can no longer be
used.
How does that work with the example presented by Marc?
I'm not sure what you don't understand. The comment in the
sample code seems
clear to me.
// "Move" `a` into `b`
// Here's what happens under the hood: the pointer `a` gets
copied (*not*
// the data on the heap, just its address) into `b`. Now both
are pointers
// to the *same* heap allocated data. But now, `b` *owns* the
heap
// allocated data; `b` is now in charge of freeing the memory
in the heap.
let b = a;
Now you have a variable b that owns the data. a is not usable
anymore.
// After the previous move, `a` can no longer be used
// Error! `a` can no longer access the data, because it no
longer owns the
// heap memory
//println!("a contains: {}", a);
// TODO ^ Try uncommenting this line
As explained here, if a is used, this is an error.
In the example a is move to b. If you had let b = &a, then b
would borrow from
a. The same way, a would not be usable anymore. The difference
with move is
that, when b is destroyed, in the first case memory is freed.
in the second
case, memory is not freed and a is "reenabled".
It is either possible to borrow once and disable who you
borrow from, or borrow
multiple time and everything become immutable for the time you
borrow.
This makes the problems mentioned in this thread effectively
impossible happen.
What if 'a' is a field of a struct in some non-trivial data
structure? How is the compiler going to statically keep track
of which field instances in this structure are not usable?
Borrowing 'a' from a struct would make the parent struct
immutable during the borrow scope of 'a', I believe.
Re: DIP74: Reference Counted Class Objects
On Sunday, 1 March 2015 at 13:13:58 UTC, bearophile wrote:
Jacob Carlborg:
@arc class Foo
{
T1 opAddRef();
T2 opRelease();
}
...
Alternative A gives a clear documentation it's a reference
counted class without having to scan the methods.
Assuming you want something like DIP74, this design design
seems safer than the design proposed in DIP74.
Bye,
bearophile
So, I don't know a huge amount about this subject. aka. (hold my
beer) but I do know that seemingly by default D has garbage
collection which is wonderful... but It is also wonderful to have
choices.
I see a story like the tortoise and the hare. Where the garbage
collector is like the hare, going really fast but then taking a
break. Reference counting is like the tortoise, a bit slower but
more predictable.
It is clear Reference counting takes more discipline, but unclear
to me how D will help/guide its users through process. it seems
like the @safe will disallow some obviously dumb things. which is
actually really awesome.
It is also unclear how combining both RC and GC will work, how
does the GC know not to scan the RC's territory? And if it does
is it actually beneficial to have RC?
As far as DIP74: In the Definition I am confused what UFCS has to
do with defining the opAddRef() and opRelease() methods. And also
why the are used in the fun() x.opAddref() example. I probably
have like 100 more dumb questions but lets leave it there. :D
Re: RCArray is unsafe
On Sunday, 1 March 2015 at 15:44:49 UTC, Marc Schütz wrote:
Walter posted an example implementation of a reference counted
array [1], that utilizes the features introduced in DIP25 [2].
Then, in the threads about reference counted objects, several
people posted examples [3, 4] that broke the suggested
optimization of eliding `opAddRef()`/`opRelease()` calls in
certain situations.
A weakness of the same kind affects DIP25, too. The core of the
problem is borrowing (ref return as in DIP25), combined with
manual (albeit hidden) memory management. An example to
illustrate:
1 > struct T {
2 > void doSomething();
3 > }
4 > struct S {
5 > RCArray!T array;
7 > }
8 > void main() {
9 > auto s = S(RCArray!T([T()])); // s.array's refcount is
10> now 1
11> foo(s, s.array[0]); // pass by ref
++ s.array[0] = myt; // Would also be invalid
12> }
13> void foo(ref S s, ref T T) {
14> s.array = RCArray!T([]); // drop the old s.array
15> t.doSomething(); // oops, t is gone
16> }
1. Assignment to RC types is not the same as assignments to
non-RC types.
2. Allocation of RC types is more than just allocating of non-RC
types.
Using the above example:
#9: The assignment and allocation does two things:
a. Decrements the current RC of s.array(here it is null so no
RC is used)
b. Increments the ref count of the allocated RCArray by one.
#11: we pass both s and a referent to inside S. This is odd
behavior and not technically required in this case(just pass S).
Of course, we can't expect such behavior to creep up in complex
code.
#14. In this case the behavior is correct. The assignment does
the following:
a. Decrements the current RC of s.array(here was 1, so now it
is 0)
b. Increments the current RC of the newly allocated RC array
and assigns it to s.array.
#15. Since T refers to the old s.array, which now has a ref count
of 0(which we can assume to then be unallocated), line 15 becomes
a run-time error.
---
How to fix?
It seems that the only way to make it work well is to use sort of
"lazy" ref counting.
Here references are created at the start of functions and
decremented at the end of functions... rather that in place(which
then causes problems with things that happen after it).
But this can't work as in the ++ I added. Here it would solve
your problem but not fix the a generalization of it.
---
Is there a solution? Essentially no. Only at the end of the
program could we possibly have the last function release all RC
counted arrays. Since it is the end of the program it is the only
time we can know that no more RC types will be used.
The problem is analogous to the multiple-inheritance problem.
---
Flow analysis can only help so far(it won't help with dynamically
allocated types that are allocated depending on some "random"
factor.
---
Note that since t is pointing to a part of S, technically we
can't release s until t is released.
So, effectively, S has to be ref counted too! And it's ref count
must check check T's ref count. If it is non-zero then the it
can't release itself(or the reference to it's array).
Also, something eventually has to release S(or which is then
s.array). When? Well, since t depends on S, it is when t is
released. So we would naturally have to inform t that it should
too release s when it is released.
So, if that sort of makes sense I'll explain the solution: It
might be a bit fuzzy but I'll try to make things very clear. [I
will also sometimes conflate the usage of concrete objects and
their corresponding abstract types. e.g., "T with t". It should
be clear context which is meant. e.g., If I say "T is released",
I mean that the object with type T was released. When they need
to be distinguished between I will do so(mainly in the code
examples]
A *reference counted* type, from here on designated as RCT, is a
type that only allows itself to be released to the heap when
nothing depends on it. [Obviously if anything depends on it when
after it is released will fail to be fulfilled]
If a RCT does not contains any references to any RCT's we will
call it a free RCT or FRCT. Such types behave in a simple way.
They can be free'ed completely immediately. Since FRCT's do not
contain any references to other RCT's all their references are
simple references that can be free'ed immediately.
We require that if a type contains a reference to a RCT then it
too is a RCT. [the relation ContainsRCT(T1,T2) ==>
ContainsRCT(T2,T1) ==> Both T1 and T2 are either RCT's or not
RCT's] This isn't a big deal but without it we end up requiring
all types to be RCT's. Here we allow some types to be not be
RCT's.
In code,
class T { } // a standard non-reference counted type.
// It can't use any RCT's as references. It is your
basic D type of class.
rcclass FRCT // rcclas
