Re: TDPL, shared data, and Phobos

2010-07-23 Thread Jérôme M. Berger
Graham St Jack wrote:
> Priority inheritance chaining goes like this:
> 
> Thread low locks mutex A, then mutex B
> 
> Thread high tries to lock mutex B, elevating low's priority to high's so
> that high can get the mutex quickly.
> 
> When thread low releases mutex B (letting high get it), the OS has
> trouble figuring out what low's priority should now be, and leaves it
> elevated until it releases all mutexes it still has (mutex A in this case).
> 
> Low is now running at a high priority, preventing thread medium from
> getting any CPU.
> 
> 
> This scenario happened for me with vxWorks some time back, and is the
> reason I no longer do much work at all while I have a mutex locked. I am
> confident that it is a real problem to this day.
> 
This is only a problem on real-time OSes where high priority
threads prevent low priority ones from running at all. On non
real-time OSes like Windows, Linux, *BSD and MacOS, low priority
threads will always get some CPU cycles too, and AFAIK thread
priorities are never elevated in the way you describe.

That being said, it is always a good practice to spend as little
time as possible holding a lock (whether a mutex or a file lock or
whatever).

Jerome
-- 
mailto:jeber...@free.fr
http://jeberger.free.fr
Jabber: jeber...@jabber.fr



signature.asc
Description: OpenPGP digital signature


Re: TDPL, shared data, and Phobos

2010-07-22 Thread Graham St Jack

On 23/07/10 10:23, Sean Kelly wrote:

awishformore Wrote:

   

On 22/07/2010 01:49, Robert Jacques wrote:
 

Have you tried core.sync.rwmutex? Also, please remember that CREW locks
are not composable and can easily lead to dead-locks.
   

Afaik, the current rwmutex is a wrapper around two separate mutexes (one
for readers, one for writers) and you have to decide whether readers or
writers get precedence, meaning that ether all writers in the queue have
to wait if just one reader has to write or all writers in the queue have
to wait if there is a single reader comes up.

This is very unlike the behaviour I would like to see; I would expect
readers and writers to be in the same queue, meaning the only difference
between the rw and the normal mutex would be that all subsequent readers
in the queue can read at the same time.
 

ReadWriteMutex exposes a read and write interface, but there certainly aren't 
two actual mutexes underneath.  It's true that the implementation doesn't 
explicitly maintain a queue, but this is intentional.  If readers and writers 
in the queue have different thread priorities set, those priorities should be 
honored, and it's pointless to write all that code in druntime when the OS 
takes care of it for us.  Instead, those waiting for access to the mutex all 
block on a condition variable and whoever wakes up first wins.  It's up the OS 
to make sure that thread priorities are honored and starvation doesn't occur.
   
It isn't clear that thread priorities will do the job here. I have been 
burned before by things like priority inheritance chaining, and other 
ways that thread priorities can be elevated for potentially long periods 
of time.


Priority inheritance chaining goes like this:

Thread low locks mutex A, then mutex B

Thread high tries to lock mutex B, elevating low's priority to high's so 
that high can get the mutex quickly.


When thread low releases mutex B (letting high get it), the OS has 
trouble figuring out what low's priority should now be, and leaves it 
elevated until it releases all mutexes it still has (mutex A in this case).


Low is now running at a high priority, preventing thread medium from 
getting any CPU.



This scenario happened for me with vxWorks some time back, and is the 
reason I no longer do much work at all while I have a mutex locked. I am 
confident that it is a real problem to this day.


--
Graham St Jack



Re: TDPL, shared data, and Phobos

2010-07-22 Thread Sean Kelly
awishformore Wrote:

> On 22/07/2010 01:49, Robert Jacques wrote:
> >
> > Have you tried core.sync.rwmutex? Also, please remember that CREW locks
> > are not composable and can easily lead to dead-locks.
> 
> Afaik, the current rwmutex is a wrapper around two separate mutexes (one 
> for readers, one for writers) and you have to decide whether readers or 
> writers get precedence, meaning that ether all writers in the queue have 
> to wait if just one reader has to write or all writers in the queue have 
> to wait if there is a single reader comes up.
> 
> This is very unlike the behaviour I would like to see; I would expect 
> readers and writers to be in the same queue, meaning the only difference 
> between the rw and the normal mutex would be that all subsequent readers 
> in the queue can read at the same time.

ReadWriteMutex exposes a read and write interface, but there certainly aren't 
two actual mutexes underneath.  It's true that the implementation doesn't 
explicitly maintain a queue, but this is intentional.  If readers and writers 
in the queue have different thread priorities set, those priorities should be 
honored, and it's pointless to write all that code in druntime when the OS 
takes care of it for us.  Instead, those waiting for access to the mutex all 
block on a condition variable and whoever wakes up first wins.  It's up the OS 
to make sure that thread priorities are honored and starvation doesn't occur.


Re: TDPL, shared data, and Phobos

2010-07-22 Thread awishformore

On 22/07/2010 01:49, Robert Jacques wrote:

On Tue, 20 Jul 2010 15:41:31 -0400, Brian Palmer
 wrote:


> It probably wasn't very clear from my simplified example, but I'm
looking to create a shared-reader-one-writer scenario. If I declare
MyValue synchronized, only one thread can be inside the get() method
at a time, which defeats the shared-reader requirement. Imagine this
is a much larger more complex data structure, where get() requires
walking through multiple levels of a tree and a binary search at the
last level.
>

Yup, I get it. But there is one point in it: write is not atomic
operation in sense that get() might return half written data, right?


No, that's why I want a read-write lock. Multiple threads can read the
data, but writes take an exclusive lock.

http://en.wikipedia.org/wiki/Readers-writer_lock


Have you tried core.sync.rwmutex? Also, please remember that CREW locks
are not composable and can easily lead to dead-locks.


Afaik, the current rwmutex is a wrapper around two separate mutexes (one 
for readers, one for writers) and you have to decide whether readers or 
writers get precedence, meaning that ether all writers in the queue have 
to wait if just one reader has to write or all writers in the queue have 
to wait if there is a single reader comes up.


This is very unlike the behaviour I would like to see; I would expect 
readers and writers to be in the same queue, meaning the only difference 
between the rw and the normal mutex would be that all subsequent readers 
in the queue can read at the same time.


/Max


Re: TDPL, shared data, and Phobos

2010-07-22 Thread Brian Palmer
Robert Jacques Wrote:

> On Tue, 20 Jul 2010 15:41:31 -0400, Brian Palmer  
>  
> wrote:
> 
> >> > It probably wasn't very clear from my simplified example, but I'm  
> >> looking to create a shared-reader-one-writer scenario. If I declare  
> >> MyValue synchronized, only one thread can be inside the get() method at  
> >> a time, which defeats the shared-reader requirement. Imagine this is a  
> >> much larger more complex data structure, where get() requires walking  
> >> through multiple levels of a tree and a binary search at the last level.
> >> >
> >>
> >> Yup, I get it. But there is one point in it: write is not atomic  
> >> operation in sense that get() might return half written data, right?
> >
> > No, that's why I want a read-write lock. Multiple threads can read the  
> > data, but writes take an exclusive lock.
> >
> > http://en.wikipedia.org/wiki/Readers-writer_lock
> 
> Have you tried core.sync.rwmutex? Also, please remember that CREW locks  
> are not composable and can easily lead to dead-locks.

lol, yes, that's how this thread started was with a discussion of 
core.sync.rwmutex.


Re: TDPL, shared data, and Phobos

2010-07-22 Thread Sean Kelly
The core stuff is in a different repository, and I haven't done all the
work to integrate the docs yet. I sent a file to do most of this to the
Phobos list though, if you're inclined to track it down. 

Brian Palmer  wrote:
> Thanks Sean, it's great to at least know where the issue is. As to my
> other question, why do the D std library docs at
> http://www.digitalmars.com/d/2.0/phobos/phobos.html have no indication
> that core.sync and the other core.* packages even exist? Are the APIs
> not stable enough yet to treat them as public? Even if documentation
> is sparse, just knowing that RWLock existed and its exposed methods
> was all I really needed to start using it, aside from the shared
> issue.
> 
> -- Brian
> 
> 
> Sean Kelly Wrote:
> 
>> The casts are necessary because I haven't yet applied 'shared' to
> > druntime.  I ran into a few issues when doing so and rolled back my
> > changes.  I'll give it another shot before the next release.


Re: TDPL, shared data, and Phobos

2010-07-21 Thread Robert Jacques
On Tue, 20 Jul 2010 15:41:31 -0400, Brian Palmer   
wrote:


> It probably wasn't very clear from my simplified example, but I'm  
looking to create a shared-reader-one-writer scenario. If I declare  
MyValue synchronized, only one thread can be inside the get() method at  
a time, which defeats the shared-reader requirement. Imagine this is a  
much larger more complex data structure, where get() requires walking  
through multiple levels of a tree and a binary search at the last level.

>

Yup, I get it. But there is one point in it: write is not atomic  
operation in sense that get() might return half written data, right?


No, that's why I want a read-write lock. Multiple threads can read the  
data, but writes take an exclusive lock.


http://en.wikipedia.org/wiki/Readers-writer_lock


Have you tried core.sync.rwmutex? Also, please remember that CREW locks  
are not composable and can easily lead to dead-locks.


Re: TDPL, shared data, and Phobos

2010-07-20 Thread Brian Palmer
> > It probably wasn't very clear from my simplified example, but I'm looking 
> > to create a shared-reader-one-writer scenario. If I declare MyValue 
> > synchronized, only one thread can be inside the get() method at a time, 
> > which defeats the shared-reader requirement. Imagine this is a much larger 
> > more complex data structure, where get() requires walking through multiple 
> > levels of a tree and a binary search at the last level.
> > 
> 
> Yup, I get it. But there is one point in it: write is not atomic operation in 
> sense that get() might return half written data, right?

No, that's why I want a read-write lock. Multiple threads can read the data, 
but writes take an exclusive lock.

http://en.wikipedia.org/wiki/Readers-writer_lock


Re: TDPL, shared data, and Phobos

2010-07-20 Thread Brian Palmer
Thanks Sean, it's great to at least know where the issue is. As to my other 
question, why do the D std library docs at 
http://www.digitalmars.com/d/2.0/phobos/phobos.html have no indication that 
core.sync and the other core.* packages even exist? Are the APIs not stable 
enough yet to treat them as public? Even if documentation is sparse, just 
knowing that RWLock existed and its exposed methods was all I really needed to 
start using it, aside from the shared issue.

-- Brian


Sean Kelly Wrote:

> The casts are necessary because I haven't yet applied 'shared' to druntime.  
> I ran into a few issues when doing so and rolled back my changes.  I'll give 
> it another shot before the next release.



Re: TDPL, shared data, and Phobos

2010-07-19 Thread Graham St Jack
On Sun, 18 Jul 2010 16:05:08 +, Sean Kelly wrote:

> Graham St Jack  wrote:
>> On Sat, 17 Jul 2010 11:42:03 -0400, Sean Kelly wrote:
>> 
>>> The casts are necessary because I haven't yet applied 'shared' to
>>> druntime.  I ran into a few issues when doing so and rolled back my
>>> changes.  I'll give it another shot before the next release.
>> 
>> I'm glad you announced you intention - I was just about to roll up my
>> sleeves and give it a go for Condition, but will wait for the next
>> release.
>> 
>> Like all my previous attempts to use shared, I have waited for quite a
>> 
>> while for things to improve, tried using shared again, hit a brick wall
>> and resorted to defeating the compiler's detection of shared data.
>> 
>> TDPL raised my hopes without actually making it clear how to use
>> synchronized classes. Alas, it seems to me that they still aren't
>> usable
>> in practice. With any luck the problems are just library issues which
>> can
>> be fixed relatively easily.
>> 
>> Like Brian Palmer, I am frustrated by the lack of documentation about
>> shared and druntime's sync package, and am happy to lend a hand if that
>> would be helpful.
>> 
>> The code I am trying to write is a simple synchronized class with a
>> Condition, but I can't create a Condition on a shared "this".
>> 
>> A cut-down version of what I want to write is:
>> 
>> synchronized class Foo {
>>   Condition mCondition;
>>   this() {
>> mCondition = cast(shared) new Condition(this);
>>   }
>>   void some_method() {
>>   }
>> }
>> 
>> I realise that Condition wants a Mutex, but a synchronized class
>> already
>> has an implicit one which is implicitly used by all the methods, so the
>> above is definitely what I want to write.
>> 
>> What I have to write instead (which avoids the compiler noticing that
>> anything is being shared) is:
>> 
>> class Foo {
>>   Mutex mMutex;
>>   Condition mCondition;
>>   this() {
>> mMutex = new Mutex();
>> mCondition = new Condition(mMutex);
>>   }
>>   void some_method() {
>> synchronized(mMutex) {
>> }
>>   }
>> }
>> 
>> The latter works just fine, but it is very disappointing after all the
>> 
>> fuss about how important shared is that you can't actually use it for
>> the
>> most mainstream of all uses - a synchronized class with a condition
>> (which is what a message queue between threads is supposed to be).
> 
> new Mutex(this) makes the mutex the object monitor, so it will be what's
> locked for synchronized functions.

That's cool. I look forward to shared not being an issue ;-)

I assume that when Condition and Mutex are shareable, I will then (from 
your other post) write:

synchronized class Foo {
  Mutex mMutex;
  Condition mCondition;
  this() {
mMutex = cast(shared) new Mutex(this);
mCondition = cast(shared) new Condition(mMutex);
  }
  void some_method() {
...
mCondition.notify; // ok because mMutex is locked
  }
}


Re: TDPL, shared data, and Phobos

2010-07-18 Thread Sean Kelly
Graham St Jack Wrote:
> 
> The code I am trying to write is a simple synchronized class with a 
> Condition, but I can't create a Condition on a shared "this".
> 
> A cut-down version of what I want to write is:
> 
> synchronized class Foo {
>   Condition mCondition;
>   this() {
> mCondition = cast(shared) new Condition(this);
>   }
>   void some_method() {
>   }
> }
> 
> I realise that Condition wants a Mutex, but a synchronized class already 
> has an implicit one which is implicitly used by all the methods, so the 
> above is definitely what I want to write.

The built-in mutex is created on first use, so what you can do is drop in 
core.sync.mutex.Mutex before this happens (the library Mutex can be assigned as 
an object monitor).  If 'shared' weren't an issue, the code would be:

synchronized class Foo {
Condition mCond;
Mutex  mLock;
this() {
mLock = new Mutex(this); // make mutex this object's monitor
mCond = new Condition(mLock); // bind condition to mLock
}

void some_method() { // locks mLock
mLock.notify(); // safe because mLock is locked
}
}



Re: TDPL, shared data, and Phobos

2010-07-18 Thread Sean Kelly
Graham St Jack  wrote:
> On Sat, 17 Jul 2010 11:42:03 -0400, Sean Kelly wrote:
> 
>> The casts are necessary because I haven't yet applied 'shared' to
>> druntime.  I ran into a few issues when doing so and rolled back my
>> changes.  I'll give it another shot before the next release.
> 
> I'm glad you announced you intention - I was just about to roll up my 
> sleeves and give it a go for Condition, but will wait for the next 
> release.
> 
> Like all my previous attempts to use shared, I have waited for quite a
> 
> while for things to improve, tried using shared again, hit a brick
> wall 
> and resorted to defeating the compiler's detection of shared data.
> 
> TDPL raised my hopes without actually making it clear how to use 
> synchronized classes. Alas, it seems to me that they still aren't
> usable 
> in practice. With any luck the problems are just library issues which
> can 
> be fixed relatively easily.
> 
> Like Brian Palmer, I am frustrated by the lack of documentation about 
> shared and druntime's sync package, and am happy to lend a hand if
> that 
> would be helpful.
> 
> The code I am trying to write is a simple synchronized class with a 
> Condition, but I can't create a Condition on a shared "this".
> 
> A cut-down version of what I want to write is:
> 
> synchronized class Foo {
>   Condition mCondition;
>   this() {
> mCondition = cast(shared) new Condition(this);
>   }
>   void some_method() {
>   }
> }
> 
> I realise that Condition wants a Mutex, but a synchronized class
> already 
> has an implicit one which is implicitly used by all the methods, so
> the 
> above is definitely what I want to write.
> 
> What I have to write instead (which avoids the compiler noticing that 
> anything is being shared) is:
> 
> class Foo {
>   Mutex mMutex;
>   Condition mCondition;
>   this() {
> mMutex = new Mutex();
> mCondition = new Condition(mMutex);
>   }
>   void some_method() {
> synchronized(mMutex) {
> }
>   }
> }
> 
> The latter works just fine, but it is very disappointing after all the
> 
> fuss about how important shared is that you can't actually use it for
> the 
> most mainstream of all uses - a synchronized class with a condition 
> (which is what a message queue between threads is supposed to be).

new Mutex(this) makes the mutex the object monitor, so it will be what's
locked for synchronized functions.


Re: TDPL, shared data, and Phobos

2010-07-17 Thread Graham St Jack
On Sat, 17 Jul 2010 11:42:03 -0400, Sean Kelly wrote:

> The casts are necessary because I haven't yet applied 'shared' to
> druntime.  I ran into a few issues when doing so and rolled back my
> changes.  I'll give it another shot before the next release.

I'm glad you announced you intention - I was just about to roll up my 
sleeves and give it a go for Condition, but will wait for the next 
release.

Like all my previous attempts to use shared, I have waited for quite a 
while for things to improve, tried using shared again, hit a brick wall 
and resorted to defeating the compiler's detection of shared data.

TDPL raised my hopes without actually making it clear how to use 
synchronized classes. Alas, it seems to me that they still aren't usable 
in practice. With any luck the problems are just library issues which can 
be fixed relatively easily.

Like Brian Palmer, I am frustrated by the lack of documentation about 
shared and druntime's sync package, and am happy to lend a hand if that 
would be helpful.

The code I am trying to write is a simple synchronized class with a 
Condition, but I can't create a Condition on a shared "this".

A cut-down version of what I want to write is:

synchronized class Foo {
  Condition mCondition;
  this() {
mCondition = cast(shared) new Condition(this);
  }
  void some_method() {
  }
}

I realise that Condition wants a Mutex, but a synchronized class already 
has an implicit one which is implicitly used by all the methods, so the 
above is definitely what I want to write.

What I have to write instead (which avoids the compiler noticing that 
anything is being shared) is:

class Foo {
  Mutex mMutex;
  Condition mCondition;
  this() {
mMutex = new Mutex();
mCondition = new Condition(mMutex);
  }
  void some_method() {
synchronized(mMutex) {
}
  }
}

The latter works just fine, but it is very disappointing after all the 
fuss about how important shared is that you can't actually use it for the 
most mainstream of all uses - a synchronized class with a condition 
(which is what a message queue between threads is supposed to be).


Re: TDPL, shared data, and Phobos

2010-07-17 Thread Sean Kelly
The casts are necessary because I haven't yet applied 'shared' to druntime.  I 
ran into a few issues when doing so and rolled back my changes.  I'll give it 
another shot before the next release.


Re: TDPL, shared data, and Phobos

2010-07-17 Thread Bane
> It probably wasn't very clear from my simplified example, but I'm looking to 
> create a shared-reader-one-writer scenario. If I declare MyValue 
> synchronized, only one thread can be inside the get() method at a time, which 
> defeats the shared-reader requirement. Imagine this is a much larger more 
> complex data structure, where get() requires walking through multiple levels 
> of a tree and a binary search at the last level.
> 

Yup, I get it. But there is one point in it: write is not atomic operation in 
sense that get() might return half written data, right?


Re: TDPL, shared data, and Phobos

2010-07-17 Thread Brian Palmer
It probably wasn't very clear from my simplified example, but I'm looking to 
create a shared-reader-one-writer scenario. If I declare MyValue synchronized, 
only one thread can be inside the get() method at a time, which defeats the 
shared-reader requirement. Imagine this is a much larger more complex data 
structure, where get() requires walking through multiple levels of a tree and a 
binary search at the last level.

-- Brian


Bane Wrote:

> I am few days old in playin with D2 and whole shared stuff, so I am probably 
> wrong in something. 
> 
> You should probably declare your example class MyValue synchronized instead 
> of shared. It implies that class is shared too, and this way all methods are 
> synchronized. In D1 you could mix synchronized and non syncrhonized methods 
> in class, in D2 its whole or nothing. This way you don't need _lock var in 
> your example.
> 
> So this would work (i guess)
> 
> synchronized class MyValue {
>  int inc() {
> return _value++;
>   }
>  int get() {
> return _value;
>  }
>  private int _value;
> }
> 
> shared MyValue sharedVal;
> 
> void main(){
>   sharedVal = new shared(MyValue );
> }
> 
>  I noticed that in D1 synchronized methods of same class share same lock, 
> while in this D2 example (when the whole class is declared synchronized), 
> each method has its own lock. 
> 
> > Also, is there any documentation on the actual semantics of shared? 
> > http://www.digitalmars.com/d/2.0/attribute.html is a blank on the subject, 
> > and the "migrating to shared" article only talks about simple global state. 
> > What are the actual semantics of shared classes, and how do they interact 
> > with other code? For instance, after much banging of my head against the 
> > desk, I finally wrote a working implementation of a simple shared 
> > multi-reader var. Obviously there are better ways to do a simple shared 
> > incrementing counter, this is just a first experiment working toward a 
> > shared mutable 512MB trie data structure that we have in our app's current 
> > C++ implementation:
> > 
> > > shared class MyValue {
> > > this() {
> > > _lock = cast(shared)new ReadWriteMutex;
> > > }
> > > 
> > > int inc() {
> > > synchronized((cast(ReadWriteMutex)_lock).writer) {
> > > return _value++;
> > > }
> > > }
> > > 
> > > int get() {
> > > synchronized((cast(ReadWriteMutex)_lock).reader) {
> > > return _value;
> > > }
> > > }
> > > 
> > > private ReadWriteMutex _lock;
> > > private int _value;
> > > }
> > > 
> > > shared MyValue sharedVal;
> > > ... seems to behave correctly with multiple threads reading and writing 
> > > ...
> > 
> > So I can maybe understand the cast(shared) in the ctor. But I have to admit 
> > I have absolutely no idea why I had to cast away the shared attribute in 
> > the inc/get methods. Is there any documentation on what's really going on 
> > in the compiler here? It's a shared method, accessing a shared instance 
> > var, why the cast? Is the compiler upset about something in the definition 
> > of ReadWriteMutex itself?
> > 
> > Also, how would one implement this as a struct? My postblit op generates 
> > compiler errors about casting between shared/unshared MyValue:
> > 
> > > shared struct MyValue {
> > >this(this) { _lock = cast(shared) new ReadWriteMutex; } // ERROR
> > >... same as above ...
> > > }
> > 
> > I recognize the possible race conditions here, but there has to be *some* 
> > way to implement a postblit op on a shared struct?
> > 
> > I hope this doesn't come across as empty complaining, I'm happy to help 
> > improve the documentation if I can.
> 



Re: TDPL, shared data, and Phobos

2010-07-13 Thread Bane
I am few days old in playin with D2 and whole shared stuff, so I am probably 
wrong in something. 

You should probably declare your example class MyValue synchronized instead of 
shared. It implies that class is shared too, and this way all methods are 
synchronized. In D1 you could mix synchronized and non syncrhonized methods in 
class, in D2 its whole or nothing. This way you don't need _lock var in your 
example.

So this would work (i guess)

synchronized class MyValue {
 int inc() {
return _value++;
  }
 int get() {
return _value;
 }
 private int _value;
}

shared MyValue sharedVal;

void main(){
  sharedVal = new shared(MyValue );
}

 I noticed that in D1 synchronized methods of same class share same lock, while 
in this D2 example (when the whole class is declared synchronized), each method 
has its own lock. 

> Also, is there any documentation on the actual semantics of shared? 
> http://www.digitalmars.com/d/2.0/attribute.html is a blank on the subject, 
> and the "migrating to shared" article only talks about simple global state. 
> What are the actual semantics of shared classes, and how do they interact 
> with other code? For instance, after much banging of my head against the 
> desk, I finally wrote a working implementation of a simple shared 
> multi-reader var. Obviously there are better ways to do a simple shared 
> incrementing counter, this is just a first experiment working toward a shared 
> mutable 512MB trie data structure that we have in our app's current C++ 
> implementation:
> 
> > shared class MyValue {
> > this() {
> > _lock = cast(shared)new ReadWriteMutex;
> > }
> > 
> > int inc() {
> > synchronized((cast(ReadWriteMutex)_lock).writer) {
> > return _value++;
> > }
> > }
> > 
> > int get() {
> > synchronized((cast(ReadWriteMutex)_lock).reader) {
> > return _value;
> > }
> > }
> > 
> > private ReadWriteMutex _lock;
> > private int _value;
> > }
> > 
> > shared MyValue sharedVal;
> > ... seems to behave correctly with multiple threads reading and writing ...
> 
> So I can maybe understand the cast(shared) in the ctor. But I have to admit I 
> have absolutely no idea why I had to cast away the shared attribute in the 
> inc/get methods. Is there any documentation on what's really going on in the 
> compiler here? It's a shared method, accessing a shared instance var, why the 
> cast? Is the compiler upset about something in the definition of 
> ReadWriteMutex itself?
> 
> Also, how would one implement this as a struct? My postblit op generates 
> compiler errors about casting between shared/unshared MyValue:
> 
> > shared struct MyValue {
> >this(this) { _lock = cast(shared) new ReadWriteMutex; } // ERROR
> >... same as above ...
> > }
> 
> I recognize the possible race conditions here, but there has to be *some* way 
> to implement a postblit op on a shared struct?
> 
> I hope this doesn't come across as empty complaining, I'm happy to help 
> improve the documentation if I can.



TDPL, shared data, and Phobos

2010-07-13 Thread Brian Palmer
This is really like 3 messages, but I didn't want to spam the list with topics, 
and maybe I'm the only one feeling this pain anyway. So first: from the 
perspective of a guy who has dabbled in D1 and D2 for years, but hasn't looked 
closed at the language in ~10 months, I loved TDPL. It convinced me that D2 is 
going to kick ass. Sadly though, I felt like I finished chapter 13 without 
gaining any real understanding of how `shared` works in D, and how I can 
effectively share mutable data when it *is* necessary. Maybe this was 
intentional, and shared is just too much of an experimental feature still to 
write a chapter on. But I decided to play around with some prototypes.

First, it took me ages to find any sign of any locking primitives beyond the 
implied Mutex in Object. Finally I resorted to doing a `grep -ri semaphore 
/usr/local/dmd2` (on OS X), and found a whole wealth of code. There's an entire 
core.sync.* package! My end goal was to find or create a read/write lock, and 
lo and behold, there's one all ready and containing unit tests in 
core/sync/rwmutex.d.

Which leads to my first questions: why are the core.* interfaces apparently not 
documented along with the std.* packages on the D web site? Is there a 
documentation resource elsewhere? Even if the DDOC is sparse, at least showing 
which classes exist under core.* would be a huge help. And are these 
implementations ready for use by my code, or are they hidden away for a reason?

Also, is there any documentation on the actual semantics of shared? 
http://www.digitalmars.com/d/2.0/attribute.html is a blank on the subject, and 
the "migrating to shared" article only talks about simple global state. What 
are the actual semantics of shared classes, and how do they interact with other 
code? For instance, after much banging of my head against the desk, I finally 
wrote a working implementation of a simple shared multi-reader var. Obviously 
there are better ways to do a simple shared incrementing counter, this is just 
a first experiment working toward a shared mutable 512MB trie data structure 
that we have in our app's current C++ implementation:

> shared class MyValue {
> this() {
> _lock = cast(shared)new ReadWriteMutex;
> }
> 
> int inc() {
> synchronized((cast(ReadWriteMutex)_lock).writer) {
> return _value++;
> }
> }
> 
> int get() {
> synchronized((cast(ReadWriteMutex)_lock).reader) {
> return _value;
> }
> }
> 
> private ReadWriteMutex _lock;
> private int _value;
> }
> 
> shared MyValue sharedVal;
> ... seems to behave correctly with multiple threads reading and writing ...

So I can maybe understand the cast(shared) in the ctor. But I have to admit I 
have absolutely no idea why I had to cast away the shared attribute in the 
inc/get methods. Is there any documentation on what's really going on in the 
compiler here? It's a shared method, accessing a shared instance var, why the 
cast? Is the compiler upset about something in the definition of ReadWriteMutex 
itself?

Also, how would one implement this as a struct? My postblit op generates 
compiler errors about casting between shared/unshared MyValue:

> shared struct MyValue {
>this(this) { _lock = cast(shared) new ReadWriteMutex; } // ERROR
>... same as above ...
> }

I recognize the possible race conditions here, but there has to be *some* way 
to implement a postblit op on a shared struct?

I hope this doesn't come across as empty complaining, I'm happy to help improve 
the documentation if I can.