Re: shared methods
Am 22.01.2014 06:16, schrieb unknown soldier:
I'm confused by shared and how to use it.
import std.stdio;
class Foo {
File logFile;
void log(in string line) shared {
synchronized(this){
logFile.writeln(line);
}
}
}
This (or the equivalent code in my full size program) won't
compile:
source/log.d(256): Error: template std.stdio.File.writeln does
not match any function template declaration. Candidates are:
/usr/include/dmd/phobos/std/stdio.d(781):
std.stdio.File.writeln(S...)(S args)
source/log.d(256): Error: template std.stdio.File.writeln(S...)(S
args) cannot deduce template function from argument types
!()(const(immutable(char)[])) shared
Why is logFile suddenly being treated as shared? This does not
make sense to me. As a programmer I have acknowledged that log()
is a shared function in the declaration 'void log(in string line)
shared', so isn't it up to me to ensure that I do proper
synchronization within the function? Why is the compiler trying
to ensure that all functions called within log() are also marked
shared?
What is the right thing to do here?
Note that I can't just mark log() as not shared; log() must be
shared because elsewhere I have:
shared Foo sfoo = ...
sfoo.log(...)
For a shared method the this pointer is also shared. That means that you
have to tell the compiler (manually) that it is now safe to use
non-shared code. You usually do this by casting the this pointer to a
unshared type.
class Foo {
File logFile;
void log(in string line) shared {
synchronized(this){
// safe from here on because synchronized
auto self = cast(Foo)this;
self.logFile.writeln(line);
}
}
}
The compiler does not do this for you, because it can not know if
accessing any member is truly thread-safe. You might share the logFile
with other instances of Foo. This would be a case where the compiler
would wrongly remove the shared from this, if it would do so
automatically inside synchronized blocks.
Re: shared methods
The compiler does not do this for you, because it can not know if accessing any member is truly thread-safe. You might share the logFile with other instances of Foo. This would be a case where the compiler would wrongly remove the shared from this, if it would do so automatically inside synchronized blocks. Makes sense; thank you!
Re: shared methods
A minor caveat is GDC treats shared as volatile, which can be not optimal. Even worse, on some platforms like itanium volatile generates fences.
Re: shared methods
Am Thu, 23 Jan 2014 19:20:34 +
schrieb "Kagamin" :
> A minor caveat is GDC treats shared as volatile, which can be not
> optimal. Even worse, on some platforms like itanium volatile
> generates fences.
Do you have more information on that issue?
There are claims that Itanium "automatically generates fences", however
that statement is not precise and I can't find a detailed description.
Itanium has special instructions which refresh the caches before
reading/writing. But the interesting question is this: Does GCC use
these instructions if you access a variable marked as volatile?
If it does, that's really a performance problem. AFAICS we need a
'volatile' in D for embedded programming which just does the following:
* Avoid removing any stores / or loads to that variable (includes
moving accesses out of loops)
* Avoid instruction reordering between accesses to volatile variables:
volatile int a, c;
int b;
a = 0;
b = 0;
c = 0;
b can be moved anywhere, but c must always come after a
Here volatile should only affect the ASM generated by the compiler, it
shouldn't care at all what the final CPU does, that's the programmers
task.
In some cases the compiler can't even know: For example some
memory areas in ARM are special and the processor won't reorder
accesses to those areas anyway, so a 'smart compiler' adding memory
barriers would actually only harm performance.
http://infocenter.arm.com/help/topic/com.arm.doc.dui0552a/CIHGEIID.html
Such a 'volatile' could be integrated with shared as it should not
cause any huger performance problems. However, if you think of this
code:
shared int X;
synchronized(x)
{
for(int i=0; i< 100; i++)
X = i;
}
it's actually valid to optimize the loop, so 'volatile' behavior is not
wanted. But what if we used atomic ops?
shared int X;
for(int i=0; i< 100; i++)
atomicSet(X, i);
Now the loop can't be optimized away. Is the compiler smart enough to
figure out that an atomic op is used and it can't optimize this? Or
would we have to mark this volatile?
Re: shared methods
for example http://software.intel.com/en-us/blogs/2007/11/30/volatile-almost-useless-for-multi-threaded-programming
Re: shared methods
http://igoro.com/archive/volatile-keyword-in-c-memory-model-explained/ As I understand, because Itanium doesn't have cache coherency, a memory fence is needed to implement volatile load and store. On x86 load and store are already volatile because of cache coherency, so fences are not needed.
Re: shared methods
Am Fri, 24 Jan 2014 22:30:13 + schrieb "Kagamin" : > http://igoro.com/archive/volatile-keyword-in-c-memory-model-explained/ > As I understand, because Itanium doesn't have cache coherency, a > memory fence is needed to implement volatile load and store. On > x86 load and store are already volatile because of cache > coherency, so fences are not needed. Seems like C#s volatile cares about "compiler optimizations and processor optimizations.". I'd rather want a volatile for D (or as a base for shared) which only cares about compiler optimizations, leaving the processor part to the programmer. (For example it isn't valid in D to access a shared variable with normal operations anyway, AFAIK. You need to use the atomicOp things and these will the worry about the hardware part, using the correct instructions and so on) See also: http://www.drdobbs.com/parallel/volatile-vs-volatile/212701484 So volatile is a completely different thing in C#/JAVA and C/C++. This is why I'm confused, C compilers are not supposed to guarantee that another thread can see all changes to a volatile variable, volatile must just prevent compiler optimizations. So if Itanium C compilers automatically use the barrier/fence instructions for volatile that's IMHO a compiler performance bug. I also can't find any information the GCC adds barriers on Itanium. Maybe this is just true for the Intel compiler, as the main source for these claims are Intel pages.
Re: shared methods
On Saturday, 25 January 2014 at 10:00:58 UTC, Johannes Pfau wrote: (For example it isn't valid in D to access a shared variable with normal operations anyway, AFAIK. You need to use the atomicOp things and these will the worry about the hardware part, using the correct instructions and so on) Is it invalid to access shared data on stack too? How about closures? See also: http://www.drdobbs.com/parallel/volatile-vs-volatile/212701484 So volatile is a completely different thing in C#/JAVA and C/C++. This is why I'm confused, C compilers are not supposed to guarantee that another thread can see all changes to a volatile variable, volatile must just prevent compiler optimizations. So if Itanium C compilers automatically use the barrier/fence instructions for volatile that's IMHO a compiler performance bug. The standard says "memory" and doesn't address cache and processor optimizations, so it probably allows different interpretations. Even if there's no fence, disallowed compiler optimizations are still a performance hit - on all platforms.
Re: shared methods
Also if you read a shared value with atomicLoad every time, this disallows caching in registers or on stack, which is also performance hit. The shared value should be read once and cached if possible.
Re: shared methods
Am Sat, 25 Jan 2014 21:41:20 + schrieb "Kagamin" : > On Saturday, 25 January 2014 at 10:00:58 UTC, Johannes Pfau wrote: > > (For example it isn't valid in D to access a shared variable > > with > > normal operations anyway, AFAIK. You need to use the atomicOp > > things > > and these will the worry about the hardware part, using the > > correct > > instructions and so on) > > Is it invalid to access shared data on stack too? How about > closures? > I'm not sure about the details - shared hasn't been completely implemented or even specified yet. AFAIK the most recent idea was that shared does basically nothing but prevents direct access to variables. Synchronized statements then remove the shared qualifier and you can access the variables as usual. Atomic OPs also work with shared variables. Using locks manually then probably requires casting away shared. However, this part of the language is really unfinished. I hoped we could at least use shared as a replacement for volatile, but as you said in your other reply we probably can't.
Re: shared methods
Am Sat, 25 Jan 2014 21:48:39 + schrieb "Kagamin" : > Also if you read a shared value with atomicLoad every time, this > disallows caching in registers or on stack, which is also > performance hit. The shared value should be read once and cached > if possible. Yes, I came to the same conclusion. If we combine volatile and shared into one qualifier we'll always have a certain performance hit. Great, now we have to convince Walter that we have to undeprecate volatile for embedded programming...
Re: shared methods
On Saturday, 25 January 2014 at 23:20:47 UTC, Johannes Pfau wrote: Yes, I came to the same conclusion. If we combine volatile and shared into one qualifier we'll always have a certain performance hit. This is what GDC does now.
