Shared
I've reread the relevant TDPL chapter and I still don't quite understand the following: 1. What is shared? Is it simply a piece of syntactic salt to make it hard to share data across threads by accident, or is there more to it? 2. Is it fully or mostly implemented?
Shared Hell
I've recently updated to DMD2.035 (from DMD2.031 because all the later
versions had issues with imports) and for the first time faced problems
with shared modifier.
I don't need shared and all my globals are __gshared (they are globally
unique instances that don't need per-thread copies).
Yet some of methods of the class hierarchy (a root singleton class and
everything which is accessible through it) are synchronized (well, you
know why). That's where the problems begin.
Marking a method as synchronized automatically makes it shared (more or
less obvious). And marking the method shared makes it unable to invoke
with non-shared instance (and __gshared != shared), meaning that I'm
unable to use my __gshared variables anymore, making this attribute
useless for any serious safe programming.
So I started with replacing __gshared with shared and quickly understood
how viral it is. Not only you mast mark all the members shared (methods
and field), instantiate classes with shared attribute, you also have to
create a duplicate all the methods to make them accessible with both
shared and non-shared (thread-local) instances:
class Array(T)
{
const(T) opIndex(uint index) const
{
return data[index];
}
T opIndex(uint index)
{
return data[index];
}
const(T) opIndex(uint index) shared const
{
return data[index];
}
shared(T) opIndex(uint index) shared
{
return data[index];
}
private T[] data;
}
And that's just opIndex. Ooops...
But not only that, every interface now have to specify the same method
twice, too:
interface Target
{
bool build();
bool build() shared;
void clean();
void clean() shared;
bool ready();
bool ready() shared;
void setBuildListener(BuildListener buildListener);
void setBuildListener(shared BuildListener buildListener) shared;
}
That's a bit frustrating. Most importantly, I don't even need shared
(__gshared was more than enough for me), yet I'm imposed on using it.
Oh, I can't use any of the druntime/Phobos classes (e.g. create an
instance of shared(Thread)), because none of them are shared-aware.
I think the design needs to be given a second thought before the plane
takes off because I'm afraid it may painfully crash.
shared library
I was able to build D runtime shared library (.so). The test app dies with segmentation fault, I am debugging it; decided to share. To reproduce my results one can load makefile from here: http://www.curoles.com/j/dso/so.mak (drop it into the runtime directory and call make -f so.mak) The problem happens in main() (defined in src/rt/dmain2.d) when first time a library function is called. The PLT entry look good, but jump to GOT seems to be wrong. I expect to see address of got.plt section (readelf -S ./test), instead the address for indirect jump is taken from some stack variable, the address is invalid and segmentation fault happens. I have written some notes here (gdb session): http://www.curoles.com/j/dso/dso.html Igor
Shared Memory
I just wanted to float an idea out there. I have been using technologies such as Terracotta and Memcache at work. Terracotta, has left me wondering about the applicability of integrating shared objects (memory) directly into a programming language. Is this anything anybody else here has thought of? Additionally, I was thinking about the concept of services vs libraries, and how shared objects might be a way to provide fast IPC while getting rid of the library concept. Thus, each service would have it's own memory space to manage and GC. One hinderence would be the versioning of the shared object, but if that object could be viewed through an interface... Just brainstorming.
Re: Shared
== Quote from dsimcha (dsim...@yahoo.com)'s article > I've reread the relevant TDPL chapter and I still don't quite understand the > following: > 1. What is shared? Is it simply a piece of syntactic salt to make it hard to > share data across threads by accident, or is there more to it? > 2. Is it fully or mostly implemented? Sorry, accidentally submitted the post before I was done. 3. How does casting to and from shared work? Under what circumstances can unshared data be cast to shared? Under what circumstances can shared data implicitly be cast to unshared?
Re: Shared
On 02/08/10 02:16, dsimcha wrote:
== Quote from dsimcha (dsim...@yahoo.com)'s article
I've reread the relevant TDPL chapter and I still don't quite understand the
following:
1. What is shared? Is it simply a piece of syntactic salt to make it hard to
share data across threads by accident, or is there more to it?
2. Is it fully or mostly implemented?
Sorry, accidentally submitted the post before I was done.
3. How does casting to and from shared work? Under what circumstances can
unshared data be cast to shared? Under what circumstances can shared data
implicitly be cast to unshared?
I too have had a lot of trouble using shared, but I am currently giving
it another serious try.
My observations so far are that the compiler's handling of it is a bit
buggy, but that it seems to more-or-less work, and will be usable when
the necessary library code is updated to use it - specifically Mutex,
Condition and concurrency's Mailbox.
You are not supposed to need to routinely cast to-and-from shared. Value
and immutable types should implicitly convert to/from shared, and
synchronized types should implicitly convert to shared. So for example,
a string is ok because it is a value and a pointer to immutable data.
I have found the following approach to work ok:
import std.stdio;
import std.traits;
import std.conv;
synchronized class Channel(T) if (!hasAliasing!T)
{
void add(T t) { }
}
alias Channel!string MyChannel1;
alias shared MyChannel1 MyChannel;
void main() {
auto channel = new MyChannel();
channel.add("hello");
}
The key trick for me was to use an alias to wrap the shared up with the
data type. For some reason the compiler didn't like it when I used a
templated type, but the second layer of aliases placated it - hopefully
that is a bug that will be fixed soon.
I hope that helps.
--
Graham St Jack
Re: Shared
On Sun, 01 Aug 2010 12:46:58 -0400, dsimcha wrote: == Quote from dsimcha (dsim...@yahoo.com)'s article I've reread the relevant TDPL chapter and I still don't quite understand the following: 1. What is shared? Is it simply a piece of syntactic salt to make it hard to share data across threads by accident, or is there more to it? 2. Is it fully or mostly implemented? Sorry, accidentally submitted the post before I was done. 3. How does casting to and from shared work? Under what circumstances can unshared data be cast to shared? Under what circumstances can shared data implicitly be cast to unshared? Let me preface this by saying I don't actually use shared on a daily basis, but I'll try and respond how I think it works: 1. It indicates to the compiler that multiple threads have direct access to the data. But more importantly, the *lack* of shared means that exactly one thread has direct access to the data. I see shared not as great a feature as unshared is. For an example of optimizations that can be had with unshared data, see the LRU cache for lock-free array appending. I think in terms of technical details, reading from/writing to a shared piece of data requires either a lock, or the compiler will insert a memory barrier around the write to ensure the write is atomic as long as the data written is small enough (I'm very fuzzy on these details, I use the term memory barrier in complete ignorance). Declaring a global variable shared also makes it a true global (not thread-local). I don't know what was decided on for shared classes/structs, I vaguely remember that the consensus was to require declaring the entire class shared at class definition, but I could be wrong. 2. I do not think it's fully implemented, but I think the intention is that it's fully implemented, so submit bugs against what it doesn't do if you find any. 3. *NO* implicit casting of shared<->unshared is allowed for references. To do so would violate the shared transitivity. It is ok to copy value-types to/from shared. Think of the relationship between unshared and shared in the same way as the relationship between mutable and immutable. I think you can cast shared to unshared if you *know* that no other thread will be able to access the data pointed at by the shared value. For instance, you can never take a reference to a shared global and cast that reference to unshared, because globals are always available to all threads. You can cast unshared to shared if you know that you have no other unshared references to the same data in your local thread. This one can be much easier to prove. Neither cast is statically checked or verified, it's up to you as the programmer to ensure these properties. -Steve
Shared pain
I had D code that provided a basis for creation of Windows services,
which I have just tried to get working with the latest D2.
No dice.
The point of failure was in this method
static void StartService()
{
if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *) &_sta[0]))
{
...
}
}
Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code
that install and remove the service worked fine, but the service control
dispatcher was clearly not happy about these structs being in TLS, and
the service crashed immediately when I tried to start it.
The class also has a couple of methods with signatures like:
extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}
That's probably not relevant, but I think it contributed to a host of
errors that finally made be take a big step back.
I re-implemented the thing in the style used for OOP in C.
struct ServiceBase
{
SERVICE_TABLE_ENTRY[2] _sta;
...
}
__gshared ServiceBase __sb;
void initialize(ServiceBase* sb) { ... }
void whatever(ServiceBase* sb ...) { ... }
and used the __sb global within the functions with the Windows signature,
which just became
extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}
Now it's working again, but I want to change it into idiomatically sound
D code.
Style suggestions please!
Steve
Migrating to Shared
I wrote a brief article which should help: http://www.digitalmars.com/d/2.0/migrate-to-shared.html
Semantics of shared
[from reddit] There was just a post to reddit announcing that thread local storage would be the default for global variables and that the 'shared' qualifier would make this happen. What I can't find is a description of typing rules surrounding 'shared'. From the discussion at reddit, it sounded like 'shared' was intended to mean 'possibly shared', with the implication that thread local objects can be treated as 'possibly shared'. The problem I see with this is that it implies that it is not safe to assign one shared reference to another, because the former may actually be thread local while the latter is actually global. This would seem to make the "maybe shared" concept pretty useless. Is this not a problem? Or if not, can someone clarify to me what the actual semantics & typing rules are? Thanks, Matt
shared class constructor
Just came across this while porting my code to enforce the use of shared.
---
module test;
shared class Foo {
this() {}
}
---
Error: cannot implicitly convert expression (this) of type shared(Foo) to
test.Foo
Of course I could just use new shared(Foo) but that would allow non-shared
instances of Foo to be created as well, which I do not want.
Shouldn't "this" resolve to the fully qualified class?
Re: Shared Hell
Denis Koroskin Wrote:
> I've recently updated to DMD2.035 (from DMD2.031 because all the later
> versions had issues with imports) and for the first time faced problems
> with shared modifier.
>
> I don't need shared and all my globals are __gshared (they are globally
> unique instances that don't need per-thread copies).
>
> Yet some of methods of the class hierarchy (a root singleton class and
> everything which is accessible through it) are synchronized (well, you
> know why). That's where the problems begin.
>
> Marking a method as synchronized automatically makes it shared (more or
> less obvious). And marking the method shared makes it unable to invoke
> with non-shared instance (and __gshared != shared), meaning that I'm
> unable to use my __gshared variables anymore, making this attribute
> useless for any serious safe programming.
>
> So I started with replacing __gshared with shared and quickly understood
> how viral it is. Not only you mast mark all the members shared (methods
> and field), instantiate classes with shared attribute, you also have to
> create a duplicate all the methods to make them accessible with both
> shared and non-shared (thread-local) instances:
>
> class Array(T)
> {
> const(T) opIndex(uint index) const
> {
> return data[index];
> }
>
> T opIndex(uint index)
> {
> return data[index];
> }
>
> const(T) opIndex(uint index) shared const
> {
> return data[index];
> }
>
> shared(T) opIndex(uint index) shared
> {
> return data[index];
> }
>
> private T[] data;
> }
>
> And that's just opIndex. Ooops...
>
> But not only that, every interface now have to specify the same method
> twice, too:
>
> interface Target
> {
> bool build();
> bool build() shared;
> void clean();
> void clean() shared;
> bool ready();
> bool ready() shared;
> void setBuildListener(BuildListener buildListener);
> void setBuildListener(shared BuildListener buildListener) shared;
> }
>
> That's a bit frustrating. Most importantly, I don't even need shared
> (__gshared was more than enough for me), yet I'm imposed on using it.
>
> Oh, I can't use any of the druntime/Phobos classes (e.g. create an
> instance of shared(Thread)), because none of them are shared-aware.
>
> I think the design needs to be given a second thought before the plane
> takes off because I'm afraid it may painfully crash.
Wow.
I certainly won't switch to D2 if every getter must be written 4 times (and 4
times more with immutable(T) ?).
I don't even understand the fundamental difference between __gshared
and shared : is this only transitivity ?
Re: Shared Hell
Denis Koroskin wrote: I've recently updated to DMD2.035 (from DMD2.031 because all the later versions had issues with imports) and for the first time faced problems with shared modifier. I don't need shared and all my globals are __gshared (they are globally unique instances that don't need per-thread copies). I don't understand. Are you running multiple threads? Are those threads accessing globals? A function that accesses shared data has to put in fences. There's no way to have the same code deal with shared and unshared code. As an escape from the type system, you can always cast away the shared-ness. But I wonder about code that both uses global variables shared across threads that don't need synchronization?
Re: Shared Hell
Denis Koroskin wrote: I've recently updated to DMD2.035 (from DMD2.031 because all the later versions had issues with imports) and for the first time faced problems with shared modifier. I don't need shared and all my globals are __gshared (they are globally unique instances that don't need per-thread copies). Yet some of methods of the class hierarchy (a root singleton class and everything which is accessible through it) are synchronized (well, you know why). That's where the problems begin. Marking a method as synchronized automatically makes it shared (more or less obvious). And marking the method shared makes it unable to invoke with non-shared instance (and __gshared != shared), meaning that I'm unable to use my __gshared variables anymore, making this attribute useless for any serious safe programming. So I started with replacing __gshared with shared and quickly understood how viral it is. Not only you mast mark all the members shared (methods and field), instantiate classes with shared attribute, you also have to create a duplicate all the methods to make them accessible with both shared and non-shared (thread-local) instances: Why can't you use a non-shared method on a shared object? The compiler could insert locking on the caller side. Why can't you use a shared method on a non-shared object? The compiler could, as an optimization, duplicate the method, minus the synchronization. Or it could leave in the locking, which is expensive but correct.
Re: Shared Hell
Walter Bright wrote: Denis Koroskin wrote: I've recently updated to DMD2.035 (from DMD2.031 because all the later versions had issues with imports) and for the first time faced problems with shared modifier. I don't need shared and all my globals are __gshared (they are globally unique instances that don't need per-thread copies). I don't understand. Are you running multiple threads? Are those threads accessing globals? A function that accesses shared data has to put in fences. There's no way to have the same code deal with shared and unshared code. Acquiring a lock on a non-shared instance is safe, just an unnecessary expense. I would have looked into optimizing this expense away rather than punting the problem to the programmer. As an escape from the type system, you can always cast away the shared-ness. But I wonder about code that both uses global variables shared across threads that don't need synchronization? Maybe the methods are mostly inherently threadsafe. Only a small portion requires locking, so it's more efficient to handle it manually.
Re: Shared Hell
On Wed, 28 Oct 2009 13:17:43 +0300, Walter Bright
wrote:
Denis Koroskin wrote:
I've recently updated to DMD2.035 (from DMD2.031 because all the later
versions had issues with imports) and for the first time faced problems
with shared modifier.
I don't need shared and all my globals are __gshared (they are
globally unique instances that don't need per-thread copies).
I don't understand. Are you running multiple threads? Are those threads
accessing globals?
Yes.
A function that accesses shared data has to put in fences. There's no
way to have the same code deal with shared and unshared code.
That's frustrating. I'd like to use the same class for both cases.
But I wonder about code that both uses global variables shared across
threads that don't need synchronization?
You missed the point. I do the synchronization myself and I'm fine with
switching to shared (I do believe it is a nice concept). The reason I use
__gshared is because shared object were garbage-collected while still
being in use a few versions of DMD back and I had no choice but to switch
to __gshared. I hope it is fixed by now.
But I still can't make my data shared, since shared is transitive (viral).
After a few hours or work I still can't even compile my code.
As an escape from the type system, you can always cast away the
shared-ness.
That's the only way I have now. Casts from shared to unshared *everywhere*:
class BuildManager : BuildListener
{
synchronized void build(shared Target target)
{
// ...
_buildingThread = new shared(Thread)(&_startBuild); // creating a
new shared Thread. Yes, shared Thread, because BuildManager is global.
//_buildingThread.start(); // Error: function
core.thread.Thread.start () is not callable using argument types () shared
(cast(Thread)_buildingThread).start(); // works, but ugly, and I
don't have a reason to hijack the type system in this case
// ...
}
}
Andrei would suggest a Shared!(T) template that would wrap an unshared
type and make all methods shared. This would work, but requires full AST
manipulation capabilities (it's clearly not enough to just mark all the
members shared). What should we do until then?
Re: Shared Hell
Denis Koroskin Wrote: > I've recently updated to DMD2.035 (from DMD2.031 because all the later > versions had issues with imports) and for the first time faced problems > with shared modifier. A quick trip over to bugzilla is all you need to see that shared is completely broken. Here's what I see as basic functionality that is broken: 3035, 3089, 3090, 3091, 3102, 3349. Half of those are bugs I created within an hour of trying to use shared for real. I also have a bug in druntime as well, but Batosz's rewrite should address that issue. Interestingly, even though I reported that bug long ago, it still bit me. It was the source of a segfault that took me 3 months to track down (mostly due to an inability to use gdb until recently). My code is littered with places that cast away shared since it was so utterly unusable when I tried. I'm still waiting for basic bugs to be closed before I try again.
Re: Shared Hell
Christopher Wright Wrote: > Denis Koroskin wrote: > > I've recently updated to DMD2.035 (from DMD2.031 because all the later > > versions had issues with imports) and for the first time faced problems > > with shared modifier. > > > > I don't need shared and all my globals are __gshared (they are globally > > unique instances that don't need per-thread copies). > > > > Yet some of methods of the class hierarchy (a root singleton class and > > everything which is accessible through it) are synchronized (well, you > > know why). That's where the problems begin. > > > > Marking a method as synchronized automatically makes it shared (more or > > less obvious). And marking the method shared makes it unable to invoke > > with non-shared instance (and __gshared != shared), meaning that I'm > > unable to use my __gshared variables anymore, making this attribute > > useless for any serious safe programming. > > > > So I started with replacing __gshared with shared and quickly understood > > how viral it is. Not only you mast mark all the members shared (methods > > and field), instantiate classes with shared attribute, you also have to > > create a duplicate all the methods to make them accessible with both > > shared and non-shared (thread-local) instances: > > Why can't you use a non-shared method on a shared object? The compiler > could insert locking on the caller side. > > Why can't you use a shared method on a non-shared object? The compiler > could, as an optimization, duplicate the method, minus the > synchronization. Or it could leave in the locking, which is expensive > but correct. The caller would have to acquire locks for all the data accessed by the non-shared method and all non-shared methods it calls. Additionally, non-shared functions can access thread-local data. Neither of those issues are easily solved. Bartosz's scheme would solve the first one due to implied ownership.
Re: Shared Hell
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
> Denis Koroskin wrote:
> > I've recently updated to DMD2.035 (from DMD2.031 because all the later
> > versions had issues with imports) and for the first time faced problems
> > with shared modifier.
> >
> > I don't need shared and all my globals are __gshared (they are globally
> > unique instances that don't need per-thread copies).
> I don't understand. Are you running multiple threads? Are those threads
> accessing globals?
> A function that accesses shared data has to put in fences. There's no
> way to have the same code deal with shared and unshared code.
> As an escape from the type system, you can always cast away the
> shared-ness. But I wonder about code that both uses global variables
> shared across threads that don't need synchronization?
I have at least one use case for __gshareds in multithreaded code. I often use
__gshared variables to hold program parameters that are only set using getopt at
program startup and never modified after the program becomes multithreaded.
That said, although I use D2 regularly, I basically have ignored shared's
existence up to this point. The semantics aren't fully implemented, so right
now
you get all the bondage and discipline of it without any of the benefits. As
far
as the problem of synchronized methods automatically being shared, here's an
easy
workaround until the rough edges of shared are worked out:
//Instead of this:
synchronized SomeType someMethod(Foo args) {
// Do stuff.
}
// Use this:
SomeType someMethod(Foo args) {
synchronized(this) {
// Do stuff.
}
}
Re: Shared Hell
Denis Koroskin Wrote:
> > As an escape from the type system, you can always cast away the
> > shared-ness.
>
> That's the only way I have now. Casts from shared to unshared *everywhere*:
>
> class BuildManager : BuildListener
> {
> synchronized void build(shared Target target)
> {
> // ...
>
> _buildingThread = new shared(Thread)(&_startBuild); // creating a
> new shared Thread. Yes, shared Thread, because BuildManager is global.
>
> //_buildingThread.start(); // Error: function
> core.thread.Thread.start () is not callable using argument types () shared
> (cast(Thread)_buildingThread).start(); // works, but ugly, and I
> don't have a reason to hijack the type system in this case
>
> // ...
> }
> }
you can use local variables to not have to cast in every statement.
class BuildManager : BuildListener
{
synchronized void build(shared Target target)
{
// ...
bt = new Thread(&_startBuild);
_buildingThread = cast(shared Thread)bt; //store as shared
bt.start(); //work with non-shared variable normally
// ...
}
}
Re: Shared Hell
Denis Koroskin wrote:
On Wed, 28 Oct 2009 13:17:43 +0300, Walter Bright
wrote:
Denis Koroskin wrote:
I've recently updated to DMD2.035 (from DMD2.031 because all the
later versions had issues with imports) and for the first time faced
problems with shared modifier.
I don't need shared and all my globals are __gshared (they are
globally unique instances that don't need per-thread copies).
I don't understand. Are you running multiple threads? Are those
threads accessing globals?
Yes.
A function that accesses shared data has to put in fences. There's no
way to have the same code deal with shared and unshared code.
That's frustrating. I'd like to use the same class for both cases.
But I wonder about code that both uses global variables shared across
threads that don't need synchronization?
You missed the point. I do the synchronization myself and I'm fine with
switching to shared (I do believe it is a nice concept). The reason I
use __gshared is because shared object were garbage-collected while
still being in use a few versions of DMD back and I had no choice but to
switch to __gshared. I hope it is fixed by now.
Which OS are you using? This is definitely a bug. If it's still there,
you can work around by adding the tls data as a "root" to the gc.
But I still can't make my data shared, since shared is transitive
(viral). After a few hours or work I still can't even compile my code.
As an escape from the type system, you can always cast away the
shared-ness.
That's the only way I have now. Casts from shared to unshared *everywhere*:
class BuildManager : BuildListener
{
synchronized void build(shared Target target)
{
// ...
_buildingThread = new shared(Thread)(&_startBuild); // creating
a new shared Thread. Yes, shared Thread, because BuildManager is global.
//_buildingThread.start(); // Error: function
core.thread.Thread.start () is not callable using argument types () shared
(cast(Thread)_buildingThread).start(); // works, but ugly, and I
don't have a reason to hijack the type system in this case
// ...
}
}
Andrei would suggest a Shared!(T) template that would wrap an unshared
type and make all methods shared. This would work, but requires full AST
manipulation capabilities (it's clearly not enough to just mark all the
members shared). What should we do until then?
shared(T) should transitively make a new type where it's all shared.
Re: Shared Hell
On Wed, 28 Oct 2009 16:19:11 +0300, dsimcha wrote:
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
Denis Koroskin wrote:
> I've recently updated to DMD2.035 (from DMD2.031 because all the later
> versions had issues with imports) and for the first time faced
problems
> with shared modifier.
>
> I don't need shared and all my globals are __gshared (they are
globally
> unique instances that don't need per-thread copies).
I don't understand. Are you running multiple threads? Are those threads
accessing globals?
A function that accesses shared data has to put in fences. There's no
way to have the same code deal with shared and unshared code.
As an escape from the type system, you can always cast away the
shared-ness. But I wonder about code that both uses global variables
shared across threads that don't need synchronization?
I have at least one use case for __gshareds in multithreaded code. I
often use
__gshared variables to hold program parameters that are only set using
getopt at
program startup and never modified after the program becomes
multithreaded.
That said, although I use D2 regularly, I basically have ignored shared's
existence up to this point. The semantics aren't fully implemented, so
right now
you get all the bondage and discipline of it without any of the
benefits. As far
as the problem of synchronized methods automatically being shared,
here's an easy
workaround until the rough edges of shared are worked out:
//Instead of this:
synchronized SomeType someMethod(Foo args) {
// Do stuff.
}
// Use this:
SomeType someMethod(Foo args) {
synchronized(this) {
// Do stuff.
}
}
Yes, I've though about it. That's probably the only workaround for now and
I'll give it. Thanks.
Re: Shared Hell
On Wed, 28 Oct 2009 20:30:45 +0300, Walter Bright
wrote:
Andrei would suggest a Shared!(T) template that would wrap an unshared
type and make all methods shared. This would work, but requires full
AST manipulation capabilities (it's clearly not enough to just mark all
the members shared). What should we do until then?
shared(T) should transitively make a new type where it's all shared.
Type, yes, but not the methods. It will make a type with *no* methods
usable (because they still accept and operate on thread-local variables).
I was hinting about template that would create a separate fully
shared-aware type so that there would be no need for code duplication.
I.e. it would transform the following class:
class Float
{
this(float value) { this.value = value; }
Float opAdd(Float other)
{
return new Vector(this.value + other.value);
}
private float value;
}
into the following:
class SharedFloat
{
this(float value) shared { this.value = value; }
shared Float opAdd(shared Float other) shared
{
return new shared Vector(this.value + other.value);
}
private shared float value;
}
This obviously requires techniques not available in D currently (AST
manipulation).
Re: Shared Hell
Walter Bright Wrote: > A function that accesses shared data has to put in fences. There's no > way to have the same code deal with shared and unshared code. Shared code perfectly deals with unshared data (it's not guaranteed that shared data is accessed by more than one thread). In other words unshared data is implicitly castable to shared.
Re: Shared Hell
Kagamin Wrote: > Walter Bright Wrote: > > > A function that accesses shared data has to put in fences. There's no > > way to have the same code deal with shared and unshared code. > > Shared code perfectly deals with unshared data (it's not guaranteed that > shared data is accessed by more than one thread). In other words unshared > data is implicitly castable to shared. Uh... this post was a little fast, ignore it, please.
Re: Shared Hell
Christopher Wright Wrote: > > A function that accesses shared data has to put in fences. There's no > > way to have the same code deal with shared and unshared code. > > Acquiring a lock on a non-shared instance is safe, just an unnecessary > expense. I would have looked into optimizing this expense away rather > than punting the problem to the programmer. Shared code can put data into *really* shared environment, which must not happen for unshared data.
Re: Shared Hell
Kagamin wrote: Christopher Wright Wrote: A function that accesses shared data has to put in fences. There's no way to have the same code deal with shared and unshared code. Acquiring a lock on a non-shared instance is safe, just an unnecessary expense. I would have looked into optimizing this expense away rather than punting the problem to the programmer. Shared code can put data into *really* shared environment, which must not happen for unshared data. Okay, but the compiler can try copying the method, removing the 'shared' storage class, and running semantic on the result. If that succeeds, you're guaranteed not to be doing anything that would be unsafe unless you're casting, and you can call the shared method on a local instance. This lets you avoid whole program analysis and blacklisting unsafe operations. In this case, the compiler can also point to the line of code that prevents the method from being marked not shared.
Re: Shared Hell
On Thu, 29 Oct 2009 07:25:20 -0400, Christopher Wright wrote: Kagamin wrote: Christopher Wright Wrote: A function that accesses shared data has to put in fences. There's no way to have the same code deal with shared and unshared code. Acquiring a lock on a non-shared instance is safe, just an unnecessary expense. I would have looked into optimizing this expense away rather than punting the problem to the programmer. Shared code can put data into *really* shared environment, which must not happen for unshared data. Okay, but the compiler can try copying the method, removing the 'shared' storage class, and running semantic on the result. If that succeeds, you're guaranteed not to be doing anything that would be unsafe unless you're casting, and you can call the shared method on a local instance. This lets you avoid whole program analysis and blacklisting unsafe operations. In this case, the compiler can also point to the line of code that prevents the method from being marked not shared. Let's say your shared method your attempting to analyze calls an obscure shared method: int foo(shared C c); How do you determine if it can be called without the 'shared' qualifier? This is one of the problems with not importing from compiled interface files. -Steve
Re: Shared Hell
Kagamin wrote: Walter Bright Wrote: A function that accesses shared data has to put in fences. There's no way to have the same code deal with shared and unshared code. Shared code perfectly deals with unshared data (it's not guaranteed that shared data is accessed by more than one thread). In other words unshared data is implicitly castable to shared. Unfortunately, there is a subtle problem with that that makes such implicit casts unsafe. Shared data can be handed off to other threads. So if thread local data is implicitly cast to shared, then it can be handed off to other threads. Meaning it is no longer guaranteed to be thread local.
Various shared bugs
So, after months of avoiding shared, I decided to see if I could remove all my casting away of shared. It looks like things are still pretty buggy (or at least not particularly easy to use). "is(T : shared)" gives a parse error "alias shared T U" silently does the wrong thing. (Use "alias shared(T) U" instead) " is not callable using argument types ()" is just awful. Typically, it means you're calling a non-shared function with a shared type or a shared function with a non-shared type. I hit into a case where I got that message with a shared type and a shared method, but I have mad no attempt to reproduce. The error "Can not implicitly convert expression of type(this) of type shared(xxx) to full.name.xxx" can pop up when defining shared this() constructor gives no line number where the offending usage occurs. I've backed out most of my pro-shared changes and will try again in a few months :(
Re: shared library
Walter, Don, and All Function codgen() in dmd/backend/cgcod.c has lines like: if (0 && config.flags3 & CFG3pic) so the PIC flag is ignored (2.040) Do I understand it right? I look on this code because value in %ebx (PIC register) does not look right when I debug application that is dynamically linked with runtime shared library. Also
Re: shared library
Walter, Don, and All Function codgen() in dmd/backend/cgcod.c has lines like: if (0 && config.flags3 & CFG3pic) so the PIC flag is ignored (2.040) Do I understand it right? I look on this code because value in %ebx (PIC register) does not look right when I debug application that is dynamically linked with runtime shared library. Thanks, Igor "Igor Lesik" wrote in message news:hmujil$nt...@digitalmars.com... > Walter, Don, and All > > Function codgen() in dmd/backend/cgcod.c has lines like: > if (0 && config.flags3 & CFG3pic) > so the PIC flag is ignored (2.040) > Do I understand it right? > > I look on this code because value in %ebx (PIC register) does not look > right when I debug > application that is dynamically linked with runtime shared library. > > Also >
Re: Shared Memory
On Thu, 16 Apr 2009 17:20:47 -0400, sesteel wrote: I just wanted to float an idea out there. I have been using technologies such as Terracotta and Memcache at work. Terracotta, has left me wondering about the applicability of integrating shared objects (memory) directly into a programming language. Is this anything anybody else here has thought of? Yes. D2 has introduced the shared storage type (though it doesn't do anything yet) and there's also Bartosz's blog (http://bartoszmilewski.wordpress.com/feed/). Additionally, I was thinking about the concept of services vs libraries, and how shared objects might be a way to provide fast IPC while getting rid of the library concept. Thus, each service would have it's own memory space to manage and GC. One hinderence would be the versioning of the shared object, but if that object could be viewed through an interface... Just brainstorming. This sounds like you're proposing the actor model, (at a coarser level than objects) and possibly also thread-local heaps. Also, for those who don't know (I didn't) Terracotta is an open-source Java clustering package, which essentially transparently converts a multi-threaded application into a cluster application. Neat.
Shared Class Variables
Evening. I'm having a bit of a problem and I'm hoping someone can help. I'm trying to create a class that is shared across threads. The only purpose of this class is to write data to somewhere, though currently a file. A single-threaded version of this works fine, however I can't seem to get the code to work correctly when dealing with multiple threads. I've gotten sharing issues, compilation issues trying to figure out how to use a shared class correctly, and currently an exception occurring during class finalization. So, my question is, what is the correct way to do this? Would a class work or would a struct be better? Perhaps a singleton? Thanks. Casey
Re: Shared pain
I'm not familiar with the API, but are you able to declare your original _sta
as immutable and call it a day? Immutable data should have the same protection
as __gshared but with no implications of bypassing the type system.
If that's not helpful, can you give more details like calling patterns? Is the
function called from one thread or from many? My quick read of the windows
documentation makes it seem like that's true. Probably unimportant, but I'd use
_sta.ptr instead of &_sta[0] because the former more clearly implies array
pointer to me, but that may be my own quirky style.
Steve Teale Wrote:
> I had D code that provided a basis for creation of Windows services,
> which I have just tried to get working with the latest D2.
>
> No dice.
>
> The point of failure was in this method
>
> static void StartService()
> {
>if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *) &_sta[0]))
>{
> ...
>}
> }
>
> Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code
> that install and remove the service worked fine, but the service control
> dispatcher was clearly not happy about these structs being in TLS, and
> the service crashed immediately when I tried to start it.
>
> The class also has a couple of methods with signatures like:
>
> extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}
>
> That's probably not relevant, but I think it contributed to a host of
> errors that finally made be take a big step back.
>
> I re-implemented the thing in the style used for OOP in C.
>
> struct ServiceBase
> {
>SERVICE_TABLE_ENTRY[2] _sta;
>...
> }
>
> __gshared ServiceBase __sb;
>
> void initialize(ServiceBase* sb) { ... }
> void whatever(ServiceBase* sb ...) { ... }
>
> and used the __sb global within the functions with the Windows signature,
> which just became
>
> extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}
>
> Now it's working again, but I want to change it into idiomatically sound
> D code.
>
> Style suggestions please!
>
> Steve
Re: Shared pain
On Thu, 18 Nov 2010 06:26:39 -0500, Steve Teale
wrote:
I had D code that provided a basis for creation of Windows services,
which I have just tried to get working with the latest D2.
No dice.
The point of failure was in this method
static void StartService()
{
if (!StartServiceCtrlDispatcherA(cast(SERVICE_TABLE_ENTRY *)
&_sta[0]))
{
...
}
}
Where _sta is an array of SERVICE_TABLE_ENTRY structs. The bits of code
that install and remove the service worked fine, but the service control
dispatcher was clearly not happy about these structs being in TLS, and
the service crashed immediately when I tried to start it.
I think this has something to do with how services start. From this page:
http://msdn.microsoft.com/en-us/library/ms685990(v=VS.85).aspx
I see that StartServiceCtrlDispatcher creates its own thread to run
ServiceMain, which most likely does not call the D thread initialization
routines (not sure). TLS may not be properly set up inside your thread
that is running ServiceMain, and probably the static this() module
functions have not been called.
The class also has a couple of methods with signatures like:
extern (Windows) static export void service_ctrl(uint dwCtrlCode) {...}
That's probably not relevant, but I think it contributed to a host of
errors that finally made be take a big step back.
I re-implemented the thing in the style used for OOP in C.
struct ServiceBase
{
SERVICE_TABLE_ENTRY[2] _sta;
...
}
__gshared ServiceBase __sb;
void initialize(ServiceBase* sb) { ... }
void whatever(ServiceBase* sb ...) { ... }
and used the __sb global within the functions with the Windows signature,
which just became
extern (Windows) export void service_ctrl(uint dwCtrlCode) {...}
Now it's working again, but I want to change it into idiomatically sound
D code.
Style suggestions please!
I think using a class in D is a good idea. But what you may need is
global functions which forward to your class methods. This is how I would
probably do it.
-Steve
Re: Shared pain
On Thu, 18 Nov 2010 08:33:12 -0500, Steven Schveighoffer wrote: > > I think using a class in D is a good idea. But what you may need is > global functions which forward to your class methods. This is how I > would probably do it. > > -Steve Steve, Thanks. I kind of went that way. Having done the laborious C OOP style thing, I put most of the methods back into the struct, just leaving the extern(Windows) functions at global scope - they referred to the __gshare'd object explicitly - I hate that. Then I hid all of the crap behind a class that sneakily referred to the __gshared struct, but at least now to an external user of the module the interface looks much the same as it did before. Now it works OK with the test case example - a service that just writes entries in the event log. However, the real service I'm trying to update works fine if I run it in debug mode (i.e. basically as just an executable), but if I run it as a service it seems to crash the first time I use a RegExp.find(). As you can imagine, this is terrible to debug - printf or similar is bad enough, but for the service I'm pushing entries into the Windows event logger system. Fortunately my D class for doing that has survived the D2 changes. I'm going to try tweaking the service so it will interact with the desktop, then at least I can revert to printf. But first I think I'll stop and do some carpentry for a couple of days. Then eventually something might dawn on me. Steve
Re: Shared pain
On Thu, 18 Nov 2010 13:53:53 -0500, Steve Teale wrote: On Thu, 18 Nov 2010 08:33:12 -0500, Steven Schveighoffer wrote: I think using a class in D is a good idea. But what you may need is global functions which forward to your class methods. This is how I would probably do it. -Steve Steve, Thanks. I kind of went that way. Having done the laborious C OOP style thing, I put most of the methods back into the struct, just leaving the extern(Windows) functions at global scope - they referred to the __gshare'd object explicitly - I hate that. Then I hid all of the crap behind a class that sneakily referred to the __gshared struct, but at least now to an external user of the module the interface looks much the same as it did before. Now it works OK with the test case example - a service that just writes entries in the event log. However, the real service I'm trying to update works fine if I run it in debug mode (i.e. basically as just an executable), but if I run it as a service it seems to crash the first time I use a RegExp.find(). As I said before, I don't know if the thread being created by the windows service procedure is properly initializing the D modules of the library/runtime. Try as the first line of ServiceMain to initialize the current thread: auto mythread = thread_attachThis(); see http://www.digitalmars.com/d/2.0/phobos/core_thread.html -Steve
Re: Shared pain
> As I said before, I don't know if the thread being created by the > windows service procedure is properly initializing the D modules of the > library/runtime. Try as the first line of ServiceMain to initialize the > current thread: > > auto mythread = thread_attachThis(); > > see http://www.digitalmars.com/d/2.0/phobos/core_thread.html > > -Steve Steve, I don't think it gets as far as ServiceMain. Anyway, I tried it there, and at the point just before the first WINAPI call, but it still crashes. I can fix it, by making my statically initialized RegExp objects __gshared or immutable, but the latter involves inserting a slew of tedious casts into my XML parser, since RexExp calls are used all over the place. See my separate post for my thoughts on that. Thanks Steve
Re: Shared pain
On Thu, 18 Nov 2010 11:26:39 +, Steve Teale wrote:
> I had D code that provided a basis for creation of Windows services,
> which I have just tried to get working with the latest D2.
>
> No dice.
>
I have made some progress in understanding this. It appears that any D
static data structures containing function pointers will now cause a
Windows service to crash.
For happiness, these must be forced into the global data segment.
I have demonstrated this within a vestigial service for a couple of cases
- plain old struct containing an int and a function pointer, and for
Regexp objects by using and not using __gshared on the offending objects.
Now that I'm reasonably sure what's happening, I ought to be able to make
these things work by making such static members immutable, and
initializing them in static this().
However, I then run into a slew of complier errors. Lets start with this
simple framework:
import std.stdio;
import std.regexp;
class ThingWithStatic
{
static RegExp rex;
string ns;
static this()
{
rex = RegExp("ab+a");
}
this()
{
ns = "abbaabba";
}
int getFromStatic() { return rex.find(ns); }
}
void main()
{
ThingWithStatic tws = new ThingWithStatic();
writefln("%d", tws.getFromStatic());
}
This compiles fine, but if I include this class in a service, and call
getFromStatic(), the service will crash. If I change to:
__gshared RegExp rex;
It compiles fine, and the service does not crash.
If I change to:
static immutable RegExp rex;
Then I get errors:
triv.d(11): Error: cannot implicitly convert expression (opCall("ab
+a",null)) of type std.regexp.RegExp to immutable(RegExp)
triv.d(19): Error: function std.regexp.RegExp.find (string string) is not
callable using argument types (string) immutable
st...@ubuntu:~/scratch$
So then I use a cast:
rex = cast(immutable(RegExp)) RegExp("ab+a");
and get down to:
triv.d(19): Error: function std.regexp.RegExp.find (string string) is not
callable using argument types (string) immutable
as somewhat wierd message in itself. So I use a cast in the call to find
():
return (cast(RegExp) rex).find(ns);
At this point it compiles, and if it's used in the service, the service
does not crash.
BUT - all these complicated casts will be extremely tedious to
administer to code that uses RexExp methods all over the place, so
__gshared is a great temptation. The compiler should try to eliminate
such temptations in the interests of safety.
Also why are these casts necessary. If I am assigning to something that
is the same type and was declared as immutable should not at least the
first cast be done implicitly. In the second case, I'm using a Regexp
object, wherever it may be stored, so why the cast? If find was pure
would the cast still be required?
Thanks
Steve
Re: Shared pain
On Fri, 19 Nov 2010 01:49:30 -0500, Steve Teale wrote: As I said before, I don't know if the thread being created by the windows service procedure is properly initializing the D modules of the library/runtime. Try as the first line of ServiceMain to initialize the current thread: auto mythread = thread_attachThis(); see http://www.digitalmars.com/d/2.0/phobos/core_thread.html -Steve Steve, I don't think it gets as far as ServiceMain. Anyway, I tried it there, and at the point just before the first WINAPI call, but it still crashes. I can fix it, by making my statically initialized RegExp objects __gshared or immutable, but the latter involves inserting a slew of tedious casts into my XML parser, since RexExp calls are used all over the place. See my separate post for my thoughts on that. OK, I'm out of ideas, sorry :( It certainly looks like there is an issue with TLS. I wouldn't suggest band-aiding things how you are doing, because some modules in phobos/druntime use TLS also. It's reasonable to assume that if your uses of TLS are failing, those may fail as well. -Steve
Re: Shared pain
A cast to immutable is required when constructing immutable objects. The type
system can't prove that's safe. It's a design limitation to keep complexity low.
http://www.digitalmars.com/d/2.0/phobos/std_regexp.html
Looking at the Regexp docs, find isn't marked const or pure. If that's an
oversite, file it in bugzilla. If it can't be const, then you should find
another approach.
When this stuff crashes for you, is it accessing a different thread's TLS or is
it simply unable to handle TLS at all?
Steve Teale Wrote:
> On Thu, 18 Nov 2010 11:26:39 +, Steve Teale wrote:
>
> > I had D code that provided a basis for creation of Windows services,
> > which I have just tried to get working with the latest D2.
> >
> > No dice.
> >
> I have made some progress in understanding this. It appears that any D
> static data structures containing function pointers will now cause a
> Windows service to crash.
>
> For happiness, these must be forced into the global data segment.
>
> I have demonstrated this within a vestigial service for a couple of cases
> - plain old struct containing an int and a function pointer, and for
> Regexp objects by using and not using __gshared on the offending objects.
>
> Now that I'm reasonably sure what's happening, I ought to be able to make
> these things work by making such static members immutable, and
> initializing them in static this().
>
> However, I then run into a slew of complier errors. Lets start with this
> simple framework:
>
> import std.stdio;
> import std.regexp;
>
> class ThingWithStatic
> {
>static RegExp rex;
>string ns;
>
>static this()
>{
> rex = RegExp("ab+a");
>}
>
>this()
>{
> ns = "abbaabba";
>}
>
>int getFromStatic() { return rex.find(ns); }
> }
>
> void main()
> {
>ThingWithStatic tws = new ThingWithStatic();
>writefln("%d", tws.getFromStatic());
> }
>
> This compiles fine, but if I include this class in a service, and call
> getFromStatic(), the service will crash. If I change to:
>
> __gshared RegExp rex;
>
> It compiles fine, and the service does not crash.
>
> If I change to:
>
>static immutable RegExp rex;
>
> Then I get errors:
>
> triv.d(11): Error: cannot implicitly convert expression (opCall("ab
> +a",null)) of type std.regexp.RegExp to immutable(RegExp)
> triv.d(19): Error: function std.regexp.RegExp.find (string string) is not
> callable using argument types (string) immutable
> st...@ubuntu:~/scratch$
>
> So then I use a cast:
>
> rex = cast(immutable(RegExp)) RegExp("ab+a");
>
> and get down to:
>
> triv.d(19): Error: function std.regexp.RegExp.find (string string) is not
> callable using argument types (string) immutable
>
> as somewhat wierd message in itself. So I use a cast in the call to find
> ():
>
> return (cast(RegExp) rex).find(ns);
>
> At this point it compiles, and if it's used in the service, the service
> does not crash.
>
> BUT - all these complicated casts will be extremely tedious to
> administer to code that uses RexExp methods all over the place, so
> __gshared is a great temptation. The compiler should try to eliminate
> such temptations in the interests of safety.
>
> Also why are these casts necessary. If I am assigning to something that
> is the same type and was declared as immutable should not at least the
> first cast be done implicitly. In the second case, I'm using a Regexp
> object, wherever it may be stored, so why the cast? If find was pure
> would the cast still be required?
>
> Thanks
> Steve
Re: Shared pain
On Fri, 19 Nov 2010 08:25:17 -0500, Jason House wrote: > A cast to immutable is required when constructing immutable objects. The > type system can't prove that's safe. It's a design limitation to keep > complexity low. > Jason, But design limitations like that will force programmers who are on a tight schedule to just say __gshared, and to hell with it. Languages can't be designed just on theory - some recognition of practicality is also required. As to the thread - I don't think that is under my control. I believe the Windows service dispatcher does its own thing. Thanks for taking the trouble Steve
Re: Shared pain
Steve Teale: > Languages can't be designed just on theory - some recognition of > practicality is also required. In the case of creation of immutable data structures it looks the opposite to me: DMD is too much pragmatic and there isn't enough theory behind it :-) > But design limitations like that will force programmers who are on a > tight schedule to just say __gshared, and to hell with it. Regarding the creation of immutable data structures, there is a proposal that is probably able to remove some of the pain: the result of strongly pure functions may become implicitly castable to immutable. Bye, bearophile
Re: Shared pain
On Fri, 19 Nov 2010 11:23:44 -0500, bearophile wrote: > Regarding the creation of immutable data structures, there is a proposal > that is probably able to remove some of the pain: the result of strongly > pure functions may become implicitly castable to immutable. > > Bye, > bearophile BP, I admire your dedication to language theory and purity, but there are many who'd translate that to impracticality and obscurity. I came to D in the first place because I found it refreshingly clear and easy to use after C++ and Java. But now it's getting painful. I've bailed out several times, just keep coming back to see how it is doing. I have a bunch of code that worked with D1 and D2 at one time. I've given up on D1, since it is now obviously legacy, but even without the complexity of supporting both, It's been real hard this visit to get things working again. What's your estimate of how long it will be before D is a stable language? Thanks Steve
Re: Shared pain
On 19-nov-10, at 17:42, Steve Teale wrote: On Fri, 19 Nov 2010 11:23:44 -0500, bearophile wrote: Regarding the creation of immutable data structures, there is a proposal that is probably able to remove some of the pain: the result of strongly pure functions may become implicitly castable to immutable. Bye, bearophile BP, I admire your dedication to language theory and purity, but there are many who'd translate that to impracticality and obscurity. I came to D in the first place because I found it refreshingly clear and easy to use after C++ and Java. But now it's getting painful. I've bailed out several times, just keep coming back to see how it is doing. I have a bunch of code that worked with D1 and D2 at one time. I've given up on D1, since it is now obviously legacy, but even without the complexity of supporting both, It's been real hard this visit to get things working again. What's your estimate of how long it will be before D is a stable language? well D1 is pretty stable I think, if you are interested in stability that is a good choice, has worked well for me. This does not mean that I will not consider D2, but D1 is my main workhorse. Fwzi Thanks Steve
Re: Shared pain
Steve Teale: > I admire your dedication to language theory and purity, but there are > many who'd translate that to impracticality and obscurity. I like that idea about immutables & strong pure functions, but it was not an idea of mine :-) That enhancement request in Bugzilla is not written by me. > I came to D in the first place because I found it refreshingly clear and > easy to use after C++ and Java. But now it's getting painful. But strongly pure functions are already present in D. So it's morally good to push its usages and implications as far as possible, otherwise we are wasting possibilities. That implicit cast idea is another implication of purity. It's a clean thing, it's not a hack or workaround or a dangerous thing or something like that, it comes from a clean & logical reasoning about consequences of purity. So it can't be compared to many other dirty things present in D2. It actually helps to clean the programs a little, because removes the need for one important use case of casts. So I can't agree. There is a large complexity difference between extending the logical implications of an already present feature, and adding a true special case. > What's your estimate of how long it will be before D is a stable language? I have no idea, I am not that expert, and probably no one knows. D is developed by a small group of devs, and despite in life you seem to "need" to be fast everything you do (for example because otherwise others do it before you and you miss the train), D is not a commercial project, so it has less timing pressures. Often commercial software projects produce a worse result, despite having more resources, because of the strict development timing requirements. If you look at the PyPy Python project it's may years out of schedule, it was started by lot of money coming from European Union, but in the end it may deliver something good any way. Open Source projects too need to respect some timing schedules, but in OS you are sometimes able to say "it will be done when it will be done," while in most commercial software projects you can't say that. The "feature" we are describing here probably needs only few lines of code to be implemented in the compiler (but I can't be sure) and it's backward-compatible (because it removes the need of a cast in some cases), so it's a kind of additive change. Bye, bearophile
Re: Shared pain
Steve Teale Wrote: > On Fri, 19 Nov 2010 08:25:17 -0500, Jason House wrote: > > > A cast to immutable is required when constructing immutable objects. The > > type system can't prove that's safe. It's a design limitation to keep > > complexity low. > > > Jason, > > But design limitations like that will force programmers who are on a > tight schedule to just say __gshared, and to hell with it. > > Languages can't be designed just on theory - some recognition of > practicality is also required. I didn't say I liked the need for the cast or even that I think D has the proper design! IIRC, the cast with object creation was in lieu of adding "unique" tepe qualifier and the extra complexity to get that right. In your example, I believe the object was created once, so a cast really isn't that bad. The bad part to me was the need to cast away immutability with every use. That's why I encouraged a bugzilla entry. I did't check this case, but Phobos had a bad track record with adopting type qualifiers after they were added to D2.
Re: Shared pain
On Fri, 19 Nov 2010 16:21:41 -0500, Jason House wrote: > Steve Teale Wrote: > >> On Fri, 19 Nov 2010 08:25:17 -0500, Jason House wrote: >> >> > A cast to immutable is required when constructing immutable objects. >> > The type system can't prove that's safe. It's a design limitation to >> > keep complexity low. >> > >> Jason, >> >> But design limitations like that will force programmers who are on a >> tight schedule to just say __gshared, and to hell with it. >> >> Languages can't be designed just on theory - some recognition of >> practicality is also required. > > I didn't say I liked the need for the cast or even that I think D has > the proper design! IIRC, the cast with object creation was in lieu of > adding "unique" tepe qualifier and the extra complexity to get that > right. In your example, I believe the object was created once, so a cast > really isn't that bad. The bad part to me was the need to cast away > immutability with every use. That's why I encouraged a bugzilla entry. I > did't check this case, but Phobos had a bad track record with adopting > type qualifiers after they were added to D2. I'm a bit calmer now I have got most of my utility modules working again. There were two circumstances where I found the casts particularly irksome. I had at one point an immutable char* that I would be passing to Win32 functions. I was assigning to it the address of a plain static character array, and I had to use a cast to do it. I could not see what harm such an assignment could do. Then I had a slew of cases where I had created immutable RegExp objects, and I had to do (cast(RegExp) rexobj).find(...). I could not see what the cast was protecting here either. I think these are bugs, if you agree I will put them in bugzilla.
Phobos and shared
Greetings
I am trying to create a multithreaded application. Right now I am finding it
difficult to work with "shared" qualifier. One of the reasons is that Phobos
library does not seem compatible with "shared" data-structures. For example:
import std.bitmanip;
shared BitArray foo;
void main() {
foo ~= true; // this does not work
(cast(BitArray)foo) ~= true; // Even casting does not help
}
I know that "shared" is a relatively new qualifier in D2. Are there plans to
make Phobos "shared" compatible?
Are there any workarounds that I am missing.
Regards
- Puneet
Re: Migrating to Shared
Walter Bright wrote: I wrote a brief article which should help: http://www.digitalmars.com/d/2.0/migrate-to-shared.html Sir, So immutable is an alias of invariant? D now has 3 keywords that have somewhat similar meaning: const invariant immutable. Thx.
Re: Migrating to Shared
The Anh Tran wrote: So immutable is an alias of invariant? D now has 3 keywords that have somewhat similar meaning: const invariant immutable. By popular demand, the invariant keyword was replaced with immutable. Invariant will eventually go away.
Re: Migrating to Shared
Walter Bright, el 12 de mayo a las 10:59 me escribiste: > The Anh Tran wrote: > >So immutable is an alias of invariant? > >D now has 3 keywords that have somewhat similar meaning: const invariant > >immutable. > > By popular demand, the invariant keyword was replaced with immutable. > Invariant will eventually go away. Better sooner than later. At least it would be nice if a deprecation warning is issued if invariant is used when immutable should go. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05)
Re: Migrating to Shared
Leandro Lucarella wrote: At least it would be nice if a deprecation warning is issued if invariant is used when immutable should go. That'll happen.
Re: Migrating to Shared
On Tue, May 12, 2009 at 1:47 PM, Walter Bright wrote: > I wrote a brief article which should help: > > http://www.digitalmars.com/d/2.0/migrate-to-shared.html > A couple typos: // should be immutable int[3], no? immutable table[3] x = [6, 123, 0x87]; // no type __gshared x;
Re: Migrating to Shared
Jarrett Billingsley wrote: A couple typos: Fixed.
Re: Migrating to Shared
Walter Bright Wrote: > I wrote a brief article which should help: > > http://www.digitalmars.com/d/2.0/migrate-to-shared.html I like this development, the only thing that makes me wary is the implied TLS behavior. I think I'd personally prefer it if tls was a keyword as well, and all globals had to have either tls or shared (or __gshared) in their type, rather than defaulting to tls if none is specified. So we'd force the programmer to explicitly define the level of sharing. Some might view that as too verbose though?
Re: Migrating to Shared
Hello Brian, I like this development, the only thing that makes me wary is the implied TLS behavior. I think I'd personally prefer it if tls was a keyword as well, and all globals had to have either tls or shared (or __gshared) in their type, rather than defaulting to tls if none is specified. So we'd force the programmer to explicitly define the level of sharing. I'm fine with the implicit but I'd like to see the explicit form as well (so they wouldn't pop up from -vtls).
Re: Migrating to Shared
BCS wrote: I'm fine with the implicit but I'd like to see the explicit form as well (so they wouldn't pop up from -vtls). You can do it explicitly with ___thread, though I was planning on deprecating that.
Re: Migrating to Shared
Walter Bright, el 12 de mayo a las 11:19 me escribiste: > Leandro Lucarella wrote: > >At least it would be nice if a deprecation > >warning is issued if invariant is used when immutable should go. > > That'll happen. Great! =) -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05)
Re: Migrating to Shared
Hello Walter, BCS wrote: I'm fine with the implicit but I'd like to see the explicit form as well (so they wouldn't pop up from -vtls). You can do it explicitly with ___thread, though I was planning on deprecating that. not soon I hope.
Re: Migrating to Shared
On Tue, 12 May 2009 10:47:02 -0700, Walter Bright wrote: > I wrote a brief article which should help: > > http://www.digitalmars.com/d/2.0/migrate-to-shared.html Good article. This is an exciting enhancment to an already exciting programming language. -- Derek Parnell Melbourne, Australia skype: derek.j.parnell
Re: Migrating to Shared
Jarrett Billingsley wrote: // should be immutable int[3], no? immutable table[3] x = [6, 123, 0x87]; Really shouldn't that be enum, unless you mysteriously needed to take the address of it?
Re: Migrating to Shared
And shared(int) is neither subtype nor supertype of int?
Re: Migrating to Shared
Robert Fraser wrote: Jarrett Billingsley wrote: // should be immutable int[3], no? immutable table[3] x = [6, 123, 0x87]; Really shouldn't that be enum, unless you mysteriously needed to take the address of it? Seeing as how the example is for immutable, I'd say enum wouldn't be quite right. -- Simen
Re: Migrating to Shared
Kagamin wrote: And shared(int) is neither subtype nor supertype of int? It's like const(int) and immutable(int).
Re: Semantics of shared
On Wed, 13 May 2009 23:44:32 -0400, Matt wrote: [from reddit] There was just a post to reddit announcing that thread local storage would be the default for global variables and that the 'shared' qualifier would make this happen. What I can't find is a description of typing rules surrounding 'shared'. From the discussion at reddit, it sounded like 'shared' was intended to mean 'possibly shared', with the implication that thread local objects can be treated as 'possibly shared'. The problem I see with this is that it implies that it is not safe to assign one shared reference to another, because the former may actually be thread local while the latter is actually global. This would seem to make the "maybe shared" concept pretty useless. Is this not a problem? Or if not, can someone clarify to me what the actual semantics & typing rules are? Thanks, Matt I'm posting Walter's reply from reddit: WalterBright 4 points 6 hours ago[-] You're right about the return types of accessors, though we plan to address this. But if I may make some corrections, C++ has four versions (none, const, volatile, and const volatile), while D has five (none, const, immutable, shared, and shared const). The shared immutable is not counted since is the same as immutable. I don't see a place for "maybe shared" that isn't already handled by simply "shared".
Re: Semantics of shared
Matt wrote: There was just a post to reddit announcing that thread local storage would be the default for global variables and that the 'shared' qualifier would make this happen. What I can't find is a description of typing rules surrounding 'shared'. From the discussion at reddit, it sounded like 'shared' was intended to mean 'possibly shared', with the implication that thread local objects can be treated as 'possibly shared'. The problem I see with this is that it implies that it is not safe to assign one shared reference to another, because the former may actually be thread local while the latter is actually global. This would seem to make the "maybe shared" concept pretty useless. Is this not a problem? Or if not, can someone clarify to me what the actual semantics & typing rules are? The addresses of thread local data cannot be implicitly cast to shared. If they could, then they could be accessed by other threads, and the whole safety of them being thread local is compromised. Shared data must be created as shared, or cast to be shared. Casting to shared is an implicitly unsafe operation, relying on the user ensuring that it is safe to do so. Thread local data can point to shared data, but cannot be implicitly cast to shared.
Re: Semantics of shared
Robert Jacques wrote: I don't see a place for "maybe shared" that isn't already handled by simply "shared". I gave a flip and incomplete answer there. I'm not sure there is even a point to a function that could handle both shared and unshared with the same code. First of all, sharing is going to need some sort of synchronization; you're going to try and minimize the amount of code that has to deal with shared. I can't see trying to run an in-place sort on a shared array, for example. Can you imagine two threads trying to sort the same array? You're going to want to approach manipulating shared data differently than unshared.
Re: Semantics of shared
On Thu, 14 May 2009 01:27:15 -0400, Walter Bright wrote: Robert Jacques wrote: I don't see a place for "maybe shared" that isn't already handled by simply "shared". I gave a flip and incomplete answer there. I'm not sure there is even a point to a function that could handle both shared and unshared with the same code. First of all, sharing is going to need some sort of synchronization; you're going to try and minimize the amount of code that has to deal with shared. I can't see trying to run an in-place sort on a shared array, for example. Can you imagine two threads trying to sort the same array? You're going to want to approach manipulating shared data differently than unshared. I agree for POD, but what classes where the synchronization is encapsulated behind a virtual function call? Also, does this mean 'scope' as a type is going away?
Re: Semantics of shared
Robert Jacques wrote: I agree for POD, but what classes where the synchronization is encapsulated behind a virtual function call? synchronization can make a shared reference "tail shared". Also, does this mean 'scope' as a type is going away? Scope never was a type, it's a storage class.
Re: Semantics of shared
Walter Bright Wrote: > Robert Jacques wrote: > > I don't see a place for "maybe shared" that isn't already handled by > > simply "shared". > > I gave a flip and incomplete answer there. > > I'm not sure there is even a point to a function that could handle both > shared and unshared with the same code. [...] > You're going to want to approach manipulating shared data differently > than unshared. Ok, so there is no cast from 'shared' to 'not shared' or vice-versa, so it's sound. Sorry, quotes like the above from Robert confused me. But now I'm confused by the idea that you wouldn't want to use the same code on shared and unshared data. The usual approach in C or C++ in dealing with shared data is to first acquire a lock and then to run code that would have been otherwise safe on the data. Is there some way to cast shared to thread local when a local has been acquired? > Can you imagine two threads trying to sort the same array? Not at the same time, but yes.
Re: Semantics of shared
Matt Wrote: > Is there some way to cast shared to thread local when a local has been > acquired? It occurs to me that your plan is probably: with an explicit cast when the lock is acquired. So in practice 'unshared' is really going to mean something more like 'exclusively owned' and these modifiers aren't really going to help with managing what data is actually thread local vs. in the global heap. The purpose of the 'shared' annotation is then just to warn you about unlocked data and to serve a similar purpose as 'volatile' for code generation. Is this about right?
Re: Semantics of shared
On Thu, 14 May 2009 02:13:37 -0400, Walter Bright wrote: Robert Jacques wrote: I agree for POD, but what classes where the synchronization is encapsulated behind a virtual function call? synchronization can make a shared reference "tail shared". I agree, but that doesn't seem answer my question. Put another way, if I have an interface I which is implemented by both a thread local class L and a shared class S, then does some function F need to know about whether the implementor of I is S or L? P.S. There will obviously be some interfaces S can't implement, but that a separate issue. Also, does this mean 'scope' as a type is going away? Scope never was a type, it's a storage class. Sorry for the confusion of terminology. However, you talk blog about using the 'scope' keyword to support escape analysis, ettc. i.e. 'scope' would become the 'const' of the shared-thread local-stack storage type system. Is this still the plan?
Re: Semantics of shared
Robert Jacques Wrote: > On Thu, 14 May 2009 02:13:37 -0400, Walter Bright > wrote: > > > Robert Jacques wrote: > >> I agree for POD, but what classes where the synchronization is > >> encapsulated behind a virtual function call? > > > > synchronization can make a shared reference "tail shared". > > I agree, but that doesn't seem answer my question. Put another way, if I > have an interface I which is implemented by both a thread local class L > and a shared class S, then does some function F need to know about whether > the implementor of I is S or L? Shared data needs fundamentally different handling than thread local data. I expect "shared I" and "__thread I" to be handled differently. You can't store an S where an L is expected... It can break code. > P.S. There will obviously be some interfaces S can't implement, but that a > separate issue. > > >> Also, does this mean 'scope' as a type is going away? Of course not. Scope storage class will remain. > > Scope never was a type, it's a storage class. > > Sorry for the confusion of terminology. However, you talk blog about using > the 'scope' keyword to support escape analysis, ettc. i.e. 'scope' would > become the 'const' of the shared-thread local-stack storage type system. > Is this still the plan?
Re: Migrating to Shared
On Tue, 12 May 2009 13:47:02 -0400, Walter Bright wrote: I wrote a brief article which should help: http://www.digitalmars.com/d/2.0/migrate-to-shared.html Regarding making globals immutable, one other option that was not identified was making it an enum. Most of the time, I would think you'd want manifest constants versus immutable globals, as they take up no static space. (not any more comments yet, still reading...) -Steve
Re: Semantics of shared
Matt Wrote: > Matt Wrote: > > > Is there some way to cast shared to thread local when a local has been > > acquired? > > It occurs to me that your plan is probably: with an explicit cast when the > lock is acquired. So in practice 'unshared' is really going to mean > something more like 'exclusively owned' and these modifiers aren't really > going to help with managing what data is actually thread local vs. in the > global heap. The purpose of the 'shared' annotation is then just to warn you > about unlocked data and to serve a similar purpose as 'volatile' for code > generation. > > Is this about right? I don't think so. What happens when the lock is released? Any residual use of the cast data is going to be incorrect (lacks a lock). What that means is that the cast is unsafe even when the data is locked! Casting is a back door that should be used with extreme care.
Re: Migrating to Shared
On Thu, 14 May 2009 08:54:21 -0400, Steven Schveighoffer wrote: On Tue, 12 May 2009 13:47:02 -0400, Walter Bright wrote: I wrote a brief article which should help: http://www.digitalmars.com/d/2.0/migrate-to-shared.html Regarding making globals immutable, one other option that was not identified was making it an enum. Most of the time, I would think you'd want manifest constants versus immutable globals, as they take up no static space. (not any more comments yet, still reading...) OK, I think there's another thread talking about this, but I was looking for the same thing... There doesn't seem to be a way to convert shared data into a form (call it "locked-shared") that then can be used in a function as if it were local data. For example, given that shared is a type constructor, I have to mark methods on a class shared in order to call them on a shared class instance. But if there was a way to lock the instance so it temporarly became local, then I could call any method (as long as it didn't squirrel away a pointer to this). Are there plans to be able to do something like this? It couldn't be simply marked local, because you don't want to be able to squirrel away a pointer to it. It would have to be marked something else... Then comes the notion of deep locking: if you lock simply the this pointer, then it becomes tail-shared as you mentioned -- all the references the object contains are still shared. You'd almost have to recursively lock everything, and then what if something has a cycle... I'm thinking this shared thing is going to be only good for small POD, as it's going to be almost untennable to deal with repeating all implementation just because something is shared or not. And even then, you probably want to copy the POD to non-shared form so you can pass it around. I don't anticipate anyone making much more than a shared int. -Steve
Re: Semantics of shared
On Thu, 14 May 2009 08:51:37 -0400, Jason House wrote: Robert Jacques Wrote: On Thu, 14 May 2009 02:13:37 -0400, Walter Bright wrote: > Robert Jacques wrote: >> I agree for POD, but what classes where the synchronization is >> encapsulated behind a virtual function call? > > synchronization can make a shared reference "tail shared". I agree, but that doesn't seem answer my question. Put another way, if I have an interface I which is implemented by both a thread local class L and a shared class S, then does some function F need to know about whether the implementor of I is S or L? Shared data needs fundamentally different handling than thread local data. I expect "shared I" and "__thread I" to be handled differently. You can't store an S where an L is expected... It can break code. P.S. There will obviously be some interfaces S can't implement, but that a separate issue. >> Also, does this mean 'scope' as a type is going away? Of course not. Scope storage class will remain. The use of scope I'm talking about (see below) isn't even implemented yet, so how can it remain? It was Walter bogged a while ago about using the scope keyword to aid escape analysis, which would provide a common type for shared-local-stack allocation. I'm not referring to the use of 'scope' to stack allocate a class. > Scope never was a type, it's a storage class. Sorry for the confusion of terminology. However, you talk blog about using the 'scope' keyword to support escape analysis, ettc. i.e. 'scope' would become the 'const' of the shared-thread local-stack storage type system. Is this still the plan?
Re: Migrating to Shared
Steven Schveighoffer wrote: I'm thinking this shared thing is going to be only good for small POD, as it's going to be almost untennable to deal with repeating all implementation just because something is shared or not. And even then, you probably want to copy the POD to non-shared form so you can pass it around. I don't anticipate anyone making much more than a shared int. Personally, I anticipate using shared for large global containers almost exclusively. I've never considered the "interconnected web of shared objects" idea to be a good one, and building shared into the language doesn't change that. For containers, a range returned by a shared container will probably have to hold the lock on that container until it is destroyed or shared won't be terribly useful. I haven't thought about this very thoroughly, but that's my initial take on the idea.
Re: Semantics of shared
Jason House Wrote: > Matt Wrote: > > Is this about right? > > I don't think so. What happens when the lock is released? Any residual use of > the cast data is going to be incorrect (lacks a lock). What that means is > that the cast is unsafe even when the data is locked! Casting is a back door > that should be used with extreme care. You're right - it's a big departure from C++ where casting is to be avoided if possible, and you're right that the cast isn't safe in general. It's semi-reasonable because it's safe unless you cast somewhere and the C++ multi-threaded programming model is unsafe anyway. But it's bad that the bugs introduced can be quite far from the casts (e.g. if cast-to-unshared data is placed in a global thread-local somewhere). But I don't see how you can do much of anything useful with shared data if something like this isn't the plan. Tracking what data is protected by lock and when is going to be outside of the scope of any type system you're going to want for D (and is going to be undecidable in general).
Re: Semantics of shared
Matt Wrote: > Jason House Wrote: > > > Matt Wrote: > > > Is this about right? > > > > I don't think so. What happens when the lock is released? Any residual use > > of the cast data is going to be incorrect (lacks a lock). What that means > > is that the cast is unsafe even when the data is locked! Casting is a back > > door that should be used with extreme care. > > You're right - it's a big departure from C++ where casting is to be avoided > if possible, and you're right that the cast isn't safe in general. It's > semi-reasonable because it's safe unless you cast somewhere and the C++ > multi-threaded programming model is unsafe anyway. But it's bad that the > bugs introduced can be quite far from the casts (e.g. if cast-to-unshared > data is placed in a global thread-local somewhere). > > But I don't see how you can do much of anything useful with shared data if > something like this isn't the plan. Tracking what data is protected by lock > and when is going to be outside of the scope of any type system you're going > to want for D (and is going to be undecidable in general). > We're all still waiting to hear how D will handle locking under the covers. I don't think locking will be used to change types. A shared variable will always be shares, even if you have the lock.
Re: Migrating to Shared
Walter Bright Wrote: > I wrote a brief article which should help: > > http://www.digitalmars.com/d/2.0/migrate-to-shared.html __gshared, __etc. These look ugly IMHO. :)
Re: Migrating to Shared
Hello bobef, Walter Bright Wrote: I wrote a brief article which should help: http://www.digitalmars.com/d/2.0/migrate-to-shared.html __gshared, __etc. These look ugly IMHO. :) I think that's the point: they look ugly because they are ugly.
Re: Semantics of shared
Robert Jacques wrote: On Thu, 14 May 2009 02:13:37 -0400, Walter Bright wrote: Robert Jacques wrote: I agree for POD, but what classes where the synchronization is encapsulated behind a virtual function call? synchronization can make a shared reference "tail shared". I agree, but that doesn't seem answer my question. Put another way, if I have an interface I which is implemented by both a thread local class L and a shared class S, then does some function F need to know about whether the implementor of I is S or L? Since a reference to thread local cannot be implicitly cast to shared, then this scenario cannot happen - i.e. a shared function is not covariant with an unshared one. P.S. There will obviously be some interfaces S can't implement, but that a separate issue. Also, does this mean 'scope' as a type is going away? Scope never was a type, it's a storage class. Sorry for the confusion of terminology. However, you talk blog about using the 'scope' keyword to support escape analysis, ettc. i.e. 'scope' would become the 'const' of the shared-thread local-stack storage type system. Is this still the plan? I'm not sure what you mean by that.
Re: Semantics of shared
Matt wrote: But now I'm confused by the idea that you wouldn't want to use the same code on shared and unshared data. The usual approach in C or C++ in dealing with shared data is to first acquire a lock and then to run code that would have been otherwise safe on the data. Is there some way to cast shared to thread local when a local has been acquired? Shared data becomes unshared for the duration of a lock on it. The problem with this is: 1. determining that there are no other shared references into that data. 2. determining that the code operating on that data doesn't squirrel away a thread local reference to it. Currently, Bartosz is working on these problems. There is no solution yet other than using an (unsafe) cast and relying on the user not to screw it up. Can you imagine two threads trying to sort the same array? Not at the same time, but yes. That's why there's no way one would do this with simply shared data. Locks would be needed, too.
Re: Semantics of shared
Matt wrote: You're right - it's a big departure from C++ where casting is to be avoided if possible, and you're right that the cast isn't safe in general. It's semi-reasonable because it's safe unless you cast somewhere and the C++ multi-threaded programming model is unsafe anyway. But it's bad that the bugs introduced can be quite far from the casts (e.g. if cast-to-unshared data is placed in a global thread-local somewhere). It's not all bad. The huge advantage with "shared" is when you do a code review, it points you to where the potential trouble spots are. With the implicit sharing in C/C++, the whole program is a trouble spot.
Re: Semantics of shared
On Thu, 14 May 2009 13:12:46 -0400, Walter Bright wrote: Robert Jacques wrote: On Thu, 14 May 2009 02:13:37 -0400, Walter Bright wrote: Robert Jacques wrote: I agree for POD, but what classes where the synchronization is encapsulated behind a virtual function call? synchronization can make a shared reference "tail shared". I agree, but that doesn't seem answer my question. Put another way, if I have an interface I which is implemented by both a thread local class L and a shared class S, then does some function F need to know about whether the implementor of I is S or L? Since a reference to thread local cannot be implicitly cast to shared, then this scenario cannot happen - i.e. a shared function is not covariant with an unshared one. P.S. There will obviously be some interfaces S can't implement, but that a separate issue. Also, does this mean 'scope' as a type is going away? Scope never was a type, it's a storage class. Sorry for the confusion of terminology. However, you talk blog about using the 'scope' keyword to support escape analysis, ettc. i.e. 'scope' would become the 'const' of the shared-thread local-stack storage type system. Is this still the plan? I'm not sure what you mean by that. I'm asking about the use of scope you blogged about: http://dobbscodetalk.com/index.php?option=com_myblog&show=Escape-Analysis.html&Itemid=29 [...] For D, we are looking at a design that creates a parameter storage class called scope: T foo(scope int* p); The presence of scope means that the function will not allow the parameter, or anything reachable through that parameter, to escape from the scope of the function. The scope storage class can be applied to the parameters or the 'this' reference (for member functions). Initially, this will be a promise by the implementor of foo(), but it should be entirely possible for the compiler to perform escape analysis using data flow analysis techniques on the implementation of foo() to ensure it. The caller of the function will know that a reference to a local variable can be safely passed as a scope parameter. A million line program can be automatically verified as being free of escaping reference bugs.
Re: Semantics of shared
Robert Jacques wrote: Sorry for the confusion of terminology. However, you talk blog about using the 'scope' keyword to support escape analysis, ettc. i.e. 'scope' would become the 'const' of the shared-thread local-stack storage type system. Is this still the plan? I'm not sure what you mean by that. I'm asking about the use of scope you blogged about: http://dobbscodetalk.com/index.php?option=com_myblog&show=Escape-Analysis.html&Itemid=29 [...] For D, we are looking at a design that creates a parameter storage class called scope: T foo(scope int* p); The presence of scope means that the function will not allow the parameter, or anything reachable through that parameter, to escape from the scope of the function. The scope storage class can be applied to the parameters or the 'this' reference (for member functions). Initially, this will be a promise by the implementor of foo(), but it should be entirely possible for the compiler to perform escape analysis using data flow analysis techniques on the implementation of foo() to ensure it. The caller of the function will know that a reference to a local variable can be safely passed as a scope parameter. A million line program can be automatically verified as being free of escaping reference bugs. We've talked about it, but am unsure as to whether it will work or not.
Re: Migrating to Shared
bobef wrote: __gshared, __etc. These look ugly IMHO. :) They're only allowed in safe mode, and are meant to inspire people to use them only when there's no other choice. Hence the unattractiveness
Re: Semantics of shared
Walter Bright Wrote: > Matt wrote: > > Shared data becomes unshared for the duration of a lock on it. Is that a statement of fact? Or is it just speculation leading to the issues below? Even if this was changed to "scope unshared", that still is really hairy since scope is a storage class. > The problem with this is: > > 1. determining that there are no other shared references into that data. > > 2. determining that the code operating on that data doesn't squirrel > away a thread local reference to it. > > Currently, Bartosz is working on these problems. There is no solution > yet other than using an (unsafe) cast and relying on the user not to > screw it up. His last blog implied he was further. I thought the recent shift to use TLS and shared inside dmd was because a design had been worked out. Without that, D hardly helps writing correct multi-threaded code :( It may be that I'm being overly pessimistic... > >> Can you imagine two threads trying to sort the same array? > > Not at the same time, but yes. > > That's why there's no way one would do this with simply shared data. > Locks would be needed, too. >
Re: Semantics of shared
Jason House wrote: His last blog implied he was further. I thought the recent shift to use TLS and shared inside dmd was because a design had been worked out. Without that, D hardly helps writing correct multi-threaded code :( It may be that I'm being overly pessimistic... Even if D goes *no further* with shared than where it is now, it still helps a *lot* in writing correct multi-threaded code. It does this by making the points where threads communicate obvious, instead of inadvertent and hidden.
Re: Migrating to Shared
BCS, el 14 de mayo a las 16:48 me escribiste: > Hello bobef, > > >Walter Bright Wrote: > >>I wrote a brief article which should help: > >>http://www.digitalmars.com/d/2.0/migrate-to-shared.html > >__gshared, __etc. These look ugly IMHO. :) > > I think that's the point: they look ugly because they are ugly. I prefer normal keywords for standard things (keywords or symbols starting with __ are historically reserved for compiler specific features or even private stuff in Python). For __gshared I agree that it's something ugly and it's fine if it looks ugly. But __traits?! Come on! __traits is beutiful, it deserves a beutiful keyword ;) -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05)
Re: Migrating to Shared
== Quote from Leandro Lucarella (llu...@gmail.com)'s article > BCS, el 14 de mayo a las 16:48 me escribiste: > > Hello bobef, > > > > >Walter Bright Wrote: > > >>I wrote a brief article which should help: > > >>http://www.digitalmars.com/d/2.0/migrate-to-shared.html > > >__gshared, __etc. These look ugly IMHO. :) > > > > I think that's the point: they look ugly because they are ugly. > I prefer normal keywords for standard things (keywords or symbols starting > with __ are historically reserved for compiler specific features or even > private stuff in Python). For __gshared I agree that it's something ugly > and it's fine if it looks ugly. > But __traits?! Come on! __traits is beutiful, it deserves a beutiful > keyword ;) True. I think I subconsciously avoid anything with an __ in front of it because to me the connotation of an __ is "Only works if the stars align just right. We're trying to scare you away from this feature for a reason. Here be monsters." If traits, excuse me, __traits, is stable enough not to deserve this connotation, then it absolutely deserves its own keyword.
Re: Migrating to Shared
Reply to dsimcha, True. I think I subconsciously avoid anything with an __ in front of it because to me the connotation of an __ is "Only works if the stars align just right. We're trying to scare you away from this feature for a reason. Here be monsters." like __gshared? If traits, excuse me, __traits, is stable enough not to deserve this connotation, then it absolutely deserves its own keyword. I think traits should get first class treatment as it really isn't being used as the backdoor hack that the __ implies.
Re: Migrating to Shared
== Quote from BCS (a...@pathlink.com)'s article > Reply to dsimcha, > > True. I think I subconsciously avoid anything with an __ in front of > > it because to me the connotation of an __ is "Only works if the stars > > align just right. We're trying to scare you away from this feature for > > a reason. Here be monsters." > like __gshared? Exactly, except __gshared deserves this treatment.
Re: Migrating to Shared
Leandro Lucarella wrote: > I prefer normal keywords for standard things (keywords or symbols starting > with __ are historically reserved for compiler specific features or even > private stuff in Python). For __gshared I agree that it's something ugly > and it's fine if it looks ugly. > > But __traits?! Come on! __traits is beutiful, it deserves a beutiful > keyword ;) I like the idea of using __ for temporary, transitional, and experimental keywords. Using __ has several advantages here: - It's unlikely to collide with existing identifiers. - You can create as many of them as you want without worrying about namespace pollution. - It's ugly as a reminder that the feature is temporary, transitional, or experimental. - It's easy to search for and easy to globally replace with a real keyword. Eventually all features using __ keywords should be removed, renamed, or modified in such as way that they do not require a keyword at all. -- Rainer Deyke - rain...@eldwood.com
Re: Migrating to Shared
Walter Bright wrote: bobef wrote: __gshared, __etc. These look ugly IMHO. :) They're only allowed in safe mode, and are meant to inspire people to use them only when there's no other choice. Hence the unattractiveness Agreed. Yet as others have said, __traits deserves a beautiful keyword.
Re: shared class constructor
On Thu, 13 Aug 2009 09:34:50 -0700, Jeremie Pelletier
wrote:
Just came across this while porting my code to enforce the use of shared.
---
module test;
shared class Foo {
this() {}
}
---
Error: cannot implicitly convert expression (this) of type shared(Foo)
to test.Foo
Of course I could just use new shared(Foo) but that would allow
non-shared instances of Foo to be created as well, which I do not want.
Shouldn't "this" resolve to the fully qualified class?
In general, shared classes right now are broken. For example, you can't
actually write any shared methods yet, (bug 3089) amoung other things.
This does look like something new, and should be added.
As for your question: given that immutable classes work the same way, I
assume it should.
Re: Various shared bugs
On Mon, 07 Dec 2009 22:45:17 -0500, Jason House wrote: > So, after months of avoiding shared, I decided to see if I could remove > all my casting away of shared. It looks like things are still pretty > buggy (or at least not particularly easy to use). > > "is(T : shared)" gives a parse error > "alias shared T U" silently does the wrong thing. >(Use "alias shared(T) U" instead) > " is not callable using argument types ()" is just awful. > Typically, it means you're calling a non-shared function with a shared > type or a shared function with a non-shared type. I hit into a case > where I got that message with a shared type and a shared method, but I > have mad no attempt to reproduce. > The error "Can not implicitly convert expression of type(this) of type > shared(xxx) to full.name.xxx" can pop up when defining shared this() > constructor gives no line number where the offending usage occurs. > > I've backed out most of my pro-shared changes and will try again in a > few months :( I have also given up on shared and am also adopting a waiting strategy. I would love to get some tips from anyone (like Walter, for example) who thinks they have a way of using shared successfully. My recent post on the subject got no meaningful responses - just one from bearophile that was supportive, but alas not helpful.
