On Wed, Oct 17, 2018 at 12:35 PM Steven Schveighoffer via
Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 10/17/18 2:46 PM, Manu wrote:
> > On Wed, Oct 17, 2018 at 10:30 AM Steven Schveighoffer via
>
> >> What the example demonstrates is that while you are trying to disallow
> >> implicit casting of a shared pointer to an unshared pointer, you have
> >> inadvertently allowed it by leaving behind an unshared pointer that is
> >> the same thing.
> >
> > This doesn't make sense... you're showing a thread-local program.
> > The thread owning the unshared pointer is entitled to the unshared
> > pointer. It can make as many copies at it likes. They are all
> > thread-local.
>
> It's assumed that shared int pointer can be passed to another thread,
> right? Do I have to write a full program to demonstrate?
And that shared(int)* provides no access. No other thread with that
pointer can do anything with it.
> > There's only one owning thread, and you can't violate that without unsafe
> > casts.
>
> The what is the point of shared? Like why would you share data that
> NOBODY CAN USE?
You can call shared methods. They promise threadsafety.
That's a small subset of the program, but that's natural; only a very
small subset of the program is safe to be called from a shared
context.
In addition, traditional unsafe interactions which may involve
acquiring locks and doing casts remain exactly the same, and the exact
same design patterns must apply which assure that the object is
handled correctly.
I'm not suggesting any changes that affect that workflow.
> At SOME POINT, shared data needs to be readable and writable. Any
> correct system is going to dictate how that works. It's a good start to
> make shared data unusable unless you cast. But then to make it
> implicitly castable from unshared defeats the whole purpose.
No. No casting! This is antiquated workflow.. I'm not trying to take
it away from you, but it's not an interesting model for the future.
`shared` can model more than just that.
You can call threadsafe methods. Shared methods explicitly dictate how
the system works, and in a very clear and obvious/intuitive way.
The implicit cast makes using threadsafe objects more convenient when
you only have one, which is extremely common.
> >> In order for a datum to be
> >> safely shared, it must be accessed with synchronization or atomics by
> >> ALL parties.
> >
> > ** Absolutely **
> >
> >> If you have one party that can simply change it without
> >> those, you will get races.
> >
> > *** THIS IS NOT WHAT I'M PROPOSING ***
> >
> > I've explained it a few times now, but people aren't reading what I
> > actually write, and just assume based on what shared already does that
> > they know what I'm suggesting.
> > You need to eject all presumptions from your mind, take the rules I
> > offer as verbatim, and do thought experiments from there.
>
> What seems to be a mystery here is how one is to actually manipulate
> shared data. If it's not usable as shared data, how does one use it?
Call shared methods. It's not like I haven't been saying this in every
post since my OP.
The only possible thing that you can safely do with a shared object is
call a method that has been carefully designed for thread-safe
calling. Any other access is invalid under any circumstance.
> >> It's true that only one thread will have thread-local access. It's not
> >> valid any more than having one mutable alias to immutable data.
> >
> > And this is why the immutable analogy is invalid. It's like const.
> > shared offers restricted access (like const), not a different class of
> > thing.
>
> No, not at all. Somehow one must manipulate shared data. If shared data
> cannot be read or written, there is no reason to share it.
Call shared methods.
> So LOGICALLY, we have to assume, yes there actually IS a way to
> manipulate shared data through these very carefully constructed and
> guarded things.
Yes, call the methods, they were carefully constructed to be threadsafe.
Only functions that have made a promise to be threadsafe and
implemented that complexity are valid interactions.
> > There is one thread with thread-local access, and many threads with
> > shared access.
> >
> > If a shared (threadsafe) method can be defeated by threadlocal access,
> > then it's **not threadsafe**, and the program is invalid.
> >
> > struct NotThreadsafe
> > {
> > int x;
> > void local()
> > {
> > ++x; // <- invalidates the method below, you violate the other
> > function's `shared` promise
> > }
> > void notThreadsafe() shared
> > {
> > atomicIncrement(&x);
> > }
> > }
>
> So the above program is invalid. Is it compilable with your added
> allowance of implicit casting to shared? If it's not compilable, why
> not?
All my examples assume my implicit conversion rule. But regardless,
under my proposal, the above program is invalid. This violates the
threadsafe promise.
> If it is compilable, how in the hell does your proposal help
> anything? I get the exact behavior today without any changes (except
> today, I need to explicitly cast, which puts the onus on me).
My proposal doesn't help this program, it's invalid. I'm just
demonstrating what an invalid program looks like.
> > struct Atomic(T)
> > {
> > void opUnary(string op : "++")() shared { atomicIncrement(&val); }
> > private T val;
> > }
> > struct Threadsafe
> > {
> > Atomic!int x;
> > void local()
> > {
> > ++x;
> > }
> > void threadsafe() shared
> > {
> > ++x;
> > }
> > }
> >
> > Naturally, local() is redundant, and it's perfectly fine for a
> > thread-local to call threadsafe() via implicit conversion.
>
> In this case, yes. But that's not because of anything the compiler can
> prove.
The compiler can't 'prove' anything at all related to threadsafety. We
need to make one assumption; that `shared` methods are expected to be
threadsafe, and from there the compiler can prove correctness with
respect to that assumption.
In this case, `Atomic.opUnary("++")` promises that it's threadsafe,
and as such, the aggregate can safely use that tool to implement
higher-level logic.
> How does Atomic work? I thought shared data was not usable? I'm being
> pedantic because every time I say "well at some point you must be able
> to modify things", you explode.
Atomic implements a safe utility using unsafe primitives (atomic
increment intrinsic).
Atomic wraps the unsafe call to an intrinsic into a box that's safe,
and can be used by clients.
In my worldview, atomic is at the bottom of the chain-of-trust. It's
effectively a @trusted implementation of a foundational tool.
Almost every low level tool is of this nature.
> Complete the sentence: "In order to read or write shared data, you have
> to ..."
Call a shared method, or at the bottom of the stack, you need to do
unsafe (@trusted?) implementations of the foundational machinery
(possibly using casts), and package into boxes that are safe to
interact with and build out from.
> > Here's another one, where only a subset of the object is modeled to be
> > threadsafe (this is particularly interesting to me):
> >
> > struct Threadsafe
> > {
> > int x;
> > Atomic!int y;
> >
> > void notThreadsafe()
> > {
> > ++x;
> > ++y;
> > }
> > void threadsafe() shared
> > {
> > ++y;
> > }
> > }
> >
> > In these examples, the thread-local function *does not* undermine the
> > threadsafety of threadsafe(), it MUST NOT undermine the threadsafety
> > of threadsafe(), or else threadsafe() **IS NOT THREADSAFE**.
> > In the second example, you can see how it's possible and useful to do
> > thread-local work without invalidating the objects threadsafety
> > commitments.
> >
> >
> > I've said this a bunch of times, there are 2 rules:
> > 1. shared inhibits read and write access to members
> > 2. `shared` methods must be threadsafe
> >
> >>From there, shared becomes interesting and useful.
> >
>
> Given rule 1, how does Atomic!int actually work, if it can't read or
> write shared members?
It's an intrinsic. You use it the same as malloc() or free(). It's a
piece of low-level mechanical tooling which you use in an unsafe way,
but you then wrap it in a layer that introduces the type-safety.
malloc() returns a void*, which you cast to the intended type, and
then perform construction.
You can't implement a typesafe new without malloc() at the bottom of the stack.
You need to raise your vision one-level higher to users of Atomic to
see interesting interactions.
> For rule 2, how does the compiler actually prove this?
>
> Any programming by convention, we can do today. We can implement
> Atomic!int with the current compiler, using unsafe casts inside @trusted
> blocks.
It can't. I don't know what can be done to mechanically enforce this
requirement, but I would suggest that it's a goal to work towards in
the future with any technology possible.
In the meantime though, if we accept that the user writing a
threadsafe tool is responsible for delivering on their promise, then
the system that emerges is widely useful. The higher-level becomes
generally interesting, the low-level will remain to be implemented by
experts, and is no change from the situation today.
The low level doesn't really care much about type-safety, it's the
high-level I'm interested in. We can tell a MUCH better story about
how users can interact with shared machinery, and we should.
