Re: Shared an non-shared
On Wednesday, October 05, 2016 21:10:01 Begah via Digitalmars-d-learn wrote: > Although the triple buffer seems a good idea, there is one > problem. > I will need three time as much ram than what i currently need. > Not to mention, every time i switch buffer i will need to copy > all changes made to the updated buffer to the next buffer to be > updated (Which i think, doing it a few hundred times a second > might become a bottleneck ). Well, I certainly can't tell what the exact best solution is in your case, but if you can't do some kind of buffer swapping, I wouldn't think that there's any point in separating the update and render threads, because if you only have one buffer to deal with, the update thread can't actually update anything while the render thread renders it, because otherwise, the render thread would be accessing data while the update thread mutated it. And properly protecting that code with a mutex would force each thread to wait for the other would pretty much negate the benefits of putting that code in separate threads. Certainly, the way that your typical grahpics stuff paints things involves some variant of double or triple buffering. But since I obviously don't know all of the details of your problem, I can't really give better advice. In any case, hopefully you understand the shared stuff and mutex stuff well enough now to figure out what works best for your particular use case. - Jonathan M Davis
Re: Shared an non-shared
On Wednesday, 5 October 2016 at 12:48:07 UTC, Jonathan M Davis wrote: On Wednesday, October 05, 2016 11:25:57 Begah via Digitalmars-d-learn wrote: [...] Unless you're writing lock-free algorithms (which really should only be done by experts, and even then, they should probably reconsider it, since they're so insanely hard to get right), _every_ variable/object that's going to be accessible from multiple threads needs to be protected by a mutex so that it's guaranteed that only one thread accesses the object at a time. That would be just as true in C/C++ as it is in D. It's just that D requires that they be marked as shared. That being said, how many objects should be protected by a given mutex depends entirely on what you're doing. In some cases, it makes sense to protect a lot of objects with the same mutex (e.g. all of the member variables of a class could be protected with a single mutex, which is what would happen with synchronized functions/classes), and in other cases, it makes sense to have as many as a mutex per variable. Having fewer mutexes is easier to handle, but it can also mean that code gets blocked waiting more. And of course, in some cases, the state in question is really spread across multiple variables, and they all need to be protected together. I really can't judge how many mutexes would be needed without knowing what you're doing. [...] Thanks, Although the triple buffer seems a good idea, there is one problem. I will need three time as much ram than what i currently need. Not to mention, every time i switch buffer i will need to copy all changes made to the updated buffer to the next buffer to be updated (Which i think, doing it a few hundred times a second might become a bottleneck ).
Re: Shared an non-shared
On Wednesday, October 05, 2016 11:25:57 Begah via Digitalmars-d-learn wrote:
> On Wednesday, 5 October 2016 at 07:36:58 UTC, Jonathan M Davis
>
> wrote:
> > On Tuesday, October 04, 2016 19:22:10 Begah via
> >
> > Digitalmars-d-learn wrote:
> >> How can I make a method that accepts being called by both a
> >> shared and non-shared object, to prevent having to copy
> >> methods and adding a "shared"?
> >
> > You could templatize them, but really, the idea is that you
> > _don't_ call much of anything on a shared object. It's not
> > thread-safe to do so unless it's protected by a mutex or
> > sychronized block. shared is as unwieldy as it is in part
> > because it's easy to use incorrectly, and most code should not
> > be using shared at all, because the stuff that actually needs
> > to be shared across threads is normally pretty minimal.
>
> Thanks for the reply,
>
> One of my problem is that, i need all of my data to be accessible
> by both threads ( I have two ). I am making a 3d application and
> decided to separate the update loop and the render loop ( I just
> created another thread for the update loop meanwhile the render
> loop has to remain on the main thread ).
>
> I just want to ensure that when my render loop ( or update loop )
> updates/render an object, the other loop cannot ( To avoid
> potential bugs whereas one loop changes the position component
> and the render loop renders the object with only the new x
> position because the y and z variable haven't been changed yet ).
>
> As i will have many of those objects, do i need to create a mutex
> for everyone of them?
Unless you're writing lock-free algorithms (which really should only be done
by experts, and even then, they should probably reconsider it, since they're
so insanely hard to get right), _every_ variable/object that's going to be
accessible from multiple threads needs to be protected by a mutex so that
it's guaranteed that only one thread accesses the object at a time. That
would be just as true in C/C++ as it is in D. It's just that D requires that
they be marked as shared. That being said, how many objects should be
protected by a given mutex depends entirely on what you're doing. In some
cases, it makes sense to protect a lot of objects with the same mutex (e.g.
all of the member variables of a class could be protected with a single
mutex, which is what would happen with synchronized functions/classes), and
in other cases, it makes sense to have as many as a mutex per variable.
Having fewer mutexes is easier to handle, but it can also mean that code
gets blocked waiting more. And of course, in some cases, the state in
question is really spread across multiple variables, and they all need to be
protected together. I really can't judge how many mutexes would be needed
without knowing what you're doing.
That being said, if you're dealing with a rendering loop where you have one
thread updating the information, and another thread rendering, you probably
want to be using double or triple buffering.
https://en.wikipedia.org/wiki/Multiple_buffering
Basically, you have one buffer (or object or group of objects or whatever is
holding the state) which is updated by one thread, and then when it's ready,
it's swapped with another buffer. So, only the swap would need to be
protected by a mutex, because each buffer would only be referenced by a
single thread at a time. So, you could probably do something as simple as
having a pointer/reference to the buffer (or whatever object is holding the
state), and you swap that. There are a variety of ways that you could do it,
but one way would be something like
// update loop
while(cond)
{
// update data in backBuffer...
// update done
synchronized(mutex)
{
auto temp = cast(Buffer)sharedBuffer;
sharedBuffer = cast(shared Buffer)backBuffer;
backBuffer = temp;
}
}
// render loop
while(cond)
{
// render...
// wait...
synchronized(mutex)
{
auto temp = cast(Buffer)sharedBuffer;
if(temp !is frontBuffer)
{
sharedBuffer = cast(shared Buffer)frontBuffer;
frontBuffer = temp;
}
}
}
This isn't necessarily the best way to do it, but in this particular scheme,
you have 3 buffers so that the render loop always has something to render
from while the update loop can keep swapping buffers as it updates and isn't
stuck waiting for the render loop to be done with the buffer, whereas the
render loop can always just render from the same buffer as long as there
isn't an update.
This does end up locking in a somewhat different way from the basic example
I gave before, because it's just the swap that's being protected, with each
thread passing ownership of a buffer via the middle/shared buffer. However,
it does rely on making sure that all references to the buffer and everything
that it references are passed to the other thread each time so that
operating on a buffer outside of
Re: Shared an non-shared
On Wednesday, 5 October 2016 at 07:36:58 UTC, Jonathan M Davis wrote: On Tuesday, October 04, 2016 19:22:10 Begah via Digitalmars-d-learn wrote: How can I make a method that accepts being called by both a shared and non-shared object, to prevent having to copy methods and adding a "shared"? You could templatize them, but really, the idea is that you _don't_ call much of anything on a shared object. It's not thread-safe to do so unless it's protected by a mutex or sychronized block. shared is as unwieldy as it is in part because it's easy to use incorrectly, and most code should not be using shared at all, because the stuff that actually needs to be shared across threads is normally pretty minimal. Thanks for the reply, One of my problem is that, i need all of my data to be accessible by both threads ( I have two ). I am making a 3d application and decided to separate the update loop and the render loop ( I just created another thread for the update loop meanwhile the render loop has to remain on the main thread ). I just want to ensure that when my render loop ( or update loop ) updates/render an object, the other loop cannot ( To avoid potential bugs whereas one loop changes the position component and the render loop renders the object with only the new x position because the y and z variable haven't been changed yet ). As i will have many of those objects, do i need to create a mutex for everyone of them?
Re: Shared an non-shared
On Tuesday, October 04, 2016 19:22:10 Begah via Digitalmars-d-learn wrote:
> How can I make a method that accepts being called by both a
> shared and non-shared object, to prevent having to copy methods
> and adding a "shared"?
You could templatize them, but really, the idea is that you _don't_ call
much of anything on a shared object. It's not thread-safe to do so unless
it's protected by a mutex or sychronized block. shared is as unwieldy as it
is in part because it's easy to use incorrectly, and most code should not be
using shared at all, because the stuff that actually needs to be shared
across threads is normally pretty minimal. The fact that most operations are
illegal on shared objects prevents misuse. But of course, it leaves the
question of how you go about actually doing anything with a shared object,
since obviously, you're going to do need to do more than just create the
thing.
TDPL talks about synchronized classes where the outer layer of shared gets
automatically cast away within the member functions of the class (since the
compiler can guarantee that only one thread at a time is in the member
functions of the class). However, synchronized classes haven't been
implemented as of yet, and we just have synchronized functions, which do
protect those functions with a mutex, but without requiring that all
functions within the class be synchronized and make it illegal to access the
member variables except via the member functions, the compiler can't cast
away the outer layer of shared, because it can't guarantee that the member
is actually protected by a mutex.
So, the way to solve this is that when you want to operate on a shared
object, you first make sure that it's protected by a mutex (like you would
in a language like C or C++), then you cast away shared to operate on the
object, and then when you're done, you make sure that no non-shared
references to the object exist, and release the mutex. e.g. something like
shared MyClass mySharedObj = getSharedObj();
synchronized(myMutex)
{
auto obj = cast(MyClass)mySharedObj;
// do stuff with obj...
// obj should be the only non-shared reference to the object
// referred to by mySharedObj when the sychronized block exits.
}
So, ultimately, the code is basically the same as what you'd do in C/C++
except that you have to cast away shared to actually do much of anything to
the object. It _is_ unfortunately more error-prone than synchronized classes
would be, because you're forced to do a manual cast, but since we don't
actually have synchronized class, you don't have much choice, and since
synchronized classes would only strip away the outer layer of shared,
there's a halfway decent chance that you'd still need to do something like
this even if we did have synchronized classes.
- Jonathan M Davis
Re: Shared an non-shared
On 10/04/2016 09:22 PM, Begah wrote:
I seem to be missing something.
It seems that if you want to create a shared object of a structure ( or
class ), then I have to copy every functions and add "shared" to it.
This seems way more work than it should.
For example why can't this simply work :
class Image {
string name;
int contents;
this(string name, int contents) {
this.name = name;
this.contents = contents;
}
void printContents() {
writeln(name, " : ", contents);
}
void changeContents(int newContents) {
this.contents = newContents;
}
}
void main()
{
Image im = new Image("Test", 15);
im.printContents();
im.changeContents(45);
im.printContents();
shared Image imShared = new shared Image("Test2", 80);
imShared.printContents();
imShared.changeContents(6);
imShared.printContents();
}
Non-`shared` methods are not required to be somehow thread-safe at all.
So they can't be allowed to be called on `shared` objects.
The compiler can't automatically ensure thread-safety, because it can't
know what can run in parallel and what can't.
How can I make a method that accepts being called by both a shared and
non-shared object, to prevent having to copy methods and adding a "shared"?
I would have thought that you can call a shared method on an unshared
object. Apparently not. There's probably a reason for that, but I don't
see it at the moment.
I am not looking for an explanation for how to handle multi-threading (
synchronization and so on ). I am looking to use pre-coded classes and
structures ( without using __gshared ) with non-shared and shared objects.
Ensure thread-safety and cast `shared` away.
