Re: core.sync.rwmutex example
On 05/10/2014 06:28 PM, TheFlyingFiddle via Digitalmars-d-learn wrote:
On Friday, 9 May 2014 at 23:12:44 UTC, Charles Hixson via
Digitalmars-d-learn wrote:
But I'm worried about the receiving end. It needs, somehow, to ensure
that the message it receives is the appropriate message, and that
other messages don't get dropped while it's waiting for the
answer...or, perhaps worse, substituted for the expected answer. If
I can depend on msg[0] of auto msg = receiveOnly!(Tid, bool) that
will allow me to check that the message was received from the proper
source
If you are worried that other messages having the same signature will
be sent from other sources than the expected source you could make use
of message tagging. Simply wrap the boolean result in a struct with a
descriptive name.
struct SharedHashMapSetCB { bool flag; }
void set (string s, uint64_t id)
{
tbl[s] = id;
send (SharedHashMapSetCB(true));
}
//On the receiving end
auto msg = receiveOnly!SharedHashMapSetCB();
But doesn't this design lock the entire hash-table while the update
is in progress? Is there a better way?
I think a shared memory hash-map is better for your use case. Working
with message passing is preferable done asynchronously. Blocking calls
(send followed by receive) is likely to be slower then simply waiting
on a semaphore.
I think you're probably right. It should be a fairly fast lookup
anyway, and since there should be only one copy of the table, I don't
think one would get much of a speedup no matter what one did. OTOH, it
can't be directly shared. What can be shared is a handle to a singleton
synchronized class. Its requirement that it can be updated, IIUC, means
that it can't be directly shared.
OTOH... I think I may need to build a mailbox class, the design I'm not
yet sure about, that acts as an interface between the major number of
threads and each other. Access to it needs to be non-blocking
(essentially) so it had probably better ONLY act as a message
switchboard. I want it to pass messages with time and origin
(thread id) prefixed to them. OTOH, I probably only need to support a
small number of message formats, perhaps two, perhaps three. And it
would be more efficient if whenever a thread polled it, it responded
with all waiting mail for that destination. So it's only sending in
response to a received message directed at it, but it holds all messages
directed at some other thread. This allows the other threads to loop
through their processing, only occasionally pausing to check whether
they have any mail or not. I'm hoping a couple of ms will be a long
enough timeout. The mailbox could spend most of it's time sitting
blocked at receive.
This isn't the design I was thinking about earlier, but it seems better,
and it's the design I originally ended up with the last time I thought
about this problem seriously, though admittedly I never got around to
implementing it. The question in my mind is is it better to build this
on top of std.concurrency, or to go back to the thread class. The code
of std.concurrency shows pretty clearly how to build a mailbox, but the
one that it builds doesn't store the things that I want, and is a lot
more flexible than I have any use for. Still, it's being maintained as
a part of the language...but it's a private class, so I can't use it
directly. But threading is difficult and error prone, so it might be
better to write a layer on top of std.concurrency, even if it means I
need to re-implement a lot of what has already been done.
--
Charles Hixson
Re: core.sync.rwmutex example
On Friday, 9 May 2014 at 23:12:44 UTC, Charles Hixson via
Digitalmars-d-learn wrote:
But I'm worried about the receiving end. It needs, somehow, to
ensure that the message it receives is the appropriate message,
and that other messages don't get dropped while it's waiting
for the answer...or, perhaps worse, substituted for the
expected answer. If I can depend on msg[0] of auto msg =
receiveOnly!(Tid, bool) that will allow me to check that the
message was received from the proper source
If you are worried that other messages having the same signature
will be sent from other sources than the expected source you
could make use of message tagging. Simply wrap the boolean result
in a struct with a descriptive name.
struct SharedHashMapSetCB { bool flag; }
void set (string s, uint64_t id)
{
tbl[s] = id;
send (SharedHashMapSetCB(true));
}
//On the receiving end
auto msg = receiveOnly!SharedHashMapSetCB();
But doesn't this design lock the entire hash-table while the
update is in progress? Is there a better way?
I think a shared memory hash-map is better for your use case.
Working with message passing is preferable done asynchronously.
Blocking calls (send followed by receive) is likely to be slower
then simply waiting on a semaphore.
Re: core.sync.rwmutex example
Hi Charles,
would the following work (just a shot in the dark) ?
//---
module test;
import std.stdio;
import std.concurrency;
void spawnedFuncFoo(Tid tid, Tid tidBar) {
receive(
(int i) {
writeln(Foo Received the number , i);
send(tidBar, i, thisTid);
auto barSuccessful = receiveOnly!(string);
writeln(Bar got my (Foo) message);
}
);
send(tid, true);
}
void spawnedFuncBar(Tid tid) {
receive(
(int i, Tid tidFoo) {
writeln(Foo passed me (Bar) the number , i);
send(tidFoo, done);
}
);
receive(
(string sig) {
writeln(Main says I'm (Bar) done.);
send(tid, 42);
}
);
}
void main() {
auto tidBar = spawn(spawnedFuncBar, thisTid);
auto tidFoo = spawn(spawnedFuncFoo, thisTid, tidBar);
send(tidFoo, 42);
auto fooWasSuccessful = receiveOnly!(bool);
assert(fooWasSuccessful);
send(tidBar, your done);
auto barWasSuccessful = receiveOnly!(int);
assert(barWasSuccessful == 42);
writeln(Successfully had two separate threads communicate
with each other);
}
//---
Re: core.sync.rwmutex example
On 05/09/2014 02:51 PM, Joshua Niehus via Digitalmars-d-learn wrote:
Hi Charles,
would the following work (just a shot in the dark) ?
//---
module test;
import std.stdio;
import std.concurrency;
void spawnedFuncFoo(Tid tid, Tid tidBar) {
receive(
(int i) {
writeln(Foo Received the number , i);
send(tidBar, i, thisTid);
auto barSuccessful = receiveOnly!(string);
writeln(Bar got my (Foo) message);
}
);
send(tid, true);
}
void spawnedFuncBar(Tid tid) {
receive(
(int i, Tid tidFoo) {
writeln(Foo passed me (Bar) the number , i);
send(tidFoo, done);
}
);
receive(
(string sig) {
writeln(Main says I'm (Bar) done.);
send(tid, 42);
}
);
}
void main() {
auto tidBar = spawn(spawnedFuncBar, thisTid);
auto tidFoo = spawn(spawnedFuncFoo, thisTid, tidBar);
send(tidFoo, 42);
auto fooWasSuccessful = receiveOnly!(bool);
assert(fooWasSuccessful);
send(tidBar, your done);
auto barWasSuccessful = receiveOnly!(int);
assert(barWasSuccessful == 42);
writeln(Successfully had two separate threads communicate
with each other);
}
//---
Thank you very much for the response.
How do you ensure that the received message (in auto barWasSuccessful =
receiveOnly!(int); ) came from the thread sent to by tidFoo? Also, why
two receives in
spawnedFuncBar? I thought that receive handled multiple different
arguments with a single receive? Are you asserting that the arguments
happen in sequence, as it appears? That doesn't look as if it would
scale will to multiple threads. And with multiple threads I would
expect the mailbox to start getting very full with inappropriate
messages that never got handled, because they came at the wrong time
(and arguably from the wrong sender, as it seems that there is an
assumption that there is no intervening thread sending any messages).
That's why I was looking for a way to ensure that the message being
received came from the appropriate sender. You can't depend on messages
to always come from the source you expect. This probably means that you
need to have only ONE receive statement, located in a loop. And some
way to check and set the status of incoming messages. Also a yield
method that lets one receive and stack any incoming messages during a
busy cycle, though I think that that's what receive does.
OTOH, I do note that you are using auto barWasSuccessful =
receiveOnly!(int); which implies that the value returned is not a void,
even though you never use the result. This fits with the claims of
TDPL, but make me increasingly dubious about the documentation. Or do I
just not understand what a function of type void is supposed to return?
The hashtable itself will be a uint64_t[string] which is confined to one
thread, so the access methods need to be rather straightforwards. I'm
thinking of:
receive
( (string s, uint64_t id) { void set(s, id); }
(string s) { void get(s); }
(Variant v) { void reportError (v); }
);
get should send the id back to the calling thread. set updates the AA
and sends true or false (success) back to the calling thread. But
doesn't this design lock the entire hashtable while the update is in
progress? Is there a better way? set will be, approx:
void set (string s, uint64_t id)
{tbl[s]=id;
send (tid, true);
}
etc. But I'm worried about the receiving end. It needs, somehow, to
ensure that the message it receives is the appropriate message, and that
other messages don't get dropped while it's waiting for the answer...or,
perhaps worse, substituted for the expected answer. If I can depend on
msg[0] of auto msg = receiveOnly!(Tid, bool) that will allow me to
check that the message was received from the proper source Also, if a
message is received that if of an inappropriate form, an exception is
thrown. Recovery looks problematic. But this is a case that should be
expected when different threads are sending messages. And if the
message is of the correct form, but from the wrong thread, how can one
re-queue it for later processing?
So far the best idea I've got is to have a dequeue of messages that are
stacked with receiveOnly!Variant. But this would need to be accesses
in various places throughout the thread, and the handling looks messy.
OTOH, it does let me save messages that come in at the wrong time where
I can deal with them later. So it's not all bad. But it gives up on the
processing that receive makes available. And note that this processing
doesn't need to be done in the hashtable function, that's pretty clear,
but wherever it's used. Which is terrible. As it is, I might be better
putting the hashtable in s synchronized class. The entire hashtable is
getting locked on every read anyway.
