On Fri, 14 May 2010 20:00:01 +0100, Daniel Ruoso - dan...@ruoso.com
<+nntp+browseruk+d52dbf78bb.daniel#ruoso....@spamgourmet.com> wrote:
Em Sex, 2010-05-14 às 18:13 +0100, nigelsande...@btconnect.com escreveu:
The point I(we)'ve been trying to make is that once you have a reentrant
interpreter, and the ability to spawn one in an OS thread,
all the other bits can be built on top. But unless you have that
ability,
whilst the others can be constructed, the cannot make
use of SMP. Eg. They cannot scale.
Okay, this is an important point... Having a reentrant interpreter is a
given already, The form I tried to implement that in SMOP was by using
CPS.
The idea of using green threads is just to enforce serialized access to
some data in order to avoid locking (locking with the amount of
polymorphism required by Perl 6 is too expensive), and then the regular
threads wouldn't share data between them.
Of course we could try to have the data structures thread-safe, I didn't
try to go that path since I was trying to get Perl 5 interoperability
and therefore used refcount garbage collector.
The other possibility would be to use processor affinity to force
different threads to spawn in the same processor and that way ensure
serialized access to the non-thread-safe data.
daniel
Sorry, but restricting yourself to one processor in order to avoid locking
makes no sense at all.
It's like making one of your 4x400m relay runners run the whole race, to
save carrying and passing the baton.
There are essentially four cases to consider:
1) IO-bound coroutines running on a single CPU machine:
Locking will have very little effect upon coroutines bound by IO.
2) CPU-bound coroutines running on a single CPU machine:
Locking would have some effect on throughput.
But why would you parallelise CPU-bound coroutines on a single core.
Even without locking, the context switching would have a negative affect
upon throughput. You'd be better to just run the routines serially.
3) IO-bound coroutines running on a multi-core machine:
Again, locking has almost no effect due to IO latency.
4) CPU-bound coroutines running on a multi-core machine:
There is no way that the absence of locking is going to compensate for
only using 1/2 of the available processor power on a 2-core or
hyper-threading processor; much less only 1/4 of the current crop quad-CPU
commodity processors.
Also, basing the Perl 6 concurrency model upon what is convenient for the
implementation of SMOP:
http://www.perlfoundation.org/perl6/index.cgi?smop
as clever as it is, and as important as that has been to the evolution of
the Perl 6 development effort, is not a good idea.
Given it's dependency list, it seems unlikely that SMOP is going to become
*the* perl interpreter in the long term. Interoperability with Perl 5 and
is reference counting should not be a high priority in the decision making
process for defining the Perl 6 concurrency model.
Ultimately, I think your apprehensions about the costs of locking are
unfounded.
If your interpreter is reentrant, then you should be able to start two
copies in two threads, without any need to add internal locking as far as
lexicals are concerned, because the scoping will prevent any attempts at
concurrent access. Put simply, neither will be able to see the variables
within the other, so even though they share the same address space, the
language scoping keeps them apart.
There are 3 exceptions to this:
1) Process global entities: file & directory handles, environment
variables etc.
These are sufficiently rare that applying per entity locking is not a
problem.
Per entity (1 bit in the status word for each scalar), user space
locking (bit test & set) requires minimal development/maintenance effort
and imposes minimal runtime performance impact, in the light of the
overhead of IO anyway.
2) Explicitly shared data.
These require (internal) locks regardless of how you wrap them. Be it
with STM--which is looking less and less viable; message passing which
works but precludes (or renders grossly inefficient), many data-parallel
algorithms. I've reached the conclusion that some provision of user
specified locking must be made available for some applications. But it
doesn't mean there isn't the scope for hiding the complexities of the
underlying (POSIX) mechanisms.
3) The tough-y: Closed-over variables.
These are tough because it exposes lexicals to sharing, but they are so
natural to use, it is hard to suggest banning their use in concurrent
routines.
However, interpreters already have to detect closed over variables in
order to 'lift' them and extend their lifetimes beyond their natural
scope. It doesn't seem it would be any harder to lift them to shared
variable status, moving them out of the thread-local lexical pads and into
the same data-space as process globals and explicitly shared data.
My currently favoured mechanism for handling shared data, is via
message-passing, but passing references to the shared data, rather than
the data itself. This seems to give the reason-ability, compose-ability
and controlled access of message passing whilst retaining the efficiency
of direct, shared-state mutability. Only the code that declares the shared
data, plus any other thread it choses to send a handle to, has any
knowledge of, and therefore access to the shared state.
Effectively, allocating a shared entity returns a handle to the underlying
state, and only the holder of that handle can access it. Such handles
would be indirect references and only usable from the thread that creates
them. When a handle is passed as a message to another thread, it is
transformed into a handle usable by the recipient thread during the
transfer and the old handle becomes invalid. Attempt to use an old handle
after it has been sent result in a runtime exception.
This gives you the lock-free shared state manipulations of message passing
and the efficiency of direct shared-state access.
Anyway, this probably already longer than anyone will read, so I stop
there and hope that I've convinced a few people not to give up the
scalability of kernel threading, and efficiency of mutable shared-state
too lightly. There are ways to making them usable. They just need to be
thrashed out.
Buk
