> On Monday, May 26, 2003, at 06:51 PM, John Macdonald wrote:
> > This is an interesting idea. I'd add forked processes
> > to the list (which includes the magic opens that fork
> > a child process on the end of a pipeline instead of
> > opening a file.
>
> I thought about that quite a bit, but fork() is a process-level thing,
> whereas even threads are more internally controllable/implementable, so
> I thought that would be too controversial. People already *know* how
> to fork processes in Perl, whereas thread syntax is newer and more
> malleable.
>
> > There is a danger in hiding the distinction between
> > them too much. They have quite difference performance
> > overheads.
>
> Some thinking out loud: a thread is significantly more expensive than a
> coroutine. Why? Because threads must be executable in parallel, so
> you're duping quite a bit of data. (You don't dup nearly as much stuff
> for a coroutine, _unless_ of course you're cloning an already-active
> coroutine.) So a conventional thread is like cloning a
> "very-top-level" coroutine.
>
> Now, if we were to say:
>
> sub foo(...) is coroutine { ... yield() ... return() }
>
> We would expect
>
> foo(...args...);
>
> to give us back that coroutine, from the last yield point. In the
> above, C<yield> yields out of the coroutine, and C<return>
> yields-without-saving-state such that the next foo() invocation will
> start from the beginning of the routine.
>
> Similarly, then, I would expect:
>
> sub foo(...) is threaded { ... yield() ... return() }
>
> foo(...args...)
>
> to start &foo as a new thread. C<yield()> would temporarily suspend
> the thread, and C<return()> would end the thread. (Note that you could
> use &_.yield to yield the current Code object, so you can have nested
> yields w/out confusion -- see C<leave>, from A6.)
The problem with unifying Coroutines with Threads is that Coroutines
are largely a data transfer/communication method, while Threads make
use of the very opposite -- that some computations are independent and
can be done in parallel.
> These are using nearly identical syntax, but there is still a clear
> semantic difference between them -- the trait C<threaded> is sufficient
> for that.
>
> We could also have things like:
>
> sub { ... }
> closure { ... }
I think you've finally gone crazy. :-)
All four of these things are closures.
> coroutine { ... }
> thread { ... }
>
> If that's preferable syntax. As long as they're similar, and share
> similar suspend/resume capabilities.
>
> OTOH, the difference between a thread and a coroutine is mostly
> internal, not external.
Again, I beg to differ. But, these are the kinds of misunderstandings
that are keeping a good coroutine proposal from getting in....
Here's how I think of things:
coroutines - Used for easy iteration over complex data structures,
pipelines, and communication of the results of certain
algorithms.
threads - Used for user interfaces, speed (on SMP systems), very
occasionally pipelines, and headaches.
I may have missed things in the threads section, because I haven't
done very much threaded programming. To be honest, my favorite use so
far has been using them to (painstakingly) emulate coroutines :-)
But I'm pretty sure these two concepts are things that we don't want
to unify, even though they both have to do with "state". I like your
musings on "state", however, and making them more explicit might
enable us to come up with some very cool ideas.
> If we define a thread as a coroutine that runs
> in parallel, the syntax might converge:
>
> sub foo() is cothread { ... yield() ... return() }
>
> # start foo() as a coroutine, (blocks until explicitly yields):
>
> my $results = foo(...);
>
> # start foo() as a parallel thread (nonblocking):
>
> my $results_junction = parallel( &foo(...), &bar(...), &baz(...) )
>
> In this situation, C<parallel> would indicate that you couldn't
> continue on until all three threads had suspended or exited -- so in
> order to have truly "top-level" parallel threads, you'd have to set
> them up so that your entire program was encapsulated within them.
> (Otherwise you'd just get temporary parallelization, which is just as
> desirable.) (You could declare a scheduler as an optional named
> parameter of C<parallel>.)
>
> So to what extent is it OK to "hide" the complexity of a coroutine, or
> a thread, in order to have the caller side interface as clean and brief
> as possible? (A question over which I remember having a vigorous but
> unanswerable debate with someone -- specifically over C++ operator
> overloading, which suffers from the problem in spades.)
There is very little complexity to a coroutine. It's a difficult
*concept*, in the ways it has traditionally been explained.
Somewhat unrelatedly, I have a mini-rant about encapsulating
coroutines inside the sub call interface. Why do so many people
consider this a good thing? I don't go around coding all of my
functions with ten C<state> variables, and I consider it a good
practice. My subs tend to do the same thing each time they're
called... which is precisely how the concept works. There are things
that are meant to keep state, and these things are called objects!
Why don't we use their interface to manage our state?
I say this because Damian's coroutine proposal could be greatly
simplified (IMHO making it clearer and easier) if calling the sub
didn't imply starting an implicit coroutine the first time. I might
write something that exemplifies this.
> I'm of mixed opinion. I like the notion of merging them, because I
> think they are largely the same concept, implemented in different ways.
> I _VERY MUCH_ like the idea of any arbitrary Code block being
> parallelizable with the addition of a single word. Few languages do a
> decent job of parallelization, and current thread syntax is often
> overly burdensome.
Hm. I've never liked programming with threads, which is a good
argument for your point. Austin's said a few things about threads
that I mostly like.
> Importantly, I hope that it _might_ be possible to, in the
> multiprocessor-laden future, automagically parallelize some coroutines
> without going full-fledged into multiple threads, which are far too
> expensive to produce any net benefit for anything but the largest
> tasks. (The trick is in the dataflow analysis -- making sure there's
> no side effects on either path that will conflict, which is bloody
> difficult, if not impossible.) Such that given:
>
> my $result = foo(...);
>
> ... more stuff ...
>
> print $result;
An important concept for optimization is "pure" routines. These are
routines that have no side-effects. It should be possible to declare
these things, for sure. If all the source is readily available, it's
also possible to deduce this from data flow analysis (we know a
routine is pure if it calls only pure routines). Even if we have a
good analyzer, declaring this kind of thing can help catch some nasty
bugs.
> foo() recognizes that it can be run in parallel with the main flow,
> which is only blocked at the C<print> statement if C<$result> isn't
> completed yet. Until such time as dataflow analysis can accomplish
> that, however, it may take a keyword:
>
> my $result = parallel foo(...);
> ...
> print $result;
>
> or
>
> my $result |||= foo(...);
> ...
> print $result;
>
>
> MikeL
Luke
