On Wednesday, 3 October 2012 at 21:02:07 UTC, David Nadlinger
wrote:
On Wednesday, 3 October 2012 at 19:42:07 UTC, dsimcha wrote:
If not, please clarify what you needed and the relevant use
cases so that I can fix std.parallelism.
In my use case, conflating the notion of a future, i.e. a value
that becomes available at some point in the future, with the
process which creates that future makes no sense.
So the "process which creates the future" is a Task that executes
in a different thread than the caller? And an alternative way
that a value might become available in the future is e.g. if it's
being retrieved from some slow I/O process like a database or
network?
For example, let's say you are writing a function which
computes a complex database query from its parameters and then
submits it to your query manager/connection pool/… for
asynchronous execution. You cannot use std.parallelism.Task in
this case, because there is no way of expressing the process
which retrieves the result as a delegate running inside a
TaskPool.
Ok, I'm confused here. Why can't the process that retrieves the
result be expressed as a delegate running in a TaskPool or a new
thread?
Or, say you want to write an "aggregator", combining the
results of several futures together, again offering the same
future interface (maybe an array of the original result types)
to consumers. Again, there is no computation-bound part to that
at all, which would make sense to run on a TaskPool – you are
only waiting on the other tasks to finish.
Maybe I'm just being naive since I don't understand the use
cases, but why couldn't you just create an array of Task objects?
The second problem with std.parallelism.Task is that your only
choice is polling (or blocking, for that matter). Yes,
callbacks are a hairy thing to do if you can't be sure what
thread they are executed on, but not having them severely
limits the power of your abstraction, especially if you are
dealing with non-CPU-bound tasks (as many of today's "modern"
use cases are).
I'm a little confused about how the callbacks would be used here.
Is the idea that some callback would be called when the task is
finished? Would it be called in the worker thread or the thread
that submitted the task to the pool? Can you provide a use case?
For example, something my mentor asked to implement for Thrift
during last year's GSoC was a feature which allows to send a
request out to a pool of servers concurrently, returning the
first one of the results (apparently, this mechanism is used as
a sharding mechanism in some situations – if a server doesn't
have the data, it simply ignores the request).
"First one of the results" == the result produced by the the
first server to return anything?
How would you implement something like that as a function
Task[] -> Task? For what it's worth, Task in C# (which is quite
universally praised for its take on the matter) also has a
»ContinueWith« method which is really just a completion
callback mechanism.
I'll look into ContinueWith and see if it's implementable in
std.parallelism without breaking anything.
std.parallelism.Task is great for expressing local
resource-intensive units of work (and fast!), but I think it is
to rigid and specialized for that case to be generally useful.
Right. I wrote std.parallelism with resource-intensive units of
work in mind because that's the use case I was familiar with. It
was designed first and foremost to make using SMP parallelism
_simple_. In hindsight I might have erred to much on the side of
making simple things simple vs. complicated things possible or
over-specialized it and avoided solving the an important, more
general problem. I'll try to understand your use cases and see
if they can be addressed without making simple things more
complicated.
I think the best way you could help me understand what I've
overlooked in std.parallelism's design is to give a quick n'
dirty example of how an API that does what you want would be
used. Even more generally, any _concise, concrete_ use cases,
even toy use cases, would be a huge help.
