On Sat, 2008-12-27 at 20:35 +0100, Alexander Larsson wrote:
On Sat, 2008-12-27 at 04:21 -0500, Freddie Unpenstein wrote:
This is a side-topic, raised by developments in handling DBus, but
something I feel is worth asking... Is there any mechanism for making
working with asynchronous stuff easier?
Quite often I've had a situation where I've needed to collect several
pieces of information from various asynchronous sources, and then act
on them as a group only once some portion of that information is
available. This is becoming increasingly important as we start working
with more and more remote DBus services, web services,
Corba/Orb/whatever-its-presently-called services, and the list goes
on. Data life cycle and flow management keeps being a problem.
You need to:
- use a signal to catch data when it becomes available
- store that data somewhere
- distribute that data to other places when it becomes available
- clean up the data when it's no longer needed
- safely handle data sources failing or disappearing
- manage re-try timeouts, age invalidation timeouts, etc.
I sort of feel that what's needed, is a GWish which is a higher-level
GBoxed embodying the whole data life cycle management, and I'm
wondering whether there are any plans or similar ideas in the works,
or even just whether anyone else has a working example of tackling a
similar situation.
My starter concept is this; GWish will
- hold a piece of data, when it's available
- can be asked to obtain its data if it doesn't have it
- can be dependant on other GWish's (dependant data)
- ask any depedant GWishes for theirs when it is asked
- invoke a callback when ALL dependant data is available
- can be given the data to hold at any time (usually by the callback)
- signal when its data is ready (ie. when its data is set)
- only ask dependant GWish's when it itself is asked
- provide a helper function to perform the ask, and then wait
- allow the data to be invalidated without destroying the GWish
- be used internally (or at least faked externally) by async
properties
- propagate data invalidation (unless blocked by a flag or ignored)
- allow some dependant data to be flagged as optional
- allow the data set callback to fail after requesting more data
- optionally count a dependency as a reference on that GWish
- use weak references to clean up non-referencing dependencies
- support relevant timeouts with automatic timeout cleanup
- optionally destroy itself if a dependancy gets destroyed
An application can then tie together various pieces of required data
from any source (either GWish-compatible or wrapped in an external
GWish), even user interaction, by having the final GWish dependant on
other GWishes, some representing other intermediate shared data, and
others representing source data, or simply stages of processing to
ensure certain events occur in order regardless of what order their
data becomes available.
This sounds quite nice actually. There are many cases where something
like this would have helped me.
Another possible name for this is GFuture, as such objects are about
information or other things that'll be availible in the future.
This sounds very similar to how it's handled in ACE
http://www.dre.vanderbilt.edu/Doxygen/Stable/ace/classACE__Future.html
One thing I did in EggDBus as compared to GIO, was to make the async
functions return a pending call id, e.g.
guint some_async_function (const gchar *stuff,
GCancellable*cancellable,
GAsyncReadyCallback callback,
gpointer user_data);
so you can write
foo-pending = some_async_function(stuff, NULL, async_ready, NULL);
and later
egg_dbus_connection_pending_call_cancel (con, foo-pending);
to cancel. Or you can do
egg_dbus_connection_pending_call_block (con, foo-pending);
to block. It's not farfetched, I think, to have a GFuture base class so
you can do
GFuture *future;
future = g_future_get (foo-pending_call);
g_object_set_data (future, other-stuff, other_stuff, g_object_unref);
and the GFuture object would be unreffed when the pending call is
finished.
In libgio, all async functions currently don't return anything. I think
we can make them return a pending call id without breaking ABI.
One major problem about doing things like this in C is that you
constantly have to define and allocate small structs to keep the data
you need for the completo operation. If we could somehow make this
easier that would be a large win. I don't know a great approach to do
that offhand though. Some form of user data is one possibility, but I'd
really like it to betype safe and without using strings as API.
If you want type safety you could subclass GFuture and do this
subclassed_future = my_subclassed_future_new (...);
g_future_set (foo-pending_call,
