[Dev] Follow up on collection notifications

[Ted Leung]

At the collections review we talked about some dimensions of notification like systems.  I'm going to rehash that as context for the rest of this message. 1. Sync versus Async - when does the message get generated?  As soon as a change happens (synchronous) or is there some delay before the message is generated (asynchronous) 2. Push versus Pull (poll) -  How does the application get the message? Callbacks are called when the message is generated? (push) Callbacks are called when the application indicates it is ready to process the notifications -- by calling some function (pull) 3. Transient versus Persistent - How is subscription information (who will be notified of changes) maintained?    I'll point out that I don't think that it is necessary that we support all possible combinations of dimensions.  This is more just to help people understand the design space of possibilities. In Chandler, we have severals ways in which the app can be notified of changes 1. Collection Notifications     * These tell you about the contents of a collection - when items are added and removed from collections and when items in a collection are changed.     * An item can subscribe to changes on a collection by adding itself to that collection's subscribers list and implementing an onCollectionEvent callback method     * When an add/remove operation happens a notification message is generated and delivered to the callbacks of all subscribers     * When an item in the collection is changed, a notification message is generated but is not delivered until mapChangesCallable is called.  In Chandler this happens in the idle loop     * This mix between sync/push and async/pull models within the collections framework is problematic -- see below for a proposed solution.     * You can change the method which will be called as a callback by adding an onCollectionEventHandler attribute (whose value is the name of the method).  It's not clear to me that we need this level of generality     * The callback API for collections has been propagated up from the low level set API.  This should probably be changed to something that doesn't expose the implementation details of collections. 2. Monitors     * Monitors tell you about changes to any attribute named 'x', regardless of the kind of the item.     * When you create a monitor you supply a callback which consists of  an item and a method on that item.     * When the the attribute is changed, the monitor code calls the callback -- Message generation is synchronous and message delivery is push based. So there are a few problems that we need to look at. 1. the mix of notification styles in the collections framework 2. different mechanism for registering callbacks 3. differing callback API's == Problem #1: the mix of notification styles We agreed at the review that we would like to have a single style for notifications.   The driving consideration is for the app to be able to process notifications when it is ready to, which points to a pull/polling style model. I have succeeded in restructuring the notification code so that all notifications (add/remove/changed) are delivered when a single function is called.   Even though some messages are generated synchrounously, they are queued until delivery, so as far as the application is concerned, notifications are now delivered push style, which makes it hard to tell whether the messages were generated synchronously or not.  This function is a module function in osaf.pim.collections: deliverNotifications(repositoryView) You must pass the current view to  deliverNotifications, since changes happen in a particular view.   On possible improvement would be to make deliverNotificaitons a method on a repositoryView, but I don't know how Andi would feel about that, and I'm not convinced that would be an improvement. I have not checked this code in, but it is passing all the existing notification tests, and Chandler appears to be running smoothly. Every view has a notification queue (transient) attached to it, and deliverNotifications walks the queue delivering the messages.  Now that we have the queue, it seems like this would be a good place to insert code to completely turn off notifications, or to try to eliminate duplicate notifications. == Problem #2: Differing callback registration Alec proposed a unified API for callback registration in the message below, but needing to understand the possible values for 3 parameters seems more complicated than calling the existing monitor and collections subscription API's, although I suppose I could be persuaded. == Problem #3: Differing callback APIs' The API's for collection callbacks and monitor callbacks methods are different.     collection_callback(self, op, item, name, other, *args)   op = the kind of change   item = the item containing a set (the collection item)   name = the name of the set attribute (always "rep" for collections)   other = the item that was changed   *args = grab bag, but mostly unused monitor_callback(self, op, item, attribute, *args)   op = the kind of change   item = the item changed   attribute = the attribute that was changed (allowing you to use the callback for more than one monitor if you checked the attribute value)   *args = grab bag You could make them more similar by: unified_callback(self, op, changedItem, changedValue, attribute=None, *args)   For collections, attribute would be None, dropping the name is fine because it is a fixed value in the collections framework.   For monitors, attribute would be the attribute that would be changed. Again, it's not clear how much work this actually saves.  But I am open to suggestions Ted ---- Ted Leung                 Open Source Applications Foundation (OSAF) PGP Fingerprint: 1003 7870 251F FA71 A59A  CEE3 BEBA 2B87 F5FC 4B42 On Sep 19, 2005, at 1:56 PM, Alec Flett wrote: Also, didn't we talk about a unified notification API that sits on top of the Monitors/watchers/notifications with similar callbacks and all that sort of thing? The way I was imagining this was a singleton notification manager class that has a single entrypoint for notification registration. It would also be responsible for queuing up notifications, and would have a method that "fires" these notifications that could be called from an OnIdle loop. What's nice is that asynchronous notifications could be handled all in one place, and it would be up to the front end's event loop to call the method that "fires" the notifications, so all the notifications would originate in the same place. It would hopefully (if this isn't too ambitious) persist the async notification list, and rely on the other subscription mechanisms (like Monitors) to persist synchronous notification lists. Here's some strawman code to get us started that uses Monitors and Collection subscribers class NotificationManager(...):     def QueueAsyncNotification(self, realTarget, realTargetMethod, ....):             def FireIdleNotifications(self, ...):             def RegisterCallback(self, targetItem, methodName, item=None, attribute=None, synchronous=True):         if item is None:             if synchronous and attribute is not None:                 Monitors.attach(target, methodName, 'set', attribute)         elsif isinstance(item, AbstractCollection):             if synchronous:                 item.subscribers.add((targetItem, methodName))             else:                 item.subscribers.add((self, 'QueueAsyncNotification', ..)) Now you could say notificationManager.RegisterCallback(self, 'onMyCollectionChanged', mycollection) Obviously there are lots of combinations of item/attribute/synchronous that we need to address here.. persisting asynchronous callbacks is going to be tricky. We'd also need API changes (like that change to AbstractCollection.subscribers) and callback parameter unification, but its a start. What do people think? Alec Ted Leung wrote: The action items that we agreed on at the collections code/design  review are here: <http://wiki.osafoundation.org/bin/view/Journal/ TedLeung20050912> One of the items is to move away from use of onValueChanged and go  back to using constructors for creating various collection kinds.   Unfortunately, this means that we wouldn't be able to use update to change existing  instances in the repository.   So we can't remove the onValueChanged  stuff, so it  doesn't make sense to duplicate the code in the collection item  constructors.  So I'm just going to leave the code the way that it is  now. Ted _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "Dev" mailing list http://lists.osafoundation.org/mailman/listinfo/dev ---- Ted Leung                 Open Source Applications Foundation (OSAF) PGP Fingerprint: 1003 7870 251F FA71 A59A  CEE3 BEBA 2B87 F5FC 4B42

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev