XWIKI-15455 </browse/XWIKI-15455> (that I have just created) invalidates both Solution A and C since it requires to store a Date along with each location, so we cannot store several locations in the same Notification Filter Preferences object.
2018-07-19 11:03 GMT+02:00 Guillaume Delhumeau < [email protected]>: > FYI, I'll continue with the solution A, which is very close to solution C, > and I'll try to use the less memory possible. > > 2018-07-18 11:27 GMT+02:00 Guillaume Delhumeau < > [email protected]>: > >> Corresponding JIRA issue: https://jira.xwiki.org/browse/XWIKI-15445 >> >> 2018-07-18 11:07 GMT+02:00 Guillaume Delhumeau < >> [email protected]>: >> >>> Hi. >>> >>> [TL;DR] >>> >>> This thread is about the way we store notification filter preferences >>> for each user. The constraint is there can be a lot of them (700 is a >>> number a user has recently reported). So how should we store them? >>> >>> [Full text] >>> >>> = Definition = >>> >>> So what is a filter preference? It's a generic object that can store >>> many elements, such as a page locations, application names, event types, >>> etc... They describe a configuration about a given filter for a given user. >>> For example, a filter preference can say "for the ScopeNotificationFilter >>> and the user A, include the location Main.WebHome" as it could be "for the >>> UserNotificationFilter and the user A, exclude the user SPAM". It's generic. >>> >>> The main usage is for page locations (ScopeNotificationFilter). By >>> default, we have the "autowatch" mode enabled. It means every time a user >>> modifies a page, a filter preference for this page and this user is >>> created. So if a user modifies 700 pages, he gets 700 filter preferences. >>> >>> = How are they stored = >>> >>> Currently, we have a simple implementation. There is a generic XClass >>> called "XWiki.Notifications.Code.NotificationFilterPreferenceClass". >>> For each preference, we add an XObject on the user page. It's that simple. >>> But it also means that if a users have 700 filter preferences, she also >>> gets 700 XObjects on her page, and 700 revisions of that page. Which is a >>> pain: it takes a lot of place in the document's cache, and it's heavy to >>> load (lot of SQL queries needed). So we have a big problem here. >>> >>> = Possible solutions = >>> >>> == A: Minimize the number of xobjects needed for ScopeNotificationFilter >>> == >>> >>> Currently, one location is represented by 1 filter preference. But most >>> filter preferences are very similar. They almost all say "for the >>> ScopeNotificationFilter, for all event types, for all applications, the >>> filter preference is enabled". The only different part is the actual >>> location. But the "location" field is itself a LIST stored with the >>> "relational storage" option. So we can take advantage of it and store >>> similar preferences into 1 single object. >>> >>> 1 object with 700 locations instead of 700 objects with 1 location. >>> >>> However, it's a bit harder than this. Event if the >>> NotificationFilterPreferences is generic and can contains many locations, >>> the ScopeNotificationFilter expect it to concern only one location (and >>> then it perform complex operations to sort the filters preferences >>> according to a hierarchy). The UI in the user profile makes the same >>> assumption so it does not handle multiple locations in the same preferences >>> object. Refactoring this is not simple and cannot be done for 10.6. >>> >>> === Variation 1: store only 1 xobject, but make the API return 700 >>> preferences objects anyway === >>> >>> This is the variation I am prototyping. Actually it's ok if the filters >>> and the UI expect only 1 location into the preferences object. All we have >>> to do is to "smash" the xobject into many NotificationFilterPreferences >>> objects that we need internally. It would simply be the responsibility of >>> the Store to detect similarities and to save the minimal amount of XObjects >>> to store a bunch of preferences. >>> >>> But it means being very smart when loading, creating, updating and >>> deleting a preference. Not having one xobject per filter preference >>> introduces complexity, and complexity can lead to bugs. Again, according to >>> the time frame, it's hard to implement. >>> >>> === Variation 2: use custom mapping === >>> >>> Probably the easiest solution that would help making less SQL queries. >>> The idea is to have a SQL table for notification filter preferences and >>> bind the XObjects to that table. It would still use a lot of place in the >>> document's cache but be more efficient on the database level. >>> >>> === Other Problem 1: it still creates page revisions === >>> >>> As long as we store the filter preferences with xobjects, we create page >>> revisions. We can get rid of those by using some internal API to not create >>> a revision when we save an xobject but I wonder if it's what users want. If >>> a user tries to rollback some changes and don't see all filter preferences >>> it concerns, I think it's not very transparent. >>> >>> === Other Problem 2: Document's cache === >>> >>> Sometime we load the a user document to get the avatar of the user, her >>> name, etc... So we load user documents very frequently, even if the user is >>> not connected! Having 700 filters in the document and cache them with the >>> document even if we don't need them is a big waste of memory. >>> >>> == B: Implement a completely new store with Hibernate == >>> >>> A bit like having a custom mapping. We could create a SQL table and >>> implement an API to handle it. Then, no xobjects would be involved. >>> >>> Some drawbacks: >>> * we need to write a custom cache as well. >>> * the user cannot modify her preferences using the wiki principles >>> (xobjects all the way). >>> >>> == C: Refactor the UI and the ScopeNotificationFilter so they do not >>> assume 1 filter preference = 1 location == >>> >>> This option is still possible. Probably the best because creating 1 >>> filter preferences object per location is an obvious waste of memory. A >>> refactoring of the UI is needed anyway, because we currently have no way to >>> remove a bunch of filter preferences easily (users have to delete the 700 >>> filters preferences manually) so we can kill 2 birds with the same stone. >>> >>> But again, it requires some work. >>> >>> = Conclusion = >>> >>> That's it. All possible solutions require development effort that is >>> hardly possible to make before 10.6 (and even 10.7, considering I would >>> probably be the one implementing it and I'm not fulltime on the subject and >>> I have holidays soon). >>> >>> Writing this email helped me to see the problem with perspective. I >>> think solution C may be the best. But any opinion is good to hear (except >>> if you propose something even more complex than I do :p). >>> >>> Thanks, >>> >>> Guillaume >>> >>> >> >> >> -- >> Guillaume Delhumeau ([email protected]) >> Research & Development Engineer at XWiki SAS >> Committer on the XWiki.org project >> > > > > -- > Guillaume Delhumeau ([email protected]) > Research & Development Engineer at XWiki SAS > Committer on the XWiki.org project > -- Guillaume Delhumeau ([email protected]) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
