Corresponding JIRA issue: https://jira.xwiki.org/browse/XWIKI-15445
2018-07-18 11:07 GMT+02:00 Guillaume Delhumeau < guillaume.delhum...@xwiki.com>: > Hi. > > [TL;DR] > > This thread is about the way we store notification filter preferences for > each user. The constraint is there can be a lot of them (700 is a number a > user has recently reported). So how should we store them? > > [Full text] > > = Definition = > > So what is a filter preference? It's a generic object that can store many > elements, such as a page locations, application names, event types, etc... > They describe a configuration about a given filter for a given user. For > example, a filter preference can say "for the ScopeNotificationFilter and > the user A, include the location Main.WebHome" as it could be "for the > UserNotificationFilter and the user A, exclude the user SPAM". It's generic. > > The main usage is for page locations (ScopeNotificationFilter). By > default, we have the "autowatch" mode enabled. It means every time a user > modifies a page, a filter preference for this page and this user is > created. So if a user modifies 700 pages, he gets 700 filter preferences. > > = How are they stored = > > Currently, we have a simple implementation. There is a generic XClass > called "XWiki.Notifications.Code.NotificationFilterPreferenceClass". For > each preference, we add an XObject on the user page. It's that simple. But > it also means that if a users have 700 filter preferences, she also gets > 700 XObjects on her page, and 700 revisions of that page. Which is a pain: > it takes a lot of place in the document's cache, and it's heavy to load > (lot of SQL queries needed). So we have a big problem here. > > = Possible solutions = > > == A: Minimize the number of xobjects needed for ScopeNotificationFilter == > > Currently, one location is represented by 1 filter preference. But most > filter preferences are very similar. They almost all say "for the > ScopeNotificationFilter, for all event types, for all applications, the > filter preference is enabled". The only different part is the actual > location. But the "location" field is itself a LIST stored with the > "relational storage" option. So we can take advantage of it and store > similar preferences into 1 single object. > > 1 object with 700 locations instead of 700 objects with 1 location. > > However, it's a bit harder than this. Event if the > NotificationFilterPreferences is generic and can contains many locations, > the ScopeNotificationFilter expect it to concern only one location (and > then it perform complex operations to sort the filters preferences > according to a hierarchy). The UI in the user profile makes the same > assumption so it does not handle multiple locations in the same preferences > object. Refactoring this is not simple and cannot be done for 10.6. > > === Variation 1: store only 1 xobject, but make the API return 700 > preferences objects anyway === > > This is the variation I am prototyping. Actually it's ok if the filters > and the UI expect only 1 location into the preferences object. All we have > to do is to "smash" the xobject into many NotificationFilterPreferences > objects that we need internally. It would simply be the responsibility of > the Store to detect similarities and to save the minimal amount of XObjects > to store a bunch of preferences. > > But it means being very smart when loading, creating, updating and > deleting a preference. Not having one xobject per filter preference > introduces complexity, and complexity can lead to bugs. Again, according to > the time frame, it's hard to implement. > > === Variation 2: use custom mapping === > > Probably the easiest solution that would help making less SQL queries. The > idea is to have a SQL table for notification filter preferences and bind > the XObjects to that table. It would still use a lot of place in the > document's cache but be more efficient on the database level. > > === Other Problem 1: it still creates page revisions === > > As long as we store the filter preferences with xobjects, we create page > revisions. We can get rid of those by using some internal API to not create > a revision when we save an xobject but I wonder if it's what users want. If > a user tries to rollback some changes and don't see all filter preferences > it concerns, I think it's not very transparent. > > === Other Problem 2: Document's cache === > > Sometime we load the a user document to get the avatar of the user, her > name, etc... So we load user documents very frequently, even if the user is > not connected! Having 700 filters in the document and cache them with the > document even if we don't need them is a big waste of memory. > > == B: Implement a completely new store with Hibernate == > > A bit like having a custom mapping. We could create a SQL table and > implement an API to handle it. Then, no xobjects would be involved. > > Some drawbacks: > * we need to write a custom cache as well. > * the user cannot modify her preferences using the wiki principles > (xobjects all the way). > > == C: Refactor the UI and the ScopeNotificationFilter so they do not > assume 1 filter preference = 1 location == > > This option is still possible. Probably the best because creating 1 filter > preferences object per location is an obvious waste of memory. A > refactoring of the UI is needed anyway, because we currently have no way to > remove a bunch of filter preferences easily (users have to delete the 700 > filters preferences manually) so we can kill 2 birds with the same stone. > > But again, it requires some work. > > = Conclusion = > > That's it. All possible solutions require development effort that is > hardly possible to make before 10.6 (and even 10.7, considering I would > probably be the one implementing it and I'm not fulltime on the subject and > I have holidays soon). > > Writing this email helped me to see the problem with perspective. I think > solution C may be the best. But any opinion is good to hear (except if you > propose something even more complex than I do :p). > > Thanks, > > Guillaume > > -- Guillaume Delhumeau (guillaume.delhum...@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
