FYI, I'll continue with the solution A, which is very close to solution C, and I'll try to use the less memory possible.
2018-07-18 11:27 GMT+02:00 Guillaume Delhumeau < guillaume.delhum...@xwiki.com>: > Corresponding JIRA issue: https://jira.xwiki.org/browse/XWIKI-15445 > > 2018-07-18 11:07 GMT+02:00 Guillaume Delhumeau < > guillaume.delhum...@xwiki.com>: > >> Hi. >> >> [TL;DR] >> >> This thread is about the way we store notification filter preferences for >> each user. The constraint is there can be a lot of them (700 is a number a >> user has recently reported). So how should we store them? >> >> [Full text] >> >> = Definition = >> >> So what is a filter preference? It's a generic object that can store many >> elements, such as a page locations, application names, event types, etc... >> They describe a configuration about a given filter for a given user. For >> example, a filter preference can say "for the ScopeNotificationFilter and >> the user A, include the location Main.WebHome" as it could be "for the >> UserNotificationFilter and the user A, exclude the user SPAM". It's generic. >> >> The main usage is for page locations (ScopeNotificationFilter). By >> default, we have the "autowatch" mode enabled. It means every time a user >> modifies a page, a filter preference for this page and this user is >> created. So if a user modifies 700 pages, he gets 700 filter preferences. >> >> = How are they stored = >> >> Currently, we have a simple implementation. There is a generic XClass >> called "XWiki.Notifications.Code.NotificationFilterPreferenceClass". For >> each preference, we add an XObject on the user page. It's that simple. But >> it also means that if a users have 700 filter preferences, she also gets >> 700 XObjects on her page, and 700 revisions of that page. Which is a pain: >> it takes a lot of place in the document's cache, and it's heavy to load >> (lot of SQL queries needed). So we have a big problem here. >> >> = Possible solutions = >> >> == A: Minimize the number of xobjects needed for ScopeNotificationFilter >> == >> >> Currently, one location is represented by 1 filter preference. But most >> filter preferences are very similar. They almost all say "for the >> ScopeNotificationFilter, for all event types, for all applications, the >> filter preference is enabled". The only different part is the actual >> location. But the "location" field is itself a LIST stored with the >> "relational storage" option. So we can take advantage of it and store >> similar preferences into 1 single object. >> >> 1 object with 700 locations instead of 700 objects with 1 location. >> >> However, it's a bit harder than this. Event if the >> NotificationFilterPreferences is generic and can contains many locations, >> the ScopeNotificationFilter expect it to concern only one location (and >> then it perform complex operations to sort the filters preferences >> according to a hierarchy). The UI in the user profile makes the same >> assumption so it does not handle multiple locations in the same preferences >> object. Refactoring this is not simple and cannot be done for 10.6. >> >> === Variation 1: store only 1 xobject, but make the API return 700 >> preferences objects anyway === >> >> This is the variation I am prototyping. Actually it's ok if the filters >> and the UI expect only 1 location into the preferences object. All we have >> to do is to "smash" the xobject into many NotificationFilterPreferences >> objects that we need internally. It would simply be the responsibility of >> the Store to detect similarities and to save the minimal amount of XObjects >> to store a bunch of preferences. >> >> But it means being very smart when loading, creating, updating and >> deleting a preference. Not having one xobject per filter preference >> introduces complexity, and complexity can lead to bugs. Again, according to >> the time frame, it's hard to implement. >> >> === Variation 2: use custom mapping === >> >> Probably the easiest solution that would help making less SQL queries. >> The idea is to have a SQL table for notification filter preferences and >> bind the XObjects to that table. It would still use a lot of place in the >> document's cache but be more efficient on the database level. >> >> === Other Problem 1: it still creates page revisions === >> >> As long as we store the filter preferences with xobjects, we create page >> revisions. We can get rid of those by using some internal API to not create >> a revision when we save an xobject but I wonder if it's what users want. If >> a user tries to rollback some changes and don't see all filter preferences >> it concerns, I think it's not very transparent. >> >> === Other Problem 2: Document's cache === >> >> Sometime we load the a user document to get the avatar of the user, her >> name, etc... So we load user documents very frequently, even if the user is >> not connected! Having 700 filters in the document and cache them with the >> document even if we don't need them is a big waste of memory. >> >> == B: Implement a completely new store with Hibernate == >> >> A bit like having a custom mapping. We could create a SQL table and >> implement an API to handle it. Then, no xobjects would be involved. >> >> Some drawbacks: >> * we need to write a custom cache as well. >> * the user cannot modify her preferences using the wiki principles >> (xobjects all the way). >> >> == C: Refactor the UI and the ScopeNotificationFilter so they do not >> assume 1 filter preference = 1 location == >> >> This option is still possible. Probably the best because creating 1 >> filter preferences object per location is an obvious waste of memory. A >> refactoring of the UI is needed anyway, because we currently have no way to >> remove a bunch of filter preferences easily (users have to delete the 700 >> filters preferences manually) so we can kill 2 birds with the same stone. >> >> But again, it requires some work. >> >> = Conclusion = >> >> That's it. All possible solutions require development effort that is >> hardly possible to make before 10.6 (and even 10.7, considering I would >> probably be the one implementing it and I'm not fulltime on the subject and >> I have holidays soon). >> >> Writing this email helped me to see the problem with perspective. I think >> solution C may be the best. But any opinion is good to hear (except if you >> propose something even more complex than I do :p). >> >> Thanks, >> >> Guillaume >> >> > > > -- > Guillaume Delhumeau (guillaume.delhum...@xwiki.com) > Research & Development Engineer at XWiki SAS > Committer on the XWiki.org project > -- Guillaume Delhumeau (guillaume.delhum...@xwiki.com) Research & Development Engineer at XWiki SAS Committer on the XWiki.org project
