Hi Dileepa OK, think I follow that ok. Looks like you're doing some decent research here and have a good handle on how to build this app.
I notice that context.io is commercial, so we'll have to take care about licensing. If the final design ends up being a polling architecture, then that will probably be some sort of domain service. I think there should be a (Java) interface to define the contract, with a fake implementation (also useful for testing) as part of the Isis codebase, but then the "real" context.io based impl can live externally up on github, as a third-party contribution. If there are any other commercial dependencies, we would also need to have these abstracted away similarly. Dan On 18 March 2014 19:48, Dileepa Jayakody <[email protected]> wrote: > Hi Dan, > > Thanks for your valuable input. > Please see my comments inline. > > On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood > <[email protected]>wrote: > > > On 14 March 2014 11:09, Dileepa Jayakody <[email protected]> > > wrote: > > > > > > > > My current idea is that it will follow below flow of operations. > > > > > > 1. User authorizes ReputationBox to connect to his mailbox to read > email > > > (probably using OAuth2) > > > > > > > I presume that this would be actioned from within ReputationBox (RB), > > ultimately by invoking an action on some domain object or service? > > > > > > At present the only thing a domain action can do is to return a URL; this > > is then opened up (by the browser, via Ajax) in a separate tab). > > > > It isn't possible to set up cookies etc on this action, and I imagine > that > > they would be needed in order to do the oauth interaction dance. > > > > AFAIK, cookies are not required for OAuth2 handshake. It simply requires > to redirect the browser to the authorization page provided by the OAuth2 > service provider. RB here will be the OAuth2 consumer, requesting the > access token from the OAuth2 service provider (Google) to access the user's > gmail access. (I'm planning to use Gmail as the email service provider with > OAuth2 support.) > > Further to perform all the email data related requests, I'm planning to use > context.io API: http://context.io/. > It is a REST email API to integrate email data into applications. > Context.io internally supports OAuth2 authorization to connect mailboxes > with applications. > > As a first iteration, I think it'd make most sense to implement this > > outside of the Isis wicket viewer (ie in a custom servlet). This custom > > servlet could use the IsisSessionTemplate class to then interact with > Isis, > > and shove the relevant credentials into an appropriate domain object. > > > > > > > > > > 2. ReputationBox performs an initial reputation-analysis process to > > > build a reputation-index over the past emails imported as a batch. > (This > > > initial reputation-index will be used as the training-data to analyse > new > > > incoming emails) > > > > > > > Once the oauth credentials have been obtained and are held within the > > Isis-managed domain, then this looks like it should probably be done > > asynchronously. > > > > Yes, the initial reputation analysis process will be done asynchronously > and the backgroundService + Quartz scheduler suggestion you have given is > most suitable to implement this. > > > > You could the backgroundService + a Quartz scheduler to pick up a request > > to perform the initial reputation analysis process, and have it store its > > results in appropriate domain objects. > > > > > > > > > > > 3. New emails are polled/ pushed to ReputationBox server and > > > reputation-analysis is performed real-time to asses the reputation. > > > > > > > I prefer the idea of pushing emails to RB, but if that's the case, then > why > > not use it everywhere (including the initial export?) That'd remove the > > need for the oauth/custom servlet stuff in (1) above. > > > > Pushing emails to the RB requires some push-based mechanism from > email-server side. I'm not sure if that is possible with any email server > to push emails to a third party application/server. Please let me know if > that's possible, in which case above OAuth2 requirement to access gmail > from RB application will be removed as you have suggested. :) > > Context.io uses a pull based mechanism to periodically retrieve email data. > > > > > > > > 4. Email reputation data is stored as a special header in the email > > > itself OR stored in a special IMAP directory in the user's mailbox > (need > > to > > > decide on the reputation data storage mechanism) > > > > > > > This also sounds like the email server should call out to the RB server. > > It'd be interesting to see how tools such as SpamAssassin [1] do this. > > > > Thanks for the pointer, I will check that. > Reputation-storage can also be done at RB server, since Isis support > persistence storage. Each user will have a reputation-account and they can > be persisted in ISIS server. The other option is to store the > reputation-data in a special IMAP directory in the user's mailbox itself. > > > > > > > > > > > > 5. ReputationBox client is either a web-app OR a plugin to an existing > > > web-mail client (eg :gmail) that represents the reputation data of the > > new > > > emails (based on the reputation data in the email the client could be > > > implemented as a priority-inbox, spam-filter, email categorizer etc) > > > > > > > > > The bit that's not clear to me is whether this is pull or push for both > the > > initial analysis and the ongoing attachment of headers. > > > I'm thinking RB server will adopt a pull based model in general using > Context.io. Context.io also have a feature called webhooks which provides > rule-based notifications pushed to the app through HTTP POST requests. I > will check the possibility of push-based implementation for RB from > context.io > > From your diagram > > on the ISIS-736 ticket it looks a bit like the RB client is actually a > > gmail plugin, and then the RB server is probably the domain surfaced by > > Isis' Restful Objects viewer. > > > > The Wicket viewer would then be an "administrative" console to also > > browse/amend reputation data. > > > > I'm thinking of developing the intial version of the RB client as a > webapp > to browse the reputation-data with emails. That way ISIS will also have > another DEMO web-application with email data. > > The final production level client will be a gmail plugin. > > > > Overall... think this is doable; just not sure if the oauth2 integration > is > > actually required. > > > > Thanks and more suggestions are welcome. I will draft a proposal and send > to this mail thread for your review. > > Regards, > Dileepa > > > > Dan > > > > > > [1] http://spamassassin.apache.org/ > > [2] > > > > > https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png > > > > > > > > > > > > > > Please see the high-level architecture diagram attached here to get a > > > better idea of my system architecture. I can also send a SRS draft if > you > > > are interested. > > > The entities I have in my mind are : email-sender, email-message, > > > reputation-profile. > > > > > > Suggestions and ideas on how I can utilize Isis framework and it's > tools > > > for my application are most welcome. > > > > > > Thanks, > > > Dileepa > > > > > > > > >> In particular, I'm interested to know what the entities are, and I'm > > also > > >> interested in about the integrations between the app and the users' > meil > > >> client. For example: how does the app get hold of these emails to > > assess > > >> reputation; is it a batch import, real-time, something else; and how > > does > > >> the Isis app then add in reputation scores later (does it interact > with > > >> the > > >> email server, perhaps). > > >> > > >> Thx > > >> Dan > > >> > > >> > > >> > > >> > > >> On 14 March 2014 09:49, Dileepa Jayakody <[email protected]> > > >> wrote: > > >> > > >> > Hi All, > > >> > > > >> > I'm Dileepa Jayakody a MSc research student from University of > > Moratuwa, > > >> > Sri Lanka. I'm doing my research project on the topic : reputation > > >> > assessment in emails. > > >> > My project goal is to introduce reputation data as a attribute to > > >> emails, > > >> > which could be used for various applications such as spam-filtering, > > >> > priority inboxes, social-networking, etc. > > >> > > > >> > I'm planning to adopt a prototype model to develop my application. > > And I > > >> > find Apache Isis a great framework to implement such applications > > mainly > > >> > focusing on my domain model. I'm interested in the GSoC idea : > build a > > >> > "real-life" app in some suitable domain, along with a semi-academic > > >> > write-up of their learning [1] and wish to seek your opinion on > > whether > > >> > implementing my project using Apache Isis can be considered a GSoC > > >> project. > > >> > I'm willing to write a paper on the project implementation, > > highlighting > > >> > the features, usage details of the framework. > > >> > > > >> > As suggested by Dan I took a look at the thesis on "Naked Objects", > > >> > chapter 7 on the implementation comparison of CarServ (conventional > vs > > >> Isis > > >> > usage). In this GSoC project idea, do you think the student must > do 2 > > >> > developments; one in conventional way and another using Isis? In > that > > >> case > > >> > it might be difficult for a research project such as mine. > > >> > > > >> > WDYT? > > >> > > > >> > Thanks, > > >> > Dileepa > > >> > > > >> > [1] https://issues.apache.org/jira/browse/ISIS-736 > > >> > > > >> > > > > > > > > >
