Thanks Dan, I will take a look at it. I'm currently drafting a proposal, will send to the list asap for comments.
On Wed, Mar 19, 2014 at 12:57 PM, Dan Haywood <[email protected]>wrote: > Dileepa, > You might want to take a look at the comment I just added to ISIS-743 [1], > answering a question raised about the proposed removal of a feature. I > used context.io to illustrate alternatives; might be helpful for you to > grok too. > Cheers > Dan > > > [1] https://issues.apache.org/jira/browse/ISIS-743 > > > On 18 March 2014 20:55, Dan Haywood <[email protected]> wrote: > > > Hi Dileepa > > OK, think I follow that ok. Looks like you're doing some decent research > > here and have a good handle on how to build this app. > > > > I notice that context.io is commercial, so we'll have to take care about > > licensing. If the final design ends up being a polling architecture, > then > > that will probably be some sort of domain service. I think there should > be > > a (Java) interface to define the contract, with a fake implementation > (also > > useful for testing) as part of the Isis codebase, but then the "real" > > context.io based impl can live externally up on github, as a third-party > > contribution. > > > > If there are any other commercial dependencies, we would also need to > have > > these abstracted away similarly. > > > > Dan > > > > > > > > > > On 18 March 2014 19:48, Dileepa Jayakody <[email protected] > >wrote: > > > >> Hi Dan, > >> > >> Thanks for your valuable input. > >> Please see my comments inline. > >> > >> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood > >> <[email protected]>wrote: > >> > >> > On 14 March 2014 11:09, Dileepa Jayakody <[email protected]> > >> > wrote: > >> > > >> > > > >> > > My current idea is that it will follow below flow of operations. > >> > > > >> > > 1. User authorizes ReputationBox to connect to his mailbox to read > >> email > >> > > (probably using OAuth2) > >> > > > >> > > >> > I presume that this would be actioned from within ReputationBox (RB), > >> > ultimately by invoking an action on some domain object or service? > >> > > >> > > >> > At present the only thing a domain action can do is to return a URL; > >> this > >> > is then opened up (by the browser, via Ajax) in a separate tab). > >> > > >> > It isn't possible to set up cookies etc on this action, and I imagine > >> that > >> > they would be needed in order to do the oauth interaction dance. > >> > > >> > AFAIK, cookies are not required for OAuth2 handshake. It simply > requires > >> to redirect the browser to the authorization page provided by the OAuth2 > >> service provider. RB here will be the OAuth2 consumer, requesting the > >> access token from the OAuth2 service provider (Google) to access the > >> user's > >> gmail access. (I'm planning to use Gmail as the email service provider > >> with > >> OAuth2 support.) > >> > >> Further to perform all the email data related requests, I'm planning to > >> use > >> context.io API: http://context.io/. > >> It is a REST email API to integrate email data into applications. > >> Context.io internally supports OAuth2 authorization to connect mailboxes > >> with applications. > >> > >> As a first iteration, I think it'd make most sense to implement this > >> > outside of the Isis wicket viewer (ie in a custom servlet). This > custom > >> > servlet could use the IsisSessionTemplate class to then interact with > >> Isis, > >> > and shove the relevant credentials into an appropriate domain object. > >> > > >> > > >> > >> > > >> > > 2. ReputationBox performs an initial reputation-analysis process to > >> > > build a reputation-index over the past emails imported as a batch. > >> (This > >> > > initial reputation-index will be used as the training-data to > analyse > >> new > >> > > incoming emails) > >> > > > >> > > >> > Once the oauth credentials have been obtained and are held within the > >> > Isis-managed domain, then this looks like it should probably be done > >> > asynchronously. > >> > > >> > Yes, the initial reputation analysis process will be done > asynchronously > >> and the backgroundService + Quartz scheduler suggestion you have given > is > >> most suitable to implement this. > >> > >> > >> > You could the backgroundService + a Quartz scheduler to pick up a > >> request > >> > to perform the initial reputation analysis process, and have it store > >> its > >> > results in appropriate domain objects. > >> > > >> > > >> > > >> > > >> > > 3. New emails are polled/ pushed to ReputationBox server and > >> > > reputation-analysis is performed real-time to asses the reputation. > >> > > > >> > > >> > I prefer the idea of pushing emails to RB, but if that's the case, > then > >> why > >> > not use it everywhere (including the initial export?) That'd remove > the > >> > need for the oauth/custom servlet stuff in (1) above. > >> > > >> > Pushing emails to the RB requires some push-based mechanism from > >> email-server side. I'm not sure if that is possible with any email > server > >> to push emails to a third party application/server. Please let me know > if > >> that's possible, in which case above OAuth2 requirement to access gmail > >> from RB application will be removed as you have suggested. :) > >> > >> Context.io uses a pull based mechanism to periodically retrieve email > >> data. > >> > >> > > >> > > >> > > 4. Email reputation data is stored as a special header in the email > >> > > itself OR stored in a special IMAP directory in the user's mailbox > >> (need > >> > to > >> > > decide on the reputation data storage mechanism) > >> > > > >> > > >> > This also sounds like the email server should call out to the RB > server. > >> > >> It'd be interesting to see how tools such as SpamAssassin [1] do this. > >> > > >> > >> Thanks for the pointer, I will check that. > >> Reputation-storage can also be done at RB server, since Isis support > >> persistence storage. Each user will have a reputation-account and they > can > >> be persisted in ISIS server. The other option is to store the > >> reputation-data in a special IMAP directory in the user's mailbox > itself. > >> > >> > > >> > > >> > > >> > > >> > > 5. ReputationBox client is either a web-app OR a plugin to an > existing > >> > > web-mail client (eg :gmail) that represents the reputation data of > the > >> > new > >> > > emails (based on the reputation data in the email the client could > be > >> > > implemented as a priority-inbox, spam-filter, email categorizer etc) > >> > > > >> > > >> > > >> > The bit that's not clear to me is whether this is pull or push for > both > >> the > >> > initial analysis and the ongoing attachment of headers. > >> > >> > >> I'm thinking RB server will adopt a pull based model in general using > >> Context.io. Context.io also have a feature called webhooks which > provides > >> rule-based notifications pushed to the app through HTTP POST requests. I > >> will check the possibility of push-based implementation for RB from > >> context.io > >> > >> From your diagram > >> > on the ISIS-736 ticket it looks a bit like the RB client is actually a > >> > gmail plugin, and then the RB server is probably the domain surfaced > by > >> > Isis' Restful Objects viewer. > >> > > >> > The Wicket viewer would then be an "administrative" console to also > >> > browse/amend reputation data. > >> > > >> > I'm thinking of developing the intial version of the RB client as a > >> webapp > >> to browse the reputation-data with emails. That way ISIS will also have > >> another DEMO web-application with email data. > >> > >> The final production level client will be a gmail plugin. > >> > >> > >> > Overall... think this is doable; just not sure if the oauth2 > >> integration is > >> > actually required. > >> > > >> > Thanks and more suggestions are welcome. I will draft a proposal and > >> send > >> to this mail thread for your review. > >> > >> Regards, > >> Dileepa > >> > >> > >> > Dan > >> > > >> > > >> > [1] http://spamassassin.apache.org/ > >> > [2] > >> > > >> > > >> > https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png > >> > > >> > > >> > > >> > > >> > > > >> > > Please see the high-level architecture diagram attached here to get > a > >> > > better idea of my system architecture. I can also send a SRS draft > if > >> you > >> > > are interested. > >> > > The entities I have in my mind are : email-sender, email-message, > >> > > reputation-profile. > >> > > > >> > > Suggestions and ideas on how I can utilize Isis framework and it's > >> tools > >> > > for my application are most welcome. > >> > > > >> > > Thanks, > >> > > Dileepa > >> > > > >> > > > >> > >> In particular, I'm interested to know what the entities are, and > I'm > >> > also > >> > >> interested in about the integrations between the app and the users' > >> meil > >> > >> client. For example: how does the app get hold of these emails to > >> > assess > >> > >> reputation; is it a batch import, real-time, something else; and > how > >> > does > >> > >> the Isis app then add in reputation scores later (does it interact > >> with > >> > >> the > >> > >> email server, perhaps). > >> > >> > >> > >> Thx > >> > >> Dan > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> On 14 March 2014 09:49, Dileepa Jayakody < > [email protected]> > >> > >> wrote: > >> > >> > >> > >> > Hi All, > >> > >> > > >> > >> > I'm Dileepa Jayakody a MSc research student from University of > >> > Moratuwa, > >> > >> > Sri Lanka. I'm doing my research project on the topic : > reputation > >> > >> > assessment in emails. > >> > >> > My project goal is to introduce reputation data as a attribute to > >> > >> emails, > >> > >> > which could be used for various applications such as > >> spam-filtering, > >> > >> > priority inboxes, social-networking, etc. > >> > >> > > >> > >> > I'm planning to adopt a prototype model to develop my > application. > >> > And I > >> > >> > find Apache Isis a great framework to implement such applications > >> > mainly > >> > >> > focusing on my domain model. I'm interested in the GSoC idea : > >> build a > >> > >> > "real-life" app in some suitable domain, along with a > semi-academic > >> > >> > write-up of their learning [1] and wish to seek your opinion on > >> > whether > >> > >> > implementing my project using Apache Isis can be considered a > GSoC > >> > >> project. > >> > >> > I'm willing to write a paper on the project implementation, > >> > highlighting > >> > >> > the features, usage details of the framework. > >> > >> > > >> > >> > As suggested by Dan I took a look at the thesis on "Naked > >> Objects", > >> > >> > chapter 7 on the implementation comparison of CarServ > >> (conventional vs > >> > >> Isis > >> > >> > usage). In this GSoC project idea, do you think the student must > >> do 2 > >> > >> > developments; one in conventional way and another using Isis? In > >> that > >> > >> case > >> > >> > it might be difficult for a research project such as mine. > >> > >> > > >> > >> > WDYT? > >> > >> > > >> > >> > Thanks, > >> > >> > Dileepa > >> > >> > > >> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736 > >> > >> > > >> > >> > >> > > > >> > > > >> > > >> > > > > >
