Dileepa, You might want to take a look at the comment I just added to ISIS-743 [1], answering a question raised about the proposed removal of a feature. I used context.io to illustrate alternatives; might be helpful for you to grok too. Cheers Dan
[1] https://issues.apache.org/jira/browse/ISIS-743 On 18 March 2014 20:55, Dan Haywood <d...@haywood-associates.co.uk> wrote: > Hi Dileepa > OK, think I follow that ok. Looks like you're doing some decent research > here and have a good handle on how to build this app. > > I notice that context.io is commercial, so we'll have to take care about > licensing. If the final design ends up being a polling architecture, then > that will probably be some sort of domain service. I think there should be > a (Java) interface to define the contract, with a fake implementation (also > useful for testing) as part of the Isis codebase, but then the "real" > context.io based impl can live externally up on github, as a third-party > contribution. > > If there are any other commercial dependencies, we would also need to have > these abstracted away similarly. > > Dan > > > > > On 18 March 2014 19:48, Dileepa Jayakody <dileepajayak...@gmail.com>wrote: > >> Hi Dan, >> >> Thanks for your valuable input. >> Please see my comments inline. >> >> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood >> <d...@haywood-associates.co.uk>wrote: >> >> > On 14 March 2014 11:09, Dileepa Jayakody <dileepajayak...@gmail.com> >> > wrote: >> > >> > > >> > > My current idea is that it will follow below flow of operations. >> > > >> > > 1. User authorizes ReputationBox to connect to his mailbox to read >> email >> > > (probably using OAuth2) >> > > >> > >> > I presume that this would be actioned from within ReputationBox (RB), >> > ultimately by invoking an action on some domain object or service? >> > >> > >> > At present the only thing a domain action can do is to return a URL; >> this >> > is then opened up (by the browser, via Ajax) in a separate tab). >> > >> > It isn't possible to set up cookies etc on this action, and I imagine >> that >> > they would be needed in order to do the oauth interaction dance. >> > >> > AFAIK, cookies are not required for OAuth2 handshake. It simply requires >> to redirect the browser to the authorization page provided by the OAuth2 >> service provider. RB here will be the OAuth2 consumer, requesting the >> access token from the OAuth2 service provider (Google) to access the >> user's >> gmail access. (I'm planning to use Gmail as the email service provider >> with >> OAuth2 support.) >> >> Further to perform all the email data related requests, I'm planning to >> use >> context.io API: http://context.io/. >> It is a REST email API to integrate email data into applications. >> Context.io internally supports OAuth2 authorization to connect mailboxes >> with applications. >> >> As a first iteration, I think it'd make most sense to implement this >> > outside of the Isis wicket viewer (ie in a custom servlet). This custom >> > servlet could use the IsisSessionTemplate class to then interact with >> Isis, >> > and shove the relevant credentials into an appropriate domain object. >> > >> > >> >> > >> > > 2. ReputationBox performs an initial reputation-analysis process to >> > > build a reputation-index over the past emails imported as a batch. >> (This >> > > initial reputation-index will be used as the training-data to analyse >> new >> > > incoming emails) >> > > >> > >> > Once the oauth credentials have been obtained and are held within the >> > Isis-managed domain, then this looks like it should probably be done >> > asynchronously. >> > >> > Yes, the initial reputation analysis process will be done asynchronously >> and the backgroundService + Quartz scheduler suggestion you have given is >> most suitable to implement this. >> >> >> > You could the backgroundService + a Quartz scheduler to pick up a >> request >> > to perform the initial reputation analysis process, and have it store >> its >> > results in appropriate domain objects. >> > >> > >> > >> > >> > > 3. New emails are polled/ pushed to ReputationBox server and >> > > reputation-analysis is performed real-time to asses the reputation. >> > > >> > >> > I prefer the idea of pushing emails to RB, but if that's the case, then >> why >> > not use it everywhere (including the initial export?) That'd remove the >> > need for the oauth/custom servlet stuff in (1) above. >> > >> > Pushing emails to the RB requires some push-based mechanism from >> email-server side. I'm not sure if that is possible with any email server >> to push emails to a third party application/server. Please let me know if >> that's possible, in which case above OAuth2 requirement to access gmail >> from RB application will be removed as you have suggested. :) >> >> Context.io uses a pull based mechanism to periodically retrieve email >> data. >> >> > >> > >> > > 4. Email reputation data is stored as a special header in the email >> > > itself OR stored in a special IMAP directory in the user's mailbox >> (need >> > to >> > > decide on the reputation data storage mechanism) >> > > >> > >> > This also sounds like the email server should call out to the RB server. >> >> It'd be interesting to see how tools such as SpamAssassin [1] do this. >> > >> >> Thanks for the pointer, I will check that. >> Reputation-storage can also be done at RB server, since Isis support >> persistence storage. Each user will have a reputation-account and they can >> be persisted in ISIS server. The other option is to store the >> reputation-data in a special IMAP directory in the user's mailbox itself. >> >> > >> > >> > >> > >> > > 5. ReputationBox client is either a web-app OR a plugin to an existing >> > > web-mail client (eg :gmail) that represents the reputation data of the >> > new >> > > emails (based on the reputation data in the email the client could be >> > > implemented as a priority-inbox, spam-filter, email categorizer etc) >> > > >> > >> > >> > The bit that's not clear to me is whether this is pull or push for both >> the >> > initial analysis and the ongoing attachment of headers. >> >> >> I'm thinking RB server will adopt a pull based model in general using >> Context.io. Context.io also have a feature called webhooks which provides >> rule-based notifications pushed to the app through HTTP POST requests. I >> will check the possibility of push-based implementation for RB from >> context.io >> >> From your diagram >> > on the ISIS-736 ticket it looks a bit like the RB client is actually a >> > gmail plugin, and then the RB server is probably the domain surfaced by >> > Isis' Restful Objects viewer. >> > >> > The Wicket viewer would then be an "administrative" console to also >> > browse/amend reputation data. >> > >> > I'm thinking of developing the intial version of the RB client as a >> webapp >> to browse the reputation-data with emails. That way ISIS will also have >> another DEMO web-application with email data. >> >> The final production level client will be a gmail plugin. >> >> >> > Overall... think this is doable; just not sure if the oauth2 >> integration is >> > actually required. >> > >> > Thanks and more suggestions are welcome. I will draft a proposal and >> send >> to this mail thread for your review. >> >> Regards, >> Dileepa >> >> >> > Dan >> > >> > >> > [1] http://spamassassin.apache.org/ >> > [2] >> > >> > >> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png >> > >> > >> > >> > >> > > >> > > Please see the high-level architecture diagram attached here to get a >> > > better idea of my system architecture. I can also send a SRS draft if >> you >> > > are interested. >> > > The entities I have in my mind are : email-sender, email-message, >> > > reputation-profile. >> > > >> > > Suggestions and ideas on how I can utilize Isis framework and it's >> tools >> > > for my application are most welcome. >> > > >> > > Thanks, >> > > Dileepa >> > > >> > > >> > >> In particular, I'm interested to know what the entities are, and I'm >> > also >> > >> interested in about the integrations between the app and the users' >> meil >> > >> client. For example: how does the app get hold of these emails to >> > assess >> > >> reputation; is it a batch import, real-time, something else; and how >> > does >> > >> the Isis app then add in reputation scores later (does it interact >> with >> > >> the >> > >> email server, perhaps). >> > >> >> > >> Thx >> > >> Dan >> > >> >> > >> >> > >> >> > >> >> > >> On 14 March 2014 09:49, Dileepa Jayakody <dileepajayak...@gmail.com> >> > >> wrote: >> > >> >> > >> > Hi All, >> > >> > >> > >> > I'm Dileepa Jayakody a MSc research student from University of >> > Moratuwa, >> > >> > Sri Lanka. I'm doing my research project on the topic : reputation >> > >> > assessment in emails. >> > >> > My project goal is to introduce reputation data as a attribute to >> > >> emails, >> > >> > which could be used for various applications such as >> spam-filtering, >> > >> > priority inboxes, social-networking, etc. >> > >> > >> > >> > I'm planning to adopt a prototype model to develop my application. >> > And I >> > >> > find Apache Isis a great framework to implement such applications >> > mainly >> > >> > focusing on my domain model. I'm interested in the GSoC idea : >> build a >> > >> > "real-life" app in some suitable domain, along with a semi-academic >> > >> > write-up of their learning [1] and wish to seek your opinion on >> > whether >> > >> > implementing my project using Apache Isis can be considered a GSoC >> > >> project. >> > >> > I'm willing to write a paper on the project implementation, >> > highlighting >> > >> > the features, usage details of the framework. >> > >> > >> > >> > As suggested by Dan I took a look at the thesis on "Naked >> Objects", >> > >> > chapter 7 on the implementation comparison of CarServ >> (conventional vs >> > >> Isis >> > >> > usage). In this GSoC project idea, do you think the student must >> do 2 >> > >> > developments; one in conventional way and another using Isis? In >> that >> > >> case >> > >> > it might be difficult for a research project such as mine. >> > >> > >> > >> > WDYT? >> > >> > >> > >> > Thanks, >> > >> > Dileepa >> > >> > >> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736 >> > >> > >> > >> >> > > >> > > >> > >> > >
