Hi Dan,
On Wed, Mar 19, 2014 at 2:32 PM, Dileepa Jayakody <[email protected] > wrote: > Thanks Dan, I will take a look at it. I'm currently drafting a proposal, > will send to the list asap for comments. > > > On Wed, Mar 19, 2014 at 12:57 PM, Dan Haywood < > [email protected]> wrote: > >> Dileepa, >> You might want to take a look at the comment I just added to ISIS-743 [1], >> answering a question raised about the proposed removal of a feature. I >> used context.io to illustrate alternatives; might be helpful for you to >> grok too. >> Cheers >> Dan >> >> >> [1] https://issues.apache.org/jira/browse/ISIS-743 >> >> >> On 18 March 2014 20:55, Dan Haywood <[email protected]> wrote: >> >> > Hi Dileepa >> > OK, think I follow that ok. Looks like you're doing some decent >> research >> > here and have a good handle on how to build this app. >> > >> > I notice that context.io is commercial, so we'll have to take care >> about >> > licensing. If the final design ends up being a polling architecture, >> then >> > that will probably be some sort of domain service. I think there >> should be >> > a (Java) interface to define the contract, with a fake implementation >> (also >> > useful for testing) as part of the Isis codebase, but then the "real" >> > context.io based impl can live externally up on github, as a >> third-party >> > contribution. >> > >> > Do you think using a commercial API such as context.io is not such good idea? If so I can look at implementing the email data handling part of RB as a separate domain service. For user authentication to email server (gmail) we can use Scribe [1] or OAuth2 extension developed for Apache Shiro [2] to perform the OAuth2 authorization. And have a domain service implemented to perform email operations. I think these can be contributed as separate domain services (integrate OAuth2 and handle email data) in Apache Isis. So the project will have several deliverables along the way. 1. OAuth2 client support in Apache Isis 2. Email data analysis domain service 3. ReputationBox as a demo application WDYT? Thanks, Dileepa [1] https://github.com/fernandezpablo85/scribe-java [2] https://github.com/bujiio/buji-pac4j > If there are any other commercial dependencies, we would also need to have >> > these abstracted away similarly. >> > >> > Dan >> > >> > >> > >> > >> > On 18 March 2014 19:48, Dileepa Jayakody <[email protected] >> >wrote: >> > >> >> Hi Dan, >> >> >> >> Thanks for your valuable input. >> >> Please see my comments inline. >> >> >> >> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood >> >> <[email protected]>wrote: >> >> >> >> > On 14 March 2014 11:09, Dileepa Jayakody <[email protected]> >> >> > wrote: >> >> > >> >> > > >> >> > > My current idea is that it will follow below flow of operations. >> >> > > >> >> > > 1. User authorizes ReputationBox to connect to his mailbox to read >> >> email >> >> > > (probably using OAuth2) >> >> > > >> >> > >> >> > I presume that this would be actioned from within ReputationBox (RB), >> >> > ultimately by invoking an action on some domain object or service? >> >> > >> >> > >> >> > At present the only thing a domain action can do is to return a URL; >> >> this >> >> > is then opened up (by the browser, via Ajax) in a separate tab). >> >> > >> >> > It isn't possible to set up cookies etc on this action, and I imagine >> >> that >> >> > they would be needed in order to do the oauth interaction dance. >> >> > >> >> > AFAIK, cookies are not required for OAuth2 handshake. It simply >> requires >> >> to redirect the browser to the authorization page provided by the >> OAuth2 >> >> service provider. RB here will be the OAuth2 consumer, requesting the >> >> access token from the OAuth2 service provider (Google) to access the >> >> user's >> >> gmail access. (I'm planning to use Gmail as the email service provider >> >> with >> >> OAuth2 support.) >> >> >> >> Further to perform all the email data related requests, I'm planning to >> >> use >> >> context.io API: http://context.io/. >> >> It is a REST email API to integrate email data into applications. >> >> Context.io internally supports OAuth2 authorization to connect >> mailboxes >> >> with applications. >> >> >> >> As a first iteration, I think it'd make most sense to implement this >> >> > outside of the Isis wicket viewer (ie in a custom servlet). This >> custom >> >> > servlet could use the IsisSessionTemplate class to then interact with >> >> Isis, >> >> > and shove the relevant credentials into an appropriate domain object. >> >> > >> >> > >> >> >> >> > >> >> > > 2. ReputationBox performs an initial reputation-analysis process >> to >> >> > > build a reputation-index over the past emails imported as a batch. >> >> (This >> >> > > initial reputation-index will be used as the training-data to >> analyse >> >> new >> >> > > incoming emails) >> >> > > >> >> > >> >> > Once the oauth credentials have been obtained and are held within the >> >> > Isis-managed domain, then this looks like it should probably be done >> >> > asynchronously. >> >> > >> >> > Yes, the initial reputation analysis process will be done >> asynchronously >> >> and the backgroundService + Quartz scheduler suggestion you have given >> is >> >> most suitable to implement this. >> >> >> >> >> >> > You could the backgroundService + a Quartz scheduler to pick up a >> >> request >> >> > to perform the initial reputation analysis process, and have it store >> >> its >> >> > results in appropriate domain objects. >> >> > >> >> > >> >> > >> >> > >> >> > > 3. New emails are polled/ pushed to ReputationBox server and >> >> > > reputation-analysis is performed real-time to asses the reputation. >> >> > > >> >> > >> >> > I prefer the idea of pushing emails to RB, but if that's the case, >> then >> >> why >> >> > not use it everywhere (including the initial export?) That'd remove >> the >> >> > need for the oauth/custom servlet stuff in (1) above. >> >> > >> >> > Pushing emails to the RB requires some push-based mechanism from >> >> email-server side. I'm not sure if that is possible with any email >> server >> >> to push emails to a third party application/server. Please let me know >> if >> >> that's possible, in which case above OAuth2 requirement to access gmail >> >> from RB application will be removed as you have suggested. :) >> >> >> >> Context.io uses a pull based mechanism to periodically retrieve email >> >> data. >> >> >> >> > >> >> > >> >> > > 4. Email reputation data is stored as a special header in the >> email >> >> > > itself OR stored in a special IMAP directory in the user's mailbox >> >> (need >> >> > to >> >> > > decide on the reputation data storage mechanism) >> >> > > >> >> > >> >> > This also sounds like the email server should call out to the RB >> server. >> >> >> >> It'd be interesting to see how tools such as SpamAssassin [1] do this. >> >> > >> >> >> >> Thanks for the pointer, I will check that. >> >> Reputation-storage can also be done at RB server, since Isis support >> >> persistence storage. Each user will have a reputation-account and they >> can >> >> be persisted in ISIS server. The other option is to store the >> >> reputation-data in a special IMAP directory in the user's mailbox >> itself. >> >> >> >> > >> >> > >> >> > >> >> > >> >> > > 5. ReputationBox client is either a web-app OR a plugin to an >> existing >> >> > > web-mail client (eg :gmail) that represents the reputation data of >> the >> >> > new >> >> > > emails (based on the reputation data in the email the client could >> be >> >> > > implemented as a priority-inbox, spam-filter, email categorizer >> etc) >> >> > > >> >> > >> >> > >> >> > The bit that's not clear to me is whether this is pull or push for >> both >> >> the >> >> > initial analysis and the ongoing attachment of headers. >> >> >> >> >> >> I'm thinking RB server will adopt a pull based model in general using >> >> Context.io. Context.io also have a feature called webhooks which >> provides >> >> rule-based notifications pushed to the app through HTTP POST requests. >> I >> >> will check the possibility of push-based implementation for RB from >> >> context.io >> >> >> >> From your diagram >> >> > on the ISIS-736 ticket it looks a bit like the RB client is actually >> a >> >> > gmail plugin, and then the RB server is probably the domain surfaced >> by >> >> > Isis' Restful Objects viewer. >> >> > >> >> > The Wicket viewer would then be an "administrative" console to also >> >> > browse/amend reputation data. >> >> > >> >> > I'm thinking of developing the intial version of the RB client as a >> >> webapp >> >> to browse the reputation-data with emails. That way ISIS will also have >> >> another DEMO web-application with email data. >> >> >> >> The final production level client will be a gmail plugin. >> >> >> >> >> >> > Overall... think this is doable; just not sure if the oauth2 >> >> integration is >> >> > actually required. >> >> > >> >> > Thanks and more suggestions are welcome. I will draft a proposal and >> >> send >> >> to this mail thread for your review. >> >> >> >> Regards, >> >> Dileepa >> >> >> >> >> >> > Dan >> >> > >> >> > >> >> > [1] http://spamassassin.apache.org/ >> >> > [2] >> >> > >> >> > >> >> >> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png >> >> > >> >> > >> >> > >> >> > >> >> > > >> >> > > Please see the high-level architecture diagram attached here to >> get a >> >> > > better idea of my system architecture. I can also send a SRS draft >> if >> >> you >> >> > > are interested. >> >> > > The entities I have in my mind are : email-sender, email-message, >> >> > > reputation-profile. >> >> > > >> >> > > Suggestions and ideas on how I can utilize Isis framework and it's >> >> tools >> >> > > for my application are most welcome. >> >> > > >> >> > > Thanks, >> >> > > Dileepa >> >> > > >> >> > > >> >> > >> In particular, I'm interested to know what the entities are, and >> I'm >> >> > also >> >> > >> interested in about the integrations between the app and the >> users' >> >> meil >> >> > >> client. For example: how does the app get hold of these emails to >> >> > assess >> >> > >> reputation; is it a batch import, real-time, something else; and >> how >> >> > does >> >> > >> the Isis app then add in reputation scores later (does it interact >> >> with >> >> > >> the >> >> > >> email server, perhaps). >> >> > >> >> >> > >> Thx >> >> > >> Dan >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> On 14 March 2014 09:49, Dileepa Jayakody < >> [email protected]> >> >> > >> wrote: >> >> > >> >> >> > >> > Hi All, >> >> > >> > >> >> > >> > I'm Dileepa Jayakody a MSc research student from University of >> >> > Moratuwa, >> >> > >> > Sri Lanka. I'm doing my research project on the topic : >> reputation >> >> > >> > assessment in emails. >> >> > >> > My project goal is to introduce reputation data as a attribute >> to >> >> > >> emails, >> >> > >> > which could be used for various applications such as >> >> spam-filtering, >> >> > >> > priority inboxes, social-networking, etc. >> >> > >> > >> >> > >> > I'm planning to adopt a prototype model to develop my >> application. >> >> > And I >> >> > >> > find Apache Isis a great framework to implement such >> applications >> >> > mainly >> >> > >> > focusing on my domain model. I'm interested in the GSoC idea : >> >> build a >> >> > >> > "real-life" app in some suitable domain, along with a >> semi-academic >> >> > >> > write-up of their learning [1] and wish to seek your opinion on >> >> > whether >> >> > >> > implementing my project using Apache Isis can be considered a >> GSoC >> >> > >> project. >> >> > >> > I'm willing to write a paper on the project implementation, >> >> > highlighting >> >> > >> > the features, usage details of the framework. >> >> > >> > >> >> > >> > As suggested by Dan I took a look at the thesis on "Naked >> >> Objects", >> >> > >> > chapter 7 on the implementation comparison of CarServ >> >> (conventional vs >> >> > >> Isis >> >> > >> > usage). In this GSoC project idea, do you think the student >> must >> >> do 2 >> >> > >> > developments; one in conventional way and another using Isis? In >> >> that >> >> > >> case >> >> > >> > it might be difficult for a research project such as mine. >> >> > >> > >> >> > >> > WDYT? >> >> > >> > >> >> > >> > Thanks, >> >> > >> > Dileepa >> >> > >> > >> >> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736 >> >> > >> > >> >> > >> >> >> > > >> >> > > >> >> > >> >> >> > >> > >> > >
