Hi Shameera, Thank you for testing it :) As much as I would like to get started with integrating the POC this month, I cannot, because I do not have the bandwidth to work on it. This being the last month of the semester, there are too many things going on, especially with the subjects I have taken. Apart from the regular assignments and classes, I have two (might become three) programming projects to develop and deliver; three end-term exams to write in this month. My last exam is on May 6th. I can start working on it from May 7th on wards.
Best regards, Siddharth Jain On Sun, Apr 3, 2016 at 11:14 PM, Shameera Rathnayaka <[email protected] > wrote: > Hi Siddharth, > > I tested your poc and it is working. Your next move should be to > intregrate this with airavata, i saw you have created a subtask for this. > Let's work on that according to your availability. You can do this part by > part, create your own working branch and work on it, let's do code review > once you finish each part. By the way nice work, seems you have understood > the problem correctly. > > Thanks, > Shameera. > > > On Sat, Mar 19, 2016 at 12:37 AM Siddharth Jain <[email protected]> > wrote: > >> Hello all, >> >> I have completed creating a stand-alone implementation/POC of job monitor in >> the monitoring module of the codebase. >> >> >> >> I did fork from a dev branch and code, but I was unable to commit. I >> tried creating patch, but the patch did not aggregate all the changes, so I >> went ahead and just created a copy of the module (I think it should be as >> good as a patch) for now and attached it to the JIRA issues: >> >> https://issues.apache.org/jira/browse/AIRAVATA-1912 >> >> https://issues.apache.org/jira/browse/AIRAVATA-1914 >> >> https://issues.apache.org/jira/browse/AIRAVATA-1915 >> >> >> >> This code addresses the sub-task: >> >> · AIRAVATA-1914 >> >> · AIRAVATA-1915 >> >> I have also flagged the above two sub-tasks as patch. >> >> >> >> Briefly this is what stand-alone implementation does right now: >> >> · Creates a fanout exchange with durable queues (will persist >> after server restarts) and auto-recovery option enabled; declares and binds >> all the consumer queues >> >> · Fetch e-mails from a dummy email address >> >> · Converts the *non-serializable* javax Message objects into >> custom *serializable* MessageExtract objects, which consists of only the >> necessary information, viz. “from”, “to”, “subject” and “content” fields, >> which are being used in the existing codebase >> >> · Serialize the messages >> >> · Send the message to the exchange, which in turn will deliver the >> message to all the queues bound to it >> >> · Two consumers threads fetch the messages from their respective >> queues >> >> · Deserialize the message and convert it into javax Message >> object, so that it can be used with the existing code >> >> · Print the received message content >> >> >> >> I have tried to keep all the components object-oriented. Following is the >> explanation of the components, >> >> Core Components: >> >> · RabbitMQEmailPublisher: RabbitMQ publisher for e-mail messages >> >> · EmailReceiver: Represents a consumer, which spawns a thread and >> keeps on receiving messages and process them. Multiple EmailReceiver can be >> instantiated and they will automatically run in different threads. >> >> · GmailSMTPMailBox: represents gmail mail box >> >> · EmailConsumer: This has the implementation of call back for >> received messages. >> >> · MessageExtract: Represents the extract from e-mail, which is >> serializable >> >> Simulation Components: >> >> · FetchPublish: Fetches emails with GmailSMTPMailBox and publishes >> to message bus with RabbitMQEmailPublisher >> >> · Simulator (also the main class to run): >> >> o Instantiates two consumers (EmailReceiver) in separate threads >> >> o Fetches email and publishes it with FetchPublish >> >> o Shuts down the consumers after a minute >> >> >> >> Following is the TO-DO list: >> >> · Use the actual email server configs >> >> o Currently I am using my own dummy account for testing, did not want >> to accidentally mess up the production mail-boxes >> >> · Use the codebase broker config >> >> o Currently I am using a local broker url and config >> >> · Fetch the actual email text/body >> >> o Currently I am using a hard-coded string for email text, as somehow >> when I extract the content, I always get multi-part content. The code you >> guys already have in the repository seems to be written with an assumption >> that the received message content will be text only, I tried creating >> an email like that, but it always has multi-part component. So once I have >> integrated the code with the existing monitor, I can get back to fetching >> the actual content >> >> · Open the inbox in read_write mode >> >> o Currently I am opening the inbox in read mode, so that the messages >> are not marked as read and so I don’t have to repeatedly create new dummy >> messages >> >> · Probably implement a RabbitMQ Factory singleton >> >> · Disable message(from message queue) auto-ack and only ack after >> processing has been completed, this might help making the workflow more >> robust >> >> · Use the codebase properties file for configuration of broker and >> email service, right now creating properties on run time >> >> · Create test-cases >> >> I looked into integrating this into the live work-flow, but following >> things kind of need time: >> >> · The message processing work-flow will have to be inverted, it’s >> kind of different than we assumed. Currently the messages are fetched >> explicitly and then processing is done. The way I have implemented the >> consumers is, the queue messages will be delivered asynchronously, so we >> don’t have to poll for it. So all the processing part will have to be wired >> with call-back function. >> >> · Setup PGA and its dependencies on local for testing >> >> · Understand in which part of the life-cycle the consumer(or >> equivalent of that) threads are to be invocated and what specific shutdown >> interrupts are they to follow and respond to >> >> · Understand properties files conventions >> >> Hence the aforementioned list with the TO-DO list will be part of >> milestone 1. >> >> >> >> Best regards, >> >> Siddharth Jain >> > -- > Shameera Rathnayaka >
