Hi Suresh, I have gone through the research paper in item 6 and try to match it to the current Airavata implementation. I can see that Airavata collects provenance data and stored them in a registry. I couldn't able to find out a place where these data has been actually used to improve the workflow processing. Xbaya can be configured with option enableProvenanceSmartRun. It seems this particular configuration is no longer used. Since provenance aware data is already available in registry, is it the responsibility of the application to use these data to improve workflow processing based on provenance aware data ?
Thanks Sanjaya On Sat, Aug 24, 2013 at 7:47 AM, Chris Mattmann <mattm...@apache.org> wrote: > Hey Sanjaya, > > You may want to check out the Traceable Product product type policy > [1] as well as OODT-139 [2] which includes the PCS services that serve > up the information and provenance. > > Cheers, > Chris > > [1] > http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examp > les/trace/ > [2] http://issues.apache.org/jira/browse/OODT-139 > > > -----Original Message----- > From: Sanjaya Medonsa <sanjaya...@gmail.com> > Reply-To: <dev@oodt.apache.org> > Date: Friday, August 23, 2013 6:30 PM > To: Airavata Dev <d...@airavata.apache.org> > Cc: "dev@oodt.apache.org" <dev@oodt.apache.org> > Subject: Re: Povenance Aware Workflow Processing > > >Thanks Suresh! I am interested in Provenance aware workflow processing > >where OODT integration could help greatly with rich metadata available. > >I'll go through the item 6 and let you know my understanding. > > > >Sanjaya > > > > > >On Thu, Aug 8, 2013 at 7:55 PM, Suresh Marru <sma...@apache.org> wrote: > > > >> Hi Sanjaya, Lewis, > >> > >> Appreciate your interest on this topic, this is a very good question and > >> sorry for the delay in responding, these days I feel I am always in > >>catch > >> up mode. > >> > >> Let me separate this discussion into four parts: Provenance Gathering, > >> Provenance Analyzing, Provenance Reasoning, Provenance Aware executions. > >> > >> Gathering: Airavata is instrumented with rich provenance centric > >> information model which captures each step and publishes to an event > >>bus. > >> This is encoded in a "workflow tracking" schema [1]. So the current > >>trunk > >> of Airavata publishes provenance rich information but not all is > >>guaranteed > >> to be captured in the registry. > >> > >> Analyzing: Significant work on provenance aspect related to Airavata is > >>a > >> by product of doctoral dissertation of Yogesh Simmhan, currently a > >>research > >> asst. professor at USC. His thesis [2] is a good source of reference on > >> this topic. Both the research and tools are in active development by > >>Prof. > >> Beth Plale's group at IU [3]. Theoretically speaking since both Karma > >>uses > >> the same workflow tracking schema and event bus, the tools should be > >>easily > >> integratable but did not have a motivating use case to do so. > >> > >> Reasoning: Phala is an extension to Airavata XBaya (its legacy LEAD > >> version) and Karma framework which provides context to the gathered > >> provenance. Couple of papers on this topic [4], [5]. > >> > >> Executions: This is an active topic within Airavata. Current release of > >> Airavata supports Provenance aware workflow executions and is discussed > >>in > >> [6]. I think the OODT and Airavata integration will greatly benefit this > >> aspect. Airavata Registry metadata cataloging is very primitive. And > >>Rich > >> data management capabilities of OODT should nicely enhance airavata > >> provenance aware executions. > >> > >> Please let me know which of the above aspects you are interested in and > >>we > >> can discuss further. > >> > >> Suresh > >> > >> [1] - > >> > >> > https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/workflow- > >>tracking/src/main/resources/schemas/workflow_tracking_types.xsd > >> [2] - http://gradworks.umi.com/3297094.pdf > >> [3] - http://d2i.indiana.edu/provenance_karma > >> [4] - > >> http://link.springer.com/content/pdf/10.1007%2F978-3-540-85502-6_18.pdf > >> [5] - > >> > >> > http://dl.acm.org/citation.cfm?id=1590963.1591575&coll=DL&dl=GUIDE&CFID=2 > >>38325269&CFTOKEN=73364358 > >> [6] - http://dl.acm.org/citation.cfm?id=2359844 > >> > >> > >> On Jul 22, 2013, at 10:57 AM, Lewis John Mcgibbney < > >> lewis.mcgibb...@gmail.com> wrote: > >> > >> > Hi Sanjaya, > >> > I am seeing some traffic floating around RE: Airavata OODT integration > >> but I am not clued up. > >> > Can you point me to the code? The issues? The mailing list threads? > >> > I am also extremely interested in provenance tracking of (record???) > >> data which passes through OODT. I would like to expose this as some > >>stage > >> to people who access the data. > >> > Thank you so much. > >> > > >> > On Monday, July 22, 2013, Sanjaya Medonsa <sanjaya...@gmail.com> > >>wrote: > >> > > Hi, > >> > > I am trying to find out ways to improve Airavata provenance aware > >> workflow > >> > > processing based on rich metadata available in Apache OODT. As part > >>of > >> this > >> > > task, I am investigating how Apache Airavata is currently performing > >> > > provenance aware workflow processing. I have looked into the > >>provenance > >> > > manager implementation, where I could see lot of data related to > >> workflow > >> > > execution is saved in registry. But I could see very little of use > >>of > >> those > >> > > data. Could some one please explain the details of provenance aware > >> > > processing in Airavata and any future plans on this area? > >> > > > >> > > Best Regards, > >> > > Sanjaya > >> > > > >> > > >> > -- > >> > Lewis > >> > > >> > >> > > >
