Hi Sanjaya-- Could you file a Jira ticket for this? We also welcome patches if you want to contribute it.
Marlon On 10/15/13 9:32 PM, Sanjaya Medonsa wrote: > Hi Suresh, > I have further looked into the code to understand, how provenance aware > workflow processing has been implemented in current code base. As you have > stated, it seems functionality is broken. In WorkflowInterpreter class, > there is a method called readProvenance (only place where ProvenanceReader > is used) . It is a private method, no longer a part of the actual > execution. I think, it's better to go back to version where provenance > aware workflow processing working as expected to get better understand on > the expected flow. Then fix the issues in current version. Could you please > let me know, possible version that this particular functionality worked as > expected ? > > Best Regards, > Sanjaya > > > On Tue, Oct 15, 2013 at 4:09 PM, Sanjaya Medonsa <[email protected]>wrote: > >> Thanks Suresh! I'll do further investigations and let the team know my >> findings. >> >> Best Regards, >> Sanjaya >> >> >> On Tue, Oct 15, 2013 at 6:52 AM, Suresh Marru <[email protected]> wrote: >> >>> Hi Sanjaya, >>> >>> XBaya and Workflow Interpreter went through some changes which probably >>> dropped the condition which makes use of the enableProvenanceSmartRun flag. >>> AFAIK, the feature did not get deprecated it should be one of those cases >>> which was not actively tested during releases and fell through the cracks. >>> Lets see if any one gets motivated to debug and fix this issue. And as >>> always, we welcome your patches. Sorry about pointing to a unverified >>> feature. >>> >>> Suresh >>> >>> On Oct 14, 2013, at 9:01 PM, Sanjaya Medonsa <[email protected]> >>> wrote: >>> >>>> Hi Suresh, >>>> I have gone through the research paper in item 6 and try to match it to >>> the current Airavata implementation. I can see that Airavata collects >>> provenance data and stored them in a registry. I couldn't able to find out >>> a place where these data has been actually used to improve the workflow >>> processing. Xbaya can be configured with option enableProvenanceSmartRun. >>> It seems this particular configuration is no longer used. Since provenance >>> aware data is already available in registry, is it the responsibility of >>> the application to use these data to improve workflow processing based on >>> provenance aware data ? >>>> Thanks >>>> Sanjaya >>>> >>>> >>>> On Sat, Aug 24, 2013 at 7:47 AM, Chris Mattmann <[email protected]> >>> wrote: >>>> Hey Sanjaya, >>>> >>>> You may want to check out the Traceable Product product type policy >>>> [1] as well as OODT-139 [2] which includes the PCS services that serve >>>> up the information and provenance. >>>> >>>> Cheers, >>>> Chris >>>> >>>> [1] >>>> >>> http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examp >>>> les/trace/ >>>> [2] http://issues.apache.org/jira/browse/OODT-139 >>>> >>>> >>>> -----Original Message----- >>>> From: Sanjaya Medonsa <[email protected]> >>>> Reply-To: <[email protected]> >>>> Date: Friday, August 23, 2013 6:30 PM >>>> To: Airavata Dev <[email protected]> >>>> Cc: "[email protected]" <[email protected]> >>>> Subject: Re: Povenance Aware Workflow Processing >>>> >>>>> Thanks Suresh! I am interested in Provenance aware workflow processing >>>>> where OODT integration could help greatly with rich metadata available. >>>>> I'll go through the item 6 and let you know my understanding. >>>>> >>>>> Sanjaya >>>>> >>>>> >>>>> On Thu, Aug 8, 2013 at 7:55 PM, Suresh Marru <[email protected]> >>> wrote: >>>>>> Hi Sanjaya, Lewis, >>>>>> >>>>>> Appreciate your interest on this topic, this is a very good question >>> and >>>>>> sorry for the delay in responding, these days I feel I am always in >>>>>> catch >>>>>> up mode. >>>>>> >>>>>> Let me separate this discussion into four parts: Provenance >>> Gathering, >>>>>> Provenance Analyzing, Provenance Reasoning, Provenance Aware >>> executions. >>>>>> Gathering: Airavata is instrumented with rich provenance centric >>>>>> information model which captures each step and publishes to an event >>>>>> bus. >>>>>> This is encoded in a "workflow tracking" schema [1]. So the current >>>>>> trunk >>>>>> of Airavata publishes provenance rich information but not all is >>>>>> guaranteed >>>>>> to be captured in the registry. >>>>>> >>>>>> Analyzing: Significant work on provenance aspect related to Airavata >>> is >>>>>> a >>>>>> by product of doctoral dissertation of Yogesh Simmhan, currently a >>>>>> research >>>>>> asst. professor at USC. His thesis [2] is a good source of reference >>> on >>>>>> this topic. Both the research and tools are in active development by >>>>>> Prof. >>>>>> Beth Plale's group at IU [3]. Theoretically speaking since both Karma >>>>>> uses >>>>>> the same workflow tracking schema and event bus, the tools should be >>>>>> easily >>>>>> integratable but did not have a motivating use case to do so. >>>>>> >>>>>> Reasoning: Phala is an extension to Airavata XBaya (its legacy LEAD >>>>>> version) and Karma framework which provides context to the gathered >>>>>> provenance. Couple of papers on this topic [4], [5]. >>>>>> >>>>>> Executions: This is an active topic within Airavata. Current release >>> of >>>>>> Airavata supports Provenance aware workflow executions and is >>> discussed >>>>>> in >>>>>> [6]. I think the OODT and Airavata integration will greatly benefit >>> this >>>>>> aspect. Airavata Registry metadata cataloging is very primitive. And >>>>>> Rich >>>>>> data management capabilities of OODT should nicely enhance airavata >>>>>> provenance aware executions. >>>>>> >>>>>> Please let me know which of the above aspects you are interested in >>> and >>>>>> we >>>>>> can discuss further. >>>>>> >>>>>> Suresh >>>>>> >>>>>> [1] - >>>>>> >>>>>> >>> https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/workflow- >>>>>> tracking/src/main/resources/schemas/workflow_tracking_types.xsd >>>>>> [2] - http://gradworks.umi.com/3297094.pdf >>>>>> [3] - http://d2i.indiana.edu/provenance_karma >>>>>> [4] - >>>>>> >>> http://link.springer.com/content/pdf/10.1007%2F978-3-540-85502-6_18.pdf >>>>>> [5] - >>>>>> >>>>>> >>> http://dl.acm.org/citation.cfm?id=1590963.1591575&coll=DL&dl=GUIDE&CFID=2 >>>>>> 38325269&CFTOKEN=73364358 >>>>>> [6] - http://dl.acm.org/citation.cfm?id=2359844 >>>>>> >>>>>> >>>>>> On Jul 22, 2013, at 10:57 AM, Lewis John Mcgibbney < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Sanjaya, >>>>>>> I am seeing some traffic floating around RE: Airavata OODT >>> integration >>>>>> but I am not clued up. >>>>>>> Can you point me to the code? The issues? The mailing list threads? >>>>>>> I am also extremely interested in provenance tracking of >>> (record???) >>>>>> data which passes through OODT. I would like to expose this as some >>>>>> stage >>>>>> to people who access the data. >>>>>>> Thank you so much. >>>>>>> >>>>>>> On Monday, July 22, 2013, Sanjaya Medonsa <[email protected]> >>>>>> wrote: >>>>>>>> Hi, >>>>>>>> I am trying to find out ways to improve Airavata provenance aware >>>>>> workflow >>>>>>>> processing based on rich metadata available in Apache OODT. As >>> part >>>>>> of >>>>>> this >>>>>>>> task, I am investigating how Apache Airavata is currently >>> performing >>>>>>>> provenance aware workflow processing. I have looked into the >>>>>> provenance >>>>>>>> manager implementation, where I could see lot of data related to >>>>>> workflow >>>>>>>> execution is saved in registry. But I could see very little of >>> use >>>>>> of >>>>>> those >>>>>>>> data. Could some one please explain the details of provenance >>> aware >>>>>>>> processing in Airavata and any future plans on this area? >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> Sanjaya >>>>>>>> >>>>>>> -- >>>>>>> Lewis >>>>>>> >>>>>> >>>> >>>> >>>
