On 8 December 2016 at 10:37, Alex Hardisty <[email protected]> wrote: > Dear developers,
Hi! > I have some questions about Scufl2 and support for it that I hope someone on > this list can answer for me please. I think they are generally of interest to > the list. Apologies for the late reply - busy December for many, I think! > The background is that I'm looking for a way of representing scientific > workflows that is portable across different execution environments i.e., such > that organisations/persons wanting to execute those workflows on their own > preferred infrastructure and technologies are not tied into Taverna. I found > in the description page of workflow bundles > (https://taverna.incubator.apache.org/documentation/scufl2/bundle) that > '...profiles define how these workflows can be realised and executed on > different environments,...'. So perhaps it is possible? > 1. Is Scufl2 development actively progressing? How widespread is > support for it? SCUFL2 as in the Workflow Bundle format is fairly stabilized - I don't think we want to change the format now. The Taverna Language API (SCUFL2 API) is also stabilized - and in latest release contains the "tavlang" tool for converting and inspecting workflows. https://taverna.incubator.apache.org/download/language/ I think the main thing remaining to look at are just code tidy: * Changing RDF handling of annotations to use Commons RDF https://issues.apache.org/jira/browse/TAVERNA-1017 * Use RO Bundle API to load/save the bundle https://issues.apache.org/jira/browse/TAVERNA-965 The above would help improve the Taverna Language API's exposure of rich annotations, and then we can call it 1.0 without really changing much from the current 0.15. > 2. Is there a version of Taverna Workbench that produces Scufl2 > workflow bundles? The latest published release that can make SCUFL2 wfbundles would be Taverna Workbench 3.0 alpha2 - https://taverna.incubator.apache.org/download/pre-apache Workflows from 2.5 can also be converted using the tavlang tool - but then can't be opened in 2.5 anymore (unless you don't change the scufl2 workflow, and extract its original .t2flow which is contained in the .wfbundle zip) > 3. Allied to question (2) are further questions about: What is the > status of Taverna Workbench 3.0? When will it be released? How is the > community support for Apache Taverna growing? The Workbench source code is now pretty much ready for a 3.1 release following an extensive license review and updates to the build system. https://github.com/apache/incubator-taverna-workbench Any developer effort would be welcome to close the last loose ends, and help push a release through! I can't say when it will be released - it depends on how many put effort in. Latest effort of the community has focused on releasing the engine, command line tool, adding docker/cwl support, as well as the Taverna Mobile app. Community-wise we have also worked towards graduation from the incubator - I guess this took some energy out of the Workbench, which to be fair was the component that needed the most work to be "releasable" after moving to Apache. For next year I think we have to choose between either the Taverna Server 3.1 or the Taverna Workbench 3.1 to be focus of a release first. > 4. Is there any work done or in progress on realising /executing Scufl2 > workflows on Apache Spark? Not that I am aware of. We have an experimental backend for Hadoop support, but it has not been looked at lately: https://github.com/apache/incubator-taverna-engine/tree/master/taverna-execution-hadoop > 5. Are there any tools/infrastructure other than the Taverna family > that can interpret and execute Scufl2? Not to my knowledge. > 6. If I wanted to write workflows in a manner independent of execution > technology, would Scufl2 be a good choice? What alternatives are there? If your workflows would be command-line centric, I would rather look at Common Workflow Language (CWL), which would give you a wide choice of compatible engine, several of them cloud-centric: http://www.commonwl.org/ We have been busy working on CWL support in Taverna as well - in two aspects: a) Running Docker-packaged command line tools b) Importing and using CWL-described Tools from the workbench Support for CWL open/save of workflow definitions would work at SCUFL2 level. developed as an additional format supported by the Taverna Language API. There a primary goal would be supporting CWL-like command line tool workflows only. Supporting other Taverna activities like REST, WSDL or interaction is also possible in CWL - but it would require more porting effort as those service types are not currently formally supported by other CWL engines -- it would be fairly easy to put the SCUFL2 activity config JSON into CWL - but only Taverna would understand it until we standardize that further. CWL is planning a research object format for capturing a CWL workflow - it really needs a "Workflow Bundle"-like format for capturing the constituent parts of a workflow and move them around as a single file that can then be archived and referenced. I can't say now how similar this will be to the wfbundle, but it will probably follow a similar structure; but with CWL YAML files instead of XML files. See for instance the CWL viewer/archiver: https://view.commonwl.org/workflows/58457819857aba0001b0077e which is being developed by a student at University of Manchester and which gives you a research object download of the workflow. While CWL is a good future goal with strong interoperability focus (as well as being hand-editable for developers); I would think the SCUFL2 Workflow Bundle will be the primary format for Taverna workflows for a bit longer as it supports the existing activity types out of the box and Taverna workflows don't do just command line tools. Being able to run CWL workflows with the Taverna Engine (through command line, server or Java API) would help bring a Java implementation of CWL to the market. There might be a point where a workflow can be both a wfbundle and CWL bundle at the same time. Again any developer effort here is welcome! -- Stian Soiland-Reyes http://orcid.org/0000-0001-9842-9718
