I have issued a pull request with a specific implementation of these ideas:
https://bitbucket.org/galaxy/galaxy-central/pull-request/227/tool-dependency-resolver-plugins/diff Please feel free to comment. -John On Fri, Sep 27, 2013 at 12:23 AM, John Chilton <chil...@msi.umn.edu> wrote: > Simon, > > What is the advantage of putting that XML definition in the tool shed? > It is not 100% true because of prior_install_required dependencies, > but for the most part sourcing/load the environment for tools is a > Galaxy problem, not so much a tool shed one. What if we did this > instead? > > Add an option to Galaxy's universe_wsgi.ini with the following default: > > tool_dependency_resolution_order = gx_package_manual, gx_package_toolshed > > Which essentially implements my idea above, with James' additional > configuration. But which can be overridden as: > > tool_dependency_resolution_order = plugin_module, gx_package_manual, > gx_package_toolshed > > If set this way then placing <requirement > package="0.5.9">bwa</requirement> in a tool will result in the module > bwa/0.5.9 being loaded if it is 'avail'able, else it will check for a > manually installed env.sh (which is where MSI is currently putting its > module loads), and else it will fallback to source the tool shed > installed dependency. > > I feel like this will give you everything you want without any extra > XML or configuration. Let me know if I am wrong. > > -John > > On Thu, Sep 26, 2013 at 11:39 PM, Guest, Simon > <simon.gu...@agresearch.co.nz> wrote: >> At Thu, 26 Sep 2013 22:03:09 -0400, >> Greg Von Kuster wrote: >>> >>> Hi John, >>> >>> On Sep 26, 2013, at 9:15 PM, John Chilton <chil...@msi.umn.edu> wrote: >>> >>> > I was not even thinking we needed to modify the tool shed to implement >>> > this. I was hoping (?) you could just modify: >>> >>> Nothing in the Tool Shed itself would be affected or require modification >>> for this new feature as this is completely on the Galaxy side. >>> >>> > >>> > lib/galaxy/tools/deps/__init__.py >>> > >>> > to implement this. If some tool contains the tag >>> > >>> > <requirement type="package" version="1.7.1">numpy</requirement> >>> > >>> > then if there is a manually installed tool_dependency in >>> > `tool_dependency_dir/numpy/1.7.1/env.sh` that would take precedence >>> > over the tool shed installed version (would that be something like >>> > `tool_dependency_dir/numpy/1.7.1/owner/name/changeset/env.sh`)? Let me >>> > know if this is way off base. >>> >>> This is a possibility perhaps, but there seems to be a potential weakness >>> in that it doesn't require the Tool Dependency object to exist since the >>> tool will function without a installed dependency from the Tool Shed. Or, >>> if the installed depndency s required, then it is meaningless because it >>> won't be used. If the former case, then the tool dependency cannot be >>> shared via the Tool Shed's dependency mechanism because none of the >>> relationships will be defined since nothing is installed. Wouldn't it be >>> better to allow the Galaxy admin to point the ToolDependency object to a >>> specified binary on disk? In this way, all relationships defined by Tool >>> Shed installations will work as expected, with all contained tools with >>> that dependency point to that same shared location on disk. >> >> Hi Greg, John, >> >> What you are discussing is pretty close to what I am after, and what I >> am prepared to spend some time working on, if that can help. This is >> what I have posted about a couple of times previously. The option to >> use a pre-installed package rather than a Galaxy installed one I think >> would be very useful in general. I think it can be done in a way that >> doesn't break the tool dependency model. >> >> I envisage providing access to existing programs on disk by loading >> the relevant environment module. Platforms will differ greatly on >> where they have programs installed (usr/local/bwa-0.5.9, >> /usr/libexec/bwa-0.5.9, etc.). However, we could arrange to have a >> conventionally named environment module available, so Galaxy just has >> to be told somehow to do a module load bwa/0.5.9, say, prior to >> trying to run that tool, which will then be found on the PATH. >> >> Hooking this in without killing the dependency on named and versioned >> toolshed repos could work like this. In parallel with the repo sets >> named, e.g. package_bwa_0_5_9, which download and build the package, >> we have a set named like native_package_bwa_0_5_9. This could have a >> tool_dependencies.xml file like this: >> >> <?xml version="1.0"?> >> <tool_dependency> >> <native_package name="bwa" version="0.5.9"> >> <actions> >> <action type="load_module"> >> <module name="bwa" version="0.5.9"</module> >> </action> >> </actions> >> <readme> >> Uses native BWA via environment module bwa/0.5.9 >> </readme> >> </native_package> >> </tool_dependency> >> >> What this does is ensure that any repo which requires version 0.5.9 of >> bwa can have this dependency resolved by either package_bwa_0_5_9, or >> native_package_bwa_0_5_9. >> >> I envisage a Galaxy configuration setting that enables native package >> and/or Galaxy package dependency resolution, and which one should be >> preferred. Of course, if one of these has been manually installed, it >> will be used as is. >> >> I would really like to be able to install tools from the toolshed, and >> have them resolve their dependencies automatically using my >> preinstalled application suite. I see that would also meet the needs >> being discussed here. >> >> How does that sound? >> >> cheers, >> Simon >> >> ======================================================================= >> Attention: The information contained in this message and/or attachments >> from AgResearch Limited is intended only for the persons or entities >> to which it is addressed and may contain confidential and/or privileged >> material. Any review, retransmission, dissemination or other use of, or >> taking of any action in reliance upon, this information by persons or >> entities other than the intended recipients is prohibited by AgResearch >> Limited. If you have received this message in error, please notify the >> sender immediately. >> ======================================================================= ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
