Hi there John Thanks for the clarification. My initial aim was, exactly, to expose those parameters that, as you say, some people don't think should be exposed.
My use case is to allow jobs with different native parameters to be scheduled on our clouster from our Galaxy server. Perhaps a better approach is to use dynamic destinations that try and estimate appropriate values for e.g. hvmem, but at present our users are used to setting such parameters themselves. Peter On 11 January 2016 at 17:32, John Chilton <jmchil...@gmail.com> wrote: > So job metrics are completely different than what you described here. > Job metrics are enabled by default and can be configured fairly > directly by modifying config/job_metrics_conf.xml. New plugins are > relatively easy to write and can be added as python files to > lib/galaxy/jobs/metrics/instrumenters/. > > What you described is what is used on usegalaxy.org to allow users to > have some input on how jobs are scheduled. I call these job resources > parameters - in job_conf.xml particular tools can be mapped to these > parameters and - yes you are correct dynamic job destination functions > can consume them. > > The dynamic job destinations definitely shouldn't be accessing these > through job.parameters keyed on __job_resource. The dynamic > destination function should just take in a parameter named > resource_params and then the function will be passed in. > > Dynamic Destinations are documented here: > > https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping > > You are correct job resource parameters have not been documented*, > there is an open issue here: > https://github.com/galaxyproject/galaxy/issues/1266 > > * It should be noted that while this a frequently requested feature > and used on usegalaxy.org - the feature is somewhat controversial. The > thinking among some is that is exposes a level of detail that Galaxy > is supposed to hide. I don't share this particular concern, but the > implementation does worry me - what it does to persisted job state and > parameter handling is fairly hacky and I'm somewhat surprised we > haven't encountered more problems. This is part of the reason not why > this hasn't been documented, but why I feel less bad that it hasn't > been documented than other stuff I have done that also hasn't been > documented :|. > > -John > > On Mon, Jan 11, 2016 at 1:56 PM, Peter van Heusden <p...@sanbi.ac.za> > wrote: > > Hi there > > > > I'm trying to understand how the job metrics configuration works. Reading > > through various examples, it looks like it works together with the > dynamic > > destination support. > > > > First on the dynamic destination support, this works, as I understand > it, by > > importing modules (files ending in .py) from the directory set as the > > rules_module, where rules_module is a parameter for the dynamic job > plugin. > > As in this snipped from the usegalaxy configuration: > > > > <plugin id="dynamic" type="runner"> > > <!-- These live in the virtualenv --> > > <param id="rules_module">usegalaxy.jobs.rules</param> > > </plugin> > > > > > > So import usegalaxy.jobs.rules, find where this resolves to as a > directory, > > find all .py files in that directory. If rules_module is not specified, > the > > default is galaxy.jobs.rules (under galaxy/lib). > > > > So these .py files are imported and within them are functions that are > used > > in dynamic destinations (those with runner="dynamic") and in turn tools > are > > directed to use these destinations. > > > > Now the job metrics configuration supplies both a user interface and > then a > > set of keys that are made accessible to dynamic runners as key value > pairs > > on the job.parameters attribute (passed in as the third argument, i.e. > job). > > Specifically the job metrics configuration exists under a key > > '__job_resource'. > > > > Is this all correct? Is there a more straightforward way of moving from > job > > resource specification to job configuration than this? > > > > Thanks, > > Peter > > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ >
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
