So job metrics are completely different than what you described here. Job metrics are enabled by default and can be configured fairly directly by modifying config/job_metrics_conf.xml. New plugins are relatively easy to write and can be added as python files to lib/galaxy/jobs/metrics/instrumenters/.
What you described is what is used on usegalaxy.org to allow users to have some input on how jobs are scheduled. I call these job resources parameters - in job_conf.xml particular tools can be mapped to these parameters and - yes you are correct dynamic job destination functions can consume them. The dynamic job destinations definitely shouldn't be accessing these through job.parameters keyed on __job_resource. The dynamic destination function should just take in a parameter named resource_params and then the function will be passed in. Dynamic Destinations are documented here: https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping You are correct job resource parameters have not been documented*, there is an open issue here: https://github.com/galaxyproject/galaxy/issues/1266 * It should be noted that while this a frequently requested feature and used on usegalaxy.org - the feature is somewhat controversial. The thinking among some is that is exposes a level of detail that Galaxy is supposed to hide. I don't share this particular concern, but the implementation does worry me - what it does to persisted job state and parameter handling is fairly hacky and I'm somewhat surprised we haven't encountered more problems. This is part of the reason not why this hasn't been documented, but why I feel less bad that it hasn't been documented than other stuff I have done that also hasn't been documented :|. -John On Mon, Jan 11, 2016 at 1:56 PM, Peter van Heusden <p...@sanbi.ac.za> wrote: > Hi there > > I'm trying to understand how the job metrics configuration works. Reading > through various examples, it looks like it works together with the dynamic > destination support. > > First on the dynamic destination support, this works, as I understand it, by > importing modules (files ending in .py) from the directory set as the > rules_module, where rules_module is a parameter for the dynamic job plugin. > As in this snipped from the usegalaxy configuration: > > <plugin id="dynamic" type="runner"> > <!-- These live in the virtualenv --> > <param id="rules_module">usegalaxy.jobs.rules</param> > </plugin> > > > So import usegalaxy.jobs.rules, find where this resolves to as a directory, > find all .py files in that directory. If rules_module is not specified, the > default is galaxy.jobs.rules (under galaxy/lib). > > So these .py files are imported and within them are functions that are used > in dynamic destinations (those with runner="dynamic") and in turn tools are > directed to use these destinations. > > Now the job metrics configuration supplies both a user interface and then a > set of keys that are made accessible to dynamic runners as key value pairs > on the job.parameters attribute (passed in as the third argument, i.e. job). > Specifically the job metrics configuration exists under a key > '__job_resource'. > > Is this all correct? Is there a more straightforward way of moving from job > resource specification to job configuration than this? > > Thanks, > Peter > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
