Re: [galaxy-dev] setting SGE arguments in galaxy (cloudman)

2014-08-14 Thread Jim McCusker
Perfect, thanks! That's in addition to the default arguments?




On Thu, Aug 14, 2014 at 2:52 PM, Enis Afgan  wrote:

> If I'm understanding your question correctly (ie, have the ability to
> specify SGE requirements on a per-tool basis), then yes. You'd do something
> like this in universe_wsgi.ini:
> [galaxy:tool_runners]
> bwa_wrapper = drmaa://-pe smp 2/
> bowtie2 = drmaa://-pe smp 2/
>
>
> On Thu, Aug 14, 2014 at 2:26 PM, Jim McCusker 
> wrote:
>
>> thanks, I ended up doing subinvocations (submitting a job from within a
>> job) because one tool in particular needs lots of memory. Is there a way to
>> set these things on a per-tool basis (which would let me remove the
>> subinvocation)?
>>
>> Jim
>>
>>
>> On Thu, Aug 14, 2014 at 2:09 PM, Enis Afgan  wrote:
>>
>>> Hi Jim
>>> Try setting it like this:
>>> default_cluster_job_runner = drmaa://-w n -l mem_free=16G/
>>>
>>> I've just tried it and worked fine.
>>>
>>> Sorry for a delay in getting back to you,
>>> Enis
>>>
>>>
>>> On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker >> > wrote:
>>>
>>>> I have a galaxy/cloudman cluster where I would like to pass a "-l
>>>> mem_free=16G" parameter when galaxy submits the job to SGE. I've tried
>>>> adding it like this:
>>>>
>>>> default_cluster_job_runner = drmaa://-l mem_free=16G/
>>>>
>>>> (it was:
>>>> default_cluster_job_runner = drmaa:///
>>>> )
>>>>
>>>> but now the jobs don't even submit. Am I wiping out other important
>>>> parameters? How should I do this?
>>>>
>>>> Thanks,
>>>> Jim
>>>>
>>>> ___
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>>   http://lists.bx.psu.edu/
>>>>
>>>> To search Galaxy mailing lists use the unified search at:
>>>>   http://galaxyproject.org/search/mailinglists/
>>>>
>>>
>>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] setting SGE arguments in galaxy (cloudman)

2014-08-14 Thread Jim McCusker
thanks, I ended up doing subinvocations (submitting a job from within a
job) because one tool in particular needs lots of memory. Is there a way to
set these things on a per-tool basis (which would let me remove the
subinvocation)?

Jim


On Thu, Aug 14, 2014 at 2:09 PM, Enis Afgan  wrote:

> Hi Jim
> Try setting it like this:
> default_cluster_job_runner = drmaa://-w n -l mem_free=16G/
>
> I've just tried it and worked fine.
>
> Sorry for a delay in getting back to you,
> Enis
>
>
> On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker 
> wrote:
>
>> I have a galaxy/cloudman cluster where I would like to pass a "-l
>> mem_free=16G" parameter when galaxy submits the job to SGE. I've tried
>> adding it like this:
>>
>> default_cluster_job_runner = drmaa://-l mem_free=16G/
>>
>> (it was:
>> default_cluster_job_runner = drmaa:///
>> )
>>
>> but now the jobs don't even submit. Am I wiping out other important
>> parameters? How should I do this?
>>
>> Thanks,
>> Jim
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] setting SGE arguments in galaxy (cloudman)

2014-08-05 Thread Jim McCusker
I have a galaxy/cloudman cluster where I would like to pass a "-l
mem_free=16G" parameter when galaxy submits the job to SGE. I've tried
adding it like this:

default_cluster_job_runner = drmaa://-l mem_free=16G/

(it was:
default_cluster_job_runner = drmaa:///
)

but now the jobs don't even submit. Am I wiping out other important
parameters? How should I do this?

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Tracking tool execution time?

2014-06-10 Thread Jim McCusker
Is there a way to look at how long a galaxy tool takes to run?

Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: Citations for tools

2014-05-27 Thread Jim McCusker
I'm a semantic web folk (I'm on the PROV Data Model), and I'm willing to
advise as needed. PROV is very generalized, but workflow provenance was one
of the primary use cases for its design (it descends from OPM, which was
almost entirely workflow provenance-oriented).

Workflow tools probably should be prov:Plan, a subclass of prov:Entity.
PROV can cover attribution of things, agents (people and orgs) acting on
behalf of other agents, and virtual and physical locations
(prov:atLocation). Tool provenance is a little smaller-scoped than workflow
invocation provenance, but obviously we want the two to mesh neatly.

Below I'm going to use Turtle to write out an example, but the tool XML can
obviously embed RDF-XML instead.

We would need to supply a little bit of our own vocabulary:

galaxy:Tool rdfs:subClassOf prov:Plan.

Citing a book in a tool might look like this:

:mytool a galaxy:Tool;
   prov:wasDerivedFrom <http://dx.doi.org/10.1119/1.15378> ; # For DOI-able
references. [1]
   dc:creator <http://tw.rpi.edu/instances/JamesMcCusker>.


[1] dx.doi.org will return RDF for the document if you ask for it. We can
use their vocabulary within our own RDF if there is no DOI for it.

@prefix doi: <http://dx.doi.org/>.
@prefix prism: <http://prismstandard.org/namespaces/basic/2.1/>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix dc: <http://purl.org/dc/terms/>.
@prefix bibo: <http://purl.org/ontology/bibo/>.
@prefix owl: <http://www.w3.org/2002/07/owl#>.

<http://id.crossref.org/contributor/milton-abramowitz-1hbgxt653tpie> a
 foaf:Person> ;
  foaf:familyName "Abramowitz" ;
  foaf:givenName "Milton" ;
  foaf:name "Milton Abramowitz" .

doi:10.1119/1.15378
  prism:doi "10.1119/1.15378" ;
  prism:startingPage "958" ;
  prism:volume "56" ;
  dc:creator <
http://id.crossref.org/contributor/milton-abramowitz-1hbgxt653tpie> ;
  dc:date "1988"^^ ;
  dc:identifier "10.1119/1.15378" ;
  dc:isPartOf <http://id.crossref.org/issn/0002-9505> ;
  dc:publisher "American Association of Physics Teachers (AAPT)" ;
  dc:title "Handbook of Mathematical Functions with Formulas, Graphs,
and Mathematical Tables" ;
  bibo:doi "10.1119/1.15378" ;
  bibo:pageStart "958" ;
  bibo:volume "56" ;
  owl:sameAs  ,  .

<http://id.crossref.org/issn/0002-9505> a bibo:Journal ;
      prism:issn "0002-9505" ;
  dc:title "American Journal of Physics" ;
  bibo:issn "0002-9505" ;
  owl:sameAs  .

Jim


On Tue, May 27, 2014 at 2:17 PM, Peter Cock wrote:

> On Tue, May 27, 2014 at 6:21 PM, Jim McCusker
>  wrote:
> > I would suggest using as much as possible from PROV, especially since
> other
> > workflow engines (Taverna and Pegasus come to mind) already support it.
> > Rather than looking for bibtex mappings in XML, we should be looking for
> > vocabularies that represent the elements we need to represent, and the
> > relevant bibtex should be generated from that. PROV and Dublin Core Terms
> > can get us most of the way there, I think.
> >
> > Jim
>
> This PROV ontology? http://www.w3.org/TR/prov-o/
> http://www.taverna.org.uk/documentation/taverna-2-x/provenance/
> https://github.com/wf4ever/taverna-prov
>
> It looks potentially relevant, but tackling a wider issue.
> Are you aware of specific examples covering tool
> citation within the context of workflow provenance?
> I think many people (myself included) would find that
> useful - a basic example for what might go into a
> Galaxy Tool's XML file?
>
> Maybe we need to CC some semantic web folk to advise...
> or schedule a get together as a BoF at GCC2014? It
> seems once a format is agreed, there is willingness
> on the Galaxy side to start coding :)
>
> Peter
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: Citations for tools

2014-05-27 Thread Jim McCusker
I would suggest using as much as possible from PROV, especially since other
workflow engines (Taverna and Pegasus come to mind) already support it.
Rather than looking for bibtex mappings in XML, we should be looking for
vocabularies that represent the elements we need to represent, and the
relevant bibtex should be generated from that. PROV and Dublin Core Terms
can get us most of the way there, I think.

Jim


On Tue, May 27, 2014 at 1:09 PM, Peter Cock wrote:

> Hi Eric,
>
> I was sure there was a Trello card for this, but I can't find it right
> now...
>
> I've pushed this idea on the mailing list before, and in person at
> the Galaxy Community Conference too. See also these threads:
>
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-December/007873.html
>
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010178.html
>
> Are you familiar enough with the area of semantic web/linked
> data to know what would be the best XML based markup to
> use for embedding the citations?
>
> i.e. We should not reinvent the wheel here ;)
>
> Peter
>
>
> On Tue, May 27, 2014 at 5:54 PM, James Taylor 
> wrote:
> > Eric, I'm very much in favor of this feature, and particularly the
> > idea of generating a list of citations from a history or workflow. I
> > imagine the only thing to quibble about will be the syntax. There are
> > already some efforts to represent bibtex in xml (e.g.
> > https://github.com/Zearin/BibTeXML), however they have always struck
> > me as overly verbose.
> >
> > -- jt
> >
> >
> > On Tue, May 27, 2014 at 12:45 PM, Eric Rasche 
> wrote:
> >> -BEGIN PGP SIGNED MESSAGE-
> >> Hash: SHA1
> >>
> >> I'd want to open up discussion on a feature I'd like to see. I'll try
> >> and implement it if I can find time this summer. I didn't see a trello
> >> card for anything like this yet, but please feel free to direct me there
> >> if I missed it.
> >>
> >>
> >> I'd like to see citations as a part of every tool.
> >>
> >> This would happen in the form of a  block in the XML, which
> >> would contain sub-elements with text. These sub-elements could be based
> >> off of BibTeX, since they have existing specs for citing things:
> >> https://en.wikipedia.org/wiki/BibTeX
> >>
> >> These citations would then be accessible in the HTML generated tool
> >> pages, or via a View/Download button somewhere on the tool page. By
> >> storing as an XML tree, we could render these citations as BibTeX
> >> entries for the LaTeX users, and I believe there are ways to convert
> >> BibTeX to EndNote XML and so on.
> >>
> >> This could be extended for use in workflows so that when you run
> >> workflows, somehow a list of citations for all tools used could be
> >> generated.
> >>
> >> Anyone have thoughts or opinions on this?
> >>
> >>
> >>
> >>
> >> Using the example bibtex entry from the wikipedia page:
> >>
> >> @Book{abramowitz+stegun,
> >>  author= "Milton {Abramowitz} and Irene A. {Stegun}",
> >>  title = "Handbook of Mathematical Functions with
> >>   Formulas, Graphs, and Mathematical Tables",
> >>  publisher = "Dover",
> >>  year  =  1964,
> >>  address   = "New York",
> >>  edition   = "ninth Dover printing, tenth GPO printing"
> >> }
> >>
> >> I imagine it'd look like the following in a real-life tool:
> >>
> >> 
> >>   ...
> >>   
> >> Milton Abramowitz and Irene A. Stegun"
> >> Handbook of Mathematical Functions with Formulas, Graphs, and
> >> Mathematical Tables
> >> Dover
> >> 1964
> >> New York
> >> ninth Dover printing, tenth GPO printing
> >>   
> >> 
> >>
> >>
> >> Cheers,
> >> Eric
> >> - --
> >> Eric Rasche
> >> Programmer II
> >> Center for Phage Technology
> >> Texas A&M University
> >> College Station, TX 77843
> >> 404-692-2048
> >> e...@tamu.edu
> >> rasche.e...@yandex.ru
> >> -BEGIN PGP SIGNATURE-
> >> Version: GnuPG v2.0.17 (GNU/Linux)
> >>
> >> iQIcBAEBAgAGBQJThMEUAAoJEMqDXdrsMcpVe9AP/1BjPbP5JdK6KDybOeV2ElvC
> >> jvmUGetAjdQzkKO1Ikeuxb46yp0j4abAGGG92AccxlBYALsT3jsPv5dYjm505vcU
> >> IwlfTBE7gc5y19x3zx1CuAd7PB11tryODz2LwXNKI75f39bNdX6Qe5aA74Vn/k8a
> >> FBeFL/EvkZwI8Z6CvuTGtZrEHvDXVvTFoxMX875f59g2vNy7W8Fh5wQCQ8qgiogi
> >> 0SWGIGZ6OJdqX60h2hOjJ06NhzBcTsjQea+AwTo5wAkyGPFJy4DV78jKOvsd5PMG
> >> qkk4U6gBcmcFolCeItlLPa/C4uiyOK2wVWHyTnEeKjdpD6dwLuemRmwtZrModbQu
> >> PPPFkKzQhUnh5Wgq8TIP4kSc2t1/jSpdV8MHqM0piqOFnR52fExPrwLn82WEFcp8
> >> tXfHHTaZj+6gL+IGswdAgqvxk1V+6QTjH57igWmYtcENpIDbhZVBh+HOD8SS2qdv
> >> ohwFg/Vn8VrCeTGGehNSBvkam0HXjXXq7mZ8W4xrOS8vxv8HPAFZHze2sPlUG/Ob
> >> WeOdiLgD2HCH95SXYDDBNFG6HwFkLETA3DjpfvgHqPdaLmEfIZW+ks1WF9RG+Pmo
> >> jhbWH8OmtTk5A/Zh63uRowMaDe7QSCruPIfIwlrgqLozOj8htMyJPBIW9oZGGkqC
> >> 7Vu5UFeL8gcigRmQv5NA
> >> =ceAo
> >> -END PGP SIGNATURE-
> >> ___
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>   http://lists.bx.psu

Re: [galaxy-dev] Galaxy REST API: multiple params per tool

2014-05-05 Thread Jim McCusker
It looks like this is already the case, but only documented in the code
(lib/galaxy/webapps/galaxy/api/workflows.py):

def _update_step_parameters(step, param_map):
"""
Update ``step`` parameters based on the user-provided ``param_map``
dict.

``param_map`` should be structured as follows::

  PARAM_MAP = {STEP_ID: PARAM_DICT, ...}
  PARAM_DICT = {NAME: VALUE, ...}

For backwards compatibility, the following (deprecated) format is
also supported for ``param_map``::

  PARAM_MAP = {TOOL_ID: PARAM_DICT, ...}

in which case PARAM_DICT affects all steps with the given tool id.
If both by-tool-id and by-step-id specifications are used, the
latter takes precedence.

Finally (again, for backwards compatibility), PARAM_DICT can also
be specified as::

  PARAM_DICT = {'param': NAME, 'value': VALUE}

Note that this format allows only one parameter to be set per step.
"""

Thanks,
Jim


On Mon, May 5, 2014 at 1:56 PM, Jim McCusker wrote:

> The current method for supplying parameters to tools using the REST API
> for workflows is fairly broken. The structure is as follow:
>
>   "parameters":{
> "tool_1": {
> "param":"param_name",
> "value":"param value"
> },
> "tool_2": {
> "param":"param_name",
> "value":"param value"
> }
>
> This structure means that it is impossible to submit more than 1 parameter
> per tool within a workflow. Tools themselves, and workflows that use them,
> have no problems with accepting multiple parameters. We should be
> supporting something more like the following:
>
>   "parameters":{
> "tool_1": {
> "param_name1":"param value 1"
> "param_name2":"param value 2"
> },
> "tool_2": {
> "tool_2_param_name1":"param value 1"
> "tool_2_param_name2":"param value 2"
> }
>
> Is there any work towards something like this taking place? I imagine that
> the REST API could support both formats (if the tool parameters dict has
> anything in it other than "param" and "value", use the new structure) for
> backwards compatibility.
>
> Thanks,
> Jim
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy REST API: multiple params per tool

2014-05-05 Thread Jim McCusker
The current method for supplying parameters to tools using the REST API for
workflows is fairly broken. The structure is as follow:

  "parameters":{
"tool_1": {
"param":"param_name",
"value":"param value"
},
"tool_2": {
"param":"param_name",
"value":"param value"
}

This structure means that it is impossible to submit more than 1 parameter
per tool within a workflow. Tools themselves, and workflows that use them,
have no problems with accepting multiple parameters. We should be
supporting something more like the following:

  "parameters":{
"tool_1": {
"param_name1":"param value 1"
"param_name2":"param value 2"
},
"tool_2": {
"tool_2_param_name1":"param value 1"
"tool_2_param_name2":"param value 2"
}

Is there any work towards something like this taking place? I imagine that
the REST API could support both formats (if the tool parameters dict has
anything in it other than "param" and "value", use the new structure) for
backwards compatibility.

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Cloudman/Galaxy autoscaling

2014-04-28 Thread Jim McCusker
I was thinking of the job idle time, where we spin up more instances if
jobs sit waiting for, say, 15 seconds instead of 60.

Jim


On Mon, Apr 28, 2014 at 2:05 PM, Dannon Baker wrote:

> The time after which it culls instances?  It is not, but because of the
> way Amazon bills for instances, you never want to kill an instance until
> the end of the hour (since, regardless of when you kill an instance, you're
> billed for the remainder of the hour).
>
> -Dannon
>
>
> On Fri, Apr 25, 2014 at 3:01 PM, Jim McCusker 
> wrote:
>
>> Thanks, is the idle time configurable?
>>
>>
>> On Fri, Apr 25, 2014 at 2:43 PM, Dannon Baker wrote:
>>
>>> You can review the exact code here( see 'slow_job_turnover') :
>>> https://bitbucket.org/galaxy/cloudman/src/7b8f04895ad309e0168cb3de66446ae20f3d8b3e/cm/services/autoscale.py?at=default
>>>
>>> But, basically, load on any particular node isn't very useful for
>>> autoscaling in this context because most jobs cannot be split among
>>> multiple nodes.  What we use is a heristic to determine churn and jobs
>>> waiting to run.  Generally speaking, if you have jobs waiting to run for a
>>> bit, your cluster should scale.
>>>
>>> -Dannon
>>>
>>>
>>> On Fri, Apr 25, 2014 at 2:37 PM, Jim McCusker <
>>> jmccus...@5amsolutions.com> wrote:
>>>
>>>> What are the exact conditions that will trigger autoscaling in a
>>>> galaxy/cloudman instance? I'm seeing 100% utilization on the head node but
>>>> no attempts by cloudman at spinning up new instances. What sort of state is
>>>> supposed to trigger that?
>>>>
>>>> Thanks,
>>>> Jim
>>>>
>>>> ___
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>>   http://lists.bx.psu.edu/
>>>>
>>>> To search Galaxy mailing lists use the unified search at:
>>>>   http://galaxyproject.org/search/mailinglists/
>>>>
>>>
>>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Cloudman/Galaxy autoscaling

2014-04-25 Thread Jim McCusker
Thanks, is the idle time configurable?


On Fri, Apr 25, 2014 at 2:43 PM, Dannon Baker wrote:

> You can review the exact code here( see 'slow_job_turnover') :
> https://bitbucket.org/galaxy/cloudman/src/7b8f04895ad309e0168cb3de66446ae20f3d8b3e/cm/services/autoscale.py?at=default
>
> But, basically, load on any particular node isn't very useful for
> autoscaling in this context because most jobs cannot be split among
> multiple nodes.  What we use is a heristic to determine churn and jobs
> waiting to run.  Generally speaking, if you have jobs waiting to run for a
> bit, your cluster should scale.
>
> -Dannon
>
>
> On Fri, Apr 25, 2014 at 2:37 PM, Jim McCusker 
> wrote:
>
>> What are the exact conditions that will trigger autoscaling in a
>> galaxy/cloudman instance? I'm seeing 100% utilization on the head node but
>> no attempts by cloudman at spinning up new instances. What sort of state is
>> supposed to trigger that?
>>
>> Thanks,
>> Jim
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Cloudman/Galaxy autoscaling

2014-04-25 Thread Jim McCusker
What are the exact conditions that will trigger autoscaling in a
galaxy/cloudman instance? I'm seeing 100% utilization on the head node but
no attempts by cloudman at spinning up new instances. What sort of state is
supposed to trigger that?

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Character encoding on the command line

2014-04-25 Thread Jim McCusker
When I pass a parameter on the command line certain characters (like @) are
transformed to X+hexadecimal. Is this some sort of sanitization step? Would
using a config file (
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Cconfigfile.3E_tag_set)
avoid that sort of mess?

I'm not sure I've seen this documented (I could be wrong though).

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy/Cloudman Compute Nodes User

2014-04-25 Thread Jim McCusker
Actually, both of these help. I'm putting a boto.conf file into
/mnt/galaxy/galaxy-app and setting the BOTO_PATH environment variable from
the python script that needs it, so it will work in dev (non-clusterman
setup) and production.

Jim


On Fri, Apr 25, 2014 at 11:20 AM, Dannon Baker wrote:

> Ahh, sorry about that, I misread the bit about it actually needing to be
> in the user's directory.  You could use a worker_post_start_script to
> configure things like this (creating/copying .boto at boot from the shared
> space in the script).  See
> https://wiki.galaxyproject.org/CloudMan/UserData for how to specify that
> in your UserData.
>
> -Dannon
>
>
>
> On Fri, Apr 25, 2014 at 11:13 AM, Dannon Baker wrote:
>
>> Hi Jim,
>>
>> /mnt/galaxy is shared via NFS with all cluster workers -- this (or a
>> subdirectory) should work for you.
>>
>> -Dannon
>>
>>
>> On Fri, Apr 25, 2014 at 10:56 AM, Jim McCusker <
>> jmccus...@5amsolutions.com> wrote:
>>
>>> Hi all,
>>>
>>> I need to add a .boto file to the user that galaxy jobs get exec'ed as
>>> on cloudman nodes. I thought it would copy over what the galaxy server has
>>> in its home directory, but that doesn't seem to happen. How can I get
>>> things like my .boto file to migrate over to new cloudman nodes when they
>>> are spun up?
>>>
>>> Thanks,
>>> Jim
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   http://lists.bx.psu.edu/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy/Cloudman Compute Nodes User

2014-04-25 Thread Jim McCusker
Hi all,

I need to add a .boto file to the user that galaxy jobs get exec'ed as on
cloudman nodes. I thought it would copy over what the galaxy server has in
its home directory, but that doesn't seem to happen. How can I get things
like my .boto file to migrate over to new cloudman nodes when they are spun
up?

Thanks,
Jim
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/