Re: [galaxy-dev] Working on the admin interface: JS vs Mako

[Martin Čech]

Hi,

Galaxy is implementing frontend with the help of these libraries:
https://github.com/galaxyproject/galaxy/tree/dev/client/galaxy/scripts/libs

Notably the MV-paradigm is being implemented in Backbone.

Good examples of applications/pieces that are written in JS are at
https://github.com/galaxyproject/galaxy/tree/dev/client/galaxy/scripts/mvc

The easiest one to look into is probably the shortest one: `citation`. If
you want to look into full backbone superpower check out `history`

Martin

On Mon, Feb 1, 2016 at 3:26 PM Ziphozakhe Mashologu 
wrote:

> Hi All
>
> In terms of the future, are we using or considering using any JS SPA
> frameworks or the route is plain JavaScript?
>
> I ask as we aim to satisfy our pursued RESTful architecture.
>
> Thanks
> Zipho
>
>
>
>
>
>
>
> On Mon, Feb 1, 2016 at 7:30 PM, Peter van Heusden  wrote:
>
>> Yeah I noticed that about the DependencyResolversView.
>>
>> In terms of writing JS - where's a good example to look at? I tried
>> digging around the client/ directories but couldn't make sense of how it
>> works.
>>
>> Thanks,
>> Peter
>>
>> On 1 February 2016 at 16:48, John Chilton  wrote:
>>
>>> Peter,
>>>
>>>   We would like to replace all the mako with JS, if I was going to put
>>> a bunch of effort into admin pages I'd start by reworking what was
>>> there to use JavaScript and the API. That is me however, I have lots
>>> of time to put into Galaxy fundamentals and refactoring. This is more
>>> work and I am happy to field PRs that modify mako for the time
>>> being.(https://github.com/galaxyproject/galaxy/pull/1632/files looks
>>> great for instance).
>>>
>>>   A middle ground between rewriting everything and still depending on
>>> mako is to dump dictionaries to JavaScript in the mako and render
>>> things in JS. This will make it easier to adapt your changes once the
>>> pages are ultimately rewritten to JavaScript SPAs. In the case 1632 -
>>> that DependencyResolversView class produces dictionaries that can be
>>> dumped as JSON easily into the page and JavaScript could be used to
>>> build up the tables in
>>> templates/webapps/tool_shed/repository/common.mako.
>>>
>>> -John
>>>
>>>
>>> On Mon, Feb 1, 2016 at 11:51 AM, Peter van Heusden 
>>> wrote:
>>> > Hi there
>>> >
>>> > Zipho and I'd would like to make some changes to the admin pages,
>>> including
>>> > the tool info page and the local data page. These are currently
>>> rendered
>>> > from mako templates - should we change these or is the interface going
>>> to be
>>> > rewritten in client side / javascript in the near(ish) future?
>>> >
>>> > Thanks,
>>> > Peter
>>> >
>>> > ___
>>> > Please keep all replies on the list by using "reply all"
>>> > in your mail client.  To manage your subscriptions to this
>>> > and other Galaxy lists, please use the interface at:
>>> >   https://lists.galaxyproject.org/
>>> >
>>> > To search Galaxy mailing lists use the unified search at:
>>> >   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Working on the admin interface: JS vs Mako

[Ziphozakhe Mashologu]

Super, thanks Martin. I will dive into these links soon and please excuse
any obvious questions from me in the process.

Thanks,
Z

On Mon, Feb 1, 2016 at 10:36 PM, Martin Čech  wrote:

> Hi,
>
> Galaxy is implementing frontend with the help of these libraries:
> https://github.com/galaxyproject/galaxy/tree/dev/client/galaxy/scripts/libs
>
> Notably the MV-paradigm is being implemented in Backbone.
>
> Good examples of applications/pieces that are written in JS are at
> https://github.com/galaxyproject/galaxy/tree/dev/client/galaxy/scripts/mvc
>
> The easiest one to look into is probably the shortest one: `citation`. If
> you want to look into full backbone superpower check out `history`
>
> Martin
>
>
> On Mon, Feb 1, 2016 at 3:26 PM Ziphozakhe Mashologu 
> wrote:
>
>> Hi All
>>
>> In terms of the future, are we using or considering using any JS SPA
>> frameworks or the route is plain JavaScript?
>>
>> I ask as we aim to satisfy our pursued RESTful architecture.
>>
>> Thanks
>> Zipho
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Feb 1, 2016 at 7:30 PM, Peter van Heusden 
>> wrote:
>>
>>> Yeah I noticed that about the DependencyResolversView.
>>>
>>> In terms of writing JS - where's a good example to look at? I tried
>>> digging around the client/ directories but couldn't make sense of how it
>>> works.
>>>
>>> Thanks,
>>> Peter
>>>
>>> On 1 February 2016 at 16:48, John Chilton  wrote:
>>>
 Peter,

   We would like to replace all the mako with JS, if I was going to put
 a bunch of effort into admin pages I'd start by reworking what was
 there to use JavaScript and the API. That is me however, I have lots
 of time to put into Galaxy fundamentals and refactoring. This is more
 work and I am happy to field PRs that modify mako for the time
 being.(https://github.com/galaxyproject/galaxy/pull/1632/files looks
 great for instance).

   A middle ground between rewriting everything and still depending on
 mako is to dump dictionaries to JavaScript in the mako and render
 things in JS. This will make it easier to adapt your changes once the
 pages are ultimately rewritten to JavaScript SPAs. In the case 1632 -
 that DependencyResolversView class produces dictionaries that can be
 dumped as JSON easily into the page and JavaScript could be used to
 build up the tables in
 templates/webapps/tool_shed/repository/common.mako.

 -John


 On Mon, Feb 1, 2016 at 11:51 AM, Peter van Heusden 
 wrote:
 > Hi there
 >
 > Zipho and I'd would like to make some changes to the admin pages,
 including
 > the tool info page and the local data page. These are currently
 rendered
 > from mako templates - should we change these or is the interface
 going to be
 > rewritten in client side / javascript in the near(ish) future?
 >
 > Thanks,
 > Peter
 >
 > ___
 > Please keep all replies on the list by using "reply all"
 > in your mail client.  To manage your subscriptions to this
 > and other Galaxy lists, please use the interface at:
 >   https://lists.galaxyproject.org/
 >
 > To search Galaxy mailing lists use the unified search at:
 >   http://galaxyproject.org/search/mailinglists/

>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Get data via the cloud?

[Scott Szakonyi]

Hi all,

Of course, we need to upload large data files. I understand that the common
way to achieve this is with an FTP server, but due to our situation setting
one up is problematic. Does Galaxy have any mechanism for retrieving data
from cloud resources such as Dropbox, Box, or Google Drive?

Thanks in advance,

-- 
Scott B. Szakonyi
Research Programmer

*Center for Research Computing*
107 Information Technology Center
Notre Dame, IN 46556
http://crc.nd.edu
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Get data via the cloud?

[Martin Čech]

Hi Scott,

Galaxy can retrieve data from any publicly available URL. Just paste the
URL into the upload modal using the 'paste/fetch data'.

This probably won't work with some of the services as they won't serve you
content without visiting their site. But it works with e.g. dropbox if you
add `dl=1` in the end of the shared url.

Martin

On Mon, Feb 1, 2016 at 4:14 PM Scott Szakonyi 
wrote:

> Hi all,
>
> Of course, we need to upload large data files. I understand that the
> common way to achieve this is with an FTP server, but due to our situation
> setting one up is problematic. Does Galaxy have any mechanism for
> retrieving data from cloud resources such as Dropbox, Box, or Google Drive?
>
> Thanks in advance,
>
> --
> Scott B. Szakonyi
> Research Programmer
>
> *Center for Research Computing*
> 107 Information Technology Center
> Notre Dame, IN 46556
> http://crc.nd.edu
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Working on the admin interface: JS vs Mako

[Ziphozakhe Mashologu]

Hi All

In terms of the future, are we using or considering using any JS SPA
frameworks or the route is plain JavaScript?

I ask as we aim to satisfy our pursued RESTful architecture.

Thanks
Zipho







On Mon, Feb 1, 2016 at 7:30 PM, Peter van Heusden  wrote:

> Yeah I noticed that about the DependencyResolversView.
>
> In terms of writing JS - where's a good example to look at? I tried
> digging around the client/ directories but couldn't make sense of how it
> works.
>
> Thanks,
> Peter
>
> On 1 February 2016 at 16:48, John Chilton  wrote:
>
>> Peter,
>>
>>   We would like to replace all the mako with JS, if I was going to put
>> a bunch of effort into admin pages I'd start by reworking what was
>> there to use JavaScript and the API. That is me however, I have lots
>> of time to put into Galaxy fundamentals and refactoring. This is more
>> work and I am happy to field PRs that modify mako for the time
>> being.(https://github.com/galaxyproject/galaxy/pull/1632/files looks
>> great for instance).
>>
>>   A middle ground between rewriting everything and still depending on
>> mako is to dump dictionaries to JavaScript in the mako and render
>> things in JS. This will make it easier to adapt your changes once the
>> pages are ultimately rewritten to JavaScript SPAs. In the case 1632 -
>> that DependencyResolversView class produces dictionaries that can be
>> dumped as JSON easily into the page and JavaScript could be used to
>> build up the tables in
>> templates/webapps/tool_shed/repository/common.mako.
>>
>> -John
>>
>>
>> On Mon, Feb 1, 2016 at 11:51 AM, Peter van Heusden 
>> wrote:
>> > Hi there
>> >
>> > Zipho and I'd would like to make some changes to the admin pages,
>> including
>> > the tool info page and the local data page. These are currently rendered
>> > from mako templates - should we change these or is the interface going
>> to be
>> > rewritten in client side / javascript in the near(ish) future?
>> >
>> > Thanks,
>> > Peter
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/mailinglists/
>>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Picard tools don't include CreateSequenceDictionary?

[Scott Szakonyi]

Hello all,

I have some biologists here at Notre Dame who use the Picard
Tools CreateSequenceDictionary application as part of their workflow. I've
tried to locate it in the toolshed but all the Picard Tools versions that
are available don't seem to include it. Does anyone know where I might find
this tool to add to Galaxy?

Thanks!

-- 
Scott B. Szakonyi
Research Programmer

*Center for Research Computing*
107 Information Technology Center
Notre Dame, IN 46556
http://crc.nd.edu
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Picard tools don't include CreateSequenceDictionary?

[Scott Szakonyi]

Hi Nicola,

Thank you very much! I'm checking with my biologists to see if any of those
tools will work out for us. I'll be sure to open an issue if not.

Best,

Scott

On Mon, Feb 1, 2016 at 10:58 AM, Nicola Soranzo  wrote:

> Hi Scott,
> it seems that devteam's picard repository in the Tool Shed (
> https://toolshed.g2.bx.psu.edu/view/devteam/picard/ ) does not have a
> tool for this, but 4 tools include an automatic conversion of a FASTA
> dataset using CreateSequenceDictionary: BedToIntervalList,
> MergeBamAlignment, ReorderSam and ValidateSamFile.
>
> If you think this is not the best approach, please open an issue at
> https://github.com/galaxyproject/tools-devteam
>
> Chees,
> Nicola
>
>
> On 01/02/16 14:02, Scott Szakonyi wrote:
>
> Hello all,
>
> I have some biologists here at Notre Dame who use the Picard
> Tools CreateSequenceDictionary application as part of their workflow. I've
> tried to locate it in the toolshed but all the Picard Tools versions that
> are available don't seem to include it. Does anyone know where I might find
> this tool to add to Galaxy?
>
> Thanks!
>
> --
> Scott B. Szakonyi
> Research Programmer
>
> *Center for Research Computing*
> 107 Information Technology Center
> Notre Dame, IN 46556
> http://crc.nd.edu
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
>
>


-- 
Scott B. Szakonyi
Research Programmer

*Center for Research Computing*
107 Information Technology Center
Notre Dame, IN 46556
http://crc.nd.edu
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy sending jobs to multiple clusters

[Ganote, Carrie L]

Hi Nikolay,
The slurm branch that John mentioned sounds great! That might be your best bet.
I didn't get drmaa to run with multiple clusters with flags, but I did 'assign' 
different job handlers to different destinations in the drmaa.py runner in 
Galaxy - but that is a bit of a hacky way to do it.

-Carrie

From: John Chilton >
Date: Monday, February 1, 2016 at 11:02 AM
To: Nikolay Aleksandrov Vazov 
>
Cc: "dannon.ba...@gmail.com" 
>, 
"galaxy-dev@lists.galaxyproject.org" 
>,
 Carrie Ganote >, Nate Coraor 
>
Subject: Re: Galaxy sending jobs to multiple clusters

Nate has a branch of slurm drmaa that allows specifying a --clusters
argument in the native specification this can be used to target
multiple hosts.

More information can be found here:

https://github.com/natefoo/slurm-drmaa

Here is how Nate uses it to configure usegalaxy.org:

https://github.com/galaxyproject/usegalaxy-playbook/blob/master/templates/galaxy/usegalaxy.org/config/job_conf.xml.j2

I guess instead of installing slurm-drmaa for a package manager or the
default source - you will just need to install Nate's version.

-John



On Wed, Jan 20, 2016 at 1:18 PM, Nikolay Aleksandrov Vazov
> wrote:
Hi, John, Dan, Carrie and all others,


I am considering a task of setting up a Galaxy instance which shall send
jobs to more than on cluster at a time. In my case I am using drmaa-python
and I was wondering if it was possible to configure multiple drmaa runners
each "pointing" at a different (slurm) control host, e.g.


local

drmaa1

drmaa2


Thanks a lot for your advice


Nikolay




===
Nikolay Vazov, PhD
Department for Research Computing, University of Oslo

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Working on the admin interface: JS vs Mako

[Peter van Heusden]

Yeah I noticed that about the DependencyResolversView.

In terms of writing JS - where's a good example to look at? I tried digging
around the client/ directories but couldn't make sense of how it works.

Thanks,
Peter

On 1 February 2016 at 16:48, John Chilton  wrote:

> Peter,
>
>   We would like to replace all the mako with JS, if I was going to put
> a bunch of effort into admin pages I'd start by reworking what was
> there to use JavaScript and the API. That is me however, I have lots
> of time to put into Galaxy fundamentals and refactoring. This is more
> work and I am happy to field PRs that modify mako for the time
> being.(https://github.com/galaxyproject/galaxy/pull/1632/files looks
> great for instance).
>
>   A middle ground between rewriting everything and still depending on
> mako is to dump dictionaries to JavaScript in the mako and render
> things in JS. This will make it easier to adapt your changes once the
> pages are ultimately rewritten to JavaScript SPAs. In the case 1632 -
> that DependencyResolversView class produces dictionaries that can be
> dumped as JSON easily into the page and JavaScript could be used to
> build up the tables in
> templates/webapps/tool_shed/repository/common.mako.
>
> -John
>
>
> On Mon, Feb 1, 2016 at 11:51 AM, Peter van Heusden 
> wrote:
> > Hi there
> >
> > Zipho and I'd would like to make some changes to the admin pages,
> including
> > the tool info page and the local data page. These are currently rendered
> > from mako templates - should we change these or is the interface going
> to be
> > rewritten in client side / javascript in the near(ish) future?
> >
> > Thanks,
> > Peter
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   https://lists.galaxyproject.org/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Current state of cloudman etc. within private AWS VPCs?

[Golhar, Ryan]

We are working in this as well and have something functioning.  I'll leave it 
to Matt to give you the details as he's the one whose got it up and running.

Please excuse any typos -- Sent from my iPhone

On Jan 29, 2016, at 1:27 PM, Enis Afgan 
> wrote:

Hi Chris,
I believe you're the first person that's tried to run this in a completely 
private VPC so I don't have experience with it but can suggest a change that 
might get over this initial hump at least.

My guess is that get_galaxy_dns method is why you're seeing this log message 
(https://github.com/galaxyproject/cloudman/blob/master/cm/controllers/root.py#L863)
 so perhaps changing that method to return an address you've configured as an 
accessible one would fix it? After changing the code, it'll also be necessary 
to update CloudMan source code in the corresponding bucket; see this page for 
more on that topic: 
https://wiki.galaxyproject.org/CloudMan/CustomizeGalaxyCloud#Using_custom_CloudMan_application

Hope this helps. Let us know how things progress and we can update the code if 
a solution is generic.

Cheers,
Enis


On Fri, Jan 29, 2016 at 10:51 AM, Chris Dagdigian 
> wrote:

Apologies if this is asked often, I did a bit of searching and google-foo but 
could not find very new news...


I've long been a fan/user of cloudman-managed Galaxy clusters but for the first 
time I need galaxy in an AWS environment where we absolutely cannot stand up 
any sort of public facing service or any sort of host with a visible EIC IP 
address. The hosts can access the internet through NAT etc. but we can't run 
them in a public VPC or otherwise expose them to the Internet at large.

My experiment last night with cloudman launching inside a private VPC (with 
NAT'ed internet access and working forward/reverse DNS resolution for all hosts 
in the subnet) seemed to have failed - the key error in the cloudman logs was 
an endless iteration of "attempting to learn public hostname" log entries.

I blew away the install when I saw that error message endlessly repeating 
because there is zero chance our hosts in this VPC subnet will ever have a 
public FDQN hostname. They all have private IPs and private (but DNS 
resolvable) hostnames.

Can anyone give a current state update on cloudman inside private VPCs? Is this 
something I should have debugged further or am I better off just rolling my own 
AMI and installing galaxy and perhaps the cloudman elements by hand?

Thanks!

-Chris




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

This message (including any attachments) may contain confidential, proprietary, 
privileged and/or private information. The information is intended to be for 
the use of the individual or entity designated above. If you are not the 
intended recipient of this message, please notify the sender immediately, and 
delete the message and any attachments. Any disclosure, reproduction, 
distribution or other use of this message or any attachments by an individual 
or entity other than the intended recipient is prohibited.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy sending jobs to multiple clusters

[John Chilton]

Nate has a branch of slurm drmaa that allows specifying a --clusters
argument in the native specification this can be used to target
multiple hosts.

More information can be found here:

https://github.com/natefoo/slurm-drmaa

Here is how Nate uses it to configure usegalaxy.org:

https://github.com/galaxyproject/usegalaxy-playbook/blob/master/templates/galaxy/usegalaxy.org/config/job_conf.xml.j2

I guess instead of installing slurm-drmaa for a package manager or the
default source - you will just need to install Nate's version.

-John



On Wed, Jan 20, 2016 at 1:18 PM, Nikolay Aleksandrov Vazov
 wrote:
> Hi, John, Dan, Carrie and all others,
>
>
> I am considering a task of setting up a Galaxy instance which shall send
> jobs to more than on cluster at a time. In my case I am using drmaa-python
> and I was wondering if it was possible to configure multiple drmaa runners
> each "pointing" at a different (slurm) control host, e.g.
>
>
> local
>
> drmaa1
>
> drmaa2
>
>
> Thanks a lot for your advice
>
>
> Nikolay
>
>
>
>
> ===
> Nikolay Vazov, PhD
> Department for Research Computing, University of Oslo
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Running Galaxy jobs remotely via ssh / rsh

[John Chilton]

I guess LIBTORQUE should be  /root/torque-6.0.0.1-1449528029_21cc3d8
then, the default permissions on /root may be too restrictive - you
may have to install this to a more traditional location such as
/usr/local/lib or /opt.

-John

On Wed, Jan 20, 2016 at 10:32 PM, D K  wrote:
> Hi John,
>
> Thanks for the clarification. I've now installed PBS/torque using the
> instructions found here:
> http://www.discngine.com/blog/2014/6/27/install-torque-on-a-single-node-centos-64
>
> I can correctly submit a job so all is good there.
>
> However, when I follow the instructions on
> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster regarding
> scrambling the eggs, e.g.:
>
> galaxy_user@galaxy_server% LIBTORQUE_DIR=/path/to/libtorque python
> scripts/scramble.py -e pbs_python
>
> I get the following:
>
> Traceback (most recent call last):
>   File "scripts/scramble.py", line 22, in 
> from galaxy.eggs.scramble import ScrambleCrate, ScrambleFailure,
> EggNotFetchable
> ImportError: cannot import name EggNotFetchable
>
> Also, what should my LIBTORQUE_DIR be? I installed this as root in
> /root/torque-6.0.0.1-1449528029_21cc3d8
>
> Thanks!
>
> On Wed, Jan 20, 2016 at 12:27 PM, John Chilton  wrote:
>>
>> Even if you just have two servers, I would strongly recommend you
>> setup a cluster distributed resource manager (DRM) like SLURM, PBS, or
>> Condor and ensuring there exists a shared file system between Galaxy
>> and the node running the jobs. You wouldn't even need to use the CLI
>> job runner - you could just use the DRMAA job runner directly in most
>> cases if you configured one of these - which is what most people use
>> to run Galaxy jobs across machines on a cluster.
>>
>> If you wish to send Galaxy jobs to a single machine without setting up
>> a DRM or if a shared file system is impossible - you can use Pulsar
>> (http://pulsar.readthedocs.org/en/latest/) for most kinds of jobs
>> (some jobs like data source jobs and upload jobs should remain on the
>> Galxay host in a such a configuration).
>>
>> -John
>>
>> On Wed, Jan 20, 2016 at 5:18 PM, D K  wrote:
>> > Hi,
>> >
>> > I would like to run the Galaxy framework on server A, while performing
>> > all
>> > of the jobs on server B using ssh.
>> >
>> > Looking at the documentation here:
>> > https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CLI,
>> > this
>> > seems like it should be possible. However, the documentation states that
>> > the
>> > cli runner requires, at a minimum, two parameters, one for the shell
>> > (which
>> > I'm selecting SecureShell) and a Job plugin. I'm not sure what this
>> > should
>> > be since the ones available are Torque, Slurm, and SlurmTorque and I'm
>> > not
>> > running any of these. Can anyone give me any hints? My current
>> > job_conf.xml
>> > looks like this:
>> >
>> > 
>> > 
>> > 
>> > 
>> > > > load="galaxy.jobs.runners.cli:ShellJobRunner"/>
>> > 
>> > 
>> > 
>> > 
>> > 
>> > 
>> >SecureShell
>> >cli
>> >computehost
>> >galaxy
>> > 
>> > 
>> > 
>> >
>> >
>> > As an alternative, in a tool xml I tried directly sshing between the
>> >  tags e.g.:
>> >
>> >   ssh galaxy@serverB 'sh $__tool_directory__/runTool.sh
>> > --verbose >
>> > $output 2>1' 
>> >
>> > This appears to work. Is it possible to "inject" this ssh command into
>> > tool
>> > execution in Galaxy?
>> >
>> > I was looking to do this to avoid having to install a scheduler like SGE
>> > or
>> > using Pulsar. Any suggestions would be greatly appreciated.
>> >
>> > Thanks!
>> >
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/