Re: [galaxy-dev] Local deployments of galaxy
Hi David There is another number, which might help you: - there are over 1000 people on the dev-mailing list (https://wiki.galaxyproject.org/GalaxyProject/Statistics#Membership) assuming there are two galaxy admins per site - wishful thinking ;) - and each site has on average two Galaxy servers running...you end up with ~1000 Galaxy servers in the Universe. Hans-Rudolf On 07/24/2014 07:02 PM, Martin Čech wrote: There is a list of the ones that reached back to us and are publicly accessible (~60): https://wiki.galaxyproject.org/PublicGalaxyServers As of private ones there is no good clue as Galaxy does not call home. M. On Thu, Jul 24, 2014 at 12:55 PM, Lapointe, David mailto:david.lapoi...@tufts.edu>> wrote: Not exactly a dev question, but I was asked recently how many sites are running galaxy. The wiki page listing deployments seems not to reflect the actual number, since may are most likely private. Is there an estimate? -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research and Geospatial Technology Services Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Run Jobs as Real User - How to Configure it for TORQUE
Nate and John, I tried with pbs-drmaa and it crashed my galaxy server when the server was started and tried to load the drmaa library . However, I am able to make the drmaa library that comes with TORQUE working with galaxy if sudo is not use.. With sudo, there are still problems I am working to resolve. Ping On Thu, Jul 24, 2014 at 10:10 AM, Nate Coraor wrote: > On Jul 22, 2014, at 2:13 PM, John Chilton wrote: > > Running jobs as the "real" user is not available with the PBS job > runner - one has to use the DRMAA interface to submit jobs as the real > user. > > > That said, the DRMAA runner plugin is compatible with Torque, you just > need to compile and use pbs-drmaa, rather than Torque's libdrmaa: > > http://apps.man.poznan.pl/trac/pbs-drmaa > > --nate > > > I have created a Trello card to add this functionality: > https://trello.com/c/OddS8bMP > > Would be happy to field pull requests to add this - because I doubt > anyone on the core team will be able to get to this anytime soon. > Sorry I don't have better news. > > -John > > On Sun, Jul 20, 2014 at 11:26 PM, Ping Luo wrote: > > I have installed pbs_python module on our cluster to interface Galaxy with > TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. > I need to run Galaxy jobs as the real user. The instruction in the user > guide is for DRMAA interface. How can I configure running jobs as real user > for TORQUE? > > thank, > > Ping > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local deployments of galaxy
There is a list of the ones that reached back to us and are publicly accessible (~60): https://wiki.galaxyproject.org/PublicGalaxyServers As of private ones there is no good clue as Galaxy does not call home. M. On Thu, Jul 24, 2014 at 12:55 PM, Lapointe, David wrote: > Not exactly a dev question, but I was asked recently how many sites are > running galaxy. The wiki page listing deployments seems not to reflect the > actual number, since may are most likely private. > > Is there an estimate? > -- > David Lapointe Ph.D. > Sr. Bioinformatics Specialist > Research and Geospatial Technology Services > Tufts Technology Services (TTS) > 16 Dearborn Road > Somerville MA 02144 > > Phone: 617-627-5319 > Fax: 617-627-3667 > http://it.tufts.edu > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Local deployments of galaxy
Not exactly a dev question, but I was asked recently how many sites are running galaxy. The wiki page listing deployments seems not to reflect the actual number, since may are most likely private. Is there an estimate? -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research and Geospatial Technology Services Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] handlers
On Jul 22, 2014, at 11:25 AM, Kandalaft, Iyad wrote:
> Your overall thought process seems correct. I suspect you still have a web
> process for galaxy that is the only process being proxied by apache?
> Make sure your universe_wsgi.ini has the option set to manage jobs in the
> database (required for multiple handlers).
> I would start with 8 handlers and work my way up (despite the python GIC
> issue). I suspect that each handler with 4 threads would easily saturate
> your 24 core server.
>
> I believe you need to set the default attribute and the tags attribute (I
> could be mistaken).
> Here's what I have and it seems to work as expected (please correct it if
> it's wrong)
>
>
>
> load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
> load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
>
>
>
>
>-q all.q -S
> /bin/bash
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Hi Donnie,
I'd second Iyad's recommandation that you do not need 23 handlers. The number
of concurrent jobs in the local runner, if you don't configure any limits, is
controlled by the number of workers you start for the local runner plugin
multiplied by the number of handlers you have configured to accept jobs going
to a destination that uses the local plugin.
A much better way to control the number of concurrent jobs on local
destinations is to use the destination_total_concurrent_jobs limit, as
explained in the advanced job config sample:
https://bitbucket.org/galaxy/galaxy-central/src/tip/job_conf.xml.sample_advanced#cl-409
You don't need to use a dynamic plugin to statically assign tools to specific
destinations. This can be done under the section of job_conf.xml:
https://wiki.galaxyproject.org/Admin/Config/Jobs#Static_Destination_Mapping
As far as load balancing goes, there is no way to do this using Galaxy's config
mechanisms. You'd need to write a dynamic job running plugin that is able to
inspect the available job destinations and choose the desired one based on
whatever criteria about the job are appropriate.
--nate
> Iyad Kandalaft
> Microbial Biodiversity Bioinformatics
> Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
> 960 Carling Ave.| 960 Ave. Carling
> Ottawa, ON| Ottawa (ON) K1A 0C6
> E-mail Address / Adresse courriel iyad.kandal...@agr.gc.ca
> Telephone | Téléphone 613-759-1228
> Facsimile | Télécopieur 613-759-1701
> Teletypewriter | Téléimprimeur 613-773-2600
> Government of Canada | Gouvernement du Canada
>
>
>
> -Original Message-
> From: galaxy-dev-boun...@lists.bx.psu.edu
> [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Shrum, Donald C
> Sent: Tuesday, July 22, 2014 10:24 AM
> To: galaxy-dev@lists.bx.psu.edu
> Subject: [galaxy-dev] handlers
>
> I could use a little bit of help in making some changes to our galaxy server.
>
>
> I'm in the process of setting up/testing a production galaxy server for our
> research computing center. Our server is setup with an apache proxy, ldap
> authentication and jobs will run as the logged in user.
>
> The server running galaxy has 24 cores. While most of the jobs submitted
> will go to either our HPC or the condor cluster there are some jobs that are
> small and should run on the galaxy server itself.
> I was planning to set up a single web handler and 23 job handlers. I don't
> expect the web server to get bogged down, especially since I have apache
> serving as a proxy.
> I expect a smaller (<100) number of users submitting many jobs.
>
> I just went with 23 handlers for no good reason other than the server has 24
> cores. Perhaps there is a better way to discern the optimum number of job
> handlers.
>
> I'd like jobs submitted to galaxy to go either to our HPC, Condor, or one of
> the 23 local workers. Can galaxy effectively load balance itself in this way?
> Does the configuration below accomplish this?
>
> universe_wsgi.ini:
> [server:handler1]
> use = egg:Paste#http
> port = 8081
> host = 127.0.0.1
> use_threadpool = true
> threadpool_workers = 10
> .
> .
> [server:handler23]
>
>
>
> job_conf.xml:
>
>
> load="galaxy.jobs.runners.local:LocalJobRunner" /> type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner" />
> ...
>
>
>
> id="dynamic" runner="dynamic"> default id="type">python />
>
>
> In my destinations.py script I point tools to the appropriate destination:
> if tool_id.startswith('upload1'):
> return JobDestination(id="local", runner="local")
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this and other Galaxy
> lists, please use the i
Re: [galaxy-dev] bioblend problem trying to use run_tool function
Hmm... the random_lines1 test case distributed with bioblend is kind
of like this but a little more complex because it includes a
conditional also:
https://github.com/jmchilton/bioblend/blob/master/tests/TestGalaxyTools.py#L68
Just using random_lines1 without specifying anything in the
conditional (so just a numeric property and dataset input) the
following snippet works for me:
from bioblend import galaxy
gi = galaxy.GalaxyInstance("localhost:8080", "1957431058be3aa3c57ca312f4a05521")
history = gi.histories.create_history(name="TestRandomLines")
dataset_id = gi.tools.paste_content("1\n2\n3",
history["id"])['outputs'][0]["id"]
from bioblend.galaxy.tools.inputs import inputs, dataset
random_lines_inputs = inputs().set("num_lines", 1).set("input",
dataset(dataset_id))
tool_outputs = gi.tools.run_tool(history_id=history["id"],
tool_id="random_lines1", tool_inputs=random_lines_inputs)
This seems very comparable to the two input tool execution you are describing.
Cleaner version of above script @
https://gist.github.com/jmchilton/38c33d365e116c750c21.
If this is not working, please make sure you are using a relatively
recent version of Galaxy (probably either of the last two releases
should work - one before that might as well) and the latest
development version of bioblend (right from github - don't know if a
full version has been released since I added tool inputs helpers).
If none of that works - please make sure the tool works through the
GUI and if it does can you share the tool XML with me?
Thanks,
-John
On Mon, Jun 30, 2014 at 9:49 AM, Olivia Doppelt-Azeroual
wrote:
> Hello John,
>
> Sorry to bother you again with this. I saw the many improvments added but I
> still can't run my tool using bioblend.
>
> The inputs names are 'input' and 'prefix' and the tool is very simple.
>
> The problem is still a mapping issue. Parameters are not assigned correctly
> in the command line. options are there but no values are there.
>
> Could you give me a use case, so I can mime it to my project ?
>
> Thanks in advance,
> --
> Olivia Doppelt-Azeroual
> CIB, Institut Pasteur, Paris
>
> Le 27/02/14 14:46, John Chilton a écrit :
>
>> I think you are right, and I am right, but you more so :). I bet the
>> inputs need to be dumped to json if files are being attached (i.e.
>> upload is being used) but should not be otherwise (because
>> content-type will be different). I wrote the client just to test
>> uploads - so this is probably the problem.
>>
>> I have created a Trello card (https://trello.com/c/67HsrgVm) to fix
>> bioblend to handle these details transparently and (per your other
>> email) to test and provide examples of using a repeat with data
>> parameters. I am certain the Galaxy API allows this - I am not sure
>> the bioblend tool client is quite there yet - it shouldn't be hard to
>> update it or provide examples - but I will need to find some time
>> which can be difficult.
>>
>> -John
>>
>> On Tue, Feb 25, 2014 at 4:41 AM, Olivia Doppelt-Azeroual
>> wrote:
>>>
>>> Dear John,
>>>
>>> I am doing tests on a local version of the last galaxy-dist version
>>> before
>>> updating our Galaxy server. And I have a question concernant the second
>>> remark you made in your last email.
>>>
>>> I've written a small script to test the run_tool function. When i put
>>> back
>>> the dumps(inputs_dict), the script fails whereas the type of my
>>> input_dict
>>> is a dictionnary.
>>>
>>> The script is the following:
>>>
>>> --
>>> from bioblend.galaxy import GalaxyInstance
>>> giLocal=GalaxyInstance('http://127.0.0.1:8080/',
>>> key='43359c833b8977b01d6237d9b839dc27')
>>> tool_inputs_loc={'input':'f597429621d6eb2b','name':"data"}
>>> print type(tool_inputs_loc)
>>>
>>> giLocal.tools.run_tool(history_id='1cd8e2f6b131e891',tool_id='export',tool_inputs=tool_inputs_loc)
>>> ---
>>>
>>> when i print the payload it is:
>>>
>>> {'tool_id': 'export', 'history_id': '1cd8e2f6b131e891', 'inputs':
>>> '{"input":
>>> "f597429621d6eb2b", "name": "data"}'}
>>>
>>> On the server side, the error is the following:
>>> --
>>> galaxy.web.framework ERROR 2014-02-25 11:35:34,865 Uncaught exception in
>>> exposed API method:
>>>
>>> Traceback (most recent call last):
>>>File
>>>
>>> "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/web/framework/__init__.py",
>>> line 200, in decorator
>>>
>>> rval = func( self, trans, *args, **kwargs)
>>>File
>>>
>>> "/home/odoppelt/localStuff_VM/tools/new_galaxy_disr/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py",
>>> line 104, in create
>>> for k, v in inputs.iteritems():
>>> AttributeError: 'unicode' object has no attribute 'iteritems'
>>>
>>>
>>>
>>> Do you have an idea ?
>>>
>>> Have a nice day,
>>> Thank you for your help,
>>> --
>>> Olivia
>>> Le 24/02/14 16:18, John Chilton a écrit :
>>>
Yes, the get_intial_value bug was a bug that has been fixed in a
recent Galaxy release (I think the mos
Re: [galaxy-dev] Run Jobs as Real User - How to Configure it for TORQUE
On Jul 22, 2014, at 2:13 PM, John Chilton wrote: > Running jobs as the "real" user is not available with the PBS job > runner - one has to use the DRMAA interface to submit jobs as the real > user. That said, the DRMAA runner plugin is compatible with Torque, you just need to compile and use pbs-drmaa, rather than Torque's libdrmaa: http://apps.man.poznan.pl/trac/pbs-drmaa --nate > > I have created a Trello card to add this functionality: > https://trello.com/c/OddS8bMP > > Would be happy to field pull requests to add this - because I doubt > anyone on the core team will be able to get to this anytime soon. > Sorry I don't have better news. > > -John > > On Sun, Jul 20, 2014 at 11:26 PM, Ping Luo wrote: >> I have installed pbs_python module on our cluster to interface Galaxy with >> TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. >> I need to run Galaxy jobs as the real user. The instruction in the user >> guide is for DRMAA interface. How can I configure running jobs as real user >> for TORQUE? >> >> thank, >> >> Ping >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Providing BLAST db in a data library
On Thu, Jul 24, 2014 at 2:50 PM, Nate Coraor wrote: > On Jul 23, 2014, at 6:42 AM, Peter Cock wrote: > >> Interesting hypothesis - you may well be right. >> >> Galaxy guys - who is the expert to talk to on this and/or where >> in the code should we be looking? >> >> Thanks, >> >> Peter > > I think there's a bit of a mixup here - Peter, I believe you were asking > if other composite types with an html primary dataset could be imported > from the history to library, but Ulf, your test was the other direction > (library->history). I'd be interested in knowing the outcome of the > history->library test as well. Good catch - yes, that was what I was asking about. Ulf? > I am woefully ignorant about the blastdbn datatype. Is the primary > file supposed to be html type but empty? The BLAST databases are 'basic' composite datatypes, of which the most commonly used example is HTML (and some bits of the base class code code seem to assume HTML). This means testing if something works with HTML is a good first step. https://github.com/peterjc/galaxy_blast/tree/master/datatypes/blast_datatypes Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Providing BLAST db in a data library
On Jul 23, 2014, at 6:42 AM, Peter Cock wrote: > Interesting hypothesis - you may well be right. > > Galaxy guys - who is the expert to talk to on this and/or where > in the code should we be looking? > > Thanks, > > Peter I think there's a bit of a mixup here - Peter, I believe you were asking if other composite types with an html primary dataset could be imported from the history to library, but Ulf, your test was the other direction (library->history). I'd be interested in knowing the outcome of the history->library test as well. I am woefully ignorant about the blastdbn datatype. Is the primary file supposed to be html type but empty? --nate > > On Wed, Jul 23, 2014 at 11:22 AM, Ulf Schaefer > wrote: >> Dear Peter >> >> Thanks for your reply. >> >> I can import an html report (e.g. FastQC output) successfully into a new >> history from a data library. But the .dat file for the html is not empty >> like the one for the blastdb. Makes me think that I could do this with a >> blast db as well, if only it would not check for size 0 at the time of >> importing it. >> >> Thanks >> Ulf > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to configure galaxy with a cluster
Hi Ben, Most of the tools you are looking for should be in the Galaxy Tool Shed: https://toolshed.g2.bx.psu.edu/ As for the earlier question regarding the resource selector drop down menu which I assume you are seeing on Galaxy Test, this is a feature currently available in the development version of Galaxy, and will be part of the next stable release: https://bitbucket.org/galaxy/galaxy-central/commits/31b2924315d0b48fa7648ab4bfd04ef754829f71 --nate On Jul 24, 2014, at 6:08 AM, 王渭巍 wrote: > Thank you, Bjoern > It turns out that I forgot start the scheduler, and now galaxy runs smoothly > with torque. > I am also trying to add more tools, some .xml config of the added tools can > be found in "tools", but some of them are missing, I don't know where are > they. I want to try do some customizing to the web interface of the tools. > Is there any guidline for me? Thanks a lot. > > Ciao, > Ben > > From: Björn Grüning > Date: 2014-07-22 16:15 > To: 王渭巍; Björn Grüning; galaxy-dev > Subject: Re: [galaxy-dev] How to configure galaxy with a cluster > Hi Ben, > > if the job is in waiting in the queue it's unlikely (not impossible) > that it is Galaxy fault. Can you recheck your Torque setup and how many > cores and memory your job has requested? > > Ciao, > Bjoern > > Am 22.07.2014 10:09, schrieb 王渭巍: > > Hi, Bjoern, > > I've tried the latest galaxy version with Torque 4.1.7, and it > > seems all right. But torque version > 4.2 won't work. > > And I tried to submit“fastqc readqc” jobs via torque (runner pbs), > > but the job is always in the queue waiting. I submited “fastqc > > readqc”local (runner local) , and the job finished successfully. So the > > question is , it seems not all the tools can be submitted via torque (or > > other resource manager), right? > > > > > > > > 王渭巍 > > > > From: Björn Grüning > > Date: 2014-07-21 01:23 > > To: 王渭巍; Björn Grüning; galaxy-dev > > Subject: Re: [galaxy-dev] How to configure galaxy with a cluster > > Hi Ben, > > > > sorry but we do not run a Torque setup. > > > > Do you have any concrete questions or error messages? > > > > Cheers, > > Bjoern > > > > Am 17.07.2014 04:10, schrieb 王渭巍: > >> Hi, Bjoern > >> Would you share your procedure to make some tools to run on a > >> cluster. > >> I have tried > >> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster using > >> Torque, but got errors. > >> I think maybe it's job_conf.xml. Would you share yours? Thanks > >> a lot > >> > >> Ben > >> > >> > >> From: Björn Grüning > >> Date: 2014-07-16 16:34 > >> To: 王渭巍; Thomas Bellembois; galaxy-dev > >> Subject: Re: [galaxy-dev] How to configure galaxy with a cluster > >> Hi Ben, > >> > >> that is not possible at the moment. The idea is to keep the > >> user-inferface as easy as possible for the user. You, as admin, can > >> decide which resource a specific tool with a specific input will use. > >> You will never see any options like that in a tool, but you can write a > >> tool by yourself if you like, or "enhance" the megablast tool. > >> > >> Cheers, > >> Bjoern > >> > >> > >> Am 16.07.2014 09:43, schrieb 王渭巍: > >>> Thanks a lot, Thomas! It really helps, I added tools section followed > >>> your suggestion... > >>> > >>> here is my job_conf.xml ( I am using Torque, I have 3 servers. One for > >>> galaxy server, two for cluster computing. ) > >>> > >>> > >>> > >>> > >>> >>> load="galaxy.jobs.runners.pbs:PBSJobRunner"/> > >>> > >>> > >>> > >>> > >>> > >>> walltime=72:00:00,nodes=1:ppn=8 > >>> 128 > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> and still no cluster options in "megablast" item. How can I see cluster > >>> options in the page, for example, the page will let me choose to use > >>> local server or a cluster. > >>> > >>> Ben > >>> > >>> > >>> > >>> From: Thomas Bellembois > >>> Date: 2014-07-15 17:41 > >>> To: galaxy-dev@lists.bx.psu.edu > >>> Subject: Re: [galaxy-dev] How to configure galaxy with a cluster > >>> Hello Ben, > >>> > >>> you can configure your Galaxy instance to use your cluster in the > >>> job_conf.xml file: > >>> > >>> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster > >>> > >>> You can set up your instance to use your cluster by default for all jobs > >>> or only for specific jobs. > >>> > >>> Here is a part of my job_conf.xml for example: > >>> > >>> > >>> > >>> >>> load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> > >>> > >>> > >>> >>> load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> -r yes -b n -cwd -S /bin/bash > >>> -V -pe galaxy 1 > >>> > >>> > >>> -r yes -b n -cwd -S /bin/bash > >>> -V -pe galaxy 12 > >>> > >>> > >>> > >>> > >>> > >>>
Re: [galaxy-dev] How to configure galaxy with a cluster
Thank you, Bjoern It turns out that I forgot start the scheduler, and now galaxy runs smoothly with torque. I am also trying to add more tools, some .xml config of the added tools can be found in "tools", but some of them are missing, I don't know where are they. I want to try do some customizing to the web interface of the tools. Is there any guidline for me? Thanks a lot. Ciao, Ben From: Björn Grüning Date: 2014-07-22 16:15 To: 王渭巍; Björn Grüning; galaxy-dev Subject: Re: [galaxy-dev] How to configure galaxy with a cluster Hi Ben, if the job is in waiting in the queue it's unlikely (not impossible) that it is Galaxy fault. Can you recheck your Torque setup and how many cores and memory your job has requested? Ciao, Bjoern Am 22.07.2014 10:09, schrieb 王渭巍: > Hi, Bjoern, > I've tried the latest galaxy version with Torque 4.1.7, and it seems > all right. But torque version > 4.2 won't work. > And I tried to submit“fastqc readqc” jobs via torque (runner pbs), > but the job is always in the queue waiting. I submited “fastqc readqc”local > (runner local) , and the job finished successfully. So the question is , it > seems not all the tools can be submitted via torque (or other resource > manager), right? > > > > 王渭巍 > > From: Björn Grüning > Date: 2014-07-21 01:23 > To: 王渭巍; Björn Grüning; galaxy-dev > Subject: Re: [galaxy-dev] How to configure galaxy with a cluster > Hi Ben, > > sorry but we do not run a Torque setup. > > Do you have any concrete questions or error messages? > > Cheers, > Bjoern > > Am 17.07.2014 04:10, schrieb 王渭巍: >> Hi, Bjoern >> Would you share your procedure to make some tools to run on a >> cluster. >> I have tried >> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster using >> Torque, but got errors. >> I think maybe it's job_conf.xml. Would you share yours? Thanks a >> lot >> >> Ben >> >> >> From: Björn Grüning >> Date: 2014-07-16 16:34 >> To: 王渭巍; Thomas Bellembois; galaxy-dev >> Subject: Re: [galaxy-dev] How to configure galaxy with a cluster >> Hi Ben, >> >> that is not possible at the moment. The idea is to keep the >> user-inferface as easy as possible for the user. You, as admin, can >> decide which resource a specific tool with a specific input will use. >> You will never see any options like that in a tool, but you can write a >> tool by yourself if you like, or "enhance" the megablast tool. >> >> Cheers, >> Bjoern >> >> >> Am 16.07.2014 09:43, schrieb 王渭巍: >>> Thanks a lot, Thomas! It really helps, I added tools section followed your >>> suggestion... >>> >>> here is my job_conf.xml ( I am using Torque, I have 3 servers. One for >>> galaxy server, two for cluster computing. ) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> walltime=72:00:00,nodes=1:ppn=8 >>> 128 >>> >>> >>> >>> >>> >>> >>> >>> and still no cluster options in "megablast" item. How can I see cluster >>> options in the page, for example, the page will let me choose to use local >>> server or a cluster. >>> >>> Ben >>> >>> >>> >>> From: Thomas Bellembois >>> Date: 2014-07-15 17:41 >>> To: galaxy-dev@lists.bx.psu.edu >>> Subject: Re: [galaxy-dev] How to configure galaxy with a cluster >>> Hello Ben, >>> >>> you can configure your Galaxy instance to use your cluster in the >>> job_conf.xml file: >>> >>> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster >>> >>> You can set up your instance to use your cluster by default for all jobs >>> or only for specific jobs. >>> >>> Here is a part of my job_conf.xml for example: >>> >>> >>> >>>>> load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> >>> >>> >>>>> load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -r yes -b n -cwd -S /bin/bash >>> -V -pe galaxy 1 >>> >>> >>> -r yes -b n -cwd -S /bin/bash >>> -V -pe galaxy 12 >>> >>> >>> >>> >>> >>> >>>>> id="toolshed.g2.bx.psu.edu/repos/bhaas/trinityrnaseq/trinityrnaseq/0.0.1" >>> destination="sge_big"/> >>> >>> >>>>> id="toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastp_wrapper/0.1.00" >>> destination="sge_big"/> >>>>> id="toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_tblastn_wrapper/0.1.00" >>> destination="sge_big"/> >>>>> id="toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastx_wrapper/0.1.00" >>> destination="sge_big"/> >>>>> id="toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastn_wrapper/0.1.00" >>> destination="sge_big"/> >>>>> id="toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_tblastx_wrapper/0.1.00" >>> destination="sge_big"/> >>>>> id="toolshed.g2.bx.psu.edu/repos/de
