Re: [galaxy-dev] [CONTENT] Re: Parallelism-enabled NCBI BLAST+ blastn does not appear to complete

2014-02-26 Thread Sanka, Ravi
Hi Peter,

This galaxy instance is connected to our SGE, and all jobs get run on it.
When we started this particular job, and it was the only job running, 4 of
the 10 tasks it was split to were able to run simultaneously, which we
viewed with qstat.

But after those four completed successfully, qstat revealed nothing
running, not even the original job. The Galaxy front-end interface still
displayed the job as running.

--
Ravi Sanka
ICS ­ Sr. Bioinformatics Engineer
J. Craig Venter Institute
301-795-7743
--




On 2/25/14 6:21 PM, "Peter Cock"  wrote:

>On Tue, Feb 25, 2014 at 10:57 PM, Sanka, Ravi  wrote:
>> Greetings,
>>
>> We are trying to enable job-splitting and merging on our Galaxy blast
>>tools.
>> We set the following parameters in the universe_wsgi.ini:
>>
>> use_tasked_jobs = True
>> local_task_queue_workers = 2
>>
>> Then we executed the tool "NCBI BLAST+ blastn" (from devteam's NCBI
>>BLAST+)
>> on a query set of 10,000 sequences. The tool's xml has the following
>> parallelism tag:
>>
>> And as expected, the job was split into 10 tasks (IDs 0-9), each one
>>with
>> 1000 sequences. Four tasks started running simultaneously.
>>
>> Those four tasks have completed, but now there is no indication that the
>> other 6 tasks have begun running, and the job on the GUI is still in the
>> yellow-state in History.
>>
>> It has been nearly an hour, and still no sign of those tasks or any job
>> running. What could be the cause of this?
>>
>
>Is this connected to a cluster (e.g. SGE), or all on one machine?
>
>We run this with a cluster, so you can use the cluster tools to
>monitor the sub-tasks Galaxy created.
>
>Peter
>
>RE: https://github.com/peterjc/galaxy_blast/issues/35 and
>http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] [CONTENT] Re: Parallelism-enabled NCBI BLAST+ blastn does not appear to complete

2014-02-26 Thread Peter Cock
Hmm.

This could be related to the problem Bjoern recently reported,
which Nate says was partly addressed in a recent fix:

http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-February/018367.html
http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-February/018382.html

What version of Galaxy do you have Ravi (i.e. what hg revision,
find out by running "hg summary" at the command line)?

Peter

On Wed, Feb 26, 2014 at 2:22 PM, Sanka, Ravi  wrote:
> Hi Peter,
>
> This galaxy instance is connected to our SGE, and all jobs get run on it.
> When we started this particular job, and it was the only job running, 4 of
> the 10 tasks it was split to were able to run simultaneously, which we
> viewed with qstat.
>
> But after those four completed successfully, qstat revealed nothing
> running, not even the original job. The Galaxy front-end interface still
> displayed the job as running.
>
> --
> Ravi Sanka
> ICS ­ Sr. Bioinformatics Engineer
> J. Craig Venter Institute
> 301-795-7743
> --
>
>
>
>
> On 2/25/14 6:21 PM, "Peter Cock"  wrote:
>
>>On Tue, Feb 25, 2014 at 10:57 PM, Sanka, Ravi  wrote:
>>> Greetings,
>>>
>>> We are trying to enable job-splitting and merging on our Galaxy blast
>>>tools.
>>> We set the following parameters in the universe_wsgi.ini:
>>>
>>> use_tasked_jobs = True
>>> local_task_queue_workers = 2
>>>
>>> Then we executed the tool "NCBI BLAST+ blastn" (from devteam's NCBI
>>>BLAST+)
>>> on a query set of 10,000 sequences. The tool's xml has the following
>>> parallelism tag:
>>>
>>> And as expected, the job was split into 10 tasks (IDs 0-9), each one
>>>with
>>> 1000 sequences. Four tasks started running simultaneously.
>>>
>>> Those four tasks have completed, but now there is no indication that the
>>> other 6 tasks have begun running, and the job on the GUI is still in the
>>> yellow-state in History.
>>>
>>> It has been nearly an hour, and still no sign of those tasks or any job
>>> running. What could be the cause of this?
>>>
>>
>>Is this connected to a cluster (e.g. SGE), or all on one machine?
>>
>>We run this with a cluster, so you can use the cluster tools to
>>monitor the sub-tasks Galaxy created.
>>
>>Peter
>>
>>RE: https://github.com/peterjc/galaxy_blast/issues/35 and
>>http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/