[galaxy-dev] Data Integration - Gencode Partition tool

[Hans-philipp Brachvogel]

Hello!

This is more or less a repost of (several) previously unanswered  
questions. Sadly it seems that one does generally not recieve an  
answer to this topic:


http://lists.bx.psu.edu/pipermail/galaxy-user/2011-October/003400.html
http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-October/007088.html
http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011478.html

So how do I properly set up the Gencode Partition tool (or the ENCODE tools)?

Another thing: the official galaxy instance seems to have the same problem:
The tool Gencode Partition returns the error:

"Error loading partitioning dataset."

which I also experience when using the the testing-bed file from  
galaxy-dist/test-data/encode_1.bed


I suppose the encode_1.bed file is not the problem, right?

Another thing is the helptext defined in the XML: it points to  
http://genome.imim.es/gencode/wiki/index.php/Collecting_Feature_Sets_from_All_Analysis_Groups

The wiki is actually closed by password...


Best,
Phil
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cleanup_datasets.py with postgresql 8.1

[Malcolm Tobias]

I've managed to resolve this issue.  I'm not sure how I ended up with this 
issue, but in case anyone else get's hit by it, here's what I've learned:

-At version 0.6, SQLAlchemy changed the format for connecting to postgresql:

[postgresql] The "postgres" dialect is now named "postgresql" 

-I somehow managed to get multiple eggs pointing to different versions of 
SQLAlchemy:

[galaxy@login002 galaxy-dist]$ ls eggs/SQLAlchemy-0.
SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/
SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/
SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg

-During my early stages of rolling out Galaxy, I managed to use the 0.7.9 
version of SQLAlchemy which forced me to change my database_connection syntax:

[galaxy@login002 galaxy-dist]$ ./run.sh --daemon 

  -this updates some of the "eggs"

  -it doesn't start; check the logs:

[galaxy@login002 galaxy-dist]$ tail paster.log
...
ile__', None ), self.config.database_engine_options, app=self )
  File "/home/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 107, 
in create_or_verify_database
% ( db_schema.version, migrate_repository.versions.latest, config_arg ) )
Exception: Your database has version '114' but this code expects version '115'. 
 Please backup your database and then migrate the schema by running 'sh 
manage_db.sh upgrade'.
Removing PID file paster.pid

[galaxy@login002 galaxy-dist]$ sh manage_db.sh upgrade
/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/url.py:105:
 SADeprecationWarning: The SQLAlchemy PostgreSQL dialect has been renamed from 
'postgres' to 'postgresql'. The new URL format is 
postgresql[+driver]://:@/

-at that point I changed my database_connection from:

database_connection = postgres://

to:

database_connection = postgresql://

and everything appeared to be happy.  Everything that is except for the cleanup 
script.

-I've now changed everything to: database_connection = postgres:// and 
everything appears to be happy.  


I really don't understand how this came out.  I was using postgresql:// for 
quite a while without issues, so it would seem that I could *not* have been 
using 0.5.6 of SQLAlchemy unless this was modified by the Galaxy community.  

Cheers,
Malcolm

On Thursday 26 September 2013 17:16:16 Jennifer Jackson wrote:
> Posting to galaxy-...@bx.psu.edu to give the 
> question better exposure to the development community.
> Please remove the galaxy-u...@bx.psu.edu 
> mailing list from all replies - we want to avoid double posts.
> 
> 
> For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>   http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> On 9/26/13 7:29 AM, Malcolm Tobias wrote:
> 
> All,
> 
> My understanding is that the cleanup_datasets.py under 
> scripts/cleanup_datasets should be compatible with 'older' versions of 
> postgresql. I'm running 8.1 under CentOS 5. When I attempt to run the 
> scripts, it fails to clean any data sets. From the logs, I'm noticing 
> messages like:
> 
> database_connection contains an unknown SQLAlchemy database dialect: 
> postgresql
> 
> In my universe_wsgi.ini I've defined the database_connection as:
> 
> database_connection = postgresql://galaxy:@localhost:5432/galaxydb
> 
> and am confident it's working as Galaxy works fine, other than this problem 
> with deleting older datasets.
> 
> The pgcleanup.py script isn't an option, as it requires postgresql >=9.1. I 
> noticed yet another version of cleanup_datasets.py under cron which looks 
> like it might be compatible with postgres, but am leary of trying it as I'm 
> not sure what it means by "1: database directory to clean". I'm not sure what 
> directory to point this to, nor am I confident that just deleting data from 
> the directory is the safest thing.
> 
> Any suggestions on how to proceed?
> 
> Thanks!
> 
> Malcolm
> 
> --
> 
> Malcolm Tobias
> 
> 314.362.1594
> 
> 
> 
> ___
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org.  Please keep all replies on the list by
> using "reply all" in your mail client.  For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>   http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
> 
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
> 
>   http://galaxyproject.org/search/mailinglists/
> 
> 
> --
> Jennifer Hillman-Jackson
> http://galaxyproject.org
> 

-- 
Malcolm Tobias
314.362.1594

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Gal

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[John Chilton]

Hey All,

Patch available with test case, let me know what I broke :).

https://bitbucket.org/jmchilton/galaxy-central-multi-input-tool-fixes-2/commits/e0016057e1648d7f4d29fbaf14252cbefff6d0bd

-John


On Fri, Oct 11, 2013 at 10:37 AM, Björn Grüning
 wrote:
> Am Freitag, den 11.10.2013, 09:18 -0500 schrieb John Chilton:
>> My plan is to refactor build_command_line out of
>> lib/galaxy/jobs/runners/__init__.py, into its own module and add unit
>> tests for this functionality. I'll issue a pull request and let people
>> test it. I am also testing Nicola's PR, I will approve it shortly.
>>
>> Nate yelled at me for committing code yesterday :) so I will let him
>> figure out when to merge it. But I will approve the pull requests that
>> I am willing to say are tested and that I will take responsibility for
>> if something breaks in them.
>
> Ok, sounds good!
>
> Thanks!
> Bjoern
>
>> -John
>>
>> On Fri, Oct 11, 2013 at 9:13 AM, Björn Grüning
>>  wrote:
>> > Ah now I got it ... there are two build_command_line functions.
>> > lib/galaxy/tools/__init__.py and lib/galaxy/jobs/runners/__init__.py.
>> >
>> > And one is adding a trailing ';'
>> >
>> > So either stripping the trailing semicolon, right after creating the
>> > command line from cheetah (patch was attached) or adding more logic to
>> > only add a ';' when no one is there.
>> >
>> > Which one is preferred, should I create a pull request?
>> >
>> > Thanks!
>> > Bjoern
>> >
>> >
>> >> If the tool produces a trailing semi-colon (and without Nicola's patch
>> >> the bowtie 2 tool definitely could), then I bet build_command_line()
>> >> is going to be what is adding the second semi-colon. It has not check
>> >> that there isn't a semi-colon at the end of the command before adding
>> >> one and extending the command. It might not be the only culprit, but
>> >> it is likely one.
>> >>
>> >> -John
>> >>
>> >> On Fri, Oct 11, 2013 at 9:02 AM, Björn Grüning
>> >>  wrote:
>> >> > Hi Nate,
>> >> >
>> >> >> Hi Björn,
>> >> >>
>> >> >> Are you certain that SGE is responsible?  I ran in to the same thing 
>> >> >> with bowtie2 with other runners.
>> >> >> The double semicolon is invalid syntax in Bourne-compatible shells and 
>> >> >> so the tool ought to be fixed not to produce them (it happens under 
>> >> >> certain combinations of options).
>> >> >> Have you noticed this happening with any other tools?
>> >> >
>> >> > Yes, but only with tools from the toolshed and there I poked people to
>> >> > remove the trailing semicolon from the wrapper. From my observation the
>> >> > tool does not produce double semicolons. Only one at the end, but the
>> >> > Job scheduler adds one, resulting in a double one with invalid syntax.
>> >> >
>> >> > I can't see where bowtie is producing a double semicolon. I really think
>> >> > SGE is adding one. At least I restructured the semicolons (attached) and
>> >> > it worked for us.
>> >> >
>> >> > Thanks Nate,
>> >> > Bjoern
>> >> >
>> >> >> --nate
>> >> >>
>> >> >> On Oct 11, 2013, at 8:57 AM, John Chilton wrote:
>> >> >>
>> >> >> > This sounds like a good way to solve the problem, I guess I would 
>> >> >> > modify
>> >> >> >
>> >> >> >
>> >> >> > On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
>> >> >> >  wrote:
>> >> >> >>
>> >> >> >> Dear all,
>> >> >> >>
>> >> >> >> some SGE instances will add automatically an semicolon add the end 
>> >> >> >> of
>> >> >> >> each command, resulting in a disrupted job, because ';;' are not
>> >> >> >> allowed.
>> >> >> >>
>> >> >> >> The latest changes to the Bowtie2 wrapper resulting in such a case 
>> >> >> >> and a
>> >> >> >> crash in our instance. An easy solution would be to fix the wrapper 
>> >> >> >> and
>> >> >> >> omitting the trailing ';' but maybe its better to fix it once for 
>> >> >> >> all in
>> >> >> >> the corresponding tools code, patched attached.
>> >> >> >>
>> >> >> >> I'm not familiar with other Job schedulers so I'm seeking for
>> >> >> >> comments ... it there any disadvantage of removing trailing ';' 
>> >> >> >> from the
>> >> >> >> command line?
>> >> >> >>
>> >> >> >> Ciao,
>> >> >> >> Bjoern
>> >> >> >>
>> >> >> >>
>> >> >> >> ___
>> >> >> >> Please keep all replies on the list by using "reply all"
>> >> >> >> in your mail client.  To manage your subscriptions to this
>> >> >> >> and other Galaxy lists, please use the interface at:
>> >> >> >>  http://lists.bx.psu.edu/
>> >> >> >>
>> >> >> >> To search Galaxy mailing lists use the unified search at:
>> >> >> >>  http://galaxyproject.org/search/mailinglists/
>> >> >> >
>> >> >> > ___
>> >> >> > Please keep all replies on the list by using "reply all"
>> >> >> > in your mail client.  To manage your subscriptions to this
>> >> >> > and other Galaxy lists, please use the interface at:
>> >> >> >  http://lists.bx.psu.edu/
>> >> >> >
>> >> >> > To search Galaxy mailing lists use the unified search at:
>> >> >> >

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[Björn Grüning]

Am Freitag, den 11.10.2013, 09:18 -0500 schrieb John Chilton:
> My plan is to refactor build_command_line out of
> lib/galaxy/jobs/runners/__init__.py, into its own module and add unit
> tests for this functionality. I'll issue a pull request and let people
> test it. I am also testing Nicola's PR, I will approve it shortly.
> 
> Nate yelled at me for committing code yesterday :) so I will let him
> figure out when to merge it. But I will approve the pull requests that
> I am willing to say are tested and that I will take responsibility for
> if something breaks in them.

Ok, sounds good!

Thanks!
Bjoern

> -John
> 
> On Fri, Oct 11, 2013 at 9:13 AM, Björn Grüning
>  wrote:
> > Ah now I got it ... there are two build_command_line functions.
> > lib/galaxy/tools/__init__.py and lib/galaxy/jobs/runners/__init__.py.
> >
> > And one is adding a trailing ';'
> >
> > So either stripping the trailing semicolon, right after creating the
> > command line from cheetah (patch was attached) or adding more logic to
> > only add a ';' when no one is there.
> >
> > Which one is preferred, should I create a pull request?
> >
> > Thanks!
> > Bjoern
> >
> >
> >> If the tool produces a trailing semi-colon (and without Nicola's patch
> >> the bowtie 2 tool definitely could), then I bet build_command_line()
> >> is going to be what is adding the second semi-colon. It has not check
> >> that there isn't a semi-colon at the end of the command before adding
> >> one and extending the command. It might not be the only culprit, but
> >> it is likely one.
> >>
> >> -John
> >>
> >> On Fri, Oct 11, 2013 at 9:02 AM, Björn Grüning
> >>  wrote:
> >> > Hi Nate,
> >> >
> >> >> Hi Björn,
> >> >>
> >> >> Are you certain that SGE is responsible?  I ran in to the same thing 
> >> >> with bowtie2 with other runners.
> >> >> The double semicolon is invalid syntax in Bourne-compatible shells and 
> >> >> so the tool ought to be fixed not to produce them (it happens under 
> >> >> certain combinations of options).
> >> >> Have you noticed this happening with any other tools?
> >> >
> >> > Yes, but only with tools from the toolshed and there I poked people to
> >> > remove the trailing semicolon from the wrapper. From my observation the
> >> > tool does not produce double semicolons. Only one at the end, but the
> >> > Job scheduler adds one, resulting in a double one with invalid syntax.
> >> >
> >> > I can't see where bowtie is producing a double semicolon. I really think
> >> > SGE is adding one. At least I restructured the semicolons (attached) and
> >> > it worked for us.
> >> >
> >> > Thanks Nate,
> >> > Bjoern
> >> >
> >> >> --nate
> >> >>
> >> >> On Oct 11, 2013, at 8:57 AM, John Chilton wrote:
> >> >>
> >> >> > This sounds like a good way to solve the problem, I guess I would 
> >> >> > modify
> >> >> >
> >> >> >
> >> >> > On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
> >> >> >  wrote:
> >> >> >>
> >> >> >> Dear all,
> >> >> >>
> >> >> >> some SGE instances will add automatically an semicolon add the end of
> >> >> >> each command, resulting in a disrupted job, because ';;' are not
> >> >> >> allowed.
> >> >> >>
> >> >> >> The latest changes to the Bowtie2 wrapper resulting in such a case 
> >> >> >> and a
> >> >> >> crash in our instance. An easy solution would be to fix the wrapper 
> >> >> >> and
> >> >> >> omitting the trailing ';' but maybe its better to fix it once for 
> >> >> >> all in
> >> >> >> the corresponding tools code, patched attached.
> >> >> >>
> >> >> >> I'm not familiar with other Job schedulers so I'm seeking for
> >> >> >> comments ... it there any disadvantage of removing trailing ';' from 
> >> >> >> the
> >> >> >> command line?
> >> >> >>
> >> >> >> Ciao,
> >> >> >> Bjoern
> >> >> >>
> >> >> >>
> >> >> >> ___
> >> >> >> Please keep all replies on the list by using "reply all"
> >> >> >> in your mail client.  To manage your subscriptions to this
> >> >> >> and other Galaxy lists, please use the interface at:
> >> >> >>  http://lists.bx.psu.edu/
> >> >> >>
> >> >> >> To search Galaxy mailing lists use the unified search at:
> >> >> >>  http://galaxyproject.org/search/mailinglists/
> >> >> >
> >> >> > ___
> >> >> > Please keep all replies on the list by using "reply all"
> >> >> > in your mail client.  To manage your subscriptions to this
> >> >> > and other Galaxy lists, please use the interface at:
> >> >> >  http://lists.bx.psu.edu/
> >> >> >
> >> >> > To search Galaxy mailing lists use the unified search at:
> >> >> >  http://galaxyproject.org/search/mailinglists/
> >> >> >
> >> >>
> >> >
> >> >
> >
> >
> >



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the un

Re: [galaxy-dev] Jobs deleted staying in 'dr' status

[Sytchev, Ilya]

On 9/12/13 10:35 AM, "Peter Cock"  wrote:

>On Thu, Sep 12, 2013 at 2:01 PM, Mathieu Bahin 
>wrote:
>> Hi all,
>>
>> We have been developing our own Galaxy instance for a while now. We
>>have a
>> cluster on which the job are sent to be executed, it is managed through
>>SGE.
>> Usually, communication between SGE and DRMAA is ok and we don't have any
>> problem with that.
>>
>> When a job is deleted by the user, most of the times, the job
>>disappears but
>> sometimes, we don't know why, the job stays and has the status 'dr'
>>within
>> SGE. If we don't kill it 'manually', it stays forever. It is not always
>>the
>> same tools which produces this error.
>> Have you any idea why how manage it ?
>
>I have noticed problem with our DRMMA/SGE setup where a
>user can cancel a large job (using the job splitter in at least some
>cases), but Galaxy does not seem to cancel the jobs on the cluster.
>I've not tried to diagnose this yet - it could be a similar issue though.

Also, in our DRMAA/LSF setup (using a fork of the latest galaxy-dist) jobs
generated by the current workflow step continue running on the cluster
after history is deleted.

Ilya


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Solenopsis invicta genome

[Jennifer Jackson]

Hi Patrick,

I think this is worth consideringand will run by team. Genomes are a bit 
on hold right now due to the move of the public server, but once that 
resumes, I have a ticket going with public requests in it. Would you 
please send me the details for this organism: exact accession(s), 
special build names/links, and such. You can send to me direct.
This is the ticket I'll be adding the info to, please don't add 
yourself, this is just so you can follow:

https://trello.com/c/kzVklAIE
(There is much more data in pipe, including genomes, tracked elsewhere).

Meanwhile, you don't need to wait (or want to!), to use the genome right 
now, do this as a _custom reference genome_ with tools. All you need is 
a fasta copy of the genome in your history. Pay attention to the 
"chromosome" identifiers and any identifiers in related GTF/BED files 
you intend to incorporate into your analysis - these almost without 
exception have to be /exactly the same /to function in downstream tool. 
Here's the how-to:

http://wiki.galaxyproject.org/Support#Custom_reference_genome
http://vimeo.com/galaxyproject/customgenome
Note that the FTP server is now: "usegalaxy.org" instead of any server 
mentioned in the films, as noted on the "Upload Data" tool form.


Thanks,

Jen
Galaxy team



On 10/11/13 5:40 AM, patrick.j...@unil.ch wrote:

Hello,

I'm working in the university of Lausanne. We are currently working 
with fire ants, Solenopsis invicta.
I was wondering if that would possible to upload Solenopsis invicta's 
genome on the Galaxy platform, which could be very useful to us and

certainly to other people.

Thank you !

Patrick Joye


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


--
Jennifer Hillman-Jackson
http://galaxyproject.org

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] masthead and history lost when embedding Galaxy in iframe

[Carl Eberhard]

Heya, Andy

Galaxy hasn't been designed to be loaded in an iframe yet.

Currently, the javascript in Galaxy makes use of the window variable top -
which points to the highest browser window in the DOM. There are many
instances where we rely on both 'top' and the link target '_top' to control
the iframes used inside the platform.

In your implementation, top won't point to the Galaxy frame - but to the
frame you've created. This can lead to pages/features that won't load (and
fail silently unfortunately).

We are currently working to remove our use of iframes within the platform
which will help with embedding Galaxy into a frame.

In the meantime, you may be able to alter those references to top to point
to parent (which is context dependent) or to a named frame (naming the
outer Galaxy frame and using that as a link target and replacement for top).


Hope that helps.
(with apologies to Andy for missing the mailing list in my first reply)


On Thu, Oct 10, 2013 at 7:56 AM, Andy Saurin wrote:

> Hi all,
>
> I really can't understand why this should be, but in recent versions of
> Galaxy if you have a simple html page with an iframe embedding Galaxy, you
> loose the masthead (no login, workflow etc) and the history panel no longer
> works.
>
> eg (newer Galaxy Versions)
>
> https://galaxy.cbio.mskcc.org/"; width="800px"
> height="600px">
> https://galaxy.cbio.mskcc.org/"; width="800px"
> height="600px">
>
> You can see the problems, whereas older versions of galaxy show no
> problems:
>
> http://galaxy.berkeleybop.org:8080/"; width="800px"
> height="600px">
>
>
> What could be causing this so that I can try to make amends?
>
> Thanks
> Andy
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Solenopsis invicta genome

[patrick . joye]

Hello,

I'm working in the university of Lausanne. We are currently working 
with fire ants, Solenopsis invicta.
I was wondering if that would possible to upload Solenopsis invicta's 
genome on the Galaxy platform, which could be very useful to us and

certainly to other people.

Thank you !

Patrick Joye


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[Nicola Soranzo]

I already fixed it one week ago:

https://bitbucket.org/nsoranzo/galaxy-central-tools/commits/74b5fd2318d33b76382b052eae6a28b035dc075f

but then Ross committed a similar fix to galaxy-central, I merged it in 
my pull request and then Ross reverted the fix!

This is the reason for rebasing pull request #222 to #234.

Ciao,
Nicola

Il 2013-10-11 16:05 Björn Grüning ha scritto:

Hi Nicola,

interesting you fixed bowtie in the same way than we did. But somehow 
I

would like to strip a trailing ';' by default in Galaxy, when it does
not break anything else.

Thanks for sharing!
Bjoern

Il giorno ven, 11/10/2013 alle 01.10 +0200, Björn Grüning ha 
scritto:

> Dear all,
>
> some SGE instances will add automatically an semicolon add the end 
of

> each command, resulting in a disrupted job, because ';;' are not
> allowed.
>
> The latest changes to the Bowtie2 wrapper resulting in such a case 
and a
> crash in our instance. An easy solution would be to fix the 
wrapper and
> omitting the trailing ';' but maybe its better to fix it once for 
all in

> the corresponding tools code, patched attached.

The Bowtie2 bug may be fixed if my pull request 234 will be merged:


https://bitbucket.org/galaxy/galaxy-central/pull-request/234/bug-fixes-for-3-tools-rebased-222

> I'm not familiar with other Job schedulers so I'm seeking for
> comments ... it there any disadvantage of removing trailing ';' 
from the

> command line?

That would be sensible too.

Ciao,
Nicola



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[Björn Grüning]

Hi Nate,

> Hi Björn,
> 
> Are you certain that SGE is responsible?  I ran in to the same thing with 
> bowtie2 with other runners.  
> The double semicolon is invalid syntax in Bourne-compatible shells and so the 
> tool ought to be fixed not to produce them (it happens under certain 
> combinations of options).  
> Have you noticed this happening with any other tools?

Yes, but only with tools from the toolshed and there I poked people to
remove the trailing semicolon from the wrapper. From my observation the
tool does not produce double semicolons. Only one at the end, but the
Job scheduler adds one, resulting in a double one with invalid syntax.

I can't see where bowtie is producing a double semicolon. I really think
SGE is adding one. At least I restructured the semicolons (attached) and
it worked for us.

Thanks Nate,
Bjoern

> --nate
> 
> On Oct 11, 2013, at 8:57 AM, John Chilton wrote:
> 
> > This sounds like a good way to solve the problem, I guess I would modify
> > 
> > 
> > On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
> >  wrote:
> >> 
> >> Dear all,
> >> 
> >> some SGE instances will add automatically an semicolon add the end of
> >> each command, resulting in a disrupted job, because ';;' are not
> >> allowed.
> >> 
> >> The latest changes to the Bowtie2 wrapper resulting in such a case and a
> >> crash in our instance. An easy solution would be to fix the wrapper and
> >> omitting the trailing ';' but maybe its better to fix it once for all in
> >> the corresponding tools code, patched attached.
> >> 
> >> I'm not familiar with other Job schedulers so I'm seeking for
> >> comments ... it there any disadvantage of removing trailing ';' from the
> >> command line?
> >> 
> >> Ciao,
> >> Bjoern
> >> 
> >> 
> >> ___
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>  http://lists.bx.psu.edu/
> >> 
> >> To search Galaxy mailing lists use the unified search at:
> >>  http://galaxyproject.org/search/mailinglists/
> > 
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >  http://lists.bx.psu.edu/
> > 
> > To search Galaxy mailing lists use the unified search at:
> >  http://galaxyproject.org/search/mailinglists/
> > 
> 




bowtie2_wrapper.xml
Description: XML document
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[John Chilton]

My plan is to refactor build_command_line out of
lib/galaxy/jobs/runners/__init__.py, into its own module and add unit
tests for this functionality. I'll issue a pull request and let people
test it. I am also testing Nicola's PR, I will approve it shortly.

Nate yelled at me for committing code yesterday :) so I will let him
figure out when to merge it. But I will approve the pull requests that
I am willing to say are tested and that I will take responsibility for
if something breaks in them.

-John

On Fri, Oct 11, 2013 at 9:13 AM, Björn Grüning
 wrote:
> Ah now I got it ... there are two build_command_line functions.
> lib/galaxy/tools/__init__.py and lib/galaxy/jobs/runners/__init__.py.
>
> And one is adding a trailing ';'
>
> So either stripping the trailing semicolon, right after creating the
> command line from cheetah (patch was attached) or adding more logic to
> only add a ';' when no one is there.
>
> Which one is preferred, should I create a pull request?
>
> Thanks!
> Bjoern
>
>
>> If the tool produces a trailing semi-colon (and without Nicola's patch
>> the bowtie 2 tool definitely could), then I bet build_command_line()
>> is going to be what is adding the second semi-colon. It has not check
>> that there isn't a semi-colon at the end of the command before adding
>> one and extending the command. It might not be the only culprit, but
>> it is likely one.
>>
>> -John
>>
>> On Fri, Oct 11, 2013 at 9:02 AM, Björn Grüning
>>  wrote:
>> > Hi Nate,
>> >
>> >> Hi Björn,
>> >>
>> >> Are you certain that SGE is responsible?  I ran in to the same thing with 
>> >> bowtie2 with other runners.
>> >> The double semicolon is invalid syntax in Bourne-compatible shells and so 
>> >> the tool ought to be fixed not to produce them (it happens under certain 
>> >> combinations of options).
>> >> Have you noticed this happening with any other tools?
>> >
>> > Yes, but only with tools from the toolshed and there I poked people to
>> > remove the trailing semicolon from the wrapper. From my observation the
>> > tool does not produce double semicolons. Only one at the end, but the
>> > Job scheduler adds one, resulting in a double one with invalid syntax.
>> >
>> > I can't see where bowtie is producing a double semicolon. I really think
>> > SGE is adding one. At least I restructured the semicolons (attached) and
>> > it worked for us.
>> >
>> > Thanks Nate,
>> > Bjoern
>> >
>> >> --nate
>> >>
>> >> On Oct 11, 2013, at 8:57 AM, John Chilton wrote:
>> >>
>> >> > This sounds like a good way to solve the problem, I guess I would modify
>> >> >
>> >> >
>> >> > On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
>> >> >  wrote:
>> >> >>
>> >> >> Dear all,
>> >> >>
>> >> >> some SGE instances will add automatically an semicolon add the end of
>> >> >> each command, resulting in a disrupted job, because ';;' are not
>> >> >> allowed.
>> >> >>
>> >> >> The latest changes to the Bowtie2 wrapper resulting in such a case and 
>> >> >> a
>> >> >> crash in our instance. An easy solution would be to fix the wrapper and
>> >> >> omitting the trailing ';' but maybe its better to fix it once for all 
>> >> >> in
>> >> >> the corresponding tools code, patched attached.
>> >> >>
>> >> >> I'm not familiar with other Job schedulers so I'm seeking for
>> >> >> comments ... it there any disadvantage of removing trailing ';' from 
>> >> >> the
>> >> >> command line?
>> >> >>
>> >> >> Ciao,
>> >> >> Bjoern
>> >> >>
>> >> >>
>> >> >> ___
>> >> >> Please keep all replies on the list by using "reply all"
>> >> >> in your mail client.  To manage your subscriptions to this
>> >> >> and other Galaxy lists, please use the interface at:
>> >> >>  http://lists.bx.psu.edu/
>> >> >>
>> >> >> To search Galaxy mailing lists use the unified search at:
>> >> >>  http://galaxyproject.org/search/mailinglists/
>> >> >
>> >> > ___
>> >> > Please keep all replies on the list by using "reply all"
>> >> > in your mail client.  To manage your subscriptions to this
>> >> > and other Galaxy lists, please use the interface at:
>> >> >  http://lists.bx.psu.edu/
>> >> >
>> >> > To search Galaxy mailing lists use the unified search at:
>> >> >  http://galaxyproject.org/search/mailinglists/
>> >> >
>> >>
>> >
>> >
>
>
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[Björn Grüning]

Ah now I got it ... there are two build_command_line functions.
lib/galaxy/tools/__init__.py and lib/galaxy/jobs/runners/__init__.py.

And one is adding a trailing ';'

So either stripping the trailing semicolon, right after creating the
command line from cheetah (patch was attached) or adding more logic to
only add a ';' when no one is there.

Which one is preferred, should I create a pull request?

Thanks!
Bjoern 


> If the tool produces a trailing semi-colon (and without Nicola's patch
> the bowtie 2 tool definitely could), then I bet build_command_line()
> is going to be what is adding the second semi-colon. It has not check
> that there isn't a semi-colon at the end of the command before adding
> one and extending the command. It might not be the only culprit, but
> it is likely one.
> 
> -John
> 
> On Fri, Oct 11, 2013 at 9:02 AM, Björn Grüning
>  wrote:
> > Hi Nate,
> >
> >> Hi Björn,
> >>
> >> Are you certain that SGE is responsible?  I ran in to the same thing with 
> >> bowtie2 with other runners.
> >> The double semicolon is invalid syntax in Bourne-compatible shells and so 
> >> the tool ought to be fixed not to produce them (it happens under certain 
> >> combinations of options).
> >> Have you noticed this happening with any other tools?
> >
> > Yes, but only with tools from the toolshed and there I poked people to
> > remove the trailing semicolon from the wrapper. From my observation the
> > tool does not produce double semicolons. Only one at the end, but the
> > Job scheduler adds one, resulting in a double one with invalid syntax.
> >
> > I can't see where bowtie is producing a double semicolon. I really think
> > SGE is adding one. At least I restructured the semicolons (attached) and
> > it worked for us.
> >
> > Thanks Nate,
> > Bjoern
> >
> >> --nate
> >>
> >> On Oct 11, 2013, at 8:57 AM, John Chilton wrote:
> >>
> >> > This sounds like a good way to solve the problem, I guess I would modify
> >> >
> >> >
> >> > On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
> >> >  wrote:
> >> >>
> >> >> Dear all,
> >> >>
> >> >> some SGE instances will add automatically an semicolon add the end of
> >> >> each command, resulting in a disrupted job, because ';;' are not
> >> >> allowed.
> >> >>
> >> >> The latest changes to the Bowtie2 wrapper resulting in such a case and a
> >> >> crash in our instance. An easy solution would be to fix the wrapper and
> >> >> omitting the trailing ';' but maybe its better to fix it once for all in
> >> >> the corresponding tools code, patched attached.
> >> >>
> >> >> I'm not familiar with other Job schedulers so I'm seeking for
> >> >> comments ... it there any disadvantage of removing trailing ';' from the
> >> >> command line?
> >> >>
> >> >> Ciao,
> >> >> Bjoern
> >> >>
> >> >>
> >> >> ___
> >> >> Please keep all replies on the list by using "reply all"
> >> >> in your mail client.  To manage your subscriptions to this
> >> >> and other Galaxy lists, please use the interface at:
> >> >>  http://lists.bx.psu.edu/
> >> >>
> >> >> To search Galaxy mailing lists use the unified search at:
> >> >>  http://galaxyproject.org/search/mailinglists/
> >> >
> >> > ___
> >> > Please keep all replies on the list by using "reply all"
> >> > in your mail client.  To manage your subscriptions to this
> >> > and other Galaxy lists, please use the interface at:
> >> >  http://lists.bx.psu.edu/
> >> >
> >> > To search Galaxy mailing lists use the unified search at:
> >> >  http://galaxyproject.org/search/mailinglists/
> >> >
> >>
> >
> >



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[John Chilton]

If the tool produces a trailing semi-colon (and without Nicola's patch
the bowtie 2 tool definitely could), then I bet build_command_line()
is going to be what is adding the second semi-colon. It has not check
that there isn't a semi-colon at the end of the command before adding
one and extending the command. It might not be the only culprit, but
it is likely one.

-John

On Fri, Oct 11, 2013 at 9:02 AM, Björn Grüning
 wrote:
> Hi Nate,
>
>> Hi Björn,
>>
>> Are you certain that SGE is responsible?  I ran in to the same thing with 
>> bowtie2 with other runners.
>> The double semicolon is invalid syntax in Bourne-compatible shells and so 
>> the tool ought to be fixed not to produce them (it happens under certain 
>> combinations of options).
>> Have you noticed this happening with any other tools?
>
> Yes, but only with tools from the toolshed and there I poked people to
> remove the trailing semicolon from the wrapper. From my observation the
> tool does not produce double semicolons. Only one at the end, but the
> Job scheduler adds one, resulting in a double one with invalid syntax.
>
> I can't see where bowtie is producing a double semicolon. I really think
> SGE is adding one. At least I restructured the semicolons (attached) and
> it worked for us.
>
> Thanks Nate,
> Bjoern
>
>> --nate
>>
>> On Oct 11, 2013, at 8:57 AM, John Chilton wrote:
>>
>> > This sounds like a good way to solve the problem, I guess I would modify
>> >
>> >
>> > On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
>> >  wrote:
>> >>
>> >> Dear all,
>> >>
>> >> some SGE instances will add automatically an semicolon add the end of
>> >> each command, resulting in a disrupted job, because ';;' are not
>> >> allowed.
>> >>
>> >> The latest changes to the Bowtie2 wrapper resulting in such a case and a
>> >> crash in our instance. An easy solution would be to fix the wrapper and
>> >> omitting the trailing ';' but maybe its better to fix it once for all in
>> >> the corresponding tools code, patched attached.
>> >>
>> >> I'm not familiar with other Job schedulers so I'm seeking for
>> >> comments ... it there any disadvantage of removing trailing ';' from the
>> >> command line?
>> >>
>> >> Ciao,
>> >> Bjoern
>> >>
>> >>
>> >> ___
>> >> Please keep all replies on the list by using "reply all"
>> >> in your mail client.  To manage your subscriptions to this
>> >> and other Galaxy lists, please use the interface at:
>> >>  http://lists.bx.psu.edu/
>> >>
>> >> To search Galaxy mailing lists use the unified search at:
>> >>  http://galaxyproject.org/search/mailinglists/
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >  http://lists.bx.psu.edu/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >  http://galaxyproject.org/search/mailinglists/
>> >
>>
>
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[Björn Grüning]

Hi Nicola,

interesting you fixed bowtie in the same way than we did. But somehow I
would like to strip a trailing ';' by default in Galaxy, when it does
not break anything else.

Thanks for sharing!
Bjoern

> Il giorno ven, 11/10/2013 alle 01.10 +0200, Björn Grüning ha scritto: 
> > Dear all,
> > 
> > some SGE instances will add automatically an semicolon add the end of
> > each command, resulting in a disrupted job, because ';;' are not
> > allowed.
> > 
> > The latest changes to the Bowtie2 wrapper resulting in such a case and a
> > crash in our instance. An easy solution would be to fix the wrapper and
> > omitting the trailing ';' but maybe its better to fix it once for all in
> > the corresponding tools code, patched attached.
> 
> The Bowtie2 bug may be fixed if my pull request 234 will be merged:
> 
> https://bitbucket.org/galaxy/galaxy-central/pull-request/234/bug-fixes-for-3-tools-rebased-222
> 
> > I'm not familiar with other Job schedulers so I'm seeking for
> > comments ... it there any disadvantage of removing trailing ';' from the
> > command line?
> 
> That would be sensible too.
> 
> Ciao,
> Nicola
> 



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[Nicola Soranzo]

Il giorno ven, 11/10/2013 alle 01.10 +0200, Björn Grüning ha scritto: 
> Dear all,
> 
> some SGE instances will add automatically an semicolon add the end of
> each command, resulting in a disrupted job, because ';;' are not
> allowed.
> 
> The latest changes to the Bowtie2 wrapper resulting in such a case and a
> crash in our instance. An easy solution would be to fix the wrapper and
> omitting the trailing ';' but maybe its better to fix it once for all in
> the corresponding tools code, patched attached.

The Bowtie2 bug may be fixed if my pull request 234 will be merged:

https://bitbucket.org/galaxy/galaxy-central/pull-request/234/bug-fixes-for-3-tools-rebased-222

> I'm not familiar with other Job schedulers so I'm seeking for
> comments ... it there any disadvantage of removing trailing ';' from the
> command line?

That would be sensible too.

Ciao,
Nicola

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Error when running paster directly

[James Taylor]

The paster script distributed with Galaxy is a stripped down Galaxy
specific version. It only supports the command serve and the arguments
--daemon or --reload.

--
James Taylor, Associate Professor, Biology/CS, Emory University


On Fri, Oct 11, 2013 at 8:36 AM, Renato Alves  wrote:
> Hi everyone,
>
> When trying to adapt the init script for debian machines (provided in
> contrib/) I started playing with paster directly. I was trying to get
> some sort of help or list of commands/params that can be passed but
> instead I was presented with the following output:
>
> $ python ./scripts/paster.py
>
> Usage: ./scripts/paster.py COMMAND
> Command 'help' not known (you may need to run setup.py egg_info)
> Traceback (most recent call last):
>   File "./scripts/paster.py", line 33, in 
> serve.run()
>   File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py",
> line 1049, in run
> invoke(command, command_name, options, args[1:])
>   File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py",
> line 1055, in invoke
> exit_code = runner.run(args)
>   File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py",
> line 367, in run
> commands = get_commands().items()
> NameError: global name 'get_commands' is not defined
>
> After inspection of the file, the get_commands function doesn't seem to
> be defined nor imported.
>
> I'm not sure how to proceed at this point.
>
> Thanks,
> Renato
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Fastx toolkit broken pipe

[UMD Bioinformatics]

Bjöern,

Here is the error message. 

An error occurred with this dataset:/bin/sh: fastq_to_fasta: command not found 
gzip: stdout: Broken pipe


On Oct 10, 2013, at 7:06 PM, Björn Grüning 
 wrote:

> Hi Ian,
> 
> what mean broken pipe error, can you post the error message here?
> 
> Thanks,
> Bjoern
> 
>> Hello,
>> 
>> I've got a local install of galaxy with fastx tools installed. I'm getting a 
>> broken pipe error but I'm not sure why. I install fastx using the compiler 
>> outside galaxy and it was working fine inside galaxy previously. Fastx works 
>> fine via the command line. I've checked the xml files and I don't see how 
>> galaxy cannot find the commands. I am running this off and external volume, 
>> but I don't think that matters.
>> 
>> Cheers
>> Ian
>> 
>> 
>> 
>> 
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  http://lists.bx.psu.edu/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[John Chilton]

Opps,  sent that last e-mail before I meant to, gmail is acting very
buggy for me this morning. I just meant to say I would modify
build_command_line in lib/galaxy/jobs/runners/__init__.py instead. It
seems like there is a lot of logic in there for this kind of thing
already. I will create a Trello card and you.

Also just say Nate's response, he is probably right so ignore this
e-mail as well :).

-John

On Fri, Oct 11, 2013 at 7:57 AM, John Chilton  wrote:
> This sounds like a good way to solve the problem, I guess I would modify
>
>
> On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
>  wrote:
>>
>> Dear all,
>>
>> some SGE instances will add automatically an semicolon add the end of
>> each command, resulting in a disrupted job, because ';;' are not
>> allowed.
>>
>> The latest changes to the Bowtie2 wrapper resulting in such a case and a
>> crash in our instance. An easy solution would be to fix the wrapper and
>> omitting the trailing ';' but maybe its better to fix it once for all in
>> the corresponding tools code, patched attached.
>>
>> I'm not familiar with other Job schedulers so I'm seeking for
>> comments ... it there any disadvantage of removing trailing ';' from the
>> command line?
>>
>> Ciao,
>> Bjoern
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[Nate Coraor]

Hi Björn,

Are you certain that SGE is responsible?  I ran in to the same thing with 
bowtie2 with other runners.  The double semicolon is invalid syntax in 
Bourne-compatible shells and so the tool ought to be fixed not to produce them 
(it happens under certain combinations of options).  Have you noticed this 
happening with any other tools?

--nate

On Oct 11, 2013, at 8:57 AM, John Chilton wrote:

> This sounds like a good way to solve the problem, I guess I would modify
> 
> 
> On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
>  wrote:
>> 
>> Dear all,
>> 
>> some SGE instances will add automatically an semicolon add the end of
>> each command, resulting in a disrupted job, because ';;' are not
>> allowed.
>> 
>> The latest changes to the Bowtie2 wrapper resulting in such a case and a
>> crash in our instance. An easy solution would be to fix the wrapper and
>> omitting the trailing ';' but maybe its better to fix it once for all in
>> the corresponding tools code, patched attached.
>> 
>> I'm not familiar with other Job schedulers so I'm seeking for
>> comments ... it there any disadvantage of removing trailing ';' from the
>> command line?
>> 
>> Ciao,
>> Bjoern
>> 
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  http://lists.bx.psu.edu/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line -> Broken bowtie2_wrapper on some SGE systems

[John Chilton]

This sounds like a good way to solve the problem, I guess I would modify


On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning
 wrote:
>
> Dear all,
>
> some SGE instances will add automatically an semicolon add the end of
> each command, resulting in a disrupted job, because ';;' are not
> allowed.
>
> The latest changes to the Bowtie2 wrapper resulting in such a case and a
> crash in our instance. An easy solution would be to fix the wrapper and
> omitting the trailing ';' but maybe its better to fix it once for all in
> the corresponding tools code, patched attached.
>
> I'm not familiar with other Job schedulers so I'm seeking for
> comments ... it there any disadvantage of removing trailing ';' from the
> command line?
>
> Ciao,
> Bjoern
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Error when running paster directly

[Renato Alves]

Hi everyone,

When trying to adapt the init script for debian machines (provided in
contrib/) I started playing with paster directly. I was trying to get
some sort of help or list of commands/params that can be passed but
instead I was presented with the following output:

$ python ./scripts/paster.py

Usage: ./scripts/paster.py COMMAND
Command 'help' not known (you may need to run setup.py egg_info)
Traceback (most recent call last):
  File "./scripts/paster.py", line 33, in 
serve.run()
  File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py",
line 1049, in run
invoke(command, command_name, options, args[1:])
  File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py",
line 1055, in invoke
exit_code = runner.run(args)
  File "/home/galaxy/galaxy_dist/lib/galaxy/util/pastescript/serve.py",
line 367, in run
commands = get_commands().items()
NameError: global name 'get_commands' is not defined

After inspection of the file, the get_commands function doesn't seem to
be defined nor imported.

I'm not sure how to proceed at this point.

Thanks,
Renato



signature.asc
Description: OpenPGP digital signature
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool Shed Tests failing with sniffer/ftype problem

[Peter Cock]

On Thu, Aug 1, 2013 at 8:43 PM, Dave Bouvier  wrote:
> Peter,
>
> As of 10290:d34d6d1da813, nsltradamus and effectivet3 should start returning
> valid test results. As for seq_rename, ...
>
>--Dave B.

Hi Dave,

That fix is on galaxy-central and the Test Tool Shed, and my tests
have been passing there :)

However, the tests are currently still failing on the main Tool Shed:
http://toolshed.g2.bx.psu.edu/view/peterjc/nlstradamus
http://toolshed.g2.bx.psu.edu/view/peterjc/effectivet3

The commit for your fix is on the galaxy-central default branch,
https://bitbucket.org/galaxy/galaxy-dist/commits/d34d6d1da813
https://bitbucket.org/galaxy/galaxy-dist/src/default/lib/galaxy/datatypes/binary.py

The fix is not on the stable branch:
https://bitbucket.org/galaxy/galaxy-dist/src/stable/lib/galaxy/datatypes/binary.py

Nor is the fix on the next-stable branch,
https://bitbucket.org/galaxy/galaxy-dist/src/next-stable/lib/galaxy/datatypes/binary.py

I'm not quite sure how fixes like this are propagated from galaxy-central
to the stable branches on galaxy-dist, but has this been forgotten about?

Thanks,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Test Toolshed Biopython package dependency & NumPy

[Peter Cock]

On Thu, Oct 10, 2013 at 10:00 AM, Peter Cock  wrote:
> On Wed, Oct 9, 2013 at 10:01 PM, Björn Grüning
>  wrote:
>> Am Mittwoch, den 09.10.2013, 19:15 +0100 schrieb Peter Cock:
>>> On Tue, Oct 8, 2013 at 9:57 PM, Dave Bouvier  wrote:
>>> > Peter, Björn,
>>> >
>>> > I have restored gfortran to the repository test host. I believe it was
>>> > inadvertently removed, possibly as a side effect of updating packages or
>>> > removing packages that were causing conflicts with the Galaxy eggs. Numpy
>>> > and biopython should now install successfully, as they do in my local
>>> > environment.
>>> >
>>> >--Dave B.
>>>
>>> Thanks Dave, fingers crossed for tonight's test run :)
>>>
>>> Bjoern - it looks like we can leave the NumPy definition as is...
>>
>> Ah sorry, totally forgot about it :(
>>
>
> Progress, Fortran is working and lapack seems to have installed, but:
> e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename
>
> Installation errors - no functional tests were run for any tools in
> this changeset revision
> Tool dependencies
> TypeNameVersion
> biopython package 1.62
> Error
> running install Numerical Python (NumPy) is not installed. This
> package is required for many Biopython features. Please install it
> before you install Biopython. You can install Biopython anyway, but
> anything dependent on NumPy will not work. If you do this, and later
> install NumPy, you should then re-install Biopython. You can find
> NumPy at http://numpy.scipy.org
> TypeNameVersion
> numpy package 1.7.1
> Error
> Running from numpy source directory.
> /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmp3kNwuR/tmp-toolshed-mtdGJ0d5H/numpy-1.7.1/numpy/distutils/system_info.py:1494:
> UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not
> found. Directories to search for the libraries can be specified in the
> numpy/distutils/site.cfg file (section [atlas]) or by setting the
> ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__)
> /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmp3kNwuR/tmp-toolshed-mtdGJ0d5H/numpy-1.7.1/numpy/distutils/system_info.py:1408:
> UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not
> found. Directories to search for the libraries can be specified in the
> numpy/distutils/site.cfg file (section [atlas]) or by setting the
> ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__)
> error: Command "/usr/bin/gfortran -Wall -Wall -shared
> build/temp.linux-x86_64-2.7/numpy/linalg/lapack_litemodule.o
> build/temp.linux-x86_64-2.7/numpy/linalg/python_xerbla.o
> -L/ToolDependencies/lapack/3.4.2/iuc/package_lapack_3_4/60957bd68fe2/lapack/lib
> -Lbuild/temp.linux-x86_64-2.7 -llapack -lblas -lgfortran -o
> build/lib.linux-x86_64-2.7/numpy/linalg/lapack_lite.so" failed with
> exit status 1
>
>
> Most of that is warnings about not using ATLAS (which we
> expect) but I'm not quite sure why it fails...

Hi Guys,

Bjoern - good news for you, the installation of NumPy can
work on the current Test Tool Shed.

Dave - bad news for you, last night I had 3 NumPy install failures
(all different) and 2 successes - which to me suggests a possible
race condition in the nightly testing?

(1) Last night on the Test Tool Shed this failed as in email above:

http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_primer_clip/834cf6618a14

(2) This failed with the simpler message:

http://testtoolshed.g2.bx.psu.edu/view/peterjc/get_orfs_or_cdss/308ec20faefb

Installation errors - no functional tests were run for any tools in
this changeset revision
Tool dependencies
TypeNameVersion
biopython package 1.62
Error
running install Numerical Python (NumPy) is not installed. This
package is required for many Biopython features. Please install it
before you install Biopython. You can install Biopython anyway, but
anything dependent on NumPy will not work. If you do this, and later
install NumPy, you should then re-install Biopython. You can find
NumPy at http://numpy.scipy.org

(3) Here the installation seems to have failed silently, leading to a
test failure at the import stage:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_select_by_id/9e9e3b860aa0

Tests that failed
Tool id: seq_select_by_id
Tool version: seq_select_by_id
Test: test_tool_00
(functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/seq_select_by_id/seq_select_by_id/0.0.6)
Stderr:
Fatal error: Exit code 1 ()
Biopython 1.54 or later is required

(4, 5) However these also depend on Biopython 1.62 and worked:

http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename/d08767ae73ce
(failed last night)

http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_filter_by_id/8a34c565a473

My guess is a race condition where multiple attempts are being made in
parallel to install NumPy (in the same place), and this

Re: [galaxy-dev] AttributeError: type object 'InvalidJobException' has no attribute 'name'

[Peter Cock]

On Thu, Oct 10, 2013 at 8:20 PM, Nate Coraor  wrote:
> Hi all,
>
> The recent changes to the DRMAA runner are for better handling
> of job-ending conditions under slurm, but it looks like SGE has
> different behavior when a job finishes.  I'll provide a fix for this
> shortly, in the meantime, it's fine to use a slightly older version
> of drmaa.py.
>
> --nate

Thanks Nate,

So far I've only seen this once so it isn't urgent for me.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/