Re: [galaxy-dev] Tophat non Sanger input

[Whyte, Jeffrey]

Anton,

If a user is running a multi-core machine, a simple method to speed up FASTQ 
-Groomer is to first split the original FASTQ file (e.g. into 10 smaller files 
if you've got a 12-core machine), run FASTQ-Groomer on each file concurrently, 
then join the 10 files back together.  This allows for parallel processing of 
the FASTQ file, rather than having FASTQ-Groomer slug it's way through with one 
processor while the other cores sit idle.  I wrote a simple bash script that 
uses "split" and "cat" to automate the process.  A file that would take two 
hours for FASTQ-Groomer now takes just over 10 min.  As a double check, I 
verify that the input FASTQ-illumina and output FASTQ-Sanger files are 
identical with the FastQC program written by Simon Andrews.  Right now, I run 
the script from the command line before putting my file into the Galaxy 
pipeline.

I'm sure there are more "refined" ways to handle this with python, but it gets 
the job done.  Thumbs up for FASTQ-groomer.

Jeff



On Sep 8, 2011, at 9:30 AM, Anton Nekrutenko wrote:

Dear Stephen (and others):

The sole reason for requiring fastq-sanger input to all of our wrappers was to 
force the users to run their data through the groomer. It is slow, but it 
checks data consistency in a way that is more robust than just checking 'four 
lines per fastq block' and prevents a lot of problems downstream. Here on 
Galaxy @ Penn State we see a lot of fastq files edited in MS Word and other 
similar horrors, which are being caught by groomer and prevent users from 
running into problems later on (and so cutting down on the support overhead - 
investigating why groomer has failed is a lot easier than researching why a 
particular set of polymorphisms derived from a Word-edited fastq file clusters 
Ukrainians with parasitic worms). In addition, even though Illumina did switch 
to Sanger encoding, there is still a lot of old data out there. However, we are 
open to suggestions ... What we are thinking of lately is switching to 
unaligned BAM for everyting. One of the benefits here is the ability to!
  add readgroups from day 1 simplifying multisample analyses down the road.

a.


Anton Nekrutenko
http://galaxyproject.org




On Sep 8, 2011, at 10:14 AM, Stephen Taylor wrote:

On 08/09/2011 14:17, Hans-Rudolf Hotz wrote:


On 09/08/2011 09:47 AM, Stephen Taylor wrote:
On 07/09/2011 20:22, Edward Kirton wrote:
seems unnecessary since illumina switched over to fastqsanger now.

http://www.illumina.com/truseq/quality_101/quality_scores.ilmn

Eventually...unfortunately we still get a lot of fastqillumina :-(


I might miss your point.but why can't you use the fastq groomer tool?


- Duplication of data (disk space usage)
- Groomer is slow and puts more demands on CPU usage where it can be done 
easily on the fly by tophat
- Consistency (bowtie does it)

>From the responses (or lack of :-)) we've been spurred on to change the 
>wrapper. If there is interest we will commit it to the code base when done.

Cheers,

Steve
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Whyte, Jeffrey]

That worked, Nate!

Thank you very much.  I really appreciate your efforts.

Jeff


From: Nate Coraor [n...@bx.psu.edu]
Sent: Wednesday, September 07, 2011 10:33 AM
To: Whyte, Jeffrey
Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

Hi Jeff,

Could you try:

% python ./scripts/fetch_eggs.py

>From the top-level galaxy-dist directory?  I have just placed eggs for
py2.7-macosx-10.6-intel-ucs2 on http://eggs.g2.bx.psu.edu/

--nate

Whyte, Jeffrey wrote:
> Thanks, Nate!  Much appreciated.
> Jeff
>
>
>
> On Sep 7, 2011, at 7:54 AM, Nate Coraor wrote:
>
> > Whyte, Jeffrey wrote:
> >> Nate,
> >>
> >> For some reason, ~/galaxy-dist wasn't in my PATH.  Here is the output when 
> >> the script is run from the ~/galaxy-dist directory:
> >>
> >> [~/galaxy-dist] myuserid 03:50 PM > python ./scripts/get_platforms.py
> >> macosx-10.6-intel-ucs2
> >> [~/galaxy-dist] myuserid 03:50 PM >
> >>
> >> Looks like it is set for Mac OS X 10.6 Snow Leopard instead of 10.7 Lion.  
> >> (?)
> >
> > This is normal since OS X has binary compatibility with older releases,
> > the Python distributors build for an older platform rather than each
> > individual platform.
> >
> > I see there's now an Intel-only build for 10.6+, I'll work on this ASAP.
> >
> >>
> >> Thanks,
> >> Jeff
> >>
> >>
> >> On Sep 6, 2011, at 10:18 AM, Nate Coraor wrote:
> >>
> >>> Whyte, Jeffrey wrote:
> >>>> Thanks for the information, Nate.
> >>>>
> >>>> I installed Python as you described, and the error I get now is pasted 
> >>>> below.  I appreciate your help.
> >>>>
> >>>> Jeff
> >>>>
> >>>> [~/galaxy-dist] myuserid 03:06 PM > ./run.sh
> >>>> Some eggs are out of date, attempting to fetch...
> >>>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> >>>> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> >>>> pysam 0.4.2 couldn't be downloaded automatically.  You can try
> >>>> building it by hand with:
> >>>> python scripts/scramble.py -e pysam
> >>>> Fetch failed.
> >>>> [~/galaxy-dist] myuserid 03:06 PM >
> >>>
> >>> Could you report back with the output of:
> >>>
> >>>   % python ./scripts/get_platforms.py
> >>>
> >>> ?
> >>>
> >>> Thanks,
> >>> --nate
> >>>
> >>>> 
> >>>> From: Nate Coraor [n...@bx.psu.edu]
> >>>> Sent: Friday, September 02, 2011 1:16 PM
> >>>> To: Whyte, Jeffrey
> >>>> Cc: galaxy-dev@lists.bx.psu.edu
> >>>> Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)
> >>>>
> >>>> Whyte, Jeffrey wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I've been having trouble running a local instance of Galaxy on a Mac 
> >>>>> Pro after upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 
> >>>>> and Mercurial is 1.9.1 for MacOS X 10.7.  I don't have any MacPorts 
> >>>>> installed.
> >>>>>
> >>>>> The error I see after running the startup script is pasted at the end 
> >>>>> of this message.  Thanks in advance for any help or advice.
> >>>>> jjw
> >>>>
> >>>> Hi Jeffrey,
> >>>>
> >>>> We haven't yet gotten our dependencies up to speed on Lion.  Could you
> >>>> grab a copy of Python from python.org and use this?  It'll install under
> >>>> /Library/Frameworks/Python.framework, just add the correct bin/
> >>>> directory to the front of your $PATH and start Galaxy as normal.
> >>>>
> >>>> Sorry for the inconvenience,
> >>>> --nate
> >>>>
> >>>>>
> >>>>> -
> >>>>>
> >>>>> [~/galaxy-dist] myuserid 10:11 AM > ./run.sh
> >>>>> Some eggs are out of date, attempting to fetch...
> >>>>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> >>>>> Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be 
> >>>>> fetched
> &

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Whyte, Jeffrey]

Nate,

For some reason, ~/galaxy-dist wasn't in my PATH.  Here is the output when the 
script is run from the ~/galaxy-dist directory:

[~/galaxy-dist] myuserid 03:50 PM > python ./scripts/get_platforms.py
macosx-10.6-intel-ucs2
[~/galaxy-dist] myuserid 03:50 PM > 

Looks like it is set for Mac OS X 10.6 Snow Leopard instead of 10.7 Lion.  (?)

Thanks,
Jeff 


On Sep 6, 2011, at 10:18 AM, Nate Coraor wrote:

> Whyte, Jeffrey wrote:
>> Thanks for the information, Nate.
>> 
>> I installed Python as you described, and the error I get now is pasted 
>> below.  I appreciate your help.
>> 
>> Jeff
>> 
>> [~/galaxy-dist] myuserid 03:06 PM > ./run.sh
>> Some eggs are out of date, attempting to fetch...
>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
>> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
>> pysam 0.4.2 couldn't be downloaded automatically.  You can try
>> building it by hand with:
>>  python scripts/scramble.py -e pysam
>> Fetch failed.
>> [~/galaxy-dist] myuserid 03:06 PM >
> 
> Could you report back with the output of:
> 
>% python ./scripts/get_platforms.py
> 
> ?
> 
> Thanks,
> --nate
> 
>> 
>> From: Nate Coraor [n...@bx.psu.edu]
>> Sent: Friday, September 02, 2011 1:16 PM
>> To: Whyte, Jeffrey
>> Cc: galaxy-dev@lists.bx.psu.edu
>> Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)
>> 
>> Whyte, Jeffrey wrote:
>>> Hi,
>>> 
>>> I've been having trouble running a local instance of Galaxy on a Mac Pro 
>>> after upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 and 
>>> Mercurial is 1.9.1 for MacOS X 10.7.  I don't have any MacPorts installed.
>>> 
>>> The error I see after running the startup script is pasted at the end of 
>>> this message.  Thanks in advance for any help or advice.
>>> jjw
>> 
>> Hi Jeffrey,
>> 
>> We haven't yet gotten our dependencies up to speed on Lion.  Could you
>> grab a copy of Python from python.org and use this?  It'll install under
>> /Library/Frameworks/Python.framework, just add the correct bin/
>> directory to the front of your $PATH and start Galaxy as normal.
>> 
>> Sorry for the inconvenience,
>> --nate
>> 
>>> 
>>> -
>>> 
>>> [~/galaxy-dist] myuserid 10:11 AM > ./run.sh
>>> Some eggs are out of date, attempting to fetch...
>>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
>>> Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched
>>> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
>>> Traceback (most recent call last):
>>> File "./scripts/fetch_eggs.py", line 30, in 
>>>   c.resolve() # Only fetch eggs required by the config
>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, 
>>> in resolve
>>>   egg.resolve()
>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, 
>>> in resolve
>>>   return self.version_conflict( e.args[0], e.args[1] )
>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, 
>>> in version_conflict
>>>   r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, 
>>> egg.fetch )
>>> File 
>>> "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py",
>>>  line 565, in resolve
>>>   raise DistributionNotFound(req)  # XXX put more info here
>>> pkg_resources.DistributionNotFound: numpy==1.6.0
>>> Fetch failed.
>>> [~/galaxy-dist] myuserid 10:12 AM >
>>> 
>>> 
>>> 
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> 
>>>  http://lists.bx.psu.edu/
>> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Whyte, Jeffrey]

Hi Nate,

Thanks for the response.  Here is the output:

[~] myuserid 01:34 PM > python ./scripts/get_platforms.py
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python:
 can't open file './scripts/get_platforms.py': [Errno 2] No such file or 
directory
[~] myuserid 01:34 PM >

Jeff


From: Nate Coraor [n...@bx.psu.edu]
Sent: Tuesday, September 06, 2011 10:18 AM
To: Whyte, Jeffrey
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

Whyte, Jeffrey wrote:
> Thanks for the information, Nate.
>
> I installed Python as you described, and the error I get now is pasted below. 
>  I appreciate your help.
>
> Jeff
>
> [~/galaxy-dist] myuserid 03:06 PM > ./run.sh
> Some eggs are out of date, attempting to fetch...
> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> pysam 0.4.2 couldn't be downloaded automatically.  You can try
> building it by hand with:
>   python scripts/scramble.py -e pysam
> Fetch failed.
> [~/galaxy-dist] myuserid 03:06 PM >

Could you report back with the output of:

% python ./scripts/get_platforms.py

?

Thanks,
--nate

> 
> From: Nate Coraor [n...@bx.psu.edu]
> Sent: Friday, September 02, 2011 1:16 PM
> To: Whyte, Jeffrey
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)
>
> Whyte, Jeffrey wrote:
> > Hi,
> >
> > I've been having trouble running a local instance of Galaxy on a Mac Pro 
> > after upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 and 
> > Mercurial is 1.9.1 for MacOS X 10.7.  I don't have any MacPorts installed.
> >
> > The error I see after running the startup script is pasted at the end of 
> > this message.  Thanks in advance for any help or advice.
> > jjw
>
> Hi Jeffrey,
>
> We haven't yet gotten our dependencies up to speed on Lion.  Could you
> grab a copy of Python from python.org and use this?  It'll install under
> /Library/Frameworks/Python.framework, just add the correct bin/
> directory to the front of your $PATH and start Galaxy as normal.
>
> Sorry for the inconvenience,
> --nate
>
> >
> > -
> >
> > [~/galaxy-dist] myuserid 10:11 AM > ./run.sh
> > Some eggs are out of date, attempting to fetch...
> > Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> > Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched
> > Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> > Traceback (most recent call last):
> >  File "./scripts/fetch_eggs.py", line 30, in 
> >c.resolve() # Only fetch eggs required by the config
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, 
> > in resolve
> >egg.resolve()
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, 
> > in resolve
> >return self.version_conflict( e.args[0], e.args[1] )
> >  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, 
> > in version_conflict
> >r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, 
> > egg.fetch )
> >  File 
> > "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py",
> >  line 565, in resolve
> >raise DistributionNotFound(req)  # XXX put more info here
> > pkg_resources.DistributionNotFound: numpy==1.6.0
> > Fetch failed.
> > [~/galaxy-dist] myuserid 10:12 AM >
> >
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> >   http://lists.bx.psu.edu/
>

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Whyte, Jeffrey]

Thanks for the information, Nate.

I installed Python as you described, and the error I get now is pasted below.  
I appreciate your help.

Jeff

[~/galaxy-dist] myuserid 03:06 PM > ./run.sh
Some eggs are out of date, attempting to fetch...
Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
pysam 0.4.2 couldn't be downloaded automatically.  You can try
building it by hand with:
  python scripts/scramble.py -e pysam
Fetch failed.
[~/galaxy-dist] myuserid 03:06 PM >

From: Nate Coraor [n...@bx.psu.edu]
Sent: Friday, September 02, 2011 1:16 PM
To: Whyte, Jeffrey
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

Whyte, Jeffrey wrote:
> Hi,
>
> I've been having trouble running a local instance of Galaxy on a Mac Pro 
> after upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 and 
> Mercurial is 1.9.1 for MacOS X 10.7.  I don't have any MacPorts installed.
>
> The error I see after running the startup script is pasted at the end of this 
> message.  Thanks in advance for any help or advice.
> jjw

Hi Jeffrey,

We haven't yet gotten our dependencies up to speed on Lion.  Could you
grab a copy of Python from python.org and use this?  It'll install under
/Library/Frameworks/Python.framework, just add the correct bin/
directory to the front of your $PATH and start Galaxy as normal.

Sorry for the inconvenience,
--nate

>
> -
>
> [~/galaxy-dist] myuserid 10:11 AM > ./run.sh
> Some eggs are out of date, attempting to fetch...
> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
> Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched
> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
> Traceback (most recent call last):
>  File "./scripts/fetch_eggs.py", line 30, in 
>c.resolve() # Only fetch eggs required by the config
>  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in 
> resolve
>egg.resolve()
>  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in 
> resolve
>return self.version_conflict( e.args[0], e.args[1] )
>  File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in 
> version_conflict
>r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, 
> egg.fetch )
>  File 
> "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py",
>  line 565, in resolve
>raise DistributionNotFound(req)  # XXX put more info here
> pkg_resources.DistributionNotFound: numpy==1.6.0
> Fetch failed.
> [~/galaxy-dist] myuserid 10:12 AM >
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>   http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)

[Whyte, Jeffrey]

Hi,

I've been having trouble running a local instance of Galaxy on a Mac Pro after 
upgrading to OS X 10.7 (Lion).  My Python version is 2.7.1 and Mercurial is 
1.9.1 for MacOS X 10.7.  I don't have any MacPorts installed.

The error I see after running the startup script is pasted at the end of this 
message.  Thanks in advance for any help or advice.
jjw

-

[~/galaxy-dist] myuserid 10:11 AM > ./run.sh
Some eggs are out of date, attempting to fetch...
Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched
Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
Traceback (most recent call last):
 File "./scripts/fetch_eggs.py", line 30, in 
   c.resolve() # Only fetch eggs required by the config
 File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in 
resolve
   egg.resolve()
 File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in 
resolve
   return self.version_conflict( e.args[0], e.args[1] )
 File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in 
version_conflict
   r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, 
egg.fetch )
 File 
"/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py",
 line 565, in resolve
   raise DistributionNotFound(req)  # XXX put more info here
pkg_resources.DistributionNotFound: numpy==1.6.0
Fetch failed.
[~/galaxy-dist] myuserid 10:12 AM >



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Special considerations for installing localinstance of Galaxy on Mac OS X (10.6.7)?

[Whyte, Jeffrey]

Assaf and Pieter,

That worked like a charm!  I installed the gsed port from MacPorts and then 
edited the shell script, "fastx_barcode_splitter_galaxy_wrapper.sh", replacing 
the two instances of "sed" with "gsed".  Ran Galaxy again, and the barcode 
splitter worked perfectly.

Thank you both for taking the time to help me troubleshoot this.  Excellent 
advice.

Jeff


On Apr 27, 2011, at 10:33 PM, Gordon, Assaf wrote:

> Regarding the fastx-splitter:
> This one is a perl+shell script, not a binary,
> 
> so try installing GNU sed (no need to override Mac's sed, the GNU sed will 
> probably be installed as "gsed"), and modify the shell script (from "sed" to 
> "gsed").
> 
> 
> On Apr 27, 2011, at 19:15, "Pieter Neerincx"  
> wrote:
> 
>> Hi Jeff,
>> 
>> On Apr 28, 2011, at 12:42 AM, Whyte, Jeffrey wrote:
>> 
>>> Hi Pieter,
>>> 
>>> Thanks to both you and Florent for the suggestion of MacPorts.  GnuPlot is 
>>> now working well in Galaxy after I installed the port.
>> 
>> :)
>> 
>>> I also appreciate the information on specifying the custom environment.  
>>> That could be related to the only other problem I am having: the FASTX 
>>> toolkit barcode splitter.  Other FASTX tools like "rename sequences" and 
>>> "trim end" work just fine, but the barcode splitter gives the following 
>>> error in Galaxy:
>>> 
>>> An error occurred running this job:sed: illegal option -- r
>>> usage: sed script [-Ealn] [-i extension] [file ...]
>>> 
>>> Unfortunately, I don't believe there is a port of the FASTX tools available 
>>> through MacPorts.  I have gone through the instructions on the Hannon Lab 
>>> webpage to try to install exactly as specified, but still get the error, 
>>> although now at least Galaxy knows where to look to find the tools.  I 
>>> don't believe this is related to the PATH, as the error in Galaxy seems to 
>>> relate to an option setting.
>> 
>> That seems correct. I just checked sed on a Mac OS X laptop and a linux 
>> server: the one distributed with Mac OS X does not have a --r option whereas 
>> the one on Linux does.
>> 
>>> From man sed on linux:
>> 
>> -r, --regexp-extended
>>use extended regular expressions in the script.
>> 
>> There is no -r in sed on Mac OS X, but from man sed:
>> 
>>-E  Interpret regular expressions as extended (modern) regular 
>> expressions rather than basic regular expressions (BRE's).  The
>>re_format(7) manual page fully describes both formats.
>> 
>> Just from the description this -E option seems extremely similar if it isn't 
>> just plain the same, so you may try to change the sed -r command into sed -E 
>> in
>> 
>>   galaxy/tools/fastx_toolkit/fastx_barcode_splitter_galaxy_wrapper.sh
>> 
>> If that doesn't work you either have to hack the code of this tool :o or try 
>> to install a compatible version of sed...
>> 
>> Cheers,
>> 
>> Pi
>> 
>>> I have run this barcode splitting on the public Galaxy server with no 
>>> problems, and I simply used the same barcode definitions and joined file in 
>>> my local instance of Galaxy, but get the error.
>>> 
>>> I'll give you a break :) and keep working on it to see if I can trace the 
>>> source of the error.
>>> 
>>> Many thanks again,
>>> Jeff
>>> 
>>> 
>>> On Apr 27, 2011, at 4:10 AM, Pieter Neerincx wrote:
>>> 
>>>> Hi Jeff,
>>>> 
>>>> 1. As Florent wrote, probably the easiest solution is to install the 
>>>> dependencies with MacPorts. Only if stuff is not available via MacPorts or 
>>>> the version available is too old / incompatible with other dependencies I 
>>>> install things manually. MacPorts will add a line like this:
>>>> -
>>>> export PATH=/opt/local/bin:/opt/local/sbin:$PATH
>>>> -
>>>> to your ~/.profile file, which is automatically sourced when you open a 
>>>> new Terminal window. Galaxy is probably running using a different account 
>>>> than yours. Usually a dedicated one or one used for the webserver like 
>>>> _www or www or apache or ... This user must have the same addition to it's 
>>>> environment in order to find gnuplot et al. To give Galaxy the same 
>>>

Re: [galaxy-dev] Special considerations for installing local instance of Galaxy on Mac OS X (10.6.7)?

[Whyte, Jeffrey]

One more note on the FASTX barcode splitter error:

After a little searching, I believe the problem is with the options used for 
sed utility, specifically the -r option

An error occurred running this job:sed: illegal option -- r

If I log in to my Linux server, and type "man sed", one of the options listed 
is, 

-r, --regexp-extended
 use extended regular expressions in the script.

If I use the terminal on my Mac, and type "man sed", ,the "-r" option is not 
listed, which is probably the source of the error.

Not sure how to fix it though.

Jeff

On Apr 27, 2011, at 4:10 AM, Pieter Neerincx wrote:

> Hi Jeff,
> 
> 1. As Florent wrote, probably the easiest solution is to install the 
> dependencies with MacPorts. Only if stuff is not available via MacPorts or 
> the version available is too old / incompatible with other dependencies I 
> install things manually. MacPorts will add a line like this:
> -
> export PATH=/opt/local/bin:/opt/local/sbin:$PATH 
> -
> to your ~/.profile file, which is automatically sourced when you open a new 
> Terminal window. Galaxy is probably running using a different account than 
> yours. Usually a dedicated one or one used for the webserver like _www or www 
> or apache or ... This user must have the same addition to it's environment in 
> order to find gnuplot et al. To give Galaxy the same environment you can 
> either add the line above to galaxy/run.sh immediately after the first line 
> that identifies the file as a shell script (#!/bin/sh) and before the code 
> that starts Galaxy. Optionally you may specify the environment in a separate 
> file and source this file. Personally I like the latter, because I can manage 
> the environment for multiple users and software packages in a single file 
> that I can call from various locations. To source a file you use the . 
> command. Hence [dot][space][path to env file]. Your galaxy/run.sh would start 
> like this:
> -
> #!/bin/sh
> 
> #
> # Source my custom environment including the path to MacPorts and other stuff 
> I installed outside the default search paths
> #
> . /path/to/our_environment.bashrc
> -
> 
> In our_environment.bashrc you would than have the same line:
> 
> export PATH=/opt/local/bin:/opt/local/sbin:$PATH
> 
> and potentially other stuff to set your environment. 
> 
> 2. For the gnuplot error below "unknown or ambiguous terminal type": It seems 
> the gnuplot script Galaxy is calling is trying to use an unsupported 
> "terminal". When you save the figure to a file a "terminal" is just a file 
> type. In order to write jpeg, tiff, png, pdf, svg, etc. images gnuplot uses 
> several libraries. When you install gnuplot it will look which libraries are 
> available and based on that some image file formats may be available and 
> others will not. It seems png support is not available. When you start a 
> gnuplot terminal and type 'set terminal' for a list, do you see png as a 
> supported terminal? If not, you should probably install libpng first and 
> re-compile gnuplot. If you install gnuplot via MacPorts though it will 
> probably already take care of this dependency and install it for you... 
> 
> Cheers,
> 
> Pi
> 
> 
> On Apr 24, 2011, at 3:57 AM, Whyte, Jeffrey wrote:
> 
>> Hi Pieter,
>> 
>> Thanks very much for taking the time to provide the advice and explanations 
>> - very helpful!  As you suggested, my search path was not defined properly 
>> to locate gnuplot.  If I typed "gnuplot" in the Terminal, I would get the 
>> message, "command not found".  Now, Gnuplot is in /usr/local/bin and this 
>> directory is listed when I type "echo $PATH".  Now when I type "gnuplot" in 
>> the terminal, I get:
>> 
>> $ gnuplot
>> 
>> G N U P L O T
>> Version 4.4 patchlevel 3
>> last modified March 2011
>> System: Darwin 10.7.0
>> 
>> Copyright (C) 1986-1993, 1998, 2004, 2007-2010
>> Thomas Williams, Colin Kelley and many others
>> 
>> gnuplot home: http://www.gnuplot.info
>> faq, bugs, etc:   type "help seeking-assistance"
>> immediate help:   type "help"
>> plot window:  hit 'h'
>> 
>> Terminal type set to 'aqua'
>> gnuplot>
>> 
>> When I start up Galaxy, and try to run the Boxplot tool, I now get a 
>> different error:
>> 
>> An error occurred running this job:
>> gnuplot> set term png size 2048,768
>> ^
>> line 0: unknown or ambig

Re: [galaxy-dev] Special considerations for installing local instance of Galaxy on Mac OS X (10.6.7)?

[Whyte, Jeffrey]

Hi Pieter,

Thanks to both you and Florent for the suggestion of MacPorts.  GnuPlot is now 
working well in Galaxy after I installed the port.  I also appreciate the 
information on specifying the custom environment.  That could be related to the 
only other problem I am having: the FASTX toolkit barcode splitter.  Other 
FASTX tools like "rename sequences" and "trim end" work just fine, but the 
barcode splitter gives the following error in Galaxy:

An error occurred running this job:sed: illegal option -- r
usage: sed script [-Ealn] [-i extension] [file ...]

Unfortunately, I don't believe there is a port of the FASTX tools available 
through MacPorts.  I have gone through the instructions on the Hannon Lab 
webpage to try to install exactly as specified, but still get the error, 
although now at least Galaxy knows where to look to find the tools.  I don't 
believe this is related to the PATH, as the error in Galaxy seems to relate to 
an option setting.

I have run this barcode splitting on the public Galaxy server with no problems, 
and I simply used the same barcode definitions and joined file in my local 
instance of Galaxy, but get the error.

I'll give you a break :) and keep working on it to see if I can trace the 
source of the error.

Many thanks again,
Jeff


On Apr 27, 2011, at 4:10 AM, Pieter Neerincx wrote:

> Hi Jeff,
> 
> 1. As Florent wrote, probably the easiest solution is to install the 
> dependencies with MacPorts. Only if stuff is not available via MacPorts or 
> the version available is too old / incompatible with other dependencies I 
> install things manually. MacPorts will add a line like this:
> -
> export PATH=/opt/local/bin:/opt/local/sbin:$PATH 
> -
> to your ~/.profile file, which is automatically sourced when you open a new 
> Terminal window. Galaxy is probably running using a different account than 
> yours. Usually a dedicated one or one used for the webserver like _www or www 
> or apache or ... This user must have the same addition to it's environment in 
> order to find gnuplot et al. To give Galaxy the same environment you can 
> either add the line above to galaxy/run.sh immediately after the first line 
> that identifies the file as a shell script (#!/bin/sh) and before the code 
> that starts Galaxy. Optionally you may specify the environment in a separate 
> file and source this file. Personally I like the latter, because I can manage 
> the environment for multiple users and software packages in a single file 
> that I can call from various locations. To source a file you use the . 
> command. Hence [dot][space][path to env file]. Your galaxy/run.sh would start 
> like this:
> -
> #!/bin/sh
> 
> #
> # Source my custom environment including the path to MacPorts and other stuff 
> I installed outside the default search paths
> #
> . /path/to/our_environment.bashrc
> -
> 
> In our_environment.bashrc you would than have the same line:
> 
> export PATH=/opt/local/bin:/opt/local/sbin:$PATH
> 
> and potentially other stuff to set your environment. 
> 
> 2. For the gnuplot error below "unknown or ambiguous terminal type": It seems 
> the gnuplot script Galaxy is calling is trying to use an unsupported 
> "terminal". When you save the figure to a file a "terminal" is just a file 
> type. In order to write jpeg, tiff, png, pdf, svg, etc. images gnuplot uses 
> several libraries. When you install gnuplot it will look which libraries are 
> available and based on that some image file formats may be available and 
> others will not. It seems png support is not available. When you start a 
> gnuplot terminal and type 'set terminal' for a list, do you see png as a 
> supported terminal? If not, you should probably install libpng first and 
> re-compile gnuplot. If you install gnuplot via MacPorts though it will 
> probably already take care of this dependency and install it for you... 
> 
> Cheers,
> 
> Pi
> 
> 
> On Apr 24, 2011, at 3:57 AM, Whyte, Jeffrey wrote:
> 
>> Hi Pieter,
>> 
>> Thanks very much for taking the time to provide the advice and explanations 
>> - very helpful!  As you suggested, my search path was not defined properly 
>> to locate gnuplot.  If I typed "gnuplot" in the Terminal, I would get the 
>> message, "command not found".  Now, Gnuplot is in /usr/local/bin and this 
>> directory is listed when I type "echo $PATH".  Now when I type "gnuplot" in 
>> the terminal, I get:
>> 
>> $ gnuplot
>> 
>> G N U P L O T
>> Version 4.4 patchlevel 3
>> last modified March 2011
>>

Re: [galaxy-dev] Special considerations for installing local instance of Galaxy on Mac OS X (10.6.7)?

[Whyte, Jeffrey]

Hi Florent,

I installed MacPorts this afternoon, but that was after I installed gnuplot 
(sudo make install).  Have you found that gnuplot works in Galaxy if installed 
through MacPorts?  Thanks for the suggestion.

Jeff


On Apr 23, 2011, at 7:03 PM, Florent Angly wrote:

> Hi Jeff
> Some of these tools can be installed easily on Mac OS using MacPorts
> port search gnuplot
> Of course, you have to install MacPorts first.
> Florent
> 
> On 23/04/11 01:25, Whyte, Jeffrey wrote:
>> Hi Everyone,
>> 
>> I'm having trouble getting some of the tools to work on my local 
>> installation of Galaxy on a Mac (OS X version 10.6.7).  The instructions on 
>> the GetGalaxy wiki are clear, and I was able to download and install from 
>> the anonymous Mercurial repository.  Galaxy starts up and runs just fine for 
>> tools like "Get Data", "FASTQ Groomer", and "FASTQ Summary Statistics".
>> 
>> The problems started when I try to run Graph/Data Display ->  Boxplot 
>> (GnuPlot).  I get the error:
>> 
>> "An error occurred running this job:/bin/sh: gnuplot: command not found
>> Error running gnuplot."
>> 
>> The FASTX Barcode Splitter also gives me an error:
>> 
>> "An error occurred running this 
>> job:/Users/Me/galaxy-dist/tools/fastx_toolkit/fastx_barcode_splitter_galaxy_wrapper.sh:
>>  line 65: fastx_barcode_splitter.pl: command not found
>> zcat: /Users/Me/galaxy-dist/database/files/000/dataset_10.dat.Z: No such 
>> file or directory
>> sed: illegal opt"
>> 
>> I'm assuming these are Tool Dependency problems, although I have tried to 
>> install the OS X versions of GnuPlot and the FASTX Toolkit and still get 
>> errors.
>> 
>> Are there any suggestions from users who are successfully running Galaxy on 
>> OS X?
>> 
>> Thanks in advance,
>> Jeff
>> 
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> 
>>   http://lists.bx.psu.edu/
> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Special considerations for installing local instance of Galaxy on Mac OS X (10.6.7)?

[Whyte, Jeffrey]

Hi Pieter,

Thanks very much for taking the time to provide the advice and explanations - 
very helpful!  As you suggested, my search path was not defined properly to 
locate gnuplot.  If I typed "gnuplot" in the Terminal, I would get the message, 
"command not found".  Now, Gnuplot is in /usr/local/bin and this directory is 
listed when I type "echo $PATH".  Now when I type "gnuplot" in the terminal, I 
get:

$ gnuplot

G N U P L O T
Version 4.4 patchlevel 3
last modified March 2011
System: Darwin 10.7.0

Copyright (C) 1986-1993, 1998, 2004, 2007-2010
Thomas Williams, Colin Kelley and many others

gnuplot home: http://www.gnuplot.info
faq, bugs, etc:   type "help seeking-assistance"
immediate help:   type "help"
plot window:  hit 'h'

Terminal type set to 'aqua'
gnuplot>

When I start up Galaxy, and try to run the Boxplot tool, I now get a different 
error:

An error occurred running this job:
gnuplot> set term png size 2048,768
^
line 0: unknown or ambiguous terminal type; type just 'set terminal' for a list

gnuplot> plot '/Users/JJWhyte/galaxy-dist/database/files/000/dataset_6.dat' 
using 1:7:11:12:9 with candlest

I think this is related to the environment, as you explained in your second 
paragraph.  I tried starting gnuplot from the Terminal first, then typing "set 
term xterm", and then exiting, but this didn't change the error.  Could you 
please explain how you set the environment in the galaxy/run.sh script?

Again, I appreciate the help.

Jeff



On Apr 23, 2011, at 8:11 AM, Pieter Neerincx wrote:

Hi Jeff,

This is indeed a dependency issue and I have seen those both on Linux and Mac 
OS X. What happens if you type gnuplot in a Terminal window yourself? If you 
also get "command not found". There is either still something missing or your 
environment is not setup properly. If the tool works if you execute it yourself 
in the Terminal, but it fails in Galaxy, you do have all the dependencies and 
it's clearly an environment setup issue: your OS will search for the binaries 
in a search path defined by the $PATH environment variable. This means that if 
you call "gnuplot" without specifying the path where it is installed, the OS 
will look in all directories specified in $PATH to find it. If it cannot find 
it, it will give you "command not found". Additionally you may have to set 
$LD_LIBRARY_PATH as well if the installed tools link to libraries installed 
outside "default" locations.

Hence if you go to the Terminal and type:

echo $PATH

You will see a colon separated list of directories that should include the 
location where the gnuplot binary is installed. If this is not the case, please 
Google for setting $PATH on Mac OS X. If gnuplot does work if you execute it on 
the commandline, but it fails in Galaxy, it most likely means the environment 
for the user used to run Galaxy is different from yours (and the location where 
gnuplot is installed is missing from $PATH). In that case you can either set 
the environment in the galaxy/run.sh script used to start Galaxy or if you can 
call a bashrc script to set the environment from galaxy/run.sh. I use the 
latter where I have one bashrc script that sets the environment for all users 
and by calling this one from galaxy/run.sh I make sure the environment for 
Galaxy is exactly the same as for my (command line) users.

Cheers,

Pi

On Apr 22, 2011, at 5:25 PM, Whyte, Jeffrey wrote:


Hi Everyone,

I'm having trouble getting some of the tools to work on my local installation 
of Galaxy on a Mac (OS X version 10.6.7).  The instructions on the GetGalaxy 
wiki are clear, and I was able to download and install from the anonymous 
Mercurial repository.  Galaxy starts up and runs just fine for tools like "Get 
Data", "FASTQ Groomer", and "FASTQ Summary Statistics".

The problems started when I try to run Graph/Data Display -> Boxplot (GnuPlot). 
 I get the error:

"An error occurred running this job:/bin/sh: gnuplot: command not found
Error running gnuplot."

The FASTX Barcode Splitter also gives me an error:

"An error occurred running this 
job:/Users/Me/galaxy-dist/tools/fastx_toolkit/fastx_barcode_splitter_galaxy_wrapper.sh:
 line 65: fastx_barcode_splitter.pl: command not found
zcat: /Users/Me/galaxy-dist/database/files/000/dataset_10.dat.Z: No such file 
or directory
sed: illegal opt"

I'm assuming these are Tool Dependency problems, although I have tried to 
install the OS X versions of GnuPlot and the FASTX Toolkit and still get errors.

Are there any suggestions from users who are successfully running Galaxy on OS 
X?

Thanks in advance,
Jeff


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your 

[galaxy-dev] Special considerations for installing local instance of Galaxy on Mac OS X (10.6.7)?

[Whyte, Jeffrey]

Hi Everyone,

I'm having trouble getting some of the tools to work on my local installation 
of Galaxy on a Mac (OS X version 10.6.7).  The instructions on the GetGalaxy 
wiki are clear, and I was able to download and install from the anonymous 
Mercurial repository.  Galaxy starts up and runs just fine for tools like "Get 
Data", "FASTQ Groomer", and "FASTQ Summary Statistics".

The problems started when I try to run Graph/Data Display -> Boxplot (GnuPlot). 
 I get the error:
 
"An error occurred running this job:/bin/sh: gnuplot: command not found
Error running gnuplot."

The FASTX Barcode Splitter also gives me an error:

"An error occurred running this 
job:/Users/Me/galaxy-dist/tools/fastx_toolkit/fastx_barcode_splitter_galaxy_wrapper.sh:
 line 65: fastx_barcode_splitter.pl: command not found
zcat: /Users/Me/galaxy-dist/database/files/000/dataset_10.dat.Z: No such file 
or directory
sed: illegal opt"

I'm assuming these are Tool Dependency problems, although I have tried to 
install the OS X versions of GnuPlot and the FASTX Toolkit and still get 
errors.  

Are there any suggestions from users who are successfully running Galaxy on OS 
X? 

Thanks in advance,
Jeff


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/