[galaxy-dev] Genome not listed
Dear developers, 1. I recently installed galaxy locally. now i am trying to run bowtie2. In reference genome section the genome is not listed. 2. Is there any way yo secure our data without using the Proxy(nginx). I Appericiate your suggestions. Thanks Sridhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] installing galaxy
hello, I am installing galaxy locally. I tried to upload a .bam file and I got this message: Traceback (most recent call last): File /home/user/galaxy_project/tools/data_source/upload.py, line 386, in module __main__() File /home/user/galaxy_project/tools/data_source/upload.py, line 375, in __main__ add_file( dataset, registry, json_file, output_path ) File /home/user/galaxy_project/tools/data_source/upload.py, line 314, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 113, in dataset_content_needs_grooming version = self._get_samtools_version() File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 97, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File /usr/lib/python2.7/subprocess.py, line 679, in __init__ errread, errwrite) File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This file uploaded successfully, to the galaxy on the net. what can it be? thanks for your help merav ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] installing galaxy
Hi, Merav. A fully functional Galaxy server relies on a lot of third party software and the error message you see is probably because you do not have samtools available - the upload process needs it to make an index for your bam file. So the solution may be to install the samtools package - eg for ubuntu something like: sudo apt-get install samtools After that, if you execute samtools at a console, you should see something like: rlazarus@iaas1-int:~$ samtools Program: samtools (Tools for alignments in the SAM format) Version: 0.1.18 (r982:295) Usage: samtools command [options] Command: viewSAM-BAM conversion sortsort alignment file mpileup multi-way pileup Please take a look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies On Wed, May 29, 2013 at 8:00 PM, מירב . mer...@gmail.com wrote: hello, I am installing galaxy locally. I tried to upload a .bam file and I got this message: Traceback (most recent call last): File /home/user/galaxy_project/tools/data_source/upload.py, line 386, in module __main__() File /home/user/galaxy_project/tools/data_source/upload.py, line 375, in __main__ add_file( dataset, registry, json_file, output_path ) File /home/user/galaxy_project/tools/data_source/upload.py, line 314, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 113, in dataset_content_needs_grooming version = self._get_samtools_version() File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 97, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File /usr/lib/python2.7/subprocess.py, line 679, in __init__ errread, errwrite) File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This file uploaded successfully, to the galaxy on the net. what can it be? thanks for your help merav ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating multiple datasets in a libset
Has the script created multiple histories? That's the intent, that each
file+workflow execution has a separate history.
And, for why that file vanished resulting in the exception -- is it
possible you ran a second instance of the script or moved the file?
-Dannon
On Wed, May 29, 2013 at 1:07 AM, neil.burd...@csiro.au wrote:
Further, when I look in the ~galaxy-dist/database/files/000 I can see both
files have been uploaded but only the second file has a history associated
to it
Thanks
Neil
-Original Message-
From: Burdett, Neil (ICT Centre, Herston - RBWH)
Sent: Wednesday, 29 May 2013 2:45 PM
To: galaxy-dev@lists.bx.psu.edu; 'Dannon Baker'; 'robert.lecl...@gmail.com
'
Subject: Re: Creating multiple datasets in a libset
Hi Guys,
Did you manage to get multiple datasets working? I can't seem to
upload multiple files. Only the last file appears in the history. I changed
my code as mentioned in the thread below in example_watch_folder.py to
add multiple files separated by a new line and increased the sleep time:
for fname in os.listdir(in_folder):
fullpath = os.path.join(in_folder, fname)
print ' fullpath is [%s] ' % fullpath
if os.path.isfile(fullpath):
data = {}
data['folder_id'] = library_folder_id
data['file_type'] = 'auto'
data['dbkey'] = ''
data['upload_option'] = 'upload_paths'
data['filesystem_paths'] =
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz\n
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz
print ' data is [%s] ' % str(data['filesystem_paths'])
data['create_type'] = 'file'
libset = submit(api_key, api_url + libraries/%s/contents
% library_id, data, return_formatted = False)
#TODO Handle this better, but the datatype isn't always
# set for the followup workflow execution without this
# pause.
time.sleep(65)
However, I get the following crash:
./example_watch_folder.py 64f3209856a3cf4f2d034a1ad5bf851c
http://barium-rbh/csiro/api/ /home/galaxy/galaxy-drop/input
/home/galaxy/galaxy-drop/output This One f2db41e1fa331b3e
fullpath is
[/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz]
data is [/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz]
url is :
http://barium-rbh/csiro/api/libraries/33b43b4e7093c91f/contents?key=64f3209856a3cf4f2d034a1ad5bf851c
data is : {'file_type': 'auto', 'dbkey': '', 'create_type': 'file',
'folder_id': 'F33b43b4e7093c91f', 'upload_option': 'upload_paths',
'filesystem_paths':
'/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz\n
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz'}
url is :
http://barium-rbh/csiro/api/workflows?key=64f3209856a3cf4f2d034a1ad5bf851c
data is : {'workflow_id': 'f2db41e1fa331b3e', 'ds_map': {'14': {'src':
'ld', 'id': 'ff5476bcf6c921fa'}}, 'history':
'141_S_0851_MRI_T2_Screening.nii.gz - apiFullCTE'}
{'outputs': ['daecbdd824e1c349', '358eb58cd5463e0d', 'c0279aab05812500'],
'history': '3cc0effd29705aa3'} url is :
http://barium-rbh/csiro/api/workflows?key=64f3209856a3cf4f2d034a1ad5bf851c
data is : {'workflow_id': 'f2db41e1fa331b3e', 'ds_map': {'14': {'src':
'ld', 'id': '79966582feb6c081'}}, 'history':
'141_S_0851_MRI_T2_Screening.nii.gz - apiFullCTE'}
{'outputs': ['19c51286b777bc04', '0f71f1fc170d4ab9', '256444f6e7017e58'],
'history': 'b701da857886499b'} Traceback (most recent call last):
File ./example_watch_folder.py, line 89, in module
main(api_key, api_url, in_folder, out_folder, data_library, workflow )
File ./example_watch_folder.py, line 75, in main
shutil.move(fullpath, os.path.join(out_folder, fname))
File /usr/lib/python2.7/shutil.py, line 299, in move
copy2(src, real_dst)
File /usr/lib/python2.7/shutil.py, line 128, in copy2
copyfile(src, dst)
File /usr/lib/python2.7/shutil.py, line 82, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory:
'/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz'
It says there is no such file, but this file has already been copied from
the input to the output directory. Any help much appreciated
Neil
--
Message: 2
Date: Mon, 29 Apr 2013 16:11:39 -0400
From: Rob Leclerc robert.lecl...@gmail.com
To: Dannon Baker dannon.ba...@gmail.com
Cc: galaxy-...@bx.psu.edu galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] Creating multiple datasets in a libset
Message-ID:
CAGkd85fHSgO2YC1T+Frctyso9G5rfQb=_mLyHGSdxPM+s3=
8...@mail.gmail.com
Content-Type: text/plain; charset=iso-8859-1
Hi Dannon,
I've written some code to (i) query a dataset to ensure that it's been
uploaded after a
[galaxy-dev] php script integration query
Hi, If I have some programs in php which I need to integrate in Galaxy as a tool, can I do it? I know if I have some programs written in Perl, python,java,C or C++, I can easily integrate them. Can I do the same with a php script? Regards, Prasun Dutta MSc (Bioinformatics) School of Biological Sciences University of Edinburgh, UK ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] php script integration query
As long as you're able to execute your php script from the command line, it should work fine. -Dannon On Wed, May 29, 2013 at 7:25 AM, Prasun Dutta prasundutt...@yahoo.co.inwrote: Hi, If I have some programs in php which I need to integrate in Galaxy as a tool, can I do it? I know if I have some programs written in Perl, python,java,C or C++, I can easily integrate them. Can I do the same with a php script? Regards, Prasun Dutta MSc (Bioinformatics) School of Biological Sciences University of Edinburgh, UK ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] advice on a mixed cloud/SGE architecture
Ok, got it. Unfortunately this isn't really possible at this time, at least not without a lot of hacking and dealing with data staging issues and delays (in downloading data from S3 to local cluster to work, uploading results back to galaxy server, etc.). Galaxy, with the exception of the LWR - http://wiki.galaxyproject.org/Admin/Config/LWR, relies on a shared filesystem between the galaxy instance and workers. I'd really recommend colocating Galaxy with the cluster, at least for now. We do want to build out the ability to use EC2 as an expansion to local clusters, but I imagine it working the other way around -- that is, you have a local instance that uses S3 as permanent storage and spin up extra nodes in EC2 to handle excess load as necessary. -Dannon On Tue, May 28, 2013 at 1:14 PM, Nikolay N. nikola...@gmail.com wrote: Hi Dannon, Yes, this is what I was thinking about. The reason is that at the moment the SGE cluster is managed separately. Also we are likely to migrate to a different cluster infrastructure (probably LSF) and I thought that if I set up a master node on AWS I could than only have to change the cluster part. The idea is that if the user metadata is stored on the AWS master node and his/her files on S3 than it would be relatively easy to just swap the cluster that is providing the raw muscle. But I don't have a good understanding of the galaxy architecture and the documentation seems a bit sparse. So any advice on this is most welcome. -Nick On Tue, May 28, 2013 at 3:58 PM, Dannon Baker dannon.ba...@gmail.comwrote: Hi Nick, Not sure I completely understand what you're trying to do here. So, you want to have a galaxy head node on AWS that dispatches jobs to a non-AWS private SGE cluster? -Dannon On Thu, May 16, 2013 at 12:18 PM, Nikolay N. nikola...@gmail.com wrote: Hi, I was contemplating setting up a galaxy master node on a aws instance but executing the workflows on an internal SGE cluster. I am not yet very familiar with galaxy so wanted to ask whether this is feasible (i.e. the API would support it). Any advice/recommendation is most welcome. many thanks, Nick ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] cufflinks in the main galaxy instance is not able to produce compatible FPKM score
Hi, Most people using the RNA-Seq toolkit use a combination of Tophat/Cufflinks/Cuffdiff. A known issue is that cufflinks and cuffdiff use different normalisation schemes ( http://seqanswers.com/forums/archive/index.php/t-26041.html). Users overcome this problem by running cufflinks with the following flags: --compatible-hits-norm --max-frag-multihits 1 However, in galaxy it is not possible to run cufflinks with these flags. Can you, perhaps, add support for them? many thanks, Nikolay ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating tool shed repository: installation of dependencies fails.
Joachim, I'm unable to duplicate that error, what revision of Galaxy and the tool shed code are you running? The path mentioned in the error message is a temporary working directory that is created for downloading and processing files needed for a tool dependency. Had the installation succeeded, the required files would have been moved to a path under /mnt/testgalaxydb/database, as expected. --Dave B. On 5/29/13 06:54:39.000, Joachim Jacob | VIB | wrote: Hi all, I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails: * Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat * Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error. The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url. * ?xml version=1.0? tool_dependency package name=blat version=1.0.0 install version=1.0 actions action type=download_by_urlhttp://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat/action action type=move_file sourceblat/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme The BLAT binary for linux x86_64 is retrieved from UCSC. /readme /package /tool_dependency * Thanks for helping me further. Cheers, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Feature Request - Reload Button
I noticed that if I touch universe_wsgi.ini from the terminal (update its timestamp) while Galaxy is running then Galaxy restarts. If this is intended, would it be possible to add a Reload Galaxy button/option in the Server section of the Admin page that mimics this feature? Obviously, you'd want some way of warning an admin if jobs are currently being executed. Thanks. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Run galaxy on a specific SGE queue
On May 22, 2013, at 11:36 AM, Sarah Maman wrote: Hi everyone, Galaxy is set up on our cluster, and so far everything is working just fine. However, we'd like to change the submission queue from the default to a specific one. To do so, we tryed to set on the universe_wsgi.ini file the default_cluster_job_runner parameter as following: default_cluster_job_runner = drmaa://-q test.q -l mem=8G -l h_vmem=10G/ When doing so, an error occured telling us the job has been rejected because 2 queues were specified: the workq (default one) and the test.q. Should be the queue setting this way ? What are we doing wrong ? Hi Sarah, Without knowing what's going on in your environment I can't say for sure, but I would guess that something else is (~/.sge_request?) is also attempting to set the queue. Are you able to use `qsub -q test.q` on the command line as the same user that runs Galaxy? Thanks, --nate thanks for your help, Sarah Sarah Maman INRA - LGC - SIGENAE Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating tool shed repository: installation of dependencies fails.
Strange. When I just wget that link in a terminal, do 'chmod +X blat' and ./blat, it just works fine. Could it be that the permissions need to be set by Galaxy? Version info of the Tool Shed: * [galaxy@galaxy galaxy-dist]$ hg summary parent: 9320:47ddf167c9f1 tip Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response. Fixes downloading tarballs from the Tool Shed when use_debug = false. branch: stable commit: 5 modified, 271 unknown update: (current) [galaxy@galaxy galaxy-dist]$ * Version info of the Galaxy integrating the tool shed repository: * parent: 9292:2cc8d10988e0 security_2013.04.08 Controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 22 unknown (new branch head) update: 11 new changesets (update) * Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 03:08 PM, Greg Von Kuster wrote: Hello Joachim, This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind? Greg Von Kuster On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote: Hi all, I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails: * Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat * Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error. The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url. * ?xml version=1.0? tool_dependency package name=blat version=1.0.0 install version=1.0 actions action type=download_by_urlhttp://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat/action action type=move_file sourceblat/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme The BLAT binary for linux x86_64 is retrieved from UCSC. /readme /package /tool_dependency * Thanks for helping me further. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating multiple datasets in a libset
Hi Neil,
I've attached my class function for uploading multiple files.
def upload_files(self, fullpaths):
Uploads files from a disk location to a Galaxy library
Accepts an array of full path filenames
Example: fullpaths = ['/home/username/file1.txt',
'/home/username/files2.txt']
if self.jsonstring == None:
self.get_library()
library_id = self.library_id
library_folder_id = self.library_folder_id
api_key = self.api_key
api_url = self.api_url
#Galaxy needs to read the pathnames as a new line delimited string
#so we do that transformation here
fullpaths_string =
for path in fullpaths:
fullpaths_string = fullpaths_string + path + \n
fullpaths_string = fullpaths_string[:-1]
data = {}
data['folder_id'] = library_folder_id
data['file_type'] = 'auto'
data['dbkey'] = ''
data['upload_option'] = 'upload_paths'
data['filesystem_paths'] = fullpaths_string
data['create_type'] = 'file'
#Start the upload. This will return right away, but it may take
awhile
libset = submit(api_key, api_url + libraries/%s/contents %
library_id, data, return_formatted = False)
#Iterate through each dataset we just uploaded and block until all
files have been written to the Galaxy database
for ds in libset:
last_filesize = 0
while True:
#If file_size != 0 and the file_size is different after a
second iteration, then we assume the disk write is finished
ds_id = ds['id']
uploaded_file = display(api_key, api_url +
'libraries/%s/contents/%s' %(library_id, ds_id), return_formatted=False)
print uploaded_file
if uploaded_file['file_size'] != 0 and
uploaded_file['file_size'] == last_filesize:
break
else:
last_filesize = uploaded_file['file_size']
time.sleep(2)
self.libset = libset
return libset
Rob Leclerc, PhD
http://www.linkedin.com/in/robleclerc https://twitter.com/#!/robleclerc
P: (US) +1-(917)-873-3037
P: (Shanghai) +86-1-(861)-612-5469
Personal Email: rob.lecl...@aya.yale.edu
On Wed, May 29, 2013 at 12:45 AM, neil.burd...@csiro.au wrote:
Hi Guys,
Did you manage to get multiple datasets working? I can't seem to
upload multiple files. Only the last file appears in the history. I changed
my code as mentioned in the thread below in example_watch_folder.py to
add multiple files separated by a new line and increased the sleep time:
for fname in os.listdir(in_folder):
fullpath = os.path.join(in_folder, fname)
print ' fullpath is [%s] ' % fullpath
if os.path.isfile(fullpath):
data = {}
data['folder_id'] = library_folder_id
data['file_type'] = 'auto'
data['dbkey'] = ''
data['upload_option'] = 'upload_paths'
data['filesystem_paths'] =
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz\n
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz
print ' data is [%s] ' % str(data['filesystem_paths'])
data['create_type'] = 'file'
libset = submit(api_key, api_url + libraries/%s/contents
% library_id, data, return_formatted = False)
#TODO Handle this better, but the datatype isn't always
# set for the followup workflow execution without this
# pause.
time.sleep(65)
However, I get the following crash:
./example_watch_folder.py 64f3209856a3cf4f2d034a1ad5bf851c
http://barium-rbh/csiro/api/ /home/galaxy/galaxy-drop/input
/home/galaxy/galaxy-drop/output This One f2db41e1fa331b3e
fullpath is
[/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz]
data is [/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz]
url is :
http://barium-rbh/csiro/api/libraries/33b43b4e7093c91f/contents?key=64f3209856a3cf4f2d034a1ad5bf851c
data is : {'file_type': 'auto', 'dbkey': '', 'create_type': 'file',
'folder_id': 'F33b43b4e7093c91f', 'upload_option': 'upload_paths',
'filesystem_paths':
'/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz\n
/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz'}
url is :
http://barium-rbh/csiro/api/workflows?key=64f3209856a3cf4f2d034a1ad5bf851c
data is : {'workflow_id': 'f2db41e1fa331b3e', 'ds_map': {'14': {'src':
'ld', 'id': 'ff5476bcf6c921fa'}}, 'history':
'141_S_0851_MRI_T2_Screening.nii.gz - apiFullCTE'}
{'outputs': ['daecbdd824e1c349', '358eb58cd5463e0d', 'c0279aab05812500'],
'history': '3cc0effd29705aa3'}
url is :
Re: [galaxy-dev] Creating tool shed repository: installation of dependencies fails.
The problem is most likely in ~/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py, line 100. 1. current_dir = os.path.abspath( os.path.join( work_dir, dir ) ) 2. while dir is set a couple of lines before as: dir = workdir (line 78). workdir is the temporary directory. 3. current_dir is then used by common_util.move_file on line 111. This explains the duplication in the path that I observed. It does not yet explain why the temporary directory is created in the wrong location. Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 03:08 PM, Greg Von Kuster wrote: Hello Joachim, This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind? Greg Von Kuster On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote: Hi all, I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails: * Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat * Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error. The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url. * ?xml version=1.0? tool_dependency package name=blat version=1.0.0 install version=1.0 actions action type=download_by_urlhttp://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat/action action type=move_file sourceblat/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme The BLAT binary for linux x86_64 is retrieved from UCSC. /readme /package /tool_dependency * Thanks for helping me further. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] organizing datasets in the history
Hi, galaxy community: I’ve set up a local galaxy for our institute, one thing that our users kept complaining is that when dealing with several dozens of datasets, it became hard to find a certain data. I’m wondering is there a way to organize data in history? Say, organizing datasets in folders just like the file arrangement in the shared data library.. Thank you all for your help. Best Regards, John Wu -- PhD Student Institute of Biomedical Science Academia Sinica ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] cufflinks in the main galaxy instance is not able to produce compatible FPKM score
These are options we'd like to add to Cufflinks, and I've created a Trello card for this enhancement: https://trello.com/c/U7nceKdj I want to add that community contributions via patch or pull request are highly encouraged and are likely to be quicker than waiting for the core team to get to this issue. Best, J. On May 29, 2013, at 8:47 AM, Nikolay N. wrote: Hi, Most people using the RNA-Seq toolkit use a combination of Tophat/Cufflinks/Cuffdiff. A known issue is that cufflinks and cuffdiff use different normalisation schemes (http://seqanswers.com/forums/archive/index.php/t-26041.html). Users overcome this problem by running cufflinks with the following flags: --compatible-hits-norm --max-frag-multihits 1 However, in galaxy it is not possible to run cufflinks with these flags. Can you, perhaps, add support for them? many thanks, Nikolay ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] organizing datasets in the history
When a user creates an account, s/he can create/copy/delete histories and copy datasets from history to history to organize data. Each history, then, is a like a library folder. Best, J. On May 29, 2013, at 11:02 AM, 吳正華 wrote: Hi, galaxy community: I’ve set up a local galaxy for our institute, one thing that our users kept complaining is that when dealing with several dozens of datasets, it became hard to find a certain data. I’m wondering is there a way to organize data in history? Say, organizing datasets in folders just like the file arrangement in the shared data library.. Thank you all for your help. Best Regards, John Wu -- PhD Student Institute of Biomedical Science Academia Sinica ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Fwd: installing galaxy
-- Forwarded message -- From: מירב . mer...@gmail.com Date: Wed, May 29, 2013 at 8:40 PM Subject: Re: [galaxy-dev] installing galaxy To: Ross ross.laza...@gmail.com It worked!! Thank you very mach [?] Merav 2013/5/29 Ross ross.laza...@gmail.com Hi, Merav. A fully functional Galaxy server relies on a lot of third party software and the error message you see is probably because you do not have samtools available - the upload process needs it to make an index for your bam file. So the solution may be to install the samtools package - eg for ubuntu something like: sudo apt-get install samtools After that, if you execute samtools at a console, you should see something like: rlazarus@iaas1-int:~$ samtools Program: samtools (Tools for alignments in the SAM format) Version: 0.1.18 (r982:295) Usage: samtools command [options] Command: viewSAM-BAM conversion sortsort alignment file mpileup multi-way pileup Please take a look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies On Wed, May 29, 2013 at 8:00 PM, מירב . mer...@gmail.com wrote: hello, I am installing galaxy locally. I tried to upload a .bam file and I got this message: Traceback (most recent call last): File /home/user/galaxy_project/tools/data_source/upload.py, line 386, in module __main__() File /home/user/galaxy_project/tools/data_source/upload.py, line 375, in __main__ add_file( dataset, registry, json_file, output_path ) File /home/user/galaxy_project/tools/data_source/upload.py, line 314, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 113, in dataset_content_needs_grooming version = self._get_samtools_version() File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 97, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File /usr/lib/python2.7/subprocess.py, line 679, in __init__ errread, errwrite) File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This file uploaded successfully, to the galaxy on the net. what can it be? thanks for your help merav ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J 328.png___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Storage question for Galaxy-Admins
Hello, I am trying to come up with a number for our local galaxy storage requirements when we go live (This is an installation for research at our university). I checked out the survey that was done a while back for the GalaxyAdmins. It seems that the bigger installations have allocated about 200TB for their installation. It would be of great help to me if I can get some feedback on storage used by local galaxy installations. If possible please use the following format: Total capacity: Used capacity: Storage Technology: Comments: Thank you all very much. -Raj ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] installing galaxy
It worked!! Thank you very mach Merav 2013/5/29 Ross ross.laza...@gmail.com Hi, Merav. A fully functional Galaxy server relies on a lot of third party software and the error message you see is probably because you do not have samtools available - the upload process needs it to make an index for your bam file. So the solution may be to install the samtools package - eg for ubuntu something like: sudo apt-get install samtools After that, if you execute samtools at a console, you should see something like: rlazarus@iaas1-int:~$ samtools Program: samtools (Tools for alignments in the SAM format) Version: 0.1.18 (r982:295) Usage: samtools command [options] Command: viewSAM-BAM conversion sortsort alignment file mpileup multi-way pileup Please take a look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies On Wed, May 29, 2013 at 8:00 PM, מירב . mer...@gmail.com wrote: hello, I am installing galaxy locally. I tried to upload a .bam file and I got this message: Traceback (most recent call last): File /home/user/galaxy_project/tools/data_source/upload.py, line 386, in module __main__() File /home/user/galaxy_project/tools/data_source/upload.py, line 375, in __main__ add_file( dataset, registry, json_file, output_path ) File /home/user/galaxy_project/tools/data_source/upload.py, line 314, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 113, in dataset_content_needs_grooming version = self._get_samtools_version() File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 97, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File /usr/lib/python2.7/subprocess.py, line 679, in __init__ errread, errwrite) File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This file uploaded successfully, to the galaxy on the net. what can it be? thanks for your help merav ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
