Hi Jim,
You message was misthreaded (perhaps a reply to another thread, with
just the subject line changed?), but I was able to dig it out.
A this time, there are no known issues with FTP Upload to the public
Main server. Any issues you have have found prior were either related to
a problem with the original file content (compression problem) or a
transitory issue with the FTP server that has since been resolved (there
has been a handful in the last few years).
The instructions to follow are here:
http://wiki.galaxyproject.org/FTPUpload
I am not exactly sure what your issue is, but any chance that you have
more than one file per archive? That will certainly cause an issue, but
usually with just the first file loading the remainder not.
Please send more details if this continues. Does the failure occur at
the FTP stage or at the point where you move from the FTP holding area
into a history?
Thanks!
Jen
Galaxy team
On 2/5/13 5:48 AM, Jim Robinson wrote:
Hi,
I am having a lot of difficulty uploading some large gzipped fastqs (~
10GB) to the public server. I have tried both ftp and "pulling" by
http URL. The upload succeeds, however I get an error as it tries to
gunzip it. I have tried more than 10 times now and succeeded once.
These files are correct and complete, and gunzip properly locally. The
error shown is usually this
empty
format: txt, database: ?
Problem decompressing gzipped data
However on 2 occasions (both ftp uploads) I got the traceback below.
Am I missing some obvious trick? I searched the archives and see
references to problems with large gzipped files but no solutions.
Thanks
Jim
Traceback (most recent call last):
File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py",
line 384, in <module>
__main__()
File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py",
line 373, in __main__
add_file( dataset, registry, json_file, output_path )
File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py",
line 270, in add_file
line_count, converted_path = sniff.convert_newlines( dataset.path,
in_place=in_place )
File "/galaxy/home/g2main/galaxy_main/lib/galaxy/datatypes/sniff.py",
line 106, in convert_newlines
shutil.move( temp_name, fname )
File "/usr/lib/python2.7/shutil.py", line 299, in move
copy2(src, real_dst)
File "/usr/lib/python2.7/shutil.py", line 128, in copy2
copyfile(src, dst)
File "/usr/lib/python2.7/shutil.py", line 84, in copyfile
copyfileobj(fsrc, fdst)
File "/usr/lib/python2.7/shutil.py", line 49, in copyfileobj
buf = fsrc.read(length)
IOError: [Errno 5] Input/output error
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
--
Jennifer Hillman-Jackson
Galaxy Support and Training
http://galaxyproject.org
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
