Hi Mike,
Thanks for the feedback. I tried a number of experiments that seem to
confirm your suspicions. Firstly, I retried the transfer with '-p 1'
specified and this succeeded (in ~14 minutes). I also tried performing
the original (failed) transfer to another server (Version 2.5) running
at a different site. This also succeeded (in ~21 minutes). Finally, I
retried the original, failed transfer with the original server.
Frustratingly, this also succeeded (in ~14 minutes).
My suspicion is that the network is a little quieter this morning, so
the transfers are completing more quickly, avoiding the problem that I
have been observing. However, I've put a support request in to have the
Version 1.12 server upgraded, which will hopefully provide a more
reliable longterm solution.
The file transfers that I am performing are part of an investigation
into failed transfers in a Globus application. If parallel streams are
the right way forward, then we will need to update the implementation in
the application source and re-distribute.
Thanks once again for your help,
George.
Cc: GT Users.
On Thu, 29 May 2008, Michael Link wrote:
I suspect the problem is in the server, as that version is over 4 years old
(it was an update to the 2.4.3 release). Is it possible to install a newer
version? The latest stable releases since then are 3.2.1 and 4.0.7.
If you can't, is there any reason not to use the -p option? If you don't
want the additional bandwidth usage caused by multiple streams, '-p 1' may
also work, as it will result in the same breakdown of the data, just over a
single stream.
Mike
George Beckett wrote:
Dear fellow GT users,
I am encountering a problem when attempting to transfer a large file using
GridFTP. The size of the file is ~5GB (4,831,840,904 bytes, to be specific)
and I try to transfer it with a command of the form:
globus-url-copy -dbg file:///<path to local file> \
gsiftp://<remote server>/<path>
The transfer progresses for approx. 25 minutes, with debugging output of
the form:
debug: data callback, no error, buffer 0xf7a89008, length 1048576,
offset=4829741056, eof=false
debug: data callback, no error, buffer 0xf7988008, length 1048576,
offset=4830789632, eof=false
debug: data callback, no error, buffer 0xf7b8a008, length 2696,
offset=4831838208, eof=true
... at which point the transfer appears to hang indefinitely.
If I check the destination, I see that the remote file has been created,
has the correct length and checksum, and has a sensible creation time.
However, the client never exits (I've left it to run for up to an hour
before aborting).
I am able to transfer smaller files (I've tried a 10 MB file) successfully,
and I've checked that there is sufficient space on the remote machine to
hold the file.
Interestingly, if I activate parallel streams:
globus-url-copy -dbg -p 4 file:///<path to local file> \
gsiftp://<remote server>/<path>
the transfer does complete successfully (in around 10 minutes), so I am
wondering if the problem is related to the transfer time, rather than the
file size.
My client is running:
> globus-url-copy -version
globus-url-copy: 3.20
on a linux machine (kernel version 2.6.9). The remote server is running a
similar version of linux and reports GridFTP Server 1.12.
Any advice on what might be causing the problem would be much appreciated.
Thanks in advance,
George.
--
------------------------------------------------------------------
Dr Mark George Beckett, Email: [EMAIL PROTECTED]
Project Manager, EPCC, Tel.: +44(0)131 650 5818
The University of Edinburgh, Fax.: +44(0)131 650 6555
James Clerk Maxwell Building, Web: www.epcc.ed.ac.uk
EDINBURGH EH9 3JZ, Scotland.
------------------------------------------------------------------
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
