Adam Bazinet wrote:
Just reading over the docs for globus-crft and have a few questions.
1) Is there ever a case where you *wouldn't* want to destroy the server?
Can it be re-used, somehow? Or is it just so you could query for the status
of a transfer after it is finished in the asynchronous method?
It is the latter. you always want to destroy, it is just a question of when. crft is different
than the rft client in that it allows you to do everything in steps. You can tell the rft service
to start a transfer from your laptop and allow globus-crft to quickly terminate (prior to transfer
completion) then throw your laptop in your car and drive to work. when you get there you can check
on the status of you transfer and give all your friends the EPR associated with your transfer so
they can check on your transfer also. At some point via some out of band mechanism you decide no
one else needs to check on your transfer so you destroy it.
2) What exactly are the options used when -ez is specified?
it is --create, --submit, and --monitor. i *think* it should be --destroy also, but looking at the
code i see that it is not.
3) -gS | --getStatusSet <int> <int> "Get the status of all the transfer
requests in the range." What range does this refer to? Does it correspond
to the number of transfers you specify in the transfers file? Is it 0 or
1-based?
it is an index into your transfer set. It kind of loses all usable meaning when you are
transferring an expanded directory (because you don't really know what file is at what index). This
is really a limitation of the service, not the client.
thanks!
Adam
On Tue, Dec 30, 2008 at 6:09 PM, <[email protected]> wrote:
I believe the java rft client is hanging because it is not properly
detecting completion. I believe this won't happen with globus-crft
-----Original Message-----
From: Adam Bazinet <[email protected]>
Sent: December 30, 2008 5:00 PM
To: [email protected]
Cc: GT User <[email protected]>
Subject: Re: [gt-user] problem with rft directory transfer
The server logs show nothing with a level of 'error'; I suppose I can turn
on info, warn, all, etc.
In the container log, the error is only tripped once. After it is tripped,
the file transfer does continue ( I can watch a directory fill up ):
gridt...@valine:>find . -name "*" | wc
108 108 1867
gridt...@valine:>find . -name "*" | wc
109 109 1903
gridt...@valine:>find . -name "*" | wc
113 113 2013
(etc.)
But the fact that it is just hanging afterwards is killing things right
now. I'll continue to look into it, but if you have ideas for things I can
try, please let me know, since this is crippling our entire system at the
moment.
Also, I know that globus-url-copy is rather fast; it's clearly the overhead
of RFT and the things you've described that are making it slow. It will be
interesting to see how globus-crft compares.
thanks,
Adam
On Tue, Dec 30, 2008 at 5:52 PM, <[email protected]> wrote:
I ment the globus container logs, but the gridftp server logs can tell
the
same thing.
How does globus-url-copy compare to scp? Alos you should try globus -crft
because part of the problem is the jvm start up time on the client.
Another
reason is the needed security delegation, and probably the largest part
is
the transactions required to enter everything in the database.
-----Original Message-----
From: Adam Bazinet <[email protected]>
Sent: December 30, 2008 2:48 PM
To: John Bresnahan <[email protected]>
Cc: GT User <[email protected]>
Subject: Re: [gt-user] problem with rft directory transfer
On Tue, Dec 30, 2008 at 2:27 PM, John Bresnahan <[email protected]
wrote:
The retries is already set to 10, correct? But the docs say this is
for
a
'non-fatal' error. The problem is that when this error occurs, the
client
processes seem to be "hung" -- there is seemingly nothing happening.
Is
there another way to specify re-tries with the rft command line
client?
Something analogous to the <maxAttempts> field in RSL?
In 4.2 every error should be considered nonfatal, so retries will
happen.
10 may not be a high enough value, can you look in the server logs to
see
if this error is tripped many times?
I assume you mean the gridftp log, which I had to turn on (hopefully the
error log will suffice)
Now I'm waiting for the error to happen again.
One other question, in the meantime. Why is RFT so slow? The same
recursive directory transfer between hosts takes all of 3 or 4 seconds,
and
with RFT each one takes several minutes- a huge difference. It's always
been this way, and I've just accepted it- but never thought to ask why.
thanks,
Adam
As you can see, one comes from 4.2.1, one from 4.2.0. Would upgrading
valine to 4.2.1 help, if you think this is a GridFTP server issue? It
sounded like you weren't so sure using globus-crft would help,
although
I
will see about using it anyway.
since this appears to be a server side issue there is no reason other
than
dumb luck that globus-crft would help. However, globus-crft tends to
be
a
better tool in general so i recommend using it.
