Re: [gt-user] globus-url-copy times out my entire system

[Foster, Ian T.]

John:

Have you tried the Globus transfer service 
(www.globus.org)? That typically works better.

Ian

On Oct 25, 2018, at 9:39 PM, John Foo 
mailto:john...@singaren.net.sg>> wrote:


Hi All!

When I do globus-url-copy on my centOS 7 machine, the transfer starts as 
normal, but occasionally the transfer will slow down to 0.00MB/s and SSH 
connect will be cut off and the entire machine will lose all its connection for 
a few seconds before recovering. Attempts to do any transfers after will result 
in the same behavior and the only fix is to reboot the machine. Has anyone 
experienced similar behaviors?

Best Regards
John Foo

Re: [gt-user] GridFTP Troubleshooting

[Foster, Ian T.]

Dear Amit:

Have you tried installing Globus Connect and using the Globus transfer service? 
Easier, faster, more reliable, …

See https://www.globus.org/data-transfer

Regards — Ian

On Aug 28, 2018, at 11:07 AM, Amit Kumar 
mailto:ihavebeenfor...@gmail.com>> wrote:

Dear GT,

I have setup GridFTP and trying to use globus-url-copy to test it and running 
it following errors, and wondering if you can help me here.

I have a InCommon cert for the gridftp host, I also have InCommonRSAServerCA 
cert in the path but I am not able to find what issue certificate is it 
complaining about?

Any help here is greatly appreciated.

Thank you,
Amit


$ globus-url-copy -vb -dbg 
gsiftp://gftp.host.xxx.edu/tmp/zero.source
 file:///tmp/zero.dest
Source: gsiftp://gftp.host.xxx.edu/tmp/
Dest:   file:///tmp/
  zero.source  ->  zero.dest
debug: starting to get 
gsiftp://gftp.host.xxx.edu/tmp/zero.source
debug: connecting to 
gsiftp://gftp.host.xxx.edu/tmp/zero.source

debug: response from 
gsiftp://gftp.host.xxx.edu/tmp/zero.source:
220 10.10.29.20 GridFTP Server 12.8 (gcc64, 1531931206-85) [Globus Toolkit 6.0] 
ready.

debug: authenticating with 
gsiftp://gftp.host.xxx.edu/tmp/zero.source
debug: fault on connection to 
gsiftp://gftp.host.xxx.edu/tmp/zero.source:
 globus_ftp_control: gss_init_sec_context failed
debug: data callback, error globus_ftp_control: gss_init_sec_context failed, 
buffer 0x2aaab0ff3010, length 0, offset=0, eof=true
debug: operation complete

error: globus_ftp_control: gss_init_sec_context failed
OpenSSL Error: s3_clnt.c:1264: in library: SSL routines, function 
ssl3_get_server_certificate: certificate verify failed
globus_gsi_callback_module: Could not verify credential
globus_gsi_callback_module: Could not verify credential: unable to get issuer 
certificate

_
Ian Foster
Director, Data Science and Learning Division; Senior Scientist; Distinguished 
Fellow, Argonne National Laboratory
Arthur Holly Compton Distinguished Service Professor of Computer Science, 
University of Chicago
Fellow, Institute for Molecular Engineering
Chief Troublemaker, Globus, www.globus.org
Check out our new book: Cloud Computing for Science and Engineering
Online at https://cloud4scieng.org
Tel: +1 630 252 4619

Re: [gt-user] Using GridFTP on Windows hosts

[Foster, Ian T.]

Have you tried installing Globus Connect Personal and using the Globus service?

On Sep 20, 2017, at 12:56 PM, Hernandez, Hugo * 
mailto:hugo.hernan...@fda.hhs.gov>> wrote:

Hello there, we are trying to install Windows client of the Globus Toolkit and 
then access the DTN GridFTP servers we have configured by using SSH as 
authentication method (we use this successfully in both Mac OS X and Linux).  
The first problem we have encountered is shown on the following pic:



We have been looking online as there is not that much documentation around so 
help will be greatly appreciated.  This is what we have done so far to install 
the Windows client:

Download the MinGW version of the Globus Toolkit package from the Globus portal 
at https://downloads.globus.org/toolkit/gt6/stable/installers/windows/ and 
download the binary zip file 
globus_toolkit-6.0.1502136246-i686-w64-mingw32-Build-13.zip. Extract the files 
to a local folder of choice.  (Ex: “C:\Program Files”)
Add the \Globus\bin to the system PATH using the steps below.


  *   From the desktop, right click the Computer icon.
  *   Choose Properties from the context menu.
  *   Click the Advanced system settings link.
  *   Click Environment Variables. In the section System Variables, find the 
PATH environment variable and select it.
  *   Click Edit. If the PATH environment variable does not exist, click New.
  *   In the Edit System Variable (or New System Variable) window, specify the 
value of the PATH environment variable Or append ";c:\\Globus\bin" to the existing value.
  *   Click OK. Close all remaining windows by clicking OK.


Once we are able to address the “Error -1, activating gass cipy module” issue, 
what is next to setup the windows client to use SSH for authentication? 
Clearly, the user should have SSH access into the corresponding DTN.

Note: we are trying to avoid users to use Cygwin but the MinGW version of the 
Globus Toolkit client for Windows.

Again, help will be greatly appreciated!

Regards,
-Hugo


Hugo R Hernandez
FDA-CFSAN | Scientific Computing Team
5001 Campus Dr
College Park, MD 20740
(301) 348-1780 – Office
hugo.hernan...@fda.hhs.gov|www.engilitycorp.com

Re: [gt-user] Catching a globus-url-copy Failure

[Foster, Ian T.]

Perhaps not helpful in your environment, but the Globus transfer service 
provides a nice API that makes this really easy.

> On Oct 1, 2015, at 5:18 PM, Ben Ferraro  wrote:
> 
> Hello,
> 
> We are currently using a globus-url-copy command for file staging in/out in 
> our current system. One problem we are facing is trying to catch a failed 
> globus-url-copy command, OR how to tell that a globus-url-command has 
> successfully transferred files. So far we have taken a look at the output 
> with the -verbose and -dbg flags, but have not found much useful information. 
> We are using a recursive file transfer (-r flag) and at any moment could be 
> transferring many sub directories of information. We would really like to 
> know if all, some, or none of the files successfully transferred. If you 
> could please point us in the right direction on how to solve this issue, any 
> information would be very helpful. Looking forward to hearing back from you.
> 
> Thank you,
> Benjamin Ferraro

Re: [gt-user] 3rd-party transfer cannot end in some circumstances

[Foster, Ian T.]

Have you considered using the Globus transfer service? 
(Globus.org)

On Jun 23, 2015, at 3:43 PM, Fabio Moreira 
mailto:souza.fab...@gmail.com>> wrote:

Hi,

My team and I have developed an web application based on jGlobus and observed 
the same issue described by Xishi. However, instead of extendedTransfer(), we 
use extendedMultipleTransfer(), since we are transferring multiple files.  What 
we observed is that the transfer hangs at 99% (observed by PerfMarkers), 
meaning that when the hanging occurs it is always when transferring the last 
part of the last file (or after its finish). It is also noteworthy that 
org.globus.ftp.MultipleTransferCompleteListener#transferComplete() is not 
informed that the file transfer has ended, even though the file has the same 
size in origin and destination.

I have tried looking jGLobus github project to check if any modifications 
implemented on version 1.8.0 to extendedTransfer() was missed in 
extendedMultipleTransfer, but github repository only tracks from jGlobus V2 
ahead.

The versions for each maven dependency is listed below.
 - JGlobus-Core : 2.0.4
 - JGlobus-GridFTP: 2.0.0
 - JGlobus-Tomcat: 2.0.4
 - myproxy: 2.0.6

Globus Toolkit version (in both endpoints): 5.2.5

___


A similar issue was identified in jglobus and fixed in the most recent release 
1.8.0.  Please see if
that version solves your problem.

Xishi PAN wrote:
> Hi,
> We are developing a transfer service with jGlobus1.7.0 and Globus
> Toolkit 4.2.1. It's a multithreaded service. Each thread has an instance of
> org.globus.ftp.GridFTPClient for 3rd-party transfers.
> In some circumstances (for example, very heavy system load), we found
> that a 3rd-party transfer cannot end occasionally even if the file has been
> completely transferred. The work thread just hung there for hours. According
> to the debug info,the sending end has already sent "226 Transfer Complete."
> and I think its monitor thread has already exited according to the debug
> message "thread dying naturally".
> I've read some source codes about the
> "org.globus.ftp.GridFTPClient.extendedTransfer()" call.I'm afraid the other
> monitor thread is still blocking so function call "extendedTransfer" cannot
> return normally.
> Is it a known issue or Is there any workaround to this problem?
> Currently we have created a guard thread to check if a transfer has been in
> idle state for too long.
> Any input will be appreciated.
> Thanks!

--
Fábio MS