Re: [gt-user] Globus GridFTP web interface.
Hello Steve, That seems very interesting. Although I haven't tested deeply Globus Online. I'm in need to provide my users a web interface to upload big data files. I thought then that Globus could be a very good option. Since it's a tested and trusted system. But I also thought that web interface in Globus Online was available. I think what I really need is a web client for Gridftp, maybe I'm wrong. I'm a bit confused and I can't really find a tool that covers my needs. That's why I'm asking. Any guidelines or information about tools will be appreciated. Thank you very much. Best regards, Guillermo. On 03/12/2013 06:21 PM, Steve Tuecke wrote: Globus Online is a service that we (University of Chicago) run for the research community. It is not software that is distributed for local operation, but rather is software as a service. Use of Globus Online for file transfer and synchronization is free. You can sign up through our web site, and create your own endpoints on servers with Globus Connect Multi-User (https://www.globusonline.org/gcmu/) and on personal computers with Globus Connect (available on the dashboard after you login). We are just beginning beta testing of additional features for big data sharing (https://www.globusonline.org/plus/) that will be available for a modest subscription fee in order to help sustain this non-profit service. Regards, -Steve On Mar 12, 2013, at 4:42 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com wrote: Hello, I'm new to Globus. I've not been capable to install Globus yet. I'm interested in installing Globus to allow my users to upload big chunks of data. I've been playing around with Globus Online and I like the web interface. Is that web interface being distributed to use it? Thank you, Best regards, Guillermo.
Re: [gt-user] Globus GridFTP web interface.
Hello Markus, Could I see the solution you implemented? Did you use any API to start developing your frontend? Thank you. Best regards, Guillermo. On 03/13/2013 10:24 AM, Markus Binsteiner wrote: Guillermo, I don't think there exists a webclient for GridFTP, apart from the one that is part of GlobusOnline. And even with GlobusOnline you can't upload files. They would have to use GlobusConnect, which they need to install on their desktop machine. GlobusConnect is basically a gridftp server, just bundled up for easy install, without having to worry (too much) about the authentication side of things. Once that is installed, users can kick off (and monitor) transfers from the GlobusOnline website. Since that will be a 3rd party gridftp transfer it'll be quite fast. But it can introduce a few problems with firewalls and such, since the ports that need to be open are often (in my experience anyway) closed by University/company firewalls. Your mileage may vary though. I guess one could write a webfrontend for gridftp, where users could upload files via http and then the portal would forward the files to the endpoint via gridftp. I implemented a similar solution as a SOAP webservice. But it'd be quite a substantial piece of work. Even if you wouldn't have to worry about how to implement the authentication part. Which you probably would have to do. So, in short: I'd recommend you have a look at GlobusConnect and accept that users need to install something on their desktop machines. Everything else will be a lot of work... Good luck! Markus On Wed, 2013-03-13 at 09:21 +0100, Guillermo Marco Puche wrote: Hello Steve, That seems very interesting. Although I haven't tested deeply Globus Online. I'm in need to provide my users a web interface to upload big data files. I thought then that Globus could be a very good option. Since it's a tested and trusted system. But I also thought that web interface in Globus Online was available. I think what I really need is a web client for Gridftp, maybe I’m wrong. I'm a bit confused and I can't really find a tool that covers my needs. That's why I'm asking. Any guidelines or information about tools will be appreciated. Thank you very much. Best regards, Guillermo. On 03/12/2013 06:21 PM, Steve Tuecke wrote: Globus Online is a service that we (University of Chicago) run for the research community. It is not software that is distributed for local operation, but rather is software as a service. Use of Globus Online for file transfer and synchronization is free. You can sign up through our web site, and create your own endpoints on servers with Globus Connect Multi-User (https://www.globusonline.org/gcmu/) and on personal computers with Globus Connect (available on the dashboard after you login). We are just beginning beta testing of additional features for big data sharing (https://www.globusonline.org/plus/) that will be available for a modest subscription fee in order to help sustain this non-profit service. Regards, -Steve On Mar 12, 2013, at 4:42 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com wrote: Hello, I'm new to Globus. I've not been capable to install Globus yet. I'm interested in installing Globus to allow my users to upload big chunks of data. I've been playing around with Globus Online and I like the web interface. Is that web interface being distributed to use it? Thank you, Best regards, Guillermo.
Re: [gt-user] Globus GridFTP web interface.
Just to add my $0.02 Xsede uses a Java (I personally detest the thing) gridftp client. You can run it locally or in a webpage. Not sure how it is licensed or how to add your own endpoints. Brock Palen www.umich.edu/~brockp CAEN Advanced Computing bro...@umich.edu (734)936-1985 On Mar 13, 2013, at 6:21 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com wrote: Hello Markus, Could I see the solution you implemented? Did you use any API to start developing your frontend? Thank you. Best regards, Guillermo. On 03/13/2013 10:24 AM, Markus Binsteiner wrote: Guillermo, I don't think there exists a webclient for GridFTP, apart from the one that is part of GlobusOnline. And even with GlobusOnline you can't upload files. They would have to use GlobusConnect, which they need to install on their desktop machine. GlobusConnect is basically a gridftp server, just bundled up for easy install, without having to worry (too much) about the authentication side of things. Once that is installed, users can kick off (and monitor) transfers from the GlobusOnline website. Since that will be a 3rd party gridftp transfer it'll be quite fast. But it can introduce a few problems with firewalls and such, since the ports that need to be open are often (in my experience anyway) closed by University/company firewalls. Your mileage may vary though. I guess one could write a webfrontend for gridftp, where users could upload files via http and then the portal would forward the files to the endpoint via gridftp. I implemented a similar solution as a SOAP webservice. But it'd be quite a substantial piece of work. Even if you wouldn't have to worry about how to implement the authentication part. Which you probably would have to do. So, in short: I'd recommend you have a look at GlobusConnect and accept that users need to install something on their desktop machines. Everything else will be a lot of work... Good luck! Markus On Wed, 2013-03-13 at 09:21 +0100, Guillermo Marco Puche wrote: Hello Steve, That seems very interesting. Although I haven't tested deeply Globus Online. I'm in need to provide my users a web interface to upload big data files. I thought then that Globus could be a very good option. Since it's a tested and trusted system. But I also thought that web interface in Globus Online was available. I think what I really need is a web client for Gridftp, maybe I’m wrong. I'm a bit confused and I can't really find a tool that covers my needs. That's why I'm asking. Any guidelines or information about tools will be appreciated. Thank you very much. Best regards, Guillermo. On 03/12/2013 06:21 PM, Steve Tuecke wrote: Globus Online is a service that we (University of Chicago) run for the research community. It is not software that is distributed for local operation, but rather is software as a service. Use of Globus Online for file transfer and synchronization is free. You can sign up through our web site, and create your own endpoints on servers with Globus Connect Multi-User (https://www.globusonline.org/gcmu/) and on personal computers with Globus Connect (available on the dashboard after you login). We are just beginning beta testing of additional features for big data sharing (https://www.globusonline.org/plus/ ) that will be available for a modest subscription fee in order to help sustain this non-profit service. Regards, -Steve On Mar 12, 2013, at 4:42 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com wrote: Hello, I'm new to Globus. I've not been capable to install Globus yet. I'm interested in installing Globus to allow my users to upload big chunks of data. I've been playing around with Globus Online and I like the web interface. Is that web interface being distributed to use it? Thank you, Best regards, Guillermo.
Re: [gt-user] Globus GridFTP web interface.
On 03/13/2013 02:56 PM, Brock Palen wrote: Just to add my $0.02 Xsede uses a Java (I personally detest the thing) gridftp client. You can run it locally or in a webpage. Not sure how it is licensed or how to add your own endpoints. I suppose you're making reference to GSI-SSH client? http://sourceforge.net/projects/gsi-sshterm/ Brock Palen www.umich.edu/~brockp CAEN Advanced Computing bro...@umich.edu (734)936-1985 On Mar 13, 2013, at 6:21 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com wrote: Hello Markus, Could I see the solution you implemented? Did you use any API to start developing your frontend? Thank you. Best regards, Guillermo. On 03/13/2013 10:24 AM, Markus Binsteiner wrote: Guillermo, I don't think there exists a webclient for GridFTP, apart from the one that is part of GlobusOnline. And even with GlobusOnline you can't upload files. They would have to use GlobusConnect, which they need to install on their desktop machine. GlobusConnect is basically a gridftp server, just bundled up for easy install, without having to worry (too much) about the authentication side of things. Once that is installed, users can kick off (and monitor) transfers from the GlobusOnline website. Since that will be a 3rd party gridftp transfer it'll be quite fast. But it can introduce a few problems with firewalls and such, since the ports that need to be open are often (in my experience anyway) closed by University/company firewalls. Your mileage may vary though. I guess one could write a webfrontend for gridftp, where users could upload files via http and then the portal would forward the files to the endpoint via gridftp. I implemented a similar solution as a SOAP webservice. But it'd be quite a substantial piece of work. Even if you wouldn't have to worry about how to implement the authentication part. Which you probably would have to do. So, in short: I'd recommend you have a look at GlobusConnect and accept that users need to install something on their desktop machines. Everything else will be a lot of work... Good luck! Markus On Wed, 2013-03-13 at 09:21 +0100, Guillermo Marco Puche wrote: Hello Steve, That seems very interesting. Although I haven't tested deeply Globus Online. I'm in need to provide my users a web interface to upload big data files. I thought then that Globus could be a very good option. Since it's a tested and trusted system. But I also thought that web interface in Globus Online was available. I think what I really need is a web client for Gridftp, maybe I’m wrong. I'm a bit confused and I can't really find a tool that covers my needs. That's why I'm asking. Any guidelines or information about tools will be appreciated. Thank you very much. Best regards, Guillermo. On 03/12/2013 06:21 PM, Steve Tuecke wrote: Globus Online is a service that we (University of Chicago) run for the research community. It is not software that is distributed for local operation, but rather is software as a service. Use of Globus Online for file transfer and synchronization is free. You can sign up through our web site, and create your own endpoints on servers with Globus Connect Multi-User (https://www.globusonline.org/gcmu/) and on personal computers with Globus Connect (available on the dashboard after you login). We are just beginning beta testing of additional features for big data sharing (https://www.globusonline.org/plus/ ) that will be available for a modest subscription fee in order to help sustain this non-profit service. Regards, -Steve On Mar 12, 2013, at 4:42 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com wrote: Hello, I'm new to Globus. I've not been capable to install Globus yet. I'm interested in installing Globus to allow my users to upload big chunks of data. I've been playing around with Globus Online and I like the web interface. Is that web interface being distributed to use it? Thank you, Best regards, Guillermo.
Re: [gt-user] Globus GridFTP web interface.
This topic of using Golbus Connect vs another form of web based client is interesting, and relevant to our situation as well. As Markus pointed out, while the Globus Connect client is extremely useful the ports used (outbound tcp 2223 and 5-51000 if I recall) are commonly blocked by organizations. I've encountered this several times when trying to get remote users set up to use our end points in Globus Online. At that point, using Globus Connect and Globus Online as a simple, easy way to move your data comes to a halt - through no fault of Globus of course, it has to use ports of course. If there were a web deployable (ruby, python, node.js or tomcat [please not tomcat]) gateway which could allow a user to authenticate to Globus Online and the endpoint, it could (I believe) help make using Globus easier for users in this situation. They could then at least be able to upload data, though if outbound ports are blocked, and traffic could only go out via 80 and 443, this would mean single stream transfers and slow transfer rates so I'm not sure how practical this is in the long run. Perhaps another option might be to allow a proxy to be configured in the Globus Connect client so that the outbound myproxy and data traffic could be directed through a single point to enable organizations to better monitor such traffic, as they are naturally reluctant to allow outbound traffic from these ports from all devices on their network. It might also be helpful to have information on the Globus Connect site for users to verify connectivity to relay.globusonline.com and something listening on port 5 so they could check via telnet relay.globusonline.org 2223 telnet somthing.listenting.com 5 (It would be nice to have a a Check Network Compatibility button on the Globus Connect Download page verify outbound ports, that's of course that is tricky. For web based tools, I've really only found services like http://www.firebind.com which use a java client to do so, or some binary client etc. the user must download which is non-optima. I guess a simple node.js, apache etc. web server running on port 2223 and 5 with links to it and a You connected might work - though since http headers would be used it might be allowed if packet inspecting firewalls allowed outbound http on ports other than 80.) I know there is not really a magic answer to all this, but I find myself coming back to these points. I would like to use Globus more, and have several instances where people need to download and upload multi terabyte sized data sets, but the number of steps currently includes: Needing a GO account, an account on the end-point, installing they GO client, having outbound ports open (which they usually don't know until this point) configuring the client, etc. I fully appreciate how easy Globus Online makes transferring data, but for many users the above is too much to ask. From a researcher or collaborator perspective I believe they tend to think why can't I just just ftp or scp and be done with it? Of course we know why, but that does not improve the above for them. Pete On Wed, Mar 13, 2013 at 10:36 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com wrote: On 03/13/2013 02:56 PM, Brock Palen wrote: Just to add my $0.02 Xsede uses a Java (I personally detest the thing) gridftp client. You can run it locally or in a webpage. Not sure how it is licensed or how to add your own endpoints. I suppose you're making reference to GSI-SSH client? http://sourceforge.net/projects/gsi-sshterm/ Brock Palenwww.umich.edu/~brockp CAEN Advanced computingbro...@umich.edu(734)936-1985 On Mar 13, 2013, at 6:21 AM, Guillermo Marco Puche guillermo.ma...@sistemasgenomicos.com guillermo.ma...@sistemasgenomicos.com wrote: Hello Markus, Could I see the solution you implemented? Did you use any API to start developing your frontend? Thank you. Best regards, Guillermo. On 03/13/2013 10:24 AM, Markus Binsteiner wrote: Guillermo, I don't think there exists a webclient for GridFTP, apart from the one that is part of GlobusOnline. And even with GlobusOnline you can't upload files. They would have to use GlobusConnect, which they need to install on their desktop machine. GlobusConnect is basically a gridftp server, just bundled up for easy install, without having to worry (too much) about the authentication side of things. Once that is installed, users can kick off (and monitor) transfers from the GlobusOnline website. Since that will be a 3rd party gridftp transfer it'll be quite fast. But it can introduce a few problems with firewalls and such, since the ports that need to be open are often (in my experience anyway) closed by University/company firewalls. Your mileage may vary though. I guess one could write a webfrontend for gridftp, where users could upload files via http and then the portal would forward the files to the endpoint via gridftp. I implemented a similar
