Re: FTP upload frustrations
> > Contradiction please. Dennie is *correct* in pointing out prevalent and irritating >performance irregularities with libURL. > > I too have had great hopes for MC FTP, but after hundreds of tests, can confirm only >too well exactly the same results. More disheartening is that one > week ago, I sent the same information to a prominent member of this list, going so >far as to ask for an MC FTP FAQ. The plea has been ignored. So you are pissed I didn't get to answer your email. I would've done it instantly have I known you were so touchy. Let me get something straight to you in case you didn't notice, I spent many hours on this list helping people out with different matters related to sockets, protocols, syntax and so on and I did it out of pleasure not obligation. A good part of your unanswered mail deals with syntax related to reading files not ftp, although I will provide instructions for whatever the library does, I will not make it my job to jump on every bad syntax issue and bring light to the masses with a FAQ. There is documentation for reading/writing files, if people invent their own syntax, let them. The library has been in testing as long as MC 2.4 has, most of the issues brought to my attention were related to HTTP and I tried dealing with them as fast as I could. People discovered ftp in the past 3 weeks or so and again, I fix bugs as soon as they are found, by me or others. Some bugs I can't fix because I can't reproduce them. You went "so far as to ask for an MC FTP FAQ". What is that supposed to mean? > From > prior experience with PERL socket programming, I've tried scripting defensive >safeguards and checks in libURL, but it just as often fails surreptitiously. > Many questions have arisen in this list over MC's FTP irregularities. As someone who >has spent too much time fighting it, my challenge is that this code > should be stabilized, and clarified via FAQ, or brought back into the shop as not >being ready for production use. You stabilize and clarify your code via FAQ, I'll let mine out to be tested and bug reported. Be convinced, I do my testing with as many servers as I can find, but real world tests with all kinds of connections (or non connections) and servers will be more valuable to everyone than a FAQ, as you imagine it. It is my expectation of libURL to be of production quality and I'm fully aware it still needs work and mostly constructive testing. > > The most successful corrective measures I have taken are (a) to increase the >socketTimeoutInterval from the default 1 millisecondsto a larger > number, like 5; and (b) to invoke libURL's "resetAll" handler before the next >PUT URL or GET... URL transfer request. ResetAll closes all sockets > -- a drastic measure and one that improved my chances of success only moderately. > > As for the first problem -- timeouts -- it is common in FTP implementations that the >connection can close before all data is sent. Sometimes the close > on the data socket returns before all data are from the TCP socket are sent, and as >ftp remote server receives a new command on the FTP command > socket it closes the data socket, thus truncating the file. I never experienced the communality of ftp servers closing connections as you did but I agree error handling needs more work. > > libURL's error handling routines are naive. The socketError call, for instance, >which is supposed to be invoked on errors, assumes that one can't open > the connection *only* if the result sent to it is the string "Error 111 reading >socket", rather than a more general failure condition. This means that all > other return codes not exactly equal to this string (and FTP flavors are free to >implement their own strings) will go undetected. For its own part the > socketError handler is never called inside libURL. At least the "try" control >structure should be implemented to catch any realtime errors, regardless of > origin. It is never used in the 1400+ lines of libURL stack script. This one you got wrong, the error you mention is a socket error I receive from the engine not ftp. Why should it be "called" and by whom? Shall I make up stories so as to protect the sensibilities of some users? > > On the same topic, the error diagnostics libURL produces are often inaccurate. If >you have a valid url such as > "ftp://user1202:[EMAIL PROTECTED]"; but your net connection is down (or perhaps >your laptop in Starbucks isn't on the Net), libURL responds to > a failed connection with the spurious "invalid URL" message that will throw off any >deeper error processing. The list of problems goes on. My mistake, I assumed I should provide error messages for computers connected to the internet but you convinced me, people do try to use ftp without a connection and I should consider them too. >50-80% of the > time, the code works 100% well. The rest of the time, it works erratically. If you >only use it once in a while, the odds of success ar
Re: FTP upload frustrations
Contradiction please. Dennie is *correct* in pointing out prevalent and irritating performance irregularities with libURL. I too have had great hopes for MC FTP, but after hundreds of tests, can confirm only too well exactly the same results. More disheartening is that one week ago, I sent the same information to a prominent member of this list, going so far as to ask for an MC FTP FAQ. The plea has been ignored. From prior experience with PERL socket programming, I've tried scripting defensive safeguards and checks in libURL, but it just as often fails surreptitiously. Many questions have arisen in this list over MC's FTP irregularities. As someone who has spent too much time fighting it, my challenge is that this code should be stabilized, and clarified via FAQ, or brought back into the shop as not being ready for production use. The most successful corrective measures I have taken are (a) to increase the socketTimeoutInterval from the default 1 millisecondsto a larger number, like 5; and (b) to invoke libURL's "resetAll" handler before the next PUT URL or GET... URL transfer request. ResetAll closes all sockets -- a drastic measure and one that improved my chances of success only moderately. As for the first problem -- timeouts -- it is common in FTP implementations that the connection can close before all data is sent. Sometimes the close on the data socket returns before all data are from the TCP socket are sent, and as ftp remote server receives a new command on the FTP command socket it closes the data socket, thus truncating the file. libURL's error handling routines are naive. The socketError call, for instance, which is supposed to be invoked on errors, assumes that one can't open the connection *only* if the result sent to it is the string "Error 111 reading socket", rather than a more general failure condition. This means that all other return codes not exactly equal to this string (and FTP flavors are free to implement their own strings) will go undetected. For its own part the socketError handler is never called inside libURL. At least the "try" control structure should be implemented to catch any realtime errors, regardless of origin. It is never used in the 1400+ lines of libURL stack script. On the same topic, the error diagnostics libURL produces are often inaccurate. If you have a valid url such as "ftp://user1202:[EMAIL PROTECTED]" but your net connection is down (or perhaps your laptop in Starbucks isn't on the Net), libURL responds to a failed connection with the spurious "invalid URL" message that will throw off any deeper error processing. The list of problems goes on. 50-80% of the time, the code works 100% well. The rest of the time, it works erratically. If you only use it once in a while, the odds of success are in your favor. In my experience, this library and its functionality, much touted in version 2.4, is undertested and undersupported. It's not hard to test FTP -- it's very easy in fact -- but it has to be done in volume under pessimistic scenarios (large file sizes; diverse FTP server environment). None of my tests used files larger than 25K of text. The FTP server (Compaq Proliant running Linux) works through other attempts, so I rule it out Besides, if it fails, the client should still know and warn the user. Real QA would be advised for the next release and as implemented, and as for critical data transfers, libURL has a credibility problem. andu <[EMAIL PROTECTED]> wrote: Dennie Hoopingarner wrote:> > OK, I admit it, I'm a complete newby when it comes to socket> programming. I am baffled by the mixed results that I get when using> the new libURL, especially for FTP.> > I can follow the "put url 'binfle:/' into url 'ftp://..." part.> When I implement it, I get partially successful results. Sometimes it> works, and sometimes it doesn't. That's what is so frustrating. If it> didn't ever work, then I could stop obsessing about it and look for> another solution. Here's what I'm trying to do.> > I have a text file on the desktop, in a folder called "webRecord." I> want to upload it to my FTP site, using this script:> > put empty into field 1> put url "binfile:/G4 Titanium/Desktop Folder/webRecord/test.txt"> into url "ftp://username:password@my! .ftp.site/test.txt"> put the result into field 1> > I click the button, and after a few seconds get the "upload complete"> message. But that's where the inconsistent results emerge.> > Sometimes the file is uploaded correctly. Sometimes I get a zero byte> file. Most of the time, though, I get nothing at all. No file is> found on the server. It seems to happen if I try more than once. If I> quit Metacard and then try again, I have better luck. Still not> consistent, though.> > I assume that I am doing something wrong. Can one of you more> knowledgeable souls help me out here, so that I can sleep at night?You're not doing anything wrong, the upload takes place properly it'sjust the result that's misplaced. It will be fi
Re: FTP upload frustrations
Dennie Hoopingarner wrote: > > OK, I admit it, I'm a complete newby when it comes to socket > programming. I am baffled by the mixed results that I get when using > the new libURL, especially for FTP. > > I can follow the "put url 'binfle:/' into url 'ftp://..."; part. > When I implement it, I get partially successful results. Sometimes it > works, and sometimes it doesn't. That's what is so frustrating. If it > didn't ever work, then I could stop obsessing about it and look for > another solution. Here's what I'm trying to do. > > I have a text file on the desktop, in a folder called "webRecord." I > want to upload it to my FTP site, using this script: > >put empty into field 1 >put url "binfile:/G4 Titanium/Desktop Folder/webRecord/test.txt" > into url "ftp://username:[EMAIL PROTECTED]/test.txt"; >put the result into field 1 > > I click the button, and after a few seconds get the "upload complete" > message. But that's where the inconsistent results emerge. > > Sometimes the file is uploaded correctly. Sometimes I get a zero byte > file. Most of the time, though, I get nothing at all. No file is > found on the server. It seems to happen if I try more than once. If I > quit Metacard and then try again, I have better luck. Still not > consistent, though. > > I assume that I am doing something wrong. Can one of you more > knowledgeable souls help me out here, so that I can sleep at night? You're not doing anything wrong, the upload takes place properly it's just the result that's misplaced. It will be fixed for the next release. > Thanks a lot! > > Dennie > > -- > > Archives: http://www.mail-archive.com/metacard@lists.runrev.com/ > Info: http://www.xworlds.com/metacard/mailinglist.htm > Please send bug reports to <[EMAIL PROTECTED]>, not this list. Archives: http://www.mail-archive.com/metacard@lists.runrev.com/ Info: http://www.xworlds.com/metacard/mailinglist.htm Please send bug reports to <[EMAIL PROTECTED]>, not this list.
FTP upload frustrations
OK, I admit it, I'm a complete newby when it comes to socket programming. I am baffled by the mixed results that I get when using the new libURL, especially for FTP. I can follow the "put url 'binfle:/' into url 'ftp://..."; part. When I implement it, I get partially successful results. Sometimes it works, and sometimes it doesn't. That's what is so frustrating. If it didn't ever work, then I could stop obsessing about it and look for another solution. Here's what I'm trying to do. I have a text file on the desktop, in a folder called "webRecord." I want to upload it to my FTP site, using this script: put empty into field 1 put url "binfile:/G4 Titanium/Desktop Folder/webRecord/test.txt" into url "ftp://username:[EMAIL PROTECTED]/test.txt"; put the result into field 1 I click the button, and after a few seconds get the "upload complete" message. But that's where the inconsistent results emerge. Sometimes the file is uploaded correctly. Sometimes I get a zero byte file. Most of the time, though, I get nothing at all. No file is found on the server. It seems to happen if I try more than once. If I quit Metacard and then try again, I have better luck. Still not consistent, though. I assume that I am doing something wrong. Can one of you more knowledgeable souls help me out here, so that I can sleep at night? Thanks a lot! Dennie -- Archives: http://www.mail-archive.com/metacard@lists.runrev.com/ Info: http://www.xworlds.com/metacard/mailinglist.htm Please send bug reports to <[EMAIL PROTECTED]>, not this list.
