Re: [PHP] Upload Progress Meter - what's the latest?
On Tue, April 25, 2006 9:55 pm, D. Dante Lorenso wrote: > Richard Lynch wrote: > that > the replies I've gotten from this list so far suggest that internals > of > PHP development are seriously opposed to such a feature? They may or may not be opposed to attempting to use server-side technology to provide client-side statistical data, for a file that the CLIENT is uploading. Which is why I suggested that the request for server-side statistical tracking of uploads in progress for the server-side statistical analysis would be a Good Thing to propose. > I am more > interested in which project extensions are leading in momentum to > solve > the problem I am trying to solve and not blaming other projects > (browsers) for not building the feature first. The momentum, as far as I know, of all the project extensions to do what you describe is zero (0). > The back-and-forth you refer to is termed 'polling'. With a protocol > like HTTP, polling is really the only option unless I implement a > socket > server which is capable of sending events. Gah! I know what it's called. Let's try AGAIN. You want to use POLLING from server to client to measure how much data the client has sent. Does not the CLIENT already *KNOW* how much data it sent? Why in the world would you generate all that traffic to compute data already in existence on the client in the first place? That is the fundamental problem to which objections are raised on this issue. It's just plain daft to poll the server to find out how much data you sent it, when you're the one sending the data. > Right. Under the callback method example, I would have access to > $_SESSION which would already contain enough information about the > logged in account for my purposes. This is trivial. If it's trivial, code up the patch and send it. >> For tying the filenames back to the users who are logged in, you'd >> have to have an onClick() on the submit button which would send some >> kind of information (filename, filesize, user) to a PHP script that >> would log the upload about to begin in a database. The onClick >> would >> then return false so that the normal form submission process would >> kick in. >> > > This will not work. And this is exactly WHY an extention/patch is > needed. Between the time PHP choses a /tmp file name and the time the > upload is completed, there is not execution of PHP code. Therefor, we > don't know the /tmp name assigned to an upload until AFTER the upload > is > completed and then it's too late. Oh but there *IS* execution of all kinds of PHP code between that time -- Just not in the script that is receiving the file. Since you're building a statistical monitoring of the process, I'm suggesting that you have ANOTHER PHP script running to examine the state of /tmp and monitor the progress from "outside" the script actually receiving the file. It won't be pretty, but it can be done with what's available in PHP today, without patches. >> You're not going to get ... accuracy ... >> It won't be so hot for tracking who's uploading what in real-time, > > It should be. Using the patch, I already have this working. I now > want > a cleaner, supported solution which doesn't involve a custom patch > which > will not be supported in the future. Although the patch I am using > works, it was build for PHP 5.0.3 RC1 and since I am using 5.1.2 you > can > understand my concerns that it will fail to continue working when the > switch is made to PHP 5.2 or PHP 6.0. If you submit a similar patch, and specifically label it as server-side statistical analysis feature for uploaded files in progress, I SUSPECT it has a much better chance of "going" than a server-side hack to solve a client-side problem. >> Another option is to provide patch to PHP which gives a special >> INPUT >> parameter to a FORM that can be used to provide a callback file to >> load, and a callback function within that file to call, and then >> modify the file upload routine to load and call that function. > > Passing this from the client form is insecure and can not be trusted. > Declaring callback functions needs to remain a server-side declaration > either through code or an INI setting, etc. You're right. That would be a gaping security hole. >> It seems genuinely useful enough to this naive reader, and should >> not >> cause an undue burden for those who don't use it. Check with >> PHP-Dev >> to see if such a patch is likely to be accepted before putting in >> TOO >> much time on it. >> > > My original email was searching for advice on which project has > momentum > and what the current state of any discussions about the issue were. > Does anybody have information on the PEAR package which appears to be > vapor? > > http://pecl.php.net/package/postparser > > I see the name Curt Zirzow, but no contact information for that > "author". He posted a few zillion messages here in PHP-General... His email has to be in the Marc archives. > Is there not a
Re: [PHP] Upload Progress Meter - what's the latest?
On Tue, April 25, 2006 9:23 pm, D. Dante Lorenso wrote: > Richard Lynch wrote: >> On Tue, April 25, 2006 5:28 pm, D. Dante Lorenso wrote: >> >>> Everything PHP returns from a server is "client side", so your >>> >> Oh yeah. I forgot to say... >> The above presumption is patently false. >> > > Let me clarify: Assuming client/server architecture, if PHP is on the > server side then something else needs to be on the client side to > interpret the output of the PHP server. That output is what PHP > returns > and it is interpreted and rendered on the client side by definition. > > Alternatively: If it is not meant for the client to receive, it is > not > (should not) be output by PHP. Guess I'd better throw away all my command-line scripts and cron jobs, and all my PHP GTK half-finished projects, and... Plus, all the times I've got a web page that initiates some asynchronous process and outputs only a tiny fraction of its output to the browser, while generating a GREAT DEAL of output elsewhere. :-) Maybe we're just posting at cross-purposes here, but PHP does a hell of a lot more than spit out HTML web pages... -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Upload Progress Meter - what's the latest?
Richard Lynch wrote: Anything you see with "PHP" "upload progress meter" together has to be some kind of hack whose under-pinning is NOT PHP at all, but is JavaScript or similar client-side technology. Not true. The graphical display is HTML, DHTML, JavaScript, etc...but the means of monitoring the progress from the server side MUST be done with PHP. I'm sorry. Your original post sounded (to me) like you wanted a progress meter on the browser (client-side) for the user to see their files uploaded. Yes, I want that also. Really, I want the feature to exist in the PHP language regardless of how I intend to use it. That's the beauty of a programming language vs a program. I want the tools to be able to build what I want however I might want it at the time. Presumably even the worst browsers have some concept of how large the file is, and how many bytes have been sent to the server. So you certainly DO NOT need server-side information to implement a client-side progress meter of the progress of a file upload! Agreed. I SHOULD not NEED this feature. Other server-side languages exist, so I technically don't NEED PHP either. I don't really want to debate whether this feature should exist or not, but it seems to me that the replies I've gotten from this list so far suggest that internals of PHP development are seriously opposed to such a feature? I am more interested in which project extensions are leading in momentum to solve the problem I am trying to solve and not blaming other projects (browsers) for not building the feature first. The code that I have seen so far requires hooks into PHP which execute callback functions periodically in order to update the progress of uploaded files as PHP processes input. Yes, which is a silly place to try and "fix" this for client-side progress meters, as you then generate traffic back-and-forth from the client to measure the number of bytes received, instead of the browser just measuring bytes sent. The back-and-forth you refer to is termed 'polling'. With a protocol like HTTP, polling is really the only option unless I implement a socket server which is capable of sending events. This is something I may be building into the project also, but aside from HOW I move the server-side collect data back to a web client, that is independent of needing to collect the information on the server side in a format which can be accessed and read easily. browser in all cases. From the server side, I need to be able to monitor * who is currently uploading files, * which files are being uploaded, * how big are the files being uploaded, * how much time has passed since upload began, and * what percentage of those files is completed I do not need to return this data to a client at all times. Sometimes I just want to use the data system administration, analysis, and tracking. The most popular need would be for returning the information to the client, however, as you suggest. You'd need a battery of meters, for all running processes, or at least all the ones currently in file upload state. I would like to have a list of all file uploads in progress, yes. A semi-frequently updated snapshot of the information I defined would be sufficient. If a callback function is invoked periodically it would be equivalent to firing an event and would allow a php program to log the data to a common data repository which could be ready by another process on the system. You'd also need tie-ins to your login/authentication to determin "who is uploading files" as the only answer inherent to PHP is "the User set in httpd.conf" Right. Under the callback method example, I would have access to $_SESSION which would already contain enough information about the logged in account for my purposes. This is trivial. How big they are relies on the browser sending the correct Content-length: for the upload. I don't know how often they get it right, but I'm betting that are some pretty common errors in this area that PHP corrects for already. So your meters would not be accurate in some (hopefully few) cases. The POST method sets the Content-length, right. So, PHP must trust that the browsers are not lying. For security purposes we don't need to worry about whether this number is wrong because it is only for display. Processing uploaded files does not occur until the file completes uploading. While I can see where this might be a useful tool to have server-side, I can suggest several possible ways to implement some portions of it without hacking PHP source. First, ... you could write a tool to watch /tmp for files and get a list of the /tmp names being uploaded. Yes, this is simple. Using filectime and filemtime would tell you the timing info -- Though Windows will suck (as usual) as it only tracks to the nearest minute or something... Yes. Using Linux. This will work. For
Re: [PHP] Upload Progress Meter - what's the latest?
Richard Lynch wrote: On Tue, April 25, 2006 5:28 pm, D. Dante Lorenso wrote: Everything PHP returns from a server is "client side", so your Oh yeah. I forgot to say... The above presumption is patently false. Let me clarify: Assuming client/server architecture, if PHP is on the server side then something else needs to be on the client side to interpret the output of the PHP server. That output is what PHP returns and it is interpreted and rendered on the client side by definition. Alternatively: If it is not meant for the client to receive, it is not (should not) be output by PHP. Dante -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Upload Progress Meter - what's the latest?
[snip] ...a pretty good discussion... [/snip] I have used output buffering to flush stuff (like lengthy data) to the client before the end of the script so that those pesky users could see something was happening (because they couldn't be bothered to watch the browser's progress meter) and at one time thought that you might be able to do something like a progress meter with that. Since the http process is stateless it would be hard to provide a truly accurate progress meter. There are too many factors at play. Using PHP in an Ajax environment might move it a little closer and is an interesting thought and depends upon the reliability of the asynchronous request. There is the old stand-by...send an animated gif to the browser with no real timing involved. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Upload Progress Meter - what's the latest?
On Tue, April 25, 2006 5:27 pm, Jochem Maas wrote: > Richard Lynch wrote: >> On Tue, April 25, 2006 12:18 pm, D. Dante Lorenso wrote: >> Why don't you ask the guys who write BROWSERS why *they* don't >> provide >> a nice API/interface to display progress, or, better yet, why the >> browser itself doesn't just do it once and for all for everybody? > > let's consider that 99% of all clients would want a 'nice' looking > progress bar (in the same way they often asking me to do something > about the std form fields, which they consider ugly). > > i.e. it would just be moving the problem somewhere else in practical > terms, in so far as we'd probably still have to figure out hacks to > make > the bl**dy thing look nice (in the eyes of the guy/girl paying the > bill ;-) Well, yes, if the browser doesn't provide "hooks" for the look n feel of the darn thing, we're not solving anything. I still don't think a progress meter for browsers is something that belongs in PHP in any way, shape, or form. Please note my other much longer post on useful server-side statistical info about file uploads, however. There *IS* a progress meter on some browsers down in the status bar, or in other places, which, apparently, the average user is too clueless to notice consciously. Though I have noted that semi-experienced users "know" it is there subconsciously if you ask them the right questions about how far along the upload is. Very weird animal, those users... :-) -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Upload Progress Meter - what's the latest?
On Tue, April 25, 2006 5:28 pm, D. Dante Lorenso wrote: > Everything PHP returns from a server is "client side", so your Oh yeah. I forgot to say... The above presumption is patently false. :-) -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Upload Progress Meter - what's the latest?
On Tue, April 25, 2006 5:28 pm, D. Dante Lorenso wrote: > Richard Lynch wrote: >> On Tue, April 25, 2006 12:18 pm, D. Dante Lorenso wrote: >> Anything you see with "PHP" "upload progress meter" together has to >> be >> some kind of hack whose under-pinning is NOT PHP at all, but is >> JavaScript or similar client-side technology. >> > > Not true. The graphical display is HTML, DHTML, JavaScript, etc...but > the means of monitoring the progress from the server side MUST be done > with PHP. I'm sorry. Your original post sounded (to me) like you wanted a progress meter on the browser (client-side) for the user to see their files uploaded. AFAIK, all the tools you referenced were written with that purpose in mind. Presumably even the worst browsers have some concept of how large the file is, and how many bytes have been sent to the server. So you certainly DO NOT need server-side information to implement a client-side progress meter of the progress of a file upload! > The code that I have seen so far requires hooks into PHP > which execute callback functions periodically in order to update the > progress of uploaded files as PHP processes input. Yes, which is a silly place to try and "fix" this for client-side progress meters, as you then generate traffic back-and-forth from the client to measure the number of bytes received, instead of the browser just measuring bytes sent. > browser in all cases. From the server side, I need to be able to > monitor > > * who is currently uploading files, > * which files are being uploaded, > * how big are the files being uploaded, > * how much time has passed since upload began, and > * what percentage of those files is completed > > I do not need to return this data to a client at all times. Sometimes > I > just want to use the data system administration, analysis, and > tracking. The most popular need would be for returning the > information > to the client, however, as you suggest. In this case, a "meter" isn't going to cut it... You'd need a battery of meters, for all running processes, or at least all the ones currently in file upload state. You'd also need tie-ins to your login/authentication to determin "who is uploading files" as the only answer inherent to PHP is "the User set in httpd.conf" Which files are being uploaded would presumably display both the original name as sent by the browser, which is not something one can trust for anything useful, and the /tmp name. How big they are relies on the browser sending the correct Content-length: for the upload. I don't know how often they get it right, but I'm betting that are some pretty common errors in this area that PHP corrects for already. So your meters would not be accurate in some (hopefully few) cases. While I can see where this might be a useful tool to have server-side, I can suggest several possible ways to implement some portions of it without hacking PHP source. First, it's a pretty safe bet that PHP uses a common prefix on the files being uploaded. So you could write a tool to watch /tmp for files and get a list of the /tmp names being uploaded. That would at least tell you how many files are currently in upload status. Using filectime and filemtime would tell you the timing info -- Though Windows will suck (as usual) as it only tracks to the nearest minute or something... For tying the filenames back to the users who are logged in, you'd have to have an onClick() on the submit button which would send some kind of information (filename, filesize, user) to a PHP script that would log the upload about to begin in a database. The onClick would then return false so that the normal form submission process would kick in. You would then have to use heuristics about the file timing to "guess" which files were most likely which -- though you could probably get pretty accurate about that part, especially if you are willing to code it so that completed uploads tie back in and eliminate incorrect guesses. You're not going to get the level of accuracy you'd have with hacked PHP source, but you can get enough accuracy for statistical analysis. It won't be so hot for tracking who's uploading what in real-time, due to the guessing part -- but after the upload finishes your information will be as accurate as it can be with hacked source. Another option is to provide patch to PHP which gives a special INPUT parameter to a FORM that can be used to provide a callback file to load, and a callback function within that file to call, and then modify the file upload routine to load and call that function. It seems genuinely useful enough to this naive reader, and should not cause an undue burden for those who don't use it. Check with PHP-Dev to see if such a patch is likely to be accepted before putting in TOO much time on it. HTH >> Why don't you ask the guys who write BROWSERS why *they* don't >> provide >> a nice API/interface to display progress, or, better yet, why the
Re: [PHP] Upload Progress Meter - what's the latest?
Richard Lynch wrote: On Tue, April 25, 2006 12:18 pm, D. Dante Lorenso wrote: I'm trying to figure out which direction the PHP community is going when it comes to an upload progress meter.Since that meter would necessarily be CLIENT side, the PHP community is pretty much ignoring it, since PHP runs on the SERVER. Everything PHP returns from a server is "client side", so your argument seems to indicate that we need to write all of our HTML using JavaScript also. No, the fact is that a client-side GUI does NOT exist, so we need to build it ourselves using the tools we have. I do not have source control access to Mozilla, IE, Safari, and Opera, but I can control my own PHP source. PHP receives uploads from clients and that client may not even be a web browser in all cases. From the server side, I need to be able to monitor * who is currently uploading files, * which files are being uploaded, * how big are the files being uploaded, * how much time has passed since upload began, and * what percentage of those files is completed I do not need to return this data to a client at all times. Sometimes I just want to use the data system administration, analysis, and tracking. The most popular need would be for returning the information to the client, however, as you suggest. Right now, PHP stops executing code from the time an upload begins until the time it is completed. In other languages like PERL as CGI, you can read STDIN directly in order to slurp in large POST data like file attachments. In such a case as that, solving this problem is trivial. Since PHP locks a programmer out of the loop during parsing and importing POST and GET variables, we need PHP code to break into that loop and return callback functions or update internal data structures directly. Part of the problem in addition to collection of the data is being able to share it with other PHP instances. Remember that PHP runs in Apache in most cases and does not have shared memory between PHP instances. Therefore, writing the upload data to an external data storage is necessary. That external storage could be Sqlite, PostgreSQL, MySQL, files, or even Memcache. I would love to see a memcache-like solution. But more importantly, I would like to see a solution which is adopted by the community rather than miserably hacked together then not supported and abandoned. Anything you see with "PHP" "upload progress meter" together has to be some kind of hack whose under-pinning is NOT PHP at all, but is JavaScript or similar client-side technology. Not true. The graphical display is HTML, DHTML, JavaScript, etc...but the means of monitoring the progress from the server side MUST be done with PHP. The code that I have seen so far requires hooks into PHP which execute callback functions periodically in order to update the progress of uploaded files as PHP processes input. Why don't you ask the guys who write BROWSERS why *they* don't provide a nice API/interface to display progress, or, better yet, why the browser itself doesn't just do it once and for all for everybody? That is a good idea, and I will recommend it for future solutions. For short term, that is not the focus of this email. Dante -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Upload Progress Meter - what's the latest?
Richard Lynch wrote: On Tue, April 25, 2006 12:18 pm, D. Dante Lorenso wrote: I'm trying to figure out which direction the PHP community is going when it comes to an upload progress meter. Since that meter would necessarily be CLIENT side, the PHP community is pretty much ignoring it, since PHP runs on the SERVER. Anything you see with "PHP" "upload progress meter" together has to be some kind of hack whose under-pinning is NOT PHP at all, but is JavaScript or similar client-side technology. Why don't you ask the guys who write BROWSERS why *they* don't provide a nice API/interface to display progress, or, better yet, why the browser itself doesn't just do it once and for all for everybody? let's consider that 99% of all clients would want a 'nice' looking progress bar (in the same way they often asking me to do something about the std form fields, which they consider ugly). i.e. it would just be moving the problem somewhere else in practical terms, in so far as we'd probably still have to figure out hacks to make the bl**dy thing look nice (in the eyes of the guy/girl paying the bill ;-) and for the record there are other languages that support a progress bar idiom ... php being the pragmatic thing that it is should imho entertain this idea more seriously (waiting for browsers to implement it or ignoring the continually recurring request for such functionality) that said Richard has a very strong argument. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Upload Progress Meter - what's the latest?
On Tue, April 25, 2006 12:18 pm, D. Dante Lorenso wrote: > I'm trying to figure out which direction the PHP community is going > when > it comes to an upload progress meter. Since that meter would necessarily be CLIENT side, the PHP community is pretty much ignoring it, since PHP runs on the SERVER. Anything you see with "PHP" "upload progress meter" together has to be some kind of hack whose under-pinning is NOT PHP at all, but is JavaScript or similar client-side technology. Why don't you ask the guys who write BROWSERS why *they* don't provide a nice API/interface to display progress, or, better yet, why the browser itself doesn't just do it once and for all for everybody? -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Upload Progress Meter - what's the latest?
All, I'm trying to figure out which direction the PHP community is going when it comes to an upload progress meter. I've just recently discovered the ease of installing packages using PECL and see that this package is defined: http://pecl.php.net/package/postparser Yet, there does not appear to be any code which I can install for this module. Meanwhile, I find this project here: http://pdoru.from.ro/ (upload-progress-meter-v4.1) I have install this patch and built an RPM to aid in mass deployment on our servers and it works well enough with a few minor unresolved bugs. I'm trying to avoid custom patches in PHP as I work on migrating toward clean PHP 5.1.2 and eliminate BC dependencies. Does anyone know anything about the upload progress meter development and can point me in a better direction? Dante -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php