Re: WE GOT A PROBLEM
Have a status bit and set it in-progress. Don't set it to complete until you are sure its done and correct. Don't remove the info. You want it for debugging and a complete history anyway. 2011/3/25 Przemysław Rejf : > Hi, How to detect file upload abort using an upload hook in perl. When > client aborts the upload i need to remove the info about the file from the > db. I can't do it size base, because before upload ends i only have an > approximate file size (request + file). I'm tryin to use mod_perl > Apache2::Connection->aborted() but so far to no effect. > -- Make a Small Loan, Make a Big Difference - Check out Kiva.org to Learn How!
Re: memory leaking on POST
What bucket type is SPOOL? I don't see it as a real bucket type. I'd like to try out the patch here, but I can't duplicate it because I don't know what a spool bucket is. Do you mean HEAP? On 11/13/06, Joe Schaefer <[EMAIL PROTECTED]> wrote: "Philip M. Gollucci" <[EMAIL PROTECTED]> writes: > Joe Schaefer wrote: > >> Until someone teaches apreq_brigade_fwrite about spool buckets, >> it probably shouldn't be used for upload brigades. Sorry about that. > I was looking at this a while ago -- APR/U have this function. Really? What's it called? > We custom coded our own -- why ? Because I didn't know apr has something like it? In any case, the "mistake" here is that apreq_brigade_fwrite doesn't consume the brigade, but leaves it "intact". I should have made that function consume the brigade instead- my bad. The fix is easy enough- just use apr_file_read instead of apr_bucket_read when dealing with spool buckets. That'd fix the memory problem for this particular use-case. I wrote a patch last night, but I haven't had the opportunity to write any tests for it yet, and probably won't until the weekend comes around. -- Joe Schaefer
brigade to stream
I have a library function that reads from a file descriptor, and I'd like to jam it into a filter. I've been thinking that GNU libc has some cool features where one can define custom streams. If one defines a read, write, seek, close and a position data structure, GNU libc lets you open a stream via those tools, producing a FILE *. Maybe that would allow me to read from a brigade as if it were a FILE * and I could pass it on to this library function. Of course its very platform specific, but that is an overrated virtue anyway. What do you all think of the specific idea, and do you have any better way for me to get access to this library? I could give the library a buffer too, but the buffer would be way too large, so I want something more stream oriented. I'd like to be able to hand off a brigade to the library, but of course nobody else has brigades. Brian
Re: how to get the temp file name
Better still I passed in the request_record as the context and saved the information in the per request config structure for the module: apreq_hook_t * my_hook_handle = apreq_hook_make(req->pool, my_hook, NULL, req); With this technique I have access to the data even after an interruption causing an EOF is received. The data hangs around until the request is completed, so I can recover from the interruption. I would like to have this hook remove itself once it sets the value. How do I remove it? I see no function for removal of a hook. Would I have to step through the hook_queue, find the one that matches the current hook, and then move the rest of them up a slot? That sounds a bit messy. On 9/12/06, Brian McQueen <[EMAIL PROTECTED]> wrote: That does work, but it won't get me where I need to be. The problem is that the spoolfile isn't set after a parsing error - there is no brigade in param->upload upon a bad parse. I did find a simple way to get a filter in there: apreq_hook_t * my_hook_handle = apreq_hook_make(req->pool, my_hook, NULL, NULL); apreq_hook_add(aph, my_hook_handle); I think I'll be able to set a note with this technique. Brian McQueen On 9/12/06, Issac Goldstand <[EMAIL PROTECTED]> wrote: > Well, the spool file is just a normal apr_file_t, and the actual file > handle is accessable via apreq_brigade_spoolfile(), so you'll likely > want to do something like: > > (untested) > char[255] filename; > apr_status_t rv; > > rv = apr_file_name_get(&filename, apreq_brigade_spoolfile(param->upload)); > > Issac > > Brian McQueen wrote: > > I am trying to figure out how to get the name of the temp file created > > by apreq when it spools to the disk. I have been through several > > tries now and the most promising seems to be making a hook and adding > > it to the parser. The hook would get the name from the brigade as the > > request is parsed, then it could put the name into a note which my > > module could get later. I'm struggling now to add my hook to the > > parser. I have a parser function, but not a parser_t at that stage of > > my code - prior to reading and parsing - and the add_hook function > > requires a parser_t. At this stage the parser is not yet set. > > > > How can I directly add a hook to the parse_multipart parser? > > Do any of you have any better ideas? > > > > Brian McQueen >
Re: how to get the temp file name
That does work, but it won't get me where I need to be. The problem is that the spoolfile isn't set after a parsing error - there is no brigade in param->upload upon a bad parse. I did find a simple way to get a filter in there: apreq_hook_t * my_hook_handle = apreq_hook_make(req->pool, my_hook, NULL, NULL); apreq_hook_add(aph, my_hook_handle); I think I'll be able to set a note with this technique. Brian McQueen On 9/12/06, Issac Goldstand <[EMAIL PROTECTED]> wrote: Well, the spool file is just a normal apr_file_t, and the actual file handle is accessable via apreq_brigade_spoolfile(), so you'll likely want to do something like: (untested) char[255] filename; apr_status_t rv; rv = apr_file_name_get(&filename, apreq_brigade_spoolfile(param->upload)); Issac Brian McQueen wrote: > I am trying to figure out how to get the name of the temp file created > by apreq when it spools to the disk. I have been through several > tries now and the most promising seems to be making a hook and adding > it to the parser. The hook would get the name from the brigade as the > request is parsed, then it could put the name into a note which my > module could get later. I'm struggling now to add my hook to the > parser. I have a parser function, but not a parser_t at that stage of > my code - prior to reading and parsing - and the add_hook function > requires a parser_t. At this stage the parser is not yet set. > > How can I directly add a hook to the parse_multipart parser? > Do any of you have any better ideas? > > Brian McQueen
how to get the temp file name
I am trying to figure out how to get the name of the temp file created by apreq when it spools to the disk. I have been through several tries now and the most promising seems to be making a hook and adding it to the parser. The hook would get the name from the brigade as the request is parsed, then it could put the name into a note which my module could get later. I'm struggling now to add my hook to the parser. I have a parser function, but not a parser_t at that stage of my code - prior to reading and parsing - and the add_hook function requires a parser_t. At this stage the parser is not yet set. How can I directly add a hook to the parse_multipart parser? Do any of you have any better ideas? Brian McQueen
Re: Parser running twice?
This scares me a bit. I removed ap_setup_client_block(), and ap_should_client_block() and the module works fine! I thought those were necessary. Now I'm not sure what they do. I'll have to go back to the basics to check that out now. Brian McQueen On 9/11/06, Joe Schaefer <[EMAIL PROTECTED]> wrote: "Brian McQueen" <[EMAIL PROTECTED]> writes: > I noticed that there are two temp files generated by apreq when I am > uploading data. When the files are really big, like gigabytes, that > becomes a problem. mod_apreq spools the entire request to a tempfile, so you have that on plus the tempfile created for each file upload. > Does it seem right that there should be two temp files? The seem to > be identical - at least when the request is completed and they are not > yet deleted, I can see that they are the same size. It looks like the > parser must be running twice for some reason. The procedure is fairly > simple, but maybe I got it wrong. I am doing these steps: > > 1) apreq_handle_apache2 - init apreq > 2) ap_setup_client_block > 3) ap_should_client_block Not necessary. If you're in a handler, call ap_discard_request_body instead and mod_apreq will not need to spool the request. -- Joe Schaefer
Parser running twice?
I noticed that there are two temp files generated by apreq when I am uploading data. When the files are really big, like gigabytes, that becomes a problem. Does it seem right that there should be two temp files? The seem to be identical - at least when the request is completed and they are not yet deleted, I can see that they are the same size. It looks like the parser must be running twice for some reason. The procedure is fairly simple, but maybe I got it wrong. I am doing these steps: 1) apreq_handle_apache2 - init apreq 2) ap_setup_client_block 3) ap_should_client_block 4) apreq_body - trigger the parse 5) apreq_param - mess with the input Is there something wrong here? Brian McQueen
Re: untrapped error in filter
Replying to myself for the education of others. The filters do not write the data to any permanent location. The filter parses the data, and the content generators, or some other specific process is responsible for writing the data to a permanent location. The temp file is of no concern to the programmer. Its used by apreq to handle large files and is not related to the final actions of the users of apreq. I need to detect the write error myself in my content generator, or wherever it is that I wrote that file. Brian McQueen On 9/6/06, Brian McQueen <[EMAIL PROTECTED]> wrote: The parser use a temp file during its handling of some data. When that temp file is moved to its final location there seems to be no error detection. We noticed the problem when we were writing the temp file to a large empty partition, but the final destination for the file was a small partition - much too small for the file. The temp file was truncated when it was moved from the temp location to the final location and no error was indicated. I'll work on a fix if I find the code. Its a bit hard to find. Brian McQueen
untrapped error in filter
The parser use a temp file during its handling of some data. When that temp file is moved to its final location there seems to be no error detection. We noticed the problem when we were writing the temp file to a large empty partition, but the final destination for the file was a small partition - much too small for the file. The temp file was truncated when it was moved from the temp location to the final location and no error was indicated. I'll work on a fix if I find the code. Its a bit hard to find. Brian McQueen
apreq_body and apreq_param
I wonder about the differences between apreq_param and apreq_body. I see that apreq_param gets a particular param and returns a param structure, which is a bit of an odd structure, but its a general way of representing the data. When using apreq_body a more intuitive, string based key/value style is provided through the returned table. The table interface is easy to use and makes sense, but I am a bit confused about the emphasis on key/value techniques with apreq_body, versus some other technique when data is retrieved via apreq_param. What is the intention behind these two different designs? Brian McQueen
