Re: debugging a timeout issue
On May 9, 2008, at 6:23 AM, Graham Dumpleton wrote: 2008/5/9 Sam Carleton [EMAIL PROTECTED]: I am a one man ISV that is using an Apache and an Apache Module. I am trying to trouble shoot a timeout issue that I cannot see, my customer is reporting the problem and he can consistently repeat the problem. Sometimes when the .Net 2.0 client makes a call to my Apache module, the .Net code times out, I am trying to figure out which side is having problems. I am thinking it is a .Net, but I wanted to make sure that I throughly confirmed that it is not Apache. I have logged the heck out of my apache module and it never gets called. I have set apache's LogLevel to debug and turned on access logging, but I never see the request come in. I see the first one, but not the second one. Is there anything else I can do to increase the logging of Apache or is it safe to say that the request is NOT leaving the .Net application? Any suggestions on how I can absolutely confirm the issue is in the .Net client? Since you see one request but not the second, one thing I would perhaps suggest doing is turn off KeepAlive and see if that makes a difference with the client. Maybe the application is trying to use HTTP/1.1 request pipelining but getting something wrong and as a result a second request over same connection is not getting passed through correctly. Turning off KeepAlive should ensure that each request uses a separate connection. And do a tcpdump / wireshark on the connection at the same time. Dw.
Re: debugging a timeout issue
On Fri, May 9, 2008 at 12:23 AM, Graham Dumpleton [EMAIL PROTECTED] wrote: Since you see one request but not the second, one thing I would perhaps suggest doing is turn off KeepAlive and see if that makes a difference with the client. I am wondering, I do not see the timeout bug but my customer is seeing it. Does it make sense that the same client application configured the same way talking to Apache configured the same way might show different KeepAlive behavior? Sam
Re: debugging a timeout issue
Sam Carleton wrote: On Fri, May 9, 2008 at 12:23 AM, Graham Dumpleton [EMAIL PROTECTED] wrote: Since you see one request but not the second, one thing I would perhaps suggest doing is turn off KeepAlive and see if that makes a difference with the client. I am wondering, I do not see the timeout bug but my customer is seeing it. Does it make sense that the same client application configured the same way talking to Apache configured the same way might show different KeepAlive behavior? Sam Especially if it is based on a different (e.g. updated) .NET framework. The previous suggestions to utilize Wireshark or even tcpdump (if in the *nix world) will help you check the connection. IF the client has a network address translation or a firewall that ends up dropping a packet, you will see that with the network trace. (Get both halves - from the client and from the server, and compare them). -- Joseph Lewis http://sharktooth.org/
Looking for reviewers for the second edition of the Apache Pocket Reference
I am nearly finished with the technical review draft of the second edition of the Apache Pocket Reference (my deadline for this is 30 May), and am looking for a number of people to review the manuscript. I would be very grateful for any volunteers and I believe that there will be some free copies of the book for reviewers from O'Reilly. O'Reilly are planning on this being published in August this year (details already on their web site and i's on Amazon for pre-order). The new edition has been completely overhauled and expanded (looks like the page count will increase from 112 to about 170), with the focus being on 2.2, and with compatibility information for users of 1.3. Following chats with some of the core developers at ApacheCon Europe last month I have decided to not bother with compatibility information for users of 2.0. As part of the research for the book I have been looking at the documentation and comparing with the source code, and have found a few errors which I am reporting to the docs project. Once I've finished the technical review draft of the book I am planning on creating a new quick reference card for Apache 2.2 and updating the old 1.3 version to remove compatibility information for 1.2. As I say, I would be very grateful for help. If you are interested, please contact me directly ([EMAIL PROTECTED]). Andrew -- Andrew Ford, DirectorPauntley Prints / Ford Mason Ltd [EMAIL PROTECTED] South Wing Compton House pauntley-prints.co.uk Compton Green, Redmarley Tel: +44 1531 829900 ford-mason.co.uk Gloucester GL19 3JB Fax: +44 1531 829901 refcards.com cronolog.org Great Britain Mobile: +44 7785 258278
Re: jumbo patch from 39380 - and moving things 'up' to mod_cache itself
What if Vary were much more than just HTTP Vary? It would be nice if the framework could support the external vary (ie, normal HTTP Vary) as well as any internal Vary. To use general mod_disk_cache structure, we currently have something sorta like this for vary metafiles: int cache_version apr_time_t expires serialized array of vary headers (ex: accept-encoding, user-agent) What if we had a serialized table instead: HTTP_VARY = accept-encoding, user-agent MY_AUTH_VARY = my_auth_cookie Where HTTP_VARY was handled by the HTTP vary provider and created a vary key based on users info, and MY_AUTH_VARY was some provider a user wrote so they could cache different content based on if a user was authenticated or not (based on a cookie). So a complete key from a request for this may be: HTTP_VARYaccept-encoding=gzipuser-agent=MozillaMY_AUTH_VARYauth=1http://doma in.com/some/url?some=querystring=1 There are cases when may want to vary based on geographical info, time of day, etc., but normal HTTP Vary does not really handle that. However, it would not be so hard to break the whol vary process into providers. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: svn commit: r654752 - in /httpd/httpd/branches/2.2.x: CHANGES STATUS modules/generators/mod_cgid.c
On Fri, May 9, 2008 at 6:57 AM, [EMAIL PROTECTED] wrote: Author: trawick Date: Fri May 9 03:57:46 2008 New Revision: 654752 URL: http://svn.apache.org/viewvc?rev=654752view=rev Log: backport from trunk: *) mod_cgid: Explicitly set permissions of the socket (ScriptSock) shared by mod_cgid and request processing threads, for OS'es such as HPUX and AIX that do not use umask for AF_UNIX socket permissions. [Eric Covener, Jeff Trawick] *) mod_cgid: Don't try to restart the daemon if it fails to initialize the socket. [Jeff Trawick] A patch for 2.2.8 is in http://people.apache.org/~trawick/cgid_socket_perms.txt and needs to be moved to http://archive.apache.org/dist/httpd/patches/apply_to_2.2.8/ (blush) I don't recall if/how I can log on to archive.apache.org or otherwise put stuff there. Offline hints and/or somebody do the dirty work?
Re: mod_mbox compile issue on SuSE Enterprise 10
On Thu, May 8, 2008 at 5:13 PM, Gregory Boyce [EMAIL PROTECTED] wrote: Ok, I found the trigger for the bug, but I'm not sure what the correct fix is. It appears that Apache on SuSE sets AP_DEBUG as part of the EXTRA_CPPFLAGS variable in config_vars.mk while Debian does not. Manually removing AP_DEBUG on a SuSE box causes the module to compile and adding it to Debian causes the compile to fail. mod_mbox_util-mod-mbox-util.o: In function `scan_dir': /root/mod_mbox/module-2.0/mod-mbox-util.c:162: undefined reference to `ap_strstr' server/util_debug.c always implements the functions to avoid the AP_DEBUG having to match. I don't understand why the symbols are being resolved when mod_mbox is linked. They will be present in httpd when mod_mbox.so is loaded.
Re: Looking for reviewers for the second edition of the Apache Pocket Reference
I'm interested Regards, Nick
Re: Looking for reviewers for the second edition of the Apache Pocket Reference
Me too! On Fri, May 9, 2008 at 8:41 AM, Nick Gearls [EMAIL PROTECTED] wrote: I'm interested Regards, Nick -- My Blogs: http://www.docunext.com/ http://www.albertlash.com/
Re: Looking for reviewers for the second edition of the Apache Pocket Reference
Good idea! :-) Iim insterested too. Iñigo I am nearly finished with the technical review draft of the second edition of the Apache Pocket Reference (my deadline for this is 30 May), and am looking for a number of people to review the manuscript. I would be very grateful for any volunteers and I believe that there will be some free copies of the book for reviewers from O'Reilly. O'Reilly are planning on this being published in August this year (details already on their web site and i's on Amazon for pre-order). The new edition has been completely overhauled and expanded (looks like the page count will increase from 112 to about 170), with the focus being on 2.2, and with compatibility information for users of 1.3. Following chats with some of the core developers at ApacheCon Europe last month I have decided to not bother with compatibility information for users of 2.0. As part of the research for the book I have been looking at the documentation and comparing with the source code, and have found a few errors which I am reporting to the docs project. Once I've finished the technical review draft of the book I am planning on creating a new quick reference card for Apache 2.2 and updating the old 1.3 version to remove compatibility information for 1.2. As I say, I would be very grateful for help. If you are interested, please contact me directly ([EMAIL PROTECTED]). Andrew -- Iñigo Medina García Librería Díaz de Santos Madrid (Spain) [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: svn commit: r654752 - in /httpd/httpd/branches/2.2.x: CHANGES STATUS modules/generators/mod_cgid.c
On Fri, May 9, 2008 at 8:27 AM, Jeff Trawick [EMAIL PROTECTED] wrote: On Fri, May 9, 2008 at 6:57 AM, [EMAIL PROTECTED] wrote: Author: trawick Date: Fri May 9 03:57:46 2008 New Revision: 654752 URL: http://svn.apache.org/viewvc?rev=654752view=rev Log: backport from trunk: *) mod_cgid: Explicitly set permissions of the socket (ScriptSock) shared by mod_cgid and request processing threads, for OS'es such as HPUX and AIX that do not use umask for AF_UNIX socket permissions. [Eric Covener, Jeff Trawick] *) mod_cgid: Don't try to restart the daemon if it fails to initialize the socket. [Jeff Trawick] A patch for 2.2.8 is in http://people.apache.org/~trawick/cgid_socket_perms.txt and needs to be moved to http://archive.apache.org/dist/httpd/patches/apply_to_2.2.8/ (blush) I don't recall if/how I can log on to archive.apache.org or otherwise put stuff there. Offline hints and/or somebody do the dirty work? Just put it in people.apache.org:/www/www.apache.org/dist/httpd/patches/apply_to_2.2.8/. The archive.apache.org/dist/ directory is an automatic copy of the stuff under www.apache.org/dist/. Joshua.
Re: svn commit: r654752 - in /httpd/httpd/branches/2.2.x: CHANGES STATUS modules/generators/mod_cgid.c
On Fri, May 9, 2008 at 9:26 AM, Joshua Slive [EMAIL PROTECTED] wrote: Just put it in people.apache.org:/www/www.apache.org/dist/httpd/patches/apply_to_2.2.8/. The archive.apache.org/dist/ directory is an automatic copy of the stuff under www.apache.org/dist/. Thanks!
Re: svn commit: r654797 - in /httpd/httpd/branches/2.2.x: CHANGES STATUS docs/manual/mod/mod_headers.xml modules/metadata/mod_headers.c
Jim Jagielski wrote: Add in r568323 and 568879. The approved patch lacked updates to the doccos and so really shouldn't have been approved as is, but what the heck, so I went ahead and pulled the doccos changes from the orig commit anyway. Also, since this is a userland change, it should really have a CHANGES entry too, so I added one of those as well :) Thanks -- apologies for not doing this too well. I'd caught r568879 and put that into the proposed 2.2.x patch, but missed the necessary CHANGES and doc alterations. Oops, and thanks for catching these and the CHANGES for mod_authn_dbd. I'll try to be more attentive in the future. Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: svn commit: r654797 - in /httpd/httpd/branches/2.2.x: CHANGES STATUS docs/manual/mod/mod_headers.xml modules/metadata/mod_headers.c
On May 9, 2008, at 1:18 PM, Chris Darroch wrote: Jim Jagielski wrote: Add in r568323 and 568879. The approved patch lacked updates to the doccos and so really shouldn't have been approved as is, but what the heck, so I went ahead and pulled the doccos changes from the orig commit anyway. Also, since this is a userland change, it should really have a CHANGES entry too, so I added one of those as well :) Thanks -- apologies for not doing this too well. I'd caught r568879 and put that into the proposed 2.2.x patch, but missed the necessary CHANGES and doc alterations. Oops, and thanks for catching these and the CHANGES for mod_authn_dbd. I'll try to be more attentive in the future. No worries! :)
Re: jumbo patch from 39380 - and moving things 'up' to mod_cache itself
Dirk-Willem van Gulik wrote: There is a lot of valuable stuff in your jumbo patch - but I am not sure what the best approach is to fold it in. Could you have have a look at the rough patch I posted earlier today - and let me know if you have any thoughts as to which parts should be moved 'up' -- and hence be of use to other caching backends as well - and which parts are pure disk optimized/specific ? Or perhaps form an option if we need multiple disk cache modules - each optimized differently (e.g. for large files on multiple spindles; versus very 'hot' cache which is virtually living on a meory disk). I think the safest option is to develop it as another disk cache module in parallel to the current one. It would definitely help if the jumbo patch could be updated alongside the current batch of changes. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: jumbo patch from 39380 - and moving things 'up' to mod_cache itself
On Fri, 9 May 2008, Graham Leggett wrote:
There is a lot of valuable stuff in your jumbo patch - but I am not sure
what the best approach is to fold it in.
Could you have have a look at the rough patch I posted earlier today - and
let me know if you have any thoughts
as to which parts should be moved 'up' -- and hence be of use to other
caching backends as well - and which
parts are pure disk optimized/specific ?
Or perhaps form an option if we need multiple disk cache modules - each
optimized differently (e.g. for large files on multiple spindles; versus
very 'hot' cache which is virtually living on a meory disk).
I think the safest option is to develop it as another disk cache module in
parallel to the current one. It would definitely help if the jumbo patch
could be updated alongside the current batch of changes.
My jumbopatch was rather singlemindedly aimed at making the disk cache
work with our usecase: caching large files on machines with
not-that-much ram (ie. 4GB DVD images on machines with 3GB RAM). The
current mod_disk_cache simply sucks at this, to the point it's
completely unusable since all it does is start caching the file once
per access, filling your disk and eating all your memory/address space
and then segfault.
With that in mind, there are a few things that without a doubt should
benefit vanilla mod_disk_cache, like the cleanup of on-disk-format
(storing of headers, data types used for sizes), cleanup of various
functions and probably other things. Those things could probably be
merged to trunk without too much debate, given that I can find time to
do it.
Also, we want the cache to be usable by other parties. We have a
preloaded library that makes rsync able to use it, and an improved
version that makes vsftpd work needs some testing before we put it to
use.
Changing the absolutely silly defaults for length and number of
subdirectories in the cache hierarchy would probably also be wise. You
should never have to delete cache directories like is being done now,
if you have to you're creating way too many (or are using a crappy
filesystem without indexed dirs). CacheDirLength 1 and CacheDirLevels
2 gives you 4096 directories (64^2), that should be more than enough
for a competent filesystem.
If I remember correctly the parts of the patch causing commotion was:
- Getting rid of the temporary files and caching directly to the
destination file (requires O_EXCL which supposedly doesn't work on
Windows, too much and ugly code and other comments).
- The background-caching-stuff. Yes, I know that forking a new
thread/process isn't pretty but that was the easy way to do it. A
service-process that does the background-caching is probably more
elegant but involved much more httpd magic that I'm clueless about.
- Using fstat() in the read-while-caching buckets, people wanted
inotify-stuff for this even though there is no support in APR for
it.
The thing is - this is what makes the thing work for using the cache
with large files, and I'm not convinced that it fundamentally breaks
stuff for small-file-cache-people.
Oh, and for a reference that the jumbo patch is at least usable: It
has served approx 4220 PB during the time we have used it, with
some 90-95% cache reuse :)
That said I'm very well aware that my code is not the ultimate
solution, it's more of a proof that there's some odds and ends missing
in apr/httpd/whatever infrastructure to do this in an elegant way.
However, obviously still possible to do in a single module without
having to patch your way through httpd.
I'm not convinced that forking the disk cache having two similar ones
tuned for different usecases is a good idea in the long run, I'm
pretty sure that the parts that needs tweaking can be solved with
config options and documentation. For a development sprint like this
it seems sane though, although one option could be to simply do a
branch of modules/cache and go wild with the controversial parts until
all parties are satisfied.
/Nikke - will have another glass of wine now, I'll fail miserably in
Guitar Hero anyhow ;)
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED]
---
When all else fails, read the manual.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Re: svn commit: r654805 [1/2] - in /httpd/httpd/branches/2.2.x/docs/manual: ./ developer/ faq/ howto/ misc/ mod/ platform/ programs/ rewrite/ ssl/ vhosts/
* [EMAIL PROTECTED] wrote: Author: jim Date: Fri May 9 06:35:27 2008 New Revision: 654805 URL: http://svn.apache.org/viewvc?rev=654805view=rev Log: Update doccos Please update your build system checkout. Yours seems outdated ;) nd -- Winnetous Erbe: http://pub.perlig.de/books.html#apache2
Re: svn commit: r647263 - in /httpd/httpd/trunk: ./ docs/manual/mod/ include/ modules/aaa/ modules/filters/ modules/http/ server/
Ruediger Pluem wrote:
Why is this needed? Should this job be performed by the
ap_keep_body_filter that should
be in our input filter chain if we want to keep the body?
Of course this depends when we call ap_parse_request_form. If we call it
during the
authn/z phase the filter chain hasn't been setup. So maybe we should
ensure that
this is the case.
This opened a can of worms - eventually the solution I found that seemed
to work well was to create a subrequest to ensure the input filters were
set up correctly, and then read the input filter stack. The content of
the input stack is doomed in this case anyway, either it contains our
login form which must be parsed and discarded, or it doesn't contain our
login form and should be discarded anyway, as authn will be denied.
If authn was approved, the kept_body filter is put in place to cap the
input filter stack, and either provide an empty body, or provide the
body that was optionally passed in the form.
Fixed in r654958.
+static int request_fixups(request_rec * r)
+{
+request_dir_conf *conf = ap_get_module_config(r-per_dir_config,
+ request_module);
+
+if (conf-keep_body) {
+ap_add_input_filter_handle(ap_keep_body_input_filter_handle,
+ NULL, r, r-connection);
+}
+
+return OK;
Why not using the insert_filter hook?
Fixed in r654952.
Regards,
Graham
--
smime.p7s
Description: S/MIME Cryptographic Signature
Re: jumbo patch from 39380 - and moving things 'up' to mod_cache itself
Niklas Edmundsson wrote: I'm not convinced that forking the disk cache having two similar ones tuned for different usecases is a good idea in the long run, I'm pretty sure that the parts that needs tweaking can be solved with config options and documentation. For a development sprint like this it seems sane though, although one option could be to simply do a branch of modules/cache and go wild with the controversial parts until all parties are satisfied. I think in the long run, the large_disk_cache can probably replace the standard disk cache we have now. Running them in parallel however gives us the chance to address the issues in the large_disk_cache, eventually we'd probably want to retire the old disk cache. I am currently preoccupied with getting mod_auth_form and friends bedded down, followed by getting the evp code in APR sorted out, but after that the plan is to look at the cache again, if someone doesn't beat me to it. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: svn commit: r654968 - in /httpd/httpd/trunk: CHANGES modules/proxy/mod_proxy_http.c server/protocol.c
On May 9, 2008, at 3:40 PM, [EMAIL PROTECTED] wrote:
Modified: httpd/httpd/trunk/modules/proxy/mod_proxy_http.c
URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/
mod_proxy_http.c?rev=654968r1=654967r2=654968view=diff
==
--- httpd/httpd/trunk/modules/proxy/mod_proxy_http.c (original)
+++ httpd/httpd/trunk/modules/proxy/mod_proxy_http.c Fri May 9
15:40:01 2008
@@ -266,6 +266,10 @@
apr_brigade_length(bb, 0, transferred);
if (transferred != -1)
conn-worker-s-transferred += transferred;
+char tmp[1024000];
+apr_size_t tlen = 1024000;
+apr_brigade_flatten(bb, tmp, tlen);
+printf(tmp, NULL);
status = ap_pass_brigade(origin-output_filters, bb);
if (status != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_ERR, status, r-server,
Please tell me that the chunk above is a mistaken commit.
Roy
Re: svn commit: r654968 - in /httpd/httpd/trunk: CHANGES modules/proxy/mod_proxy_http.c server/protocol.c
Roy T. Fielding wrote: Please tell me that the chunk above is a mistaken commit. The chunk above is a mistaken commit. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: jumbo patch from 39380 - and moving things 'up' to mod_cache itself
William A. Rowe, Jr. wrote: I think the bigger idea that mod_cache must handle all rfc related issues is key. mem and disk cache should never have had substantial differences in behavior, but they do. So the more we can consolidate in mod_cache w.r.t. the requests themselves, the less the backend old-disk, large-disk, mem, memcached or other providers will break things. So proving that the all this logic belongs in neither the old-disk or large-disk cache is a very useful exercise. Definitely. The large_disk_cache however was a redesign of some of the underlying principles of the existing disk_cache, and as a result it needed time to be bedded down and fully agreed to by everybody. Not having it in the tree at all means that it may lose out on Dirk's improvements. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
