Re: [users@httpd] Apache stops responding until being reloaded.
06.10.2011 22:09, William A. Rowe Jr. пишет: Looks like you have a vulnerable flavor of APR, be watchful (perhaps with mod_log_forensic) of autoindex requests which contain P=*?*?*?*?... style patterns. No.. lynx "http://testdomain.name?P=*?*?*?*?"; to this server --> LA not up. - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] serving pre-compressed content
On 06.10.2011 18:36, Xavier Noria wrote: > Does anyone have a well-tested and idiomatic Apache configuration to serve > pre-compressed content? Vary header, Content-Type header, browser gotchas, > and everything robustly sorted out? Yes, see below. > The situation is that you have foo.css and foo.css.gz on disk, and want > Apache to serve foo.css.gz directly if asked for foo.css and the clients > accepts the compressed content. Note, that because of mime type handling our convention for the filename is foo.gz.css. Also my recipe expects you to have both files available, foo.css and foo.gz.css. > I guess that's going to be based on content negotiation with MultiViews, but > a google search suggests there are practical issues to get this right that > may not be obvious at first. Not necessary. > And also a curiosity: After all these years, why doesn't mod_deflate do this > by itself? Compressing with max ratio to disk on a first request, and > serving the cached .gz in subsequent requests? Patches welcome! Now to the config and how it works: General description 1) mod_rewrite checks, whether the browser accepts gzipped content (RewriteCond against Accept-Encoding request header) 2) If so, check whether the URL belongs to the range of URLs, for which we provide pre compressed static content (prefix and suffix matches, you might need to adjust) You can also use "-f" to check for existance, but I prefer a more tightly managed environment, were it is clear that for certain URL pattern all files exist compressed and uncompressed and wedon't have to do a "-f" for each request. 3) If 1) and 2) are true, insert ".gz" before the file name suffix, adjust the URL accordingly and set a marker environment variable we can later check. 4) Now if the env var in 3) is set, use mod_headers toset the Content-Encoding response header to "gzip". Why does it work? By changing the request URL we let Apache send the compressed file. The content Type doesn't change, because the filesuffix hasn't changes (foo.gz.css). Browsers who don't accept gzipped content should not indicate gzip support in the Accept-Encoding request header so we send uncompressed content. Caches need to be made aware of whether the response is compressed or not. For this one uses the Vary response header. Fortunately since we make our response depending on the Accept-Encoding request header using mod_rewrite, mod_rewrite automaticcaly adds it to Vary for us. Nice. Note: mod_deflate is not involved. Following is a commented example config snippet: # This needs mod_rewrite and mod_headers # loaded as modules. # Make static content available. # Not needed if already mounted elsewhere. Alias /myapp/static /opt/myapp/static/ # Activate mod_rewrite and debug logging. # Not needed if mod_rewrite is already # activated for this VHost elsewhere. RewriteEngine On # Configure RewriteLog according to your needs # RewriteLog ... # RewriteLogLevel ... # Flip in compressed content if allowed. # Assumes all the compressed files are on disk # having the correct names: # something.css -> something.gz.css # something.js -> something.gz.js # 1) Check whether browser accepts "gzip" encoding RewriteCond %{HTTP:Accept-Encoding} gzip # 2) Check whether request belongs to our #static URLs and has the right suffix #If yes, add ".gz" to URL before existing suffix #and remember this in our custom environment variable. RewriteRule (/myapp/static/.*)\.(css|js)$ $1.gz.$2 [E=gz:1] # Fix returned encoding header if we use the gzippped file. Header set Content-Encoding gzip env=gz # Notes: # # - Be careful when introducing loops for rewrite rules: # The new .gz.js etc. file would again match the rule # leading to unterminated recursion. # Make regexp more precise in that case (not allowing the .gz.) # to match again. # # - Content-Type header works OK, because file suffix hasn't changed. # This would not work for files without suffix, because then # we end up with a ".gz" suffix! # # - Vary header is automatically extended with "Accept-Encoding" # by mod_rewrite because of using the "Accept-Encoding" header # in the RewriteCond # # - Old-style "Accept-Encoding: x-gzip" in request also works. # The "gzip" is a sub pattern match (not anchored). # # Open Questions: # # - Is there any interoperability issue when mod_deflate is # activated in addition (double compress or similar). # If so, try to set env var "no-gzip" to deactivate mod_deflate # for those requests. Example shell script to generate the compressed content (in addition to the uncompressed already existing files), here for *.css and *.js: CONTENT_DIR=/opt/myapp/static/ for suffix in css js do for file in \ `find $CONTENT_DIR -type f -name "*.$suffix" -a ! -name "*.gz.*"` do gzfile=`echo $file | sed -e 's#\.'$suffix'#.gz.'$suffix'#'` gzip --best -c $file > $gzfile chmod 644 $gzfile echo === $file $gzfile === ls -ld $file
Re: [users@httpd] Could Apache login support CAPTCHA and lockout?
On 2011-10-04 14:44, Neal Rhodes wrote: We have bunches of web applications which use the regular Apache login protection, Do you mean HTTP Basic Auth, as defined in RFC 2616 ? and they won't run unless REMOTE_USER is set by the Apache login. require valid-user require valid-user AuthName O-Visitor AuthUserFile /usr/appl/cgi/.htpasswd AuthType Basic Yes, this is HTTP Basic AUTH. It says so right there. Looking at improving security, it would seem that it would be much harder to conduct brute-force attacks on these systems if we could configure Apache login to do two things: You can't. There is no "login", just an Authorization: header which has to be sent for every page that requires it. A. Present the CAPTCHA style validation prompt as part of the login, to make it difficult for scripted attacks to proceed; B. Lockout an individual username in the .htpasswd file after X failed login attempts. Actual login-ness (a state of logged in being different from a state of not being logged in) must be achieved through non-HTTP means, possibly supported by HTTP features such as cookies. -- J.
Re: [users@httpd] Apache stops responding until being reloaded.
On 10/6/2011 8:22 AM, stal...@locum.ru wrote: > > Some time Apache stops responding until being reloaded. on server work > mod_php5 and wsgi > for django app. os debian 6.0.2. kernel 2.6.32-5-amd64 > > apache2 -v > Server version: Apache/2.2.16 (Debian) > Server built: Aug 8 2011 14:38:30 > > apache2 -V > Server version: Apache/2.2.16 (Debian) > Server built: Aug 8 2011 14:38:30 > Server's Module Magic Number: 20051115:24 > Server loaded: APR 1.4.2, APR-Util 1.3.9 Looks like you have a vulnerable flavor of APR, be watchful (perhaps with mod_log_forensic) of autoindex requests which contain P=*?*?*?*?... style patterns. - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Apache stops responding until being reloaded.
06.10.2011 20:29, Tom Evans пишет: On Thu, Oct 6, 2011 at 3:11 PM, stal...@locum.ru wrote: fox fix problem with reload or fix high load? To fix problems with load, buy a bigger server, or serve less requests (by caching etc). load high only if connections count == max_clients value. increase value fix problem, but i cannot increase value Infinitely
[users@httpd] serving pre-compressed content
Does anyone have a well-tested and idiomatic Apache configuration to serve pre-compressed content? Vary header, Content-Type header, browser gotchas, and everything robustly sorted out? The situation is that you have foo.css and foo.css.gz on disk, and want Apache to serve foo.css.gz directly if asked for foo.css and the clients accepts the compressed content. I guess that's going to be based on content negotiation with MultiViews, but a google search suggests there are practical issues to get this right that may not be obvious at first. And also a curiosity: After all these years, why doesn't mod_deflate do this by itself? Compressing with max ratio to disk on a first request, and serving the cached .gz in subsequent requests?
Re: [users@httpd] Re: Vanity URL Rewrites Best Practices?
On Thu, Oct 6, 2011 at 3:00 PM, Nick Tkach wrote: > Just to answer my own question, in case it helps someone else down the > road, what I was missing was that inside the curly braces you need to > dereference the variable with % rather than $. So the right version > of what I posted above is: > > RewriteMap vanmap txt:/tmp/map.txt > > RewriteCond %{REQUEST_URI} ^/mmh/ > RewriteCond %{QUERY_STRING} \%2F(\w+)\.xml$ > > RewriteRule .* /mmh/${vanmap:%1}? [L,NC,R=302] > Since this is for posterity, I have to say 'No!'. In both the RewriteRule and the RewriteCond, %N refers to back references in the last matched RewriteCond, and $N refers to back references in the RewriteRule. It doesn't matter one jot that it is 'inside the curly braces', it matters where the data comes from. This is mentioned in both the RewriteCond and the RewriteRule documentation: http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewritecond http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewriterule Cheers Tom - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Apache stops responding until being reloaded.
On Thu, Oct 6, 2011 at 3:11 PM, stal...@locum.ru wrote: > fox fix problem with reload or fix high load? > To fix problems with load, buy a bigger server, or serve less requests (by caching etc). Cheers Tom - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Cat request + query for rewrite?
On Thu, Oct 6, 2011 at 5:21 PM, Nick Tkach wrote: > Is there some way to make a RewriteRule concat together two > back-references? What I want to do is take the REQUEST_URI and the > QUERY_STRING of a request and use the two together as a key into a > map. > > So if I have a request like this: > > http://mysite.com/bar/startpage?arg1=a&arg2=b > > and I want to rewrite that to > > http://mysite.com/foo/endpage?arg3=c&arg4=d > > Given an entry in the map file like > > bar/startpage?arg1=a&arg2=b foo/endpage?arg3=c&arg4=d > > > I was trying some combination like > > RewriteMap map2 txt:/redirects-map.txt > > RewriteCond %{REQUEST_URI} ^/bar/startpage > RewriteCond %{QUERY_STRING} arg1=a&arg2=b > RewriteCond %{REQUEST_URI} (.*) > RewriteCond %{QUERY_STRING} (.*) > RewriteRule .* ${map2:%1%2}? [L,NC,R=302] > > > That seems to only try to do the map lookup based on the QUERY_STRING. > Obviously the second match is over-riding the first, but I'm not > quite sure how to stop that. Yes, and there is no way around that. What you can do instead is capture the REQUEST_URI as a RewriteRule backref: RewriteCond %{REQUEST_URI} ^/bar/startpage RewriteCond %{QUERY_STRING} arg1=a&arg2=b RewriteCond %{QUERY_STRING} (.*) RewriteRule (.*) ${map2:$1%1}? [L,NC,R=302] Cheers Tom - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
[users@httpd] Cat request + query for rewrite?
Is there some way to make a RewriteRule concat together two back-references? What I want to do is take the REQUEST_URI and the QUERY_STRING of a request and use the two together as a key into a map. So if I have a request like this: http://mysite.com/bar/startpage?arg1=a&arg2=b and I want to rewrite that to http://mysite.com/foo/endpage?arg3=c&arg4=d Given an entry in the map file like bar/startpage?arg1=a&arg2=bfoo/endpage?arg3=c&arg4=d I was trying some combination like RewriteMap map2 txt:/redirects-map.txt RewriteCond %{REQUEST_URI} ^/bar/startpage RewriteCond %{QUERY_STRING} arg1=a&arg2=b RewriteCond %{REQUEST_URI} (.*) RewriteCond %{QUERY_STRING} (.*) RewriteRule .* ${map2:%1%2}? [L,NC,R=302] That seems to only try to do the map lookup based on the QUERY_STRING. Obviously the second match is over-riding the first, but I'm not quite sure how to stop that. - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Apache stops responding until being reloaded.
fox fix problem with reload or fix high load? 06.10.2011 17:46, Jim Jagielski пишет: Bump up MaxClients. On Oct 6, 2011, at 9:22 AM, stal...@locum.ru wrote: Some time Apache stops responding until being reloaded. on server work mod_php5 and wsgi for django app. os debian 6.0.2. kernel 2.6.32-5-amd64 apache2 -v Server version: Apache/2.2.16 (Debian) Server built: Aug 8 2011 14:38:30 apache2 -V Server version: Apache/2.2.16 (Debian) Server built: Aug 8 2011 14:38:30 Server's Module Magic Number: 20051115:24 Server loaded: APR 1.4.2, APR-Util 1.3.9 Compiled using: APR 1.4.2, APR-Util 1.3.9 Architecture: 64-bit Server MPM: Prefork threaded: no forked: yes (variable process count) Server compiled with -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="/etc/apache2" -D SUEXEC_BIN="/usr/lib/apache2/suexec" -D DEFAULT_PIDLOG="/var/run/apache2.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="/var/run/apache2/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="mime.types" -D SERVER_CONFIG_FILE="apache2.conf" and Moreover, when connections count reaches maxclients (512) value, it causes server high load. LA up to 50-60 - The official User-To-User support forum of the Apache HTTP Server Project. Seehttp://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org - The official User-To-User support forum of the Apache HTTP Server Project. Seehttp://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Re: Vanity URL Rewrites Best Practices?
On Wed, Oct 5, 2011 at 2:48 PM, Nick Tkach wrote: > On Mon, Oct 3, 2011 at 1:41 PM, Andrew Schulman > wrote: >> >> > For example, >> > >> > http://foo.com/mmh/maintenance_plan/tip?contentCategoryType=MaintenanceTip&id=%2Fwww%2Favm_webapps%2Fmmh%2Fmaintenance-tips%2Fcontent%2Fafter_blizzard.xml >> > >> > Being sent to >> > >> > http://foo.com/mmh/articles/authored/after-blizzard >> > >> > We've got a very frequent process where we'll get a huge block of rewrites >> > like this that vary just by the last part and so far just keep going >> > through >> > and adding dozens and dozens of new rewrite rules each time. Surely there >> > has to be a better way? (Ideally that just involve Apache changes and not >> > code changes on the back-end) >> >> Are the requests similar enough that you can write a single regular >> expression, >> or maybe two or three, that extracts the useful part from the URL in every >> case? >> For example, >> >> RewriteCond %{REQUEST_URI} ^/mmh/ >> RewriteCond %{QUERY_STRING} \%2F(\w+)\.xml$ >> RewriteRule .* /mmh/articles/authored/%1 >> >> If you can describe all of the requests in this way or something like it, >> you're >> done. If not, if each request is so different that it needs its own regular >> expression, then it seems you're doomed to keep doing it as you are now. > > Yes, I'm very nearly there, thanks! The only thing is, I'm not quite > sure how to combine what you have here with a RewriteMap. If I had > something like this: > > RewriteMap vanmap txt:/tmp/map.txt > > RewriteCond %{REQUEST_URI} ^/mmh/ > RewriteCond %{QUERY_STRING} \%2F(\w+)\.xml$ > > > Then can I just do a rewrite rule at the end like this? > > RewriteRule .* /mmh/${vanmap:$1}? [L,NC,R=302] > > > Because I tried that and it keeps failing to pull any kind of key out > of the REQUEST_URI. > >> >> Good luck, >> Andrew. >> >> >> - >> The official User-To-User support forum of the Apache HTTP Server Project. >> See http://httpd.apache.org/userslist.html> for more info. >> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org >> " from the digest: users-digest-unsubscr...@httpd.apache.org >> For additional commands, e-mail: users-h...@httpd.apache.org >> > Just to answer my own question, in case it helps someone else down the road, what I was missing was that inside the curly braces you need to dereference the variable with % rather than $. So the right version of what I posted above is: RewriteMap vanmap txt:/tmp/map.txt RewriteCond %{REQUEST_URI} ^/mmh/ RewriteCond %{QUERY_STRING} \%2F(\w+)\.xml$ RewriteRule .* /mmh/${vanmap:%1}? [L,NC,R=302] - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Apache stops responding until being reloaded.
Bump up MaxClients. On Oct 6, 2011, at 9:22 AM, stal...@locum.ru wrote: > > Some time Apache stops responding until being reloaded. on server work > mod_php5 and wsgi for django app. os debian 6.0.2. kernel 2.6.32-5-amd64 > > apache2 -v > Server version: Apache/2.2.16 (Debian) > Server built: Aug 8 2011 14:38:30 > > apache2 -V > Server version: Apache/2.2.16 (Debian) > Server built: Aug 8 2011 14:38:30 > Server's Module Magic Number: 20051115:24 > Server loaded: APR 1.4.2, APR-Util 1.3.9 > Compiled using: APR 1.4.2, APR-Util 1.3.9 > Architecture: 64-bit > Server MPM: Prefork > threaded: no >forked: yes (variable process count) > Server compiled with > -D APACHE_MPM_DIR="server/mpm/prefork" > -D APR_HAS_SENDFILE > -D APR_HAS_MMAP > -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) > -D APR_USE_SYSVSEM_SERIALIZE > -D APR_USE_PTHREAD_SERIALIZE > -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT > -D APR_HAS_OTHER_CHILD > -D AP_HAVE_RELIABLE_PIPED_LOGS > -D DYNAMIC_MODULE_LIMIT=128 > -D HTTPD_ROOT="/etc/apache2" > -D SUEXEC_BIN="/usr/lib/apache2/suexec" > -D DEFAULT_PIDLOG="/var/run/apache2.pid" > -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" > -D DEFAULT_LOCKFILE="/var/run/apache2/accept.lock" > -D DEFAULT_ERRORLOG="logs/error_log" > -D AP_TYPES_CONFIG_FILE="mime.types" > -D SERVER_CONFIG_FILE="apache2.conf" > > > and > Moreover, when connections count reaches maxclients (512) value, it causes > server high load. LA up to 50-60 > > - > The official User-To-User support forum of the Apache HTTP Server Project. > See http://httpd.apache.org/userslist.html> for more info. > To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org > " from the digest: users-digest-unsubscr...@httpd.apache.org > For additional commands, e-mail: users-h...@httpd.apache.org > - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
[users@httpd] Apache stops responding until being reloaded.
Some time Apache stops responding until being reloaded. on server work mod_php5 and wsgi for django app. os debian 6.0.2. kernel 2.6.32-5-amd64 apache2 -v Server version: Apache/2.2.16 (Debian) Server built: Aug 8 2011 14:38:30 apache2 -V Server version: Apache/2.2.16 (Debian) Server built: Aug 8 2011 14:38:30 Server's Module Magic Number: 20051115:24 Server loaded: APR 1.4.2, APR-Util 1.3.9 Compiled using: APR 1.4.2, APR-Util 1.3.9 Architecture: 64-bit Server MPM: Prefork threaded: no forked: yes (variable process count) Server compiled with -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="/etc/apache2" -D SUEXEC_BIN="/usr/lib/apache2/suexec" -D DEFAULT_PIDLOG="/var/run/apache2.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="/var/run/apache2/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="mime.types" -D SERVER_CONFIG_FILE="apache2.conf" and Moreover, when connections count reaches maxclients (512) value, it causes server high load. LA up to 50-60 - The official User-To-User support forum of the Apache HTTP Server Project. See http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org " from the digest: users-digest-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org