We had this problem too and so stopped using mod_disk_cache for a few months.
I was under the impression it had been fixed and so
we've recently turned it back on. We're on Apache/2.2.26.
From: Mike Rumph [mailto:mike.ru...@oracle.com]
Sent: Wednesday, February 12, 2014 4:54 PM
To: users@ht
We had a lot of problems w/SSI, deflate and caching. The deflate and the SSI
and the cacheing just fought each other. Sometimes it would work, sometimes we
got corrupted junk - but it was usually more messed up that just some dashes
like you have.
But if you've turned off mod_cache, deleted t
Are you using mod_cache?
ently accessed objects AND
- in mod_disk_cache for slightly slower access (but nearly unlimited storage
space)?
Seems this would be good but does the overhead of polling both caches result in
everything being slower?
Sample config below.
Thanks,
Geoff Millikan
---== Sample Config Sn
ently accessed objects AND
- in mod_disk_cache for slightly slower access (but nearly unlimited storage
space)?
Seems this would be good but does the overhead of polling both caches result in
everything being slower?
Sample config below.
Thanks,
Geoff Millikan
---== Sample Config Sn
d and fast access
- and mod_disk_cache for slightly slower access (but nearly unlimited storage
space)?
Seems this would be good but maybe the overhead of polling both caches results
in everything being slower?
Anyone with experience here? Sample config below.
Thanks,
Geoff Mil
ssing.
Got it.
Must all pages served via mod_cache have the "Age" header?
Thanks,
Geoff Millikan
PS. Happy Thanksgiving tomorrow to all the USA-based folks.
-
To unsubscribe, e-mail: users-unsubscr...@httpd.apach
I think I found it but anyone want to confirm Apache doesn't unset the "Age"
header when mod_cache is used because the HTTP 1.1 spec
requires it, "An HTTP/1.1 server that includes a cache MUST include an Age
header field in every response generated from its own
cache."
Hi all,
Is it impossible for mod_headers to remove the "Age" header from items served
out of mod_cache? (Example below)
Thanks,
Geoff Millikan
Request URL: http://www.t1shopper.com/ssi/broadband.css
Request Method: GET
Status Code: 200 OK
---== Request Headers ==---
Accept:text
Since this is a common request I'd love to see someone put together a "best
practices, solutions and examples" post for the community.
> I would recommend managing it at network level, with iptables for example
Agreed, and while it's off topic for this list take a look at "iptables
hashlimit."
e www.analog.cx) to shift
the UTC time into the human's local time as needed for the human's viewing
pleasure.
Hope this helps,
Geoff Millikan
http://www.t1shopper.com/
**Story is illustrative only, actual Apache workflow for time handling under
the hood probably different.
-
Submitted bug report for Apache 2.2.22 at link below. We've turned of
mod_cache on our production web servers until it's resolved
because we can't have phantom 403 pages going out to customers. Considering
jumping to 2.4.2...
https://issues.apache.org/bugzilla/show_bug.cgi?id=53350
Two things are going amiss with Apache here.
1. Apache is intermittently issuing a 403 "Directory index forbidden by Options
directive" but it never should. For example the two
below links work fine. But look at the log entry showing the 403 error.
Crazy. I cannot duplicate the error at all
e performance chart:
http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/topic/com.ibm.websphere.express.doc/info/exp/ae/rprf_ssl.html
Thanks,
Geoff Millikan
http://www.t1shopper.com/
PS. I'm no SSL/TLS guru - I'm just fiddling with settings and testing but I
wanted to share our results in the hop
at about 2 to 4 GB (we average 8MM
hits/month) so we like this better.
root /usr/bin/find /var/httpd/proxy/* -type f -mtime +30 -exec rm -rf {} \;
Remember to change the "/var/httpd/proxy/*" directory to point to wherever your
cache is.
Happy Web Serving,
>> When both RequestReadTimeout and Timeout values are set, the smaller
>> of the two takes precedence, right? For example, if Timeout 6 and
>> RequestReadTimeout header=10 body=30 then Apache will close the connection
>> at 6 seconds and the RequestReadTimeout will never be activated, right?
When both RequestReadTimeout and Timeout values are set, the smaller of the two
takes precedence, right? For example, if Timeout 6
and RequestReadTimeout header=10 body=30 then Apache will close the connection
at 6 seconds and the RequestReadTimeout will never be
activated, right?
http://httpd.
>> Without the ServerName, the vhost entry matches everything.
>
> Everything, or the ServerName defined globally?
You're right, good catch. Without the ServerName, the vhost entry matches only
whatever the ServerName is defined as globally
(which is the ServerName outside any containers).
B
Poop. It's a simple oversight. Sorry everyone. Tom Evans example works just
fine. The example from the original post would have
worked with the addition of a *ServerName*. Without the ServerName, the vhost
entry matches everything.
#This doesn't work
RewriteEngine On
RewriteRule .* htt
> You misunderstood what Eric said. The first vhost absolutely can be
> used to redirect requests to another vhost.
> I don't think I can make it any clearer than that - hope that helps.
Wow, thank you! And just for kicks, look back at the original post - you can
see the configuration you propo
> Incorrect. The first vhost is ALWAYS used when no vhost with a
> matching host name is found - it is the catch all vhost, by
> definition.
Yes, I totally agree. But, the first vhost cannot be used to *redirect
requests to another vhost.* Per Eric Covener, "You can't
make the catch-all vhost a
> Not sure what you mean by crash...
If I put "ServerAlias *" into httpd.conf and try to restart Apache, it will not
start. There's no error log as to why it will not
start.
> The first-listed vhost in a set of namevirtualhosts is the default.
Agreed. However it was said:
> You can't mak
> Eric C: You can't make the catch-all vhost also be a reachable name-based
> vhost, with those rewrite rules. Create an additional one
> solely for that purpose.
> Jeroen G: Get rid of all the rewrite junk and just set a dummy servername;
> this will catch ALL undefined hostnames.
> Then Redi
> Everything looks correct with httpd -S on both servers.
Except for the Warning: DocumentRoot [/var/www/html/example.com/store] does not
exist.
Wouldn't that produce 403 errors when you tried to list directory contents?
-
The
> Get rid of all the rewrite junk and just set a dummy servername; this
> will catch ALL undefined hostnames.
> Then Redirect / to the correct vhost.
Yep, we tried something like this (see below) and it crashed Apache even though
the syntax is correct.
#virtual host entry #1
#vi
>> default server www.t1shopper.com (/etc/httpd/conf/httpd.conf:663)
>> port 80 namevhost www.t1shopper.com (/etc/httpd/conf/httpd.conf:663)
>
>Is this the host that loops?
Yes. When I uncomment the first entry below, the infinite loop
happens on the second entry but then
again, that's the onl
> In my case, it appears Apache selects the chunked encoding automagically
> when the size of the page goes over a certain size. Apache doesn't seem
> to set the smaller pages to the chunked encoding (even though they are
> gzipped).
No, this was wrong. Mod_file_cache was tricking me (again).
>Looks fine -- apachectl/apache2ctl/httpd -S output?
$ /usr/sbin/httpd -S
VirtualHost configuration:
wildcard NameVirtualHosts and _default_ servers:
*:443 is a NameVirtualHost
default server www.t1shopper.com (/etc/httpd/conf/httpd.conf:1399)
port 443 namevhost www.t1shopper.com (/etc/http
> missing NameVirtualHost *:80? That would be the easiest way to get
> sent back to the 1st vhost after the redirect.
Eric, we've got that already but I'm hoping it's some simple oversight like
that which is causing this.
NameVirtualHost *:80
Listen 80
NameVirtualHost *:443
Listen 443
-
> Actually on second thought, the rewrite rule in the first host
> should be like this to avoid loop:
>
> RewriteEngine On
> RewriteCond %{HTTP_HOST} !^www\.mydomain\.com [OR]
> RewriteCond %{HTTP_HOST} !^mydomain\.com
> RewriteRule .* http://www.mydomain.com%{REQUES
> Try making the first one _default_ host
>
Thanks but since we're doing name-based virtual hosting I don't believe that
will apply to us because the manual says the _default_
setting only applies to IP virtual hosting but let me know if you have thoughts
otherwise!
"The string _default_ is u
I want to make a catch-all virtual host (like the manual mentions below) which
redirects any errant hostnames like
http://oopsie.mydomain.com/ to our main hostname at http://www.mydomain.com/
But the below example doesn't work - I'm getting an
infinite redirect from http://www.mydomain.com/ rig
> Can you provide some more information to be able
> to reproduce your test over here?
Just make a web page like the one described. Enable mod_deflate and load the
page in your favorite browser that has debugging
(Firebug, IE9, Chrome, etc). In my case, it appears Apache selects the chunked
e
> My test showed (according to Firebug) that the 15 MB
> page downloaded in 618ms.
Should clarify that on disk, the page was 14,254,523 bytes but after deflating,
I downloaded a mere 314 bytes of headers
(uncompressed) plus the 41,841 byte response body (compressed) for a total
payload of 42,
> Goal is to get the HEAD of HTML documents in the client side as soon
> as possible ...thus having a more responsive page...
Agreed!
> Can anyone confirm or deny this...
+1
I ran a quick test on a 10MB file that looks like this:
About 15 megabytes of dummy ascii text here...
And my FF4
> ...is it possible that mod_deflate works by chunks...
Why are you doing this? It's not to increase client-side performance because
correct me if I'm wrong here but it's been my
understanding that the web browser cannot start decompressing the page until it
receives the final chunk. Based on
f the page and free up
the database (and RAM) to do other things.
Thanks,
Geoff Millikan @ http://www.t1shopper.com/
-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/usersli
> This went well until there were not too many files on the file system...
Our experience here has been this: We're on ext3 and caching (on disk) several
hundred thousand files using mod_disk_cache. We got
worried we were going to have too many files but using "df -i" below shows
we're at about
> I will mod fstab and reboot one server at a time...
I thought there was a way of dropping atime in real time without a reboot so
you can test performance differences without rebooting
but I could be mistaken. Might want to check out using relatime but I have no
experience with that. I do kno
> I'm not seeing anything
> in the access log file except the reference to the 500 error
> Running
> a php cli on the php file does not return any errors
Check the error log. Sounds like a permissions issue. Check the owner of the
file. And recursive permissions. Are directories
all 0755?
>
> I commented out mod_dir and Apache would not start
Yep, you'll need that one. You probably will not need the below list but you
kinda have to read up on what each module supports and
see if you're using any of those things.
mod_userdir.so
mod_usertrack.so
mod_status.so
mod_logio.so
mod_cgi.so
> We also use New Relic and new PHP performance tuning tool, its very cool...
Oh, and remember to remove/comment out any PHP modules you aren't using. Every
PHP module you load (like the GD image module which
is loaded by default that people don't often use) is loaded up into the Apache
server
> Either the documentation at link below needs updating or
> there's a bug in Apache 2.2.17.
Or there's a user error. :-)
I tested once more before submitting bug report and now it's working, just as
documented. No changes on our side. I think our
servers can smell it when I'm posting here
> Does anyone have this working?
Yes. But, as noted under separate cover, it does not return a 404 status code
in 2.2.17 even when a local URL-path is used like in
the below. The lack of the 404 response will get you dinged by your favorite
search engine as the Apache documentation mentions.
E
doesn't work with mod_file_Cahce.
Lots of bugs. So have to compile from source on this. We're on CentOS 5 too.
Lots of people will suggest the Worker MPM due to it's lower RAM usage however
we haven't done it as the PHP web site
> I am trying to calculate how much memory a server running worker would use.
> With around 700
> 'ps' reports around 10 child processes using 30-100M...
It would be nice if it were that easy. Remember a lot of the processes are
reusing components. Do a 'pmap' on one of the Apache
process IDs
> In my .htaccess file i denied access by many ips.
> I want to log if that ip is trying to access our site,
> in a special log file for audit purpose.
Not that I'm aware of. Suggest doing something like this:
http://lmgtfy.com/?q=load+apache+log+into+database
> I thought that FIPS is within mod_ssl, right?
Doubt it:
http://en.wikipedia.org/wiki/OpenSSL#FIPS_140-2_compliance
-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/use
k on this list, I'll
submit bug report.
Thanks,
Geoff Millikan
http://httpd.apache.org/docs/current/mod/core.html#errordocument
-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd
Maybe this?
"If httpd was compiled against an SSL library which did not support the
FIPS_mode flag, SSLFIPS on will fail."
http://httpd.apache.org/docs/2.2/mod/mod_ssl.html#sslfips
-
The official User-To-User support forum of
> there are still a large number of "non modern" clients out there, including
> anyone
> using the default browser on an Android phone and anyone with any flavor of
> IE
> on WindowsXP
Maybe it's marketing hype but commercial link below shows Subject Alternative
Name certificates being su
We're getting a normal "200" response when our custom "/error/404.php" is
served but when a *local URL-path* is used such as
"ErrorDocument 404 /error/404.php" I expected Apache 2.2.17 to return the
original 404 error status code.
Am I mistaken?
http://httpd.apache.org/docs/current/mod/core.h
> Is there anyway I can have a custom message
> displayed when a site listed in the above file is blocked?
Yes, make a custom webpage for the 401 response:
http://httpd.apache.org/docs/2.2/mod/core.html#errordocument
-
The offi
Dear List,
We were running this:
htcacheclean -n -t -i -d720 -p /var/httpd/proxy -l 20480M
But we found it:
1. Takes ~100MB of virtual memory to run
2. Doesn't have logic (documented) to what's being deleted
3. Takes many hours to run even when not deleting anything (even without the
-n swi
Dear List,
We were running this:
htcacheclean -n -t -i -d720 -p /var/httpd/proxy -l 20480M
But we found it:
1. Takes ~100MB of virtual memory to run
2. Doesn't have logic (documented) to what's being deleted
3. Takes many hours to run even when not deleting anything (even without the
-n swi
>Is the best way to do this like:
> AddDefaultCharset text/html
> SetEnvIfNoCase Accept "xhtml" ua_supports_xhtml=1
> Header set Content-Type "application/xhtml+xml" env=ua_supports_xhtml
Not sure if above method is "best" but we tested and it "works" but since we're
using mod_cache it doesn't w
Dear List,
For the browsers that support XHTML we'll set the Content-Type as
"application/xhtml+xml" otherwise we'll send the same XHTML
document as Content-Type "text/html." Is the best way to do this like:
AddDefaultCharset text/html
SetEnvIfNoCase Accept "xhtml" ua_supports_xhtml=1
Header s
After researching let me write note to self: Apache doesn't let me unset the
Age header using "Header unset Age" as a global config
because even though it would appear to be waste of overhead to calculate it,
and a waste of bandwidth to send it, RFC2616 requires
it because I'm using mod_disk_cac
> fwliw, I've looked before and it is "on purpose" that those requests
> aren't cached; I don't recall the exact reason
I saw something about some caching issues where
http://www.t1shopper.com
was getting properly 301 redirected to
http://www.t1shopper.com/
but mod_cache was doing something odd
To close this out: The symptoms as described in this thread and submitted as a
bug at
https://issues.apache.org/bugzilla/show_bug.cgi?id=51057
just point to the real issue which is with mod_dir (DirectoryIndex) and is
already a known bug but the way it was written up, I
don't believe the scope
> Patches welcome.
My skill at cheeky comments exceeds that of my Apache dev skills which isn't
saying much.
> Or if there's already a patch but it's languishing
> unloved, give us a friendly prod on the dev list!
Searched below for info on patches with no success. Where does one find
patche
Do we give out t-shirts or coffee mugs if someone stumps the list? ;-)
-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-ma
Dear List,
This is cached by mod_disk_cache:
http://www.t1shopper.com/mod_disk_cache_bug/index.php
This isn't cached:
http://www.t1shopper.com/mod_disk_cache_bug/
Anyone know what might cause this?
Thanks,
Geoff
---= Log showing page being written to host's cache =---
mod_cache.c(131): Addin
This issue impacts all index.html, index.php, index.shtml, etc files even
the "homepage." For example:
This is cached: http://www.mycompany.com/index.php
This *is not* cached: http://www.mycompany.com/
-
The official User-To-
Is this a known bug?
At URL #1 below the content isn't cached (or served out of cache):
http://www.t1shopper.com/mod_disk_cache_bug/
At URL #2, we just add the "index.php" into the URL string and the cache is
working properly (see log below).
http://www.t1shopper.com/mod_disk_cache_bug/index.php
hould be (see screenshot):
> https://issues.apache.org/bugzilla/attachment.cgi?id=25524
>
> Thanks,
>
> Geoff Millikan
>
> "Most people say that is it is the intellect which makes a
> great scientist.
> They are wrong: it is character." -Albert Einstein
>
>
on the way out? Simulated load would
> help determine this.
Just the pages in cache are bad. The whole page isn't bad, just the section
where the server-side-include should be (see screenshot):
https://issues.apache.org/bugzilla/attachment.cgi?id=25524
Thanks,
Geoff Millikan
"M
> > Just in case your filesystem doesn't support them, have you tried
> > disabling sendfile and mmap?
>
> Since all our files are being read directly off the drive
> (we're not using a NFS or CIFS mount) this shouldn't
> be an issue (at least on RedHat Enterprise
> Linux). But we'll give it a
> Just in case your filesystem doesn't support them, have you tried
> disabling sendfile and mmap?
Since all our files are being read directly off the drive (we're not using a
NFS or CIFS mount) this shouldn't be an issue (at least on RedHat Enterprise
Linux). But we'll give it a shot and report
Submitted bug report 6 days ago but no movement. Tips?
https://issues.apache.org/bugzilla/show_bug.cgi?id=49385
Thanks!
> -Original Message-
> From: Geoff Millikan
> Sent: Thursday, June 03, 2010 2:25 PM
> To: users@httpd.apache.org
> Subject: [us...@httpd] mod_dis
Dear List,
It appears that when mod_disk_cache reads server side includes to create its
final cached web page, it sometimes corrupts the included file.
I think the issue may be that the included file is getting DEFLATEd and Apache
is intermittently forgetting to ungzip it prior to putting it in
Is this the wrong forum for this question?
> The HTTP spec linked below says, "Servers specify explicit
> expiration times using *either* the Expires header, or
> the max-age directive of the Cache-Control header."
>
> So why then does mod_expires set *both* the Expires HTTP
> header *and*
The HTTP spec linked below says, "Servers specify explicit expiration times
using *either* the Expires header, or the max-age directive of the
Cache-Control header."
So why then does mod_expires set *both* the Expires HTTP header *and* the
max-age directive of the Cache-Control HTTP header? It's
Dear List,
You can see the corrupted/binary garbled mess on the page at the link here:
http://www.t1shopper.com/tools/calculate/index.shtml
The scrambled mess is outputted right at the server side include like this:
Everything else on the page looks fine, it's only the included file that's
Why not change the timezone on the whole server, not just Apache?
smime.p7s
Description: S/MIME cryptographic signature
> Are there any lists of common robots on the net? Are there
> some regular expressions or searches that would help? Are
> there known IP addresses that are safe to discard?
I believe your question is off topic for this forum however I'll share our
joy with you.
Some are known by hostname:
htt
> add Operating System wide firewall rules to
> disallow more than N number of concurrent TCP connections to port 80
> from a single IP address.
You using iptables? What rules did you end up using to accomplish this?
smime.p7s
Description: S/MIME cryptographic signature
How big do we set our mod_disk_cache cache? 100 Megs? 1 Gig?
"Determining how frequently to run htcacheclean and what target size to use
for the cache is somewhat complex and trial and error may be needed to
select optimal values."
http://httpd.apache.org/docs/2.1/caching.html#disk
smime.p7s
> If yes, then try this:
> http://rafael/Cambifon/indice.pl
Make that:
http://rafael:8080/Cambifon/indice.pl
smime.p7s
Description: S/MIME cryptographic signature
>From the command line, can you ping rafael like this:
prompt> ping rafael
If not, then you need to figure out what the FQDN is. Something like this:
prompt> ping rafael.mycompany.com
If yes, then try this:
http://rafael/Cambifon/indice.pl
smime.p7s
Description: S/MIME cryptographic si
Question: What are the rules of thumb for setting the size of the cache?
Fact: The memory usage on the server is below for reference.
Fact: htcacheclean will continuously prune the cache to a certain size.
Fact: When the cache got to 587,692 items taking up 2.2GB of disk space I
started gettin
> http://lmgtfy.com/?q=apache+status+page
Dude, that was harsh, but that's the funniest site I've seen in 6 months,
including The Onion.
smime.p7s
Description: S/MIME cryptographic signature
> +1. Isn't that documented? I'm surprised!
Suppose so, as a n00b though, I thought maybe cutting the default time of 15
in half would do it. It didn't. With many people on fast Internet
connections now, even 1 or 2 second KeepAliveTimeout's seem to work good.
> Interesting! Do you have any
> Fair enough, but if your testing was of ~30 requests, and we
> are believing that
> the typical browser is making 6 simultaneous connections,
> then it sounds like
> the real magic was 6 * fudge factor of 5 ;-P
LOL, yes, I totally agree and I kept thinking that no optimization guide has
MinSpa
> if your server averages 300 simulatious connections, you need
> to start with 300 servers, and you never want it to drop
> below that number.
Your experience might show otherwise however based on our experience - if we
averaged 300 new customers/min at once (not 300 requests/sec) a
MinSpareServe
Here's three things I wish I had read about Apache optimization that are not
commonly discussed (we found out by years of trial and error). These three
settings are worth more than all the other optimization we've done (e.g.
SendBufferSize, AcceptFilter http data, EnableMMAP On, EnableSendfile On,
What order does Apache write the log entries?
I'm assuming that if Apache serves 100 responses all in the same time second
window (100 responses per second) I suppose it's going to write them in the
order the response happened and not scramble them up willy-nilly. In other
words, although there
> Well, for sequence and order any quantity that only increases (or only
> decreases, for that matter) each time it is sampled, would serve.
> Such as a gadget that just hands back the next integer in series every
> time it is queried (properly interlocked across threads/processes).
Agreed, good p
Dan,
Yes, I suppose adding milliseconds to the request header using mod_headers
with a custom header name like "Milliseconds" would work (example below).
It would have to be a custom header because it appears that milliseconds
cannot be appended to any "normal" date time fields since the HTTP 1.1
Using %{UNIQUE_ID}e from mod_unique_id is a *great* suggestion if our
interest was merely identifying each log line uniquely. However the initial
question is focused more on the sequence and order of the requests. Thus, I
believe a time-based solution is what we're after.
This isn't a Linux fo
Dear List,
Has anyone figured out how to add milliseconds to the date and time the
request was made for logging? Something like the below Common Log Format
where the ":1234" represents microseconds:
127.0.0.1 - - [18/Feb/2010:17:02:37:1234 +] "GET /favicon.ico HTTP/1.1"
200 145 "-" "Mozilla/
Thanks. Apache 2.2 rocks and it's still rocking the free world after 15
years this February. Here's to another 15 years - keep up the good work
Apache Project!
smime.p7s
Description: S/MIME cryptographic signature
> [The Apache documentation] should more clearly state that
> it generates ETags for static files only. The resource in
> question is not really a static file.
I imagine there's nothing I can do to help get the 2.2 docs updated but if
there is, I'm happy to help.
> The cache doesn't know the f
> I think this is a long-standing limitation, since the etag can't be
> easily precomputed for the overall response (all the included stuff
> smushed together from potential exec's, etc)
I understand however the FileETag documentation doesn't even *hint* at such
a limitation.
By comparison, mod_d
like this:
How can I get Etag response header to show up on shtml files?
Best Regards,
Geoff Millikan
Newbury Park, CA, USA 91320
http://www.t1shopper.com/
---== httpd.conf Apache 2.2.3 (RedHat5) ==---
FileETag All
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
--== Headers
95 matches
Mail list logo
