Re: Fast by default
4 - 1 compression ratio is fine unless you are serving lots of rich content, which generally will see no performance gain if not reduced performance. As pointed out this option is not a one size fits all arrangement. Shouldn't the default be the best config for everyone based upon the lowest common denominator? On Tue, Jun 1, 2010 at 9:04 AM, William A. Rowe Jr. wr...@rowe-clan.net wrote: [...] Plus deflate may provide no benefit, and degrade performance, if the CPU utilization is a greater concern than bandwidth utilization. The CPU utilization is an interesting topic for me because I've been working on a related type of transform (minify of dynamic content) and I've gotten into the habit of measuring nanoseconds of processing time per byte of content. With that in mind, I just added in some instrumentation around the zlib calls in mod_deflate and found that the compression takes 30-60 ns/byte for buffers of a few KB. (This is with Apache 2.2.14 and zlib 1.2.3 on a 2.8GHz x64 processor.) To put that number in perspective, if you were to devote the equivalent of one CPU core worth of processing power on a web server host to compression, 30ns/byte means the host would be able to do real-time compression of ~266Mb/s of outbound traffic. Assume that your monthly bandwidth pricing is $10 per 95th percentile peak Mbps. Assume further that by turning on deflate you can reduce outbound bandwidth usage by 75% (i.e., you're getting 4:1 compression). Thus the CPU core that you've devoted completely to deflate processing will save you ~$2000 per month in bandwidth pricing. If the CPU core costs less than $24000 per year (amortized capital cost plus power, cooling, support, data center space, marginal cost of additional sysadmins needed to support each additional server, etc, etc), then you still come out ahead by turning on deflate. A few additional thoughts: 1. Speeding up the deflate implementation would be an unqualified win. Supposedly the recently-released zlib 1.2.5 is faster, but I don't have any data on it. 2. The best practice I've found for implementing compression in a web server is to do the compression in the server closest to the network edge. E.g., if you have a web server fronted by a proxy cache, do the compression dynamically in the proxy cache rather than the web server. That way the cache doesn't have to store multiple variants of each object. Similarly, if you're using a CDN for content that can benefit from gzip, ask your CDN if they can do conditional compression for non-problematic user-agents on-the-fly. -Brian -- Jeffrey Burgoyne Chief Technology Officer KCSI Keenuh Consulting Services Inc www.keenuh.com burgo...@keenuh.com
Re: Fast by default
On Wed, Jun 2, 2010 at 3:29 AM, Jeffrey E Burgoyne burgo...@keenuh.com wrote: 4 - 1 compression ratio is fine unless you are serving lots of rich content, which generally will see no performance gain if not reduced performance. The rich content doesn't need to go through the deflate filter, though, so you needn't (and shouldn't) incur any CPU overhead for it. The approach I've used successfully on large sites in the past is to do on-the-fly compression when: 1. The response content-type is HTML, CSS, or JavaScript (plus plain text and XML, for sites serving up those in nontrivial amounts) 2. and the request header contains an Accept-Encoding indicating that the client can handle gzip 3. and the User-Agent is not one of the ones with known problems handling compressed content (IE =5 and Netscape =4, if I remember correctly) The only thing stopping me from suggesting this as a default is the concern that others have noted regarding downstream caches. (I think compression probably ought to be a Transfer-Encoding rather than a Content-Encoding, but that's a whole other debate entirely.) -Brian
Re: Fast by default
Brian, Thanks for great analysis - it's quite useful to see CPU question put in real life perspective. I need to read up on downstream caches problem a bit more, but can you explain how you worked around those problems in your practice? Sergey On Wed, Jun 2, 2010 at 7:11 AM, Brian Pane brianp...@gmail.com wrote: On Wed, Jun 2, 2010 at 3:29 AM, Jeffrey E Burgoyne burgo...@keenuh.com wrote: 4 - 1 compression ratio is fine unless you are serving lots of rich content, which generally will see no performance gain if not reduced performance. The rich content doesn't need to go through the deflate filter, though, so you needn't (and shouldn't) incur any CPU overhead for it. The approach I've used successfully on large sites in the past is to do on-the-fly compression when: 1. The response content-type is HTML, CSS, or JavaScript (plus plain text and XML, for sites serving up those in nontrivial amounts) 2. and the request header contains an Accept-Encoding indicating that the client can handle gzip 3. and the User-Agent is not one of the ones with known problems handling compressed content (IE =5 and Netscape =4, if I remember correctly) The only thing stopping me from suggesting this as a default is the concern that others have noted regarding downstream caches. (I think compression probably ought to be a Transfer-Encoding rather than a Content-Encoding, but that's a whole other debate entirely.) -Brian
Re: Fast by default
Sander, Sorry, missed your reply originally. I would assume that it is possible to make configuration conditional and enable mod_gzip if zlib is available to help distribution builders like Red Hat make this decision to include the module? BTW, it beats me why zlib is not in distros by default - gzip seems like a very commonly used process ;) P.S. this problem does not affect mod_expires and mod_rewrite I was originally talking about - both don't need any additional libraries as far as I know. Sergey On Wed, Jun 2, 2010 at 12:33 AM, Sander Temme scte...@apache.org wrote: All, I was once offered money to provide a high-performance Apache configuration file for a website. When I pointed out that I would need to come in, analyze their app and its performance, and then iteratively tune the config accordingly, I was given to understand that this was not necessary. Just send us the config, please. They were highly miffed when I didn't lay that particular flavor of golden egg for them. I actually got fired from that gig. On Jun 1, 2010, at 5:50 AM, Plüm, Rüdiger, VF-Group wrote: And others have argued well to leave it off by default. My personal opinion is that we should leave it disabled by default for the reasons (CPU, Proxies, Browser behaviour, ETAG problem) mentioned by others. I thought it isn't in the default build because it requires an external library that may not be on the system. If this is not of concern, we might as well turn it on in the default build. Same for mod_ssl. Generally, I think we see the build and runtime configuration as a starting point from which to create your own environment, not a canonical default that is right for all (or even most) users. Distributors (Red Hat et. al.) should (and do) build httpd according to the capabilities of their environment. If that environment includes libz and openssl, no reason why packagers shouldn't build those modules. Including those features is good for their customers. As others have pointed out, a lot of performance tuning is highly site and situation specific. Once again the default configuration file cannot be expected to cover all possible situations. Deflate, caching, load balancing, proxying, content segregation, etc. can only be optimally configured only in the context of the individual deployment. If there were a silver bullet to making the web server fast, don't you think we would have fired it some time in the past 15 years? There is no such thing. The only way to get a handle on it is to read the books, figure it out, or hire a consultant. But don't expect him to lay any golden performance eggs. S. -- Sander Temme scte...@apache.org PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF
Re: Fast by default
Not really - I just know that they serve many users and just wonder how it's done and why that knowledge is not public. Just to give you a perspective on decision making in large companies - if you ordered a work from creative agency, they probably asked you for a list of supported browsers. Guess how long is the list... There is always a trade-off in some way or another - I tried to use the web on Wii and on iPad and the web world looks quite differently there. I'm not saying that this is the degree most popular web server should go for, just an illustration. Sergey On Tue, Jun 1, 2010 at 10:04 PM, HyperHacker hyperhac...@gmail.com wrote: On Tue, Jun 1, 2010 at 16:25, Sergey Chernyshev sergey.chernys...@gmail.com wrote: This sounds scary! How do large companies enable gzip then? How many hoops do they jump through? sounds like those hoops are in thousands! And I don't understand how one company's setup would be different from another still, even if situation is that bad as you describe it. What kind of trade-offs do large companies go for when they enable gzip? more overall traffic? no cache? Sergey On Tue, Jun 1, 2010 at 6:17 PM, toki...@aol.com wrote: There is zero reason for us to avoid putting deflate into the default configuration. Sorry. There ARE (good) reasons to avoid doing so. I'm the one who wrote the FIRST mod_gzip module for Apache 1.x series so you would think I'd be a strong advocate of 'auto-enablement' by default, but I am NOT. There is HOMEWORK involved here and most users will get into deep tapioca unless they understand all the (ongoing) issues. it is also very arguable that we should leave it off. Yes, it is. I think others have argued well to enable it by default. Disagree. I haven't seen the 'good' argument for 'auto-enablement' yet. Some of the reasons to NOT 'go there' are coming out in other similar threads right now... Here's a clip from the (concurrent) message thread entitled... 'Canned deflate conf in manual - time to drop the NS4/Vary' [snip] Don't forget the ongoing issue that if you ONLY vary on 'Accept-Encoding' then almost ALL browsers will then refuse to cache a response entity LOCALLY and the pain factor moves directly to the Proxy/Content Server(s). If you vary on 'User-Agent' ( No longer reasonable because of the abuse of that header 'out there'? ) then the browsers WILL cache responses locally and the pain is reduced at the Proxy/Content server level, but pie is not free at a truck stop and there are then OTHER issues to deal with. The OTHER 'ongoing issue' regarding compression is that, to this day, it still ONLY works for a limited set of MIME types. The 'Accept-Encoding: gzip,deflate' header coming from ALL major browser is still mostly a LIE. It would seem to indicate that the MIME type doesn't matter and it will 'decode' for ANY MIME type but nothing could be further from the truth. There is no browser on the planet that will 'Accept-Encoding' for ANY/ALL mime type(s). If you are going to turn compression ON by default, without the user having to make any decisions for their particular environment, then part of the decision for the default config has to be 'Which MIME types?' text/plain and/or text/html only? SOME browsers can 'Accept-Encoding' on the ever-increasing .js Javascript backloads but some CANNOT. These 2 issues alone are probably enough to justify keeping compression OFF by default. A lot of people that use Apache won't even be able to get their heads around either one of these 'issues' and they really SHOULD do a little homework before turning it ON. Someone already quoted that... 'people expect the default config to just WORK without major issues'. That's exactly what you have now. It's not 'broken'. Why change it? Kevin Kiley [snip] -Original Message- From: Greg Stein gst...@gmail.com To: dev@httpd.apache.org Sent: Tue, Jun 1, 2010 7:40 am Subject: Re: Fast by default Geez, Eric. No wonder people don't want to contribute to httpd, when they run into an attitude like yours. That dismissiveness makes me embarressed for our community. There is zero reason for us to avoid putting deflate into the default configuration. It is also very arguable that we should leave it off. I think others have argued well to enable it by default, while you've simply dismissed them with your holier-than-thou attitude and lack of any solid rationale. -g On May 31, 2010 8:06 PM, Eric Covener cove...@gmail.com wrote: On Mon, May 31, 2010 at 8:30 PM, Bryan McQuade bmcqu...@google.com wrote: I propose providing an... An additional httpd.conf doesn't sound valuable to me. What slice of non-savvy users would scrutinize an alternate config file, can replace the config file of their webserver, isn't using
Re: Fast by default
Sorry for saying this again - regarding cache (expiration), I'm not talking about blindly enabling expiration on content - I'm talking about enabling the module so users CAN configure their sites. Right now they don't have this option with majority of the hasting providers simply because the module is not compiled. Sergey On Tue, Jun 1, 2010 at 12:04 PM, William A. Rowe Jr. wr...@rowe-clan.netwrote: On 6/1/2010 7:05 AM, Eric Covener wrote: Typically, you would want to front a mod_deflate with an HTTP cache, such as mod_cache (or equivalent). Here mod_cache only makes sense if you have the disk space to support it, and there is no real one-size-fits-all cache setup. This said, our default config is 15 years old, and attempts to disable deflate for browsers that don't support it, like Netscape 4. Unless there are modern browsers that have broken protocol support for transfer encoding, these obsolete examples need to be removed. These go together a bit Vary, the current workarounds for old browsers cause Vary: User-Agent which is quite a buzzkill for a cache! Both is out of the question, for the reason you mention. I would argue for neither cache nor deflate, but for a bit different reason. But I see no reason to leave these out of modules 'most', to be ready for the user to deploy them. Cache should be disabled by default simply because it confuses beginning users, and we can't expect they will read all the relevant docs about setting proper expiries on their content. They will undoubtedly be lost, just as we've all been lost reconfiguring our servers and not realizing the 'refresh' wasn't a full refresh on IE, FF or what have you. It will generate too many invalid bug reports and user list queries. The current user list traffic around the cache module seems quite reasonable. Deflate should be disabled simply because the user can't trivially inspect the stream in forensics, and it interacts in odd ways once cache is enabled. I like to think of a fresh httpd install as something the common user can wrap their head around, and we even get user confusion around chunking. Plus deflate may provide no benefit, and degrade performance, if the CPU utilization is a greater concern than bandwidth utilization.
Re: Fast by default ( FWIW - Some tests )
I think you're exactly right about Refresh button - it actually behaves differently in different browsers. In my opinion we should change the test with initiation by click instead as majority of the use-cases are about caching assets between pages when user goes from page to page. I wonder if we can create a test using Browserscope: http://www.browserscope.org/ http://www.browserscope.org/Just for reference - here's the post by Steve Souders about writing such tests: http://www.stevesouders.com/blog/2010/05/12/autohead-my-first-browserscope-user-test/ Here's the API: http://www.browserscope.org/user/tests/howto I believe it's well suited for such tests, but didn't spend enough time with them. http://www.stevesouders.com/blog/2010/05/12/autohead-my-first-browserscope-user-test/ Sergey On Wed, Jun 2, 2010 at 3:11 AM, toki...@aol.com wrote: Well, FWIW, I got curious about what the state of affairs is TODAY with this 'Vary: Accept-Encoding' deal and whether or not certain 'modern' browsers will or won't actually CACHE the responses... so I took a few minutes and just did some simple tests with what I had available here in front of me today. These tests don't go anywhere near finding out what MIME types may or may not be fully supported for decoding by these 'modern' browsers. They ONLY test whether responses with 'Vary: Accept-Encoding' seem to be CACHED locally by the browser(s). The RESULTS, as they relate to this 'Fast by default' THREAD, don't really change any of the arguments much except that maybe things really HAVE gotten a little better in the last few years. IMHO, even these results ( interesting as they may be ) by no means 'justify' turning 'deflate' ON as a default. ** SUMMARY When it comes to JUST the issue of whether a compressed response from Apache 2.x with a 'Vary: Accept-Encoding' header is or is not CACHED locally by the browser(s)... the news is GOOD for Firefox 3.0.18 and MSIE 7 ( Version: 7.0.6000.16982 )... but I'm not sure what to make of the 'results' for Apple's Safari Browser ( Version: 525.28.3 - MacBook ). It would APPEAR, at first glance, that Safari reverts to the old MSIE 5/6 behavior and is REFUSING to cache any response with a 'Vary: Accept-Encoding' header... but monkeying around with the 'History' list produces confusing results. Maybe it did (cache), maybe it didn't. Hard to tell, really. ** DETAIL No kernel debugger was available on the machine in front of me so these were just presentation level 'see what happens' tests. The 'hello.htm' test file was just a 37,709 byte text/html file with no graphics. The actual content was just the first 9 or 10 pages of the RFC2616 document. It was consistently compressed to 9,923 bytes on all '200' responses. *** FIREFOX ( Version: 3.0.18 ) First request for 'hello.htm' using Firefox version 3.0.18... Apache 2.0.63 Server log reports... 65.70.XX.XXX [02/Jun/2010:06:09:46 +0100] GET /hello.htm HTTP/1.1 200 10410 - Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.18) Gecko/2010020220 Firefox/3.0.18 (.NET CLR 3.5.30729) Apache 2.0.63 Server responds with... 200 OK Date: Wed, 02 Jun 2010 05:08:20 GMT Server: Apache/2.0.63 (FreeBSD) Last-Modified: Wed, 02 Jun 2010 04:58:56 GMT ETag: 152887c-907a-f2776400 Accept-Ranges: bytes Vary: Accept-Encoding Content-Encoding: gzip Content-Length: 9923 Connection: close Content-Type: text/html COMPRESSED CONTENT FOLLOWS ** Second request for the same 'hello.htm' document a moment later... ** Initiated by pressing the Firefox 3.0.18 Toolbar 'Refresh' button... Apache 2.0.63 Server log reports... 65.70.XX.XXX [02/Jun/2010:06:12:08 +0100] GET /hello.htm HTTP/1.1 304 335 - Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.18) Gecko/2010020220 Firefox/3.0.18 (.NET CLR 3.5.30729) It would appear that Firefox 3.0.18 did, in fact, cache the initial 200 response that arrived with the 'Vary: Accept-Encoding' and 'did the right thing' when the Toolbar Refresh button was clicked. A conditional GET request was sent and '304 Not Modified' was returned. *** MSIE 7 ( Version: 7.0.6000.16982 ) First request for the same 'hello.htm' page used for the Firefox test... Apache 2.0.63 Server log reports... 65.70.XX.XXX [02/Jun/2010:06:26:17 +0100] GET /hello.htm HTTP/1.1 200 10410 - Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30618) Apache 2.0.63 Server delivers the same response headers and compressed content as the Firefox test. The response headers contained 'Vary: Accept-Encoding' and 'Content-Encoding: gzip'. The only headers that were different were 'Date:' and 'Etag:'. ** Second request for the same 'hello.htm' document a moment later... ** Initiated by pressing the MSIE 7 Toolbar 'Refresh' button... Apache 2.0.63 Server log reports... 65.70.XX.XXX [02/Jun/2010:06:27:26 +0100] GET
Re: Fast by default
On Wed, Jun 2, 2010 at 10:59 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Sorry for saying this again - regarding cache (expiration), I'm not talking about blindly enabling expiration on content - I'm talking about enabling the module so users CAN configure their sites. Right now they don't have this option with majority of the hasting providers simply because the module is not compiled. Which providers don't have mod_expires?Dreamhost does but that's the extent I can test. -- Eric Covener cove...@gmail.com
Re: Fast by default
On Wed, Jun 2, 2010 at 10:35 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Sorry, missed your reply originally. I would assume that it is possible to make configuration conditional and enable mod_gzip if zlib is available to help distribution builders like Red Hat make this decision to include the module? Can you elaborate on this part? mod_gzip is not part of the Apache HTTP Server source distribution -- it's third-party. I'm assuming you meant mod_deflate. mod_deflate, OTOH, is part of Apache, but it's also a part of every binary distribution I can find to check (Ubuntu, RedHat, Novell, Solaris, z/OS, ...). Which distribution builders do you think have this requirement? -- Eric Covener cove...@gmail.com
Re: Fast by default
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Eric Covener wrote: mod_gzip is not part of the Apache HTTP Server source distribution -- it's third-party. I'm assuming you meant mod_deflate. It should. It's great. - -- Arturo Buanzo Busleiman Independent Linux and Security Consultant - OWASP - SANS - OISSG http://www.buanzo.com.ar/pro/eng.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkwGfxgACgkQAlpOsGhXcE3AfgCeJWsgr+9wPoc11vO0MaHefcGE HA4AnRJPu1RAHIcWqdErItai+q25nJom =JvJc -END PGP SIGNATURE-
Re: Fast by default
On 6/2/2010 9:59 AM, Sergey Chernyshev wrote: Right now they don't have this option with majority of the h[o]sting providers simply because the module is not compiled. That is a pretty sweeping generalization. There is no picture here of which do and which do not, nor is there an understanding of why the hosting providers would have elected not to allow these. Hosting providers must balance their demands on ram (cache), bandwidth and the rest, in a way that doesn't lend itself to casual users meddling with such settings. If you aren't able to provision software as root, there are good reasons that you shouldn't be tweaking certain modules in httpd, either. Most of the open issues I'm aware of for untrusted administrators have to do with further limiting the options available to such hosting account users, not granting them permissions to set up unwise configurations that will impact other users of the same boxes. We can certainly ensure that modules 'most' includes such things as the expires and deflate modules, if it does not today. Although I'm not a fan of changing those elections for 2.2, since users are used to what they are enabling and disabling. For 2.4? Sure, let's make sure that happens. I suspect you may be overthinking the problem as something that the httpd project can solve for end users of our users. We can't. Our users are the administrators who provision our software, and what they allow to shared hosting users is entirely up to them.
Re: Fast by default
Yes, I meant mod_detlate, sorry. I stand corrected. Will have to check the distros. Why is it not enabled by default in the source distribution then if all the packagers have it enabled. Sergey On Wed, Jun 2, 2010 at 11:49 AM, Eric Covener cove...@gmail.com wrote: On Wed, Jun 2, 2010 at 10:35 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Sorry, missed your reply originally. I would assume that it is possible to make configuration conditional and enable mod_gzip if zlib is available to help distribution builders like Red Hat make this decision to include the module? Can you elaborate on this part? mod_gzip is not part of the Apache HTTP Server source distribution -- it's third-party. I'm assuming you meant mod_deflate. mod_deflate, OTOH, is part of Apache, but it's also a part of every binary distribution I can find to check (Ubuntu, RedHat, Novell, Solaris, z/OS, ...). Which distribution builders do you think have this requirement? -- Eric Covener cove...@gmail.com
Re: Fast by default
On Wed, Jun 2, 2010 at 1:59 PM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Why is it not enabled by default in the source distribution then if all the packagers have it enabled. Assuming you mean compiled by default. When you build Apache and don't ask for any specific modules, and don't ask for most or all modules, you get a very minimal httpd with a small number of modules statically compiled into your httpd binary. It has external dependencies, and it's not required for a minimal server, so it's not in this lowest common denominator bucket of enabled modules. OT: Please consider posting in-line instead of at the top of each thread, top-posting makes the discussion very difficult to follow. -- Eric Covener cove...@gmail.com
Re: Fast by default
On Wed, Jun 2, 2010 at 2:09 PM, Eric Covener cove...@gmail.com wrote: On Wed, Jun 2, 2010 at 1:59 PM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Why is it not enabled by default in the source distribution then if all the packagers have it enabled. Assuming you mean compiled by default. Yes I meant compiled by default. I'm sorry, I keep missing the terminology again. When you build Apache and don't ask for any specific modules, and don't ask for most or all modules, you get a very minimal httpd with a small number of modules statically compiled into your httpd binary. It has external dependencies, and it's not required for a minimal server, so it's not in this lowest common denominator bucket of enabled modules. I believe it's possible to check for dependencies and if they are available, then have stuff enabled? In this case it only concerns zlib, for expires and mod_rewrite there are actually no dependencies as I understand, right? OT: Please consider posting in-line instead of at the top of each thread, top-posting makes the discussion very difficult to follow. Sure, I will. Sergey -- Eric Covener cove...@gmail.com
Re: Fast by default
On Wed, Jun 2, 2010 at 12:52 PM, William A. Rowe Jr. wr...@rowe-clan.netwrote: On 6/2/2010 9:59 AM, Sergey Chernyshev wrote: Right now they don't have this option with majority of the h[o]sting providers simply because the module is not compiled. That is a pretty sweeping generalization. There is no picture here of which do and which do not, nor is there an understanding of why the hosting providers would have elected not to allow these. Hosting providers must balance their demands on ram (cache), bandwidth and the rest, in a way that doesn't lend itself to casual users meddling with such settings. If you aren't able to provision software as root, there are good reasons that you shouldn't be tweaking certain modules in httpd, either. Most of the open issues I'm aware of for untrusted administrators have to do with further limiting the options available to such hosting account users, not granting them permissions to set up unwise configurations that will impact other users of the same boxes. We can certainly ensure that modules 'most' includes such things as the expires and deflate modules, if it does not today. Although I'm not a fan of changing those elections for 2.2, since users are used to what they are enabling and disabling. For 2.4? Sure, let's make sure that happens. I'm not a person to show you urgency here, but I have a feeling that 2.4 is 2-3 years away from production web sites (correct me if I'm wrong here). And it might be too long to wait for these fundamental performance optimizations. I think the question of compiling in all the modules is more pressing, but default configuration can be pushed to 2.4 while being an advisory in 2.2. I suspect you may be overthinking the problem as something that the httpd project can solve for end users of our users. We can't. Our users are the administrators who provision our software, and what they allow to shared hosting users is entirely up to them. No, I'm not overthinking it - as Bryan mentioned before, hosting providers are not configuring stuff and relying on httpd project to provide viable defaults. They are in business of selling services, not development and best practices on the web and that slows down improvements (in this case performance). It's another part of performance movement to help them decide and show that gzip saves more money in traffic then it takes in CPU, for example. Thank you, Sergey
Re: Fast by default
On 6/2/2010 2:06 PM, Sergey Chernyshev wrote: I'm not a person to show you urgency here, but I have a feeling that 2.4 is 2-3 years away from production web sites (correct me if I'm wrong here). And it might be too long to wait for these fundamental performance optimizations. I rather think it will over the next 12 months. 2.4 will not be radically different (unlike the 1.3 - 2.0 transition) but will offer such blindingly superior code improvements to the mod_proxy_* and mod_*_cache that users really won't want to stay behind (much like the 2.0 - 2.2 transition). My theory, YMMV. I think the question of compiling in all the modules is more pressing, but default configuration can be pushed to 2.4 while being an advisory in 2.2. We do, in the tarball, plus we host it at http://httpd.apache.org/docs/2.2/ No, I'm not overthinking it - as Bryan mentioned before, hosting providers are not configuring stuff and relying on httpd project to provide viable defaults. We do not distribute an application which is suitable for modification by untrusted individuals within a shared mass hosting environment. Legitimate hosting providers must do their research and understand the product, it's not ment for such consumption out of the box. Neither are the root-owned files of your typical operating system, either.
Re: Fast by default
On Wed, Jun 2, 2010 at 3:24 PM, William A. Rowe Jr. wr...@rowe-clan.netwrote: Legitimate hosting providers must do their research and understand the product, it's not ment for such consumption out of the box. I do not agree here - you're not asking hosting providers to do research on every aspect of HTTP. Why do you think that correct mime-types or file security setup is OK to have as default while correct compression is not? I understand that httpd as a product can be tweaked in millions of different ways, but see no reason why it can't adhere to quality and best practices by default. Neither are the root-owned files of your typical operating system, either. BTW, root-owned files of your typical operating system have some things configured by default and distro-packager (for example Red Hat, in case of Linux) is taking care of best practices for performance, security and so on while sys-admins are configuring them further for users to be secure and so on. Sergey
Re: Fast by default (FWIW - Some tests)
Bryan McQuade wrote... thanks! it is really great that you did this investigation. You're welcome, but I wouldn't really call that an 'investigation'. More like just a quick 'observation'. RE: checking to see if in cache, try typing the URL into the nav bar and hitting enter rather than reloading. Tried that ( with Safari ). No conditional GET was sent. Behavior was the same as pressing the 'Refresh' Toolbar button. The entire document was 'Reloaded' and not 'Refreshed'. OT: If this discussion is going to continue let's agree that there IS, in fact, a difference between saying 'Reload' and 'Refresh'. On most browsers, the Toolbar button is SUPPOSED to behave as a 'Refresh' option. A browser is supposed to check its local cache and issue a 'conditional GET' request if it has a non-expired copy of the entity onboard. A RELOAD is when this local-cache-check process is 'skipped' and a browser simply disregards the content of its cache and RELOADS the page with no conditional GET request. CTRL-R is the industry standard browser RELOAD command. Works the same for both the MSIE and Mozilla/Firefox browser lineage. On Apple/Safari it's COMMAND-R. Interesting side note: Official documentation for Safari actually says the Toolbar Button is SUPPOSED to be the 'Refresh' option ( NOT RELOAD ), just like other browsers, and that pressing SHIFT-Refresh is the 'official' way to force a page to RELOAD instead of REFRESHING. If that is actually the case then the quick Safari test I did really would seem to indicate that the response with the 'Vary: Accept-Encoding' header was NOT CACHED. most browsers use a more aggressive reload algorithm (bypassing the cache for hte html) on reload. Of course. See above about the established standards and the difference between 'Refresh' and 'Reload'. also could you set an explicit resource expiration? otherwise you're operating at the whim of caching heuristics which aren't explicitly specified by the HTTP RFC. That is exactly why I did NOT add any resource control directives. The point of the test(s) was to observe the DEFAULT caching behavior. Try setting an Expires or Cache-Control: max-age header on your response. See http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.2.4 for info on heuristics. See above. I'm perfectly familiar with the directives. NOT using any of them is/was part of the testing. In my tests most browsers implement the 10% rule but not all. YOUR tests? Exactly what tests are you referring to? Are you saying you already have some detailed caching tests for X number of browsers? Do you have any 'tests' of your own that involve 'Vary:' headers and local caching behavior? Can you share any of those results or would that violate Google's information sharing policy? If you have time can you also test with safari4? Perhaps there was an issue in 3.x that was fixed in 4.x. I am not an Apple person. I do not personally own any Apple hardware. The MacBook I used for a test is a loaner from a client. I will not/cannot change their current software configuration(s). Bryan, don't take this the wrong way, but everyone is perfectly aware of who you are and who you work for and what your/their agenda is. I'm not criticizing that in any way. You have every right to be here contributing to an open-source project... ...but remember that SOME of us just do this as a HOBBY. People like you are being PAID to be here and it's part of YOUR JOB to discover/know the answers to some of the questions. I. myself, am simply curious. There is no paycheck behind anything I do with regards to Apache. As an employee of Google I would imagine you have far more resources than I do up to and including any number of machines that you could configure with any browser you want for your OWN testing. one thing that's also useful is to first load a page to put the resource in the browser cache, then shut down the browser and restart it to try to load that page again. this makes sure that the resource was really persisted to the disk cache and isn't just being reserved out of some temporary in memory cache. Great idea... you should make that part of YOUR testing. My curiosity has already been satisfied. Things are a little better than the were a few years ago. That was all I wanted to know yesterday. Now it's back to my REAL job, which has nothing to do with Content-encoding on the World Wide Web. thanks again! very cool that you did this. -bryan Again... you are welcome. Let me/us know how your OWN testing goes there at Google. Yours Kevin
Re: Fast by default
No, I'm not overthinking it - as Bryan mentioned before, hosting providers are not configuring stuff and relying on httpd project to provide viable defaults. Some of them behave this way, but I haven't seen any indication that's it's due to not being in some no-argument build. And if it was, springing such a change vicariously in the service stream doesn't sound wise. Maybe they're pushing users to higher priced hosting packages? They are in business of selling services, not development and best practices I don't see where development would be equired, but if your service is web-hosting you're most certainly in the business of building, maintaining, and configuring webservers. You of course have to know the best practices around the service you sell if you want to stay in business more than 6 months. -- Eric Covener cove...@gmail.com
Re: Fast by default
On Jun 2, 2010, at 12:06 PM, Sergey Chernyshev wrote: No, I'm not overthinking it - as Bryan mentioned before, hosting providers are not configuring stuff and relying on httpd project to provide viable defaults. Have you considered that the default settings are fine for the vast majority of deployments? Even for hosting services? They are in business of selling services, not development and best practices on the web and that slows down improvements (in this case performance). What slows down improvements for everyone, hosters and users alike, is the putrid mess in existence on the client side and the requirement that we target the lowest common denominator. As we have once again learned this week, even Deflate is not something that we can just enable in the expectation that it will work with all clients. Your hoster is likely not interested in fielding angry phone calls from their customers, whose customers' pages inexplicably don't load right because of a problem in the way they handle transfer encoding. That's a business decision for them: wetware staffing the phone or a potential increase in bandwidth. If your performance requirements outstrip the (technical or service) capabilities of your hoster, you'll have to look for other options. It seems to me that the hosting market is extremely commoditized at this point: there should be plenty of choice. If you still need more performance, you'll get your own infrastructure which you can then configure to your heart's desire. It's another part of performance movement to help them decide and show that gzip saves more money in traffic then it takes in CPU, for example. Documentation and education are good. Improving the documentation is especially good, since it ships with every server. I believe you have already started working on that: good idea. S. -- Sander Temme scte...@apache.org PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF
Re: Fast by default
On 6/1/2010 3:30 AM, Bryan McQuade wrote: I had a conversation with a well known hosting provider recently and they told me they use the default Apache configuration for their shared hosting service. When I asked if they provide gzip as an option for their users, they said no, since it was not enabled by default. When I explained to them that enabling gzip has significant benefits for end users they were very interested in turning on gzip. This company just used the default Apache config, assuming that it was reasonably well tuned by default. You can claim that they're making bad, uninformed decisions, or whatever you want to, but the fact remains: some Apache users assume that the default config is a reasonably good config, and use it as-is. There are two classes of users out there: power users that want to tune every knob manually, and typical users that just want Apache to work out of the box. Apache developers on this list are part of the former group, which I assume is why the default Apache config is very simple. It's designed by power users, for power users. But there are non-power users out there. It would be nice if the Apache community provided configuration hints for these users. In 2010, IMO there is no good reason to have gzip disabled by default. Almost all websites enable it. There are a handful of prominent websites that do not. I've had conversations with a few of these sites. Most of them have not turned it on because they don't understand what it does, not because they don't have enough CPU. gzip has been used on the web now for well over 10 years. Only *very* old browsers, proxies, etc don't have perfect support for gzip. I can respect that the default httpd.conf is designed to be simple and minimalist. But it would be helpful to have an additional example configuration in svn trunk and as part of Apache releases, that enable things like mod_deflate. The current comments in httpd.conf explain that there are additional directives and you have been warned but IMO this is not very helpful or specific. We can do better. I propose providing an additional httpd.conf in the svn trunk and as part of future Apache releases that enables modules and directives that are commonly recommended on Apache performance tuning websites. This includes mod_deflate, mod_expires, etc. This will allow power users to continue to start with the current httpd.conf while typical users can just use the well optimized configuration. Hopefully this suggestion isn't too controversial. If there are concerns about some of the specific directives suggested in this thread, I'm sure we can work those out through discussion. Can we agree that it would be useful to provide an additional configuration file for non-power users that enables commonly recommended modules by default? +1 The PHP project does something similar, and so did MySQL (they might still - haven't done a source install there lately). One could point out that most out-of-the-box users (the linux ones, at least) use their distribution's binaries (which is another story) but I don't feel that that alone justifies not doing something like this. I propose that the recommended production deployment httpd.conf enable the features that we feel should be adopted by the internet community, as a whole, regardless of individual use cases, and we can put a prominent notice that it can and should be tuned by system administrators. In addition to folks who still use it out-of-the-box we'll also get lots of power-users looking at what we consider to be new and important features for the future of the community. I'd also like to see TLS/SNI as a recommended feature for this proposed conf. Issac
Re: Fast by default
On 01.06.2010 07:19, Jerome Renard wrote: In 2010, IMO there is no good reason to have gzip disabled by default. Almost all websites enable it. There are a handful of prominent websites that do not. I've had conversations with a few of these sites. Most of them have not turned it on because they don't understand what it does, not because they don't have enough CPU. gzip has been used on the web now for well over 10 years. Only *very* old browsers, proxies, etc don't have perfect support for gzip. I remember having a lot of troubles with some combinations of Squid + IE (6 or so) where the compressed content was just never gunziped. Was I the only one to get such problems ? There are still problems with caching proxies, and the behaviour of mod_deflate has changed several times, even in the 2.2.x lifetime. There was a lot of discussion about it. As a starter you can read my last comment in https://issues.apache.org/bugzilla/show_bug.cgi?id=49358 and then follow the links given there. In short: mod_deflate uses Content-Encoding instead of Transfer-Encoding which is supported by the Browsers, but leads to tricky problems when chosing the right ETag (content equivalence, cache revalidation requests). Regards, Rainer
Re: Fast by default
On 01 Jun 2010, at 2:30 AM, Bryan McQuade wrote: I had a conversation with a well known hosting provider recently and they told me they use the default Apache configuration for their shared hosting service. When I asked if they provide gzip as an option for their users, they said no, since it was not enabled by default. When I explained to them that enabling gzip has significant benefits for end users they were very interested in turning on gzip. This company just used the default Apache config, assuming that it was reasonably well tuned by default. You can claim that they're making bad, uninformed decisions, or whatever you want to, but the fact remains: some Apache users assume that the default config is a reasonably good config, and use it as-is. The very definition of tuned means tailored for your local setup. The default httpd configuration works reasonably well out the box. It is only when your site has special needs that it should start changing the setup, and the site should understand what their needs are and whether it is appropriate to turn it on. Zooming into mod_deflate, mod_deflate only makes sense if you have the CPU to support it. If you don't have enough CPU support (think virtualised hosts), mod_deflate will be a performance drag, not a boost. Typically, you would want to front a mod_deflate with an HTTP cache, such as mod_cache (or equivalent). Here mod_cache only makes sense if you have the disk space to support it, and there is no real one-size-fits-all cache setup. The next problem is that you only want to enable mod_deflate on compressible content - that means not images for most people, but might not be. Again, not every site has the same content, and therefore not every site has the same setup for mod_delate. This said, our default config is 15 years old, and attempts to disable deflate for browsers that don't support it, like Netscape 4. Unless there are modern browsers that have broken protocol support for transfer encoding, these obsolete examples need to be removed. Regards, Graham --
Re: Fast by default
- Graham Leggett minf...@sharp.fm wrote: The very definition of tuned means tailored for your local setup. It's actually quite hard to get this thought accross. I think we should put it in a reboot of the performance ``optimization'' documentation. The default httpd configuration works reasonably well out the box. It is only when your site has special needs that it should start changing the setup, and the site should understand what their needs are and whether it is appropriate to turn it on. Zooming into mod_deflate, mod_deflate only makes sense if you have the CPU to support it. If you don't have enough CPU support (think virtualised hosts), mod_deflate will be a performance drag, not a boost. Typically, you would want to front a mod_deflate with an HTTP cache, such as mod_cache (or equivalent). Here mod_cache only makes sense if you have the disk space to support it, and there is no real one-size-fits-all cache setup. We arrive at one of my pet-peeve topic: Sane defaults. mod_cache's default values for CacheDirLevels and CacheDirLength will tear your file-system apart. And that is pretty much what: http://httpd.apache.org/docs/2.2/caching.html says... The next problem is that you only want to enable mod_deflate on compressible content - that means not images for most people, but might not be. Again, not every site has the same content, and therefore not every site has the same setup for mod_delate. This said, our default config is 15 years old, and attempts to disable deflate for browsers that don't support it, like Netscape 4. Unless there are modern browsers that have broken protocol support for transfer encoding, these obsolete examples need to be removed. An entirely different topic, but related to your proposal is that mod_deflate suggests to use a mod_filter based setup, but there's no example on how to configure that. (See my attached patch. Then you might understand why ;) Regards, Graham -- Bye, -- Igor Galić Tel: +43 (0) 699 122 96 338 Fax: +43(0) 1 91 333 41 Mail: i.ga...@brainsware.org URL: http://brainsware.org/ Index: mod/mod_deflate.xml === --- mod/mod_deflate.xml (revision 949964) +++ mod/mod_deflate.xml (working copy) @@ -44,32 +44,20 @@ AddOutputFilterByType DEFLATE text/html text/plain text/xml /example -pThe following configuration, while resulting in more compressed content, +pThe following configuration, while being the recommended setup using mod_filter, +results in more compressed content than the above but is also much more complicated. Do not use this unless you fully understand all the configuration details./p exampletitleCompress everything except images/title lt;Location /gt;br / indent -# Insert filterbr / -SetOutputFilter DEFLATEbr / -br / -# Netscape 4.x has some problems...br / -BrowserMatch ^Mozilla/4 gzip-only-text/htmlbr / -br / -# Netscape 4.06-4.08 have some more problemsbr / -BrowserMatch ^Mozilla/4\.0[678] no-gzipbr / -br / -# MSIE masquerades as Netscape, but it is finebr / -BrowserMatch \bMSIE !no-gzip !gzip-only-text/htmlbr / -# Don't compress imagesbr / -SetEnvIfNoCase Request_URI \br / -indent - \.(?:gif|jpe?g|png)$ no-gzip dont-varybr / -/indent -br / -# Make sure proxies don't deliver the wrong contentbr / -Header append Vary User-Agent env=!dont-varybr / + # Declare filter, add mod_deflate for all text/* MIME Typesbr / + FilterDeclare COMPRESSbr / + FilterProvider COMPRESS DEFLATE resp=Content-Type $text/br / + FilterChain COMPRESSbr / + FilterProtocol COMPRESS change=yes;byteranges=no + Header append Vary User-Agent env=!dont-varybr / /indent lt;/Locationgt; /example
Re: Fast by default
Typically, you would want to front a mod_deflate with an HTTP cache, such as mod_cache (or equivalent). Here mod_cache only makes sense if you have the disk space to support it, and there is no real one-size-fits-all cache setup. This said, our default config is 15 years old, and attempts to disable deflate for browsers that don't support it, like Netscape 4. Unless there are modern browsers that have broken protocol support for transfer encoding, these obsolete examples need to be removed. These go together a bit Vary, the current workarounds for old browsers cause Vary: User-Agent which is quite a buzzkill for a cache! -- Eric Covener cove...@gmail.com
Re: Fast by default
Geez, Eric. No wonder people don't want to contribute to httpd, when they run into an attitude like yours. That dismissiveness makes me embarressed for our community. There is zero reason for us to avoid putting deflate into the default configuration. It is also very arguable that we should leave it off. I think others have argued well to enable it by default, while you've simply dismissed them with your holier-than-thou attitude and lack of any solid rationale. -g On May 31, 2010 8:06 PM, Eric Covener cove...@gmail.com wrote: On Mon, May 31, 2010 at 8:30 PM, Bryan McQuade bmcqu...@google.com wrote: I propose providing an... An additional httpd.conf doesn't sound valuable to me. What slice of non-savvy users would scrutinize an alternate config file, can replace the config file of their webserver, isn't using a webserver packaged by their OS, and wouldn't have just gotten the same information today from the manual and 400,000 other websites? There's currently no ifModule bloat in the default conf, but you're welcome to submit a patch that adds one for deflate or expires (latter seems more unwise to me). See the supplemental configuration section of the generated config. This doesn't address mass-vhost companies failing to allow deflate because it's not in the no-args HTTPD ./configure , which sounds far-fetched to me. I can't recall a users@ or #httpd user implying being subjected to such a thing with their own build or with cheap hosting. -- Eric Covener cove...@gmail.com
RE: Fast by default
From: Greg SteinSent: Dienstag, 1. Juni 2010 14:40 To: dev@httpd.apache.org Subject: Re: Fast by default Geez, Eric. No wonder people don't want to contribute to httpd, when they run into an attitude like yours. That dismissiveness makes me embarressed for our community. There is zero reason for us to avoid putting deflate into the default configuration. It is also very arguable that we should leave it off. I think others have argued well to enable it by default, while you've simply dismissed them with your holier-than-thou attitude and lack of any solid rationale. And others have argued well to leave it off by default. My personal opinion is that we should leave it disabled by default for the reasons (CPU, Proxies, Browser behaviour, ETAG problem) mentioned by others. Regards Rüdiger -g On May 31, 2010 8:06 PM, Eric Covener cove...@gmail.com wrote: On Mon, May 31, 2010 at 8:30 PM, Bryan McQuade bmcqu...@google.com wrote: I propose providing an... An additional httpd.conf doesn't sound valuable to me. What slice of non-savvy users would scrutinize an alternate config file, can replace the config file of their webserver, isn't using a webserver packaged by their OS, and wouldn't have just gotten the same information today from the manual and 400,000 other websites? There's currently no ifModule bloat in the default conf, but you're welcome to submit a patch that adds one for deflate or expires (latter seems more unwise to me). See the supplemental configuration section of the generated config. This doesn't address mass-vhost companies failing to allow deflate because it's not in the no-args HTTPD ./configure , which sounds far-fetched to me. I can't recall a users@ or #httpd user implying being subjected to such a thing with their own build or with cheap hosting. -- Eric Covener cove...@gmail.com
Re: Fast by default
On 6/1/2010 7:05 AM, Eric Covener wrote: Typically, you would want to front a mod_deflate with an HTTP cache, such as mod_cache (or equivalent). Here mod_cache only makes sense if you have the disk space to support it, and there is no real one-size-fits-all cache setup. This said, our default config is 15 years old, and attempts to disable deflate for browsers that don't support it, like Netscape 4. Unless there are modern browsers that have broken protocol support for transfer encoding, these obsolete examples need to be removed. These go together a bit Vary, the current workarounds for old browsers cause Vary: User-Agent which is quite a buzzkill for a cache! Both is out of the question, for the reason you mention. I would argue for neither cache nor deflate, but for a bit different reason. But I see no reason to leave these out of modules 'most', to be ready for the user to deploy them. Cache should be disabled by default simply because it confuses beginning users, and we can't expect they will read all the relevant docs about setting proper expiries on their content. They will undoubtedly be lost, just as we've all been lost reconfiguring our servers and not realizing the 'refresh' wasn't a full refresh on IE, FF or what have you. It will generate too many invalid bug reports and user list queries. The current user list traffic around the cache module seems quite reasonable. Deflate should be disabled simply because the user can't trivially inspect the stream in forensics, and it interacts in odd ways once cache is enabled. I like to think of a fresh httpd install as something the common user can wrap their head around, and we even get user confusion around chunking. Plus deflate may provide no benefit, and degrade performance, if the CPU utilization is a greater concern than bandwidth utilization.
Re: Fast by default
There is zero reason for us to avoid putting deflate into the default configuration. Sorry. There ARE (good) reasons to avoid doing so. I'm the one who wrote the FIRST mod_gzip module for Apache 1.x series so you would think I'd be a strong advocate of 'auto-enablement' by default, but I am NOT. There is HOMEWORK involved here and most users will get into deep tapioca unless they understand all the (ongoing) issues. it is also very arguable that we should leave it off. Yes, it is. I think others have argued well to enable it by default. Disagree. I haven't seen the 'good' argument for 'auto-enablement' yet. Some of the reasons to NOT 'go there' are coming out in other similar threads right now... Here's a clip from the (concurrent) message thread entitled... 'Canned deflate conf in manual - time to drop the NS4/Vary' [snip] Don't forget the ongoing issue that if you ONLY vary on 'Accept-Encoding' then almost ALL browsers will then refuse to cache a response entity LOCALLY and the pain factor moves directly to the Proxy/Content Server(s). If you vary on 'User-Agent' ( No longer reasonable because of the abuse of that header 'out there'? ) then the browsers WILL cache responses locally and the pain is reduced at the Proxy/Content server level, but pie is not free at a truck stop and there are then OTHER issues to deal with. The OTHER 'ongoing issue' regarding compression is that, to this day, it still ONLY works for a limited set of MIME types. The 'Accept-Encoding: gzip,deflate' header coming from ALL major browser is still mostly a LIE. It would seem to indicate that the MIME type doesn't matter and it will 'decode' for ANY MIME type but nothing could be further from the truth. There is no browser on the planet that will 'Accept-Encoding' for ANY/ALL mime type(s). If you are going to turn compression ON by default, without the user having to make any decisions for their particular environment, then part of the decision for the default config has to be 'Which MIME types?' text/plain and/or text/html only? SOME browsers can 'Accept-Encoding' on the ever-increasing .js Javascript backloads but some CANNOT. These 2 issues alone are probably enough to justify keeping compression OFF by default. A lot of people that use Apache won't even be able to get their heads around either one of these 'issues' and they really SHOULD do a little homework before turning it ON. Someone already quoted that... 'people expect the default config to just WORK without major issues'. That's exactly what you have now. It's not 'broken'. Why change it? Kevin Kiley [snip] -Original Message- From: Greg Stein gst...@gmail.com To: dev@httpd.apache.org Sent: Tue, Jun 1, 2010 7:40 am Subject: Re: Fast by default Geez, Eric. No wonder people don't want to contribute to httpd, when they run into an attitude like yours. That dismissiveness makes me embarressed for our community. There is zero reason for us to avoid putting deflate into the default configuration. It is also very arguable that we should leave it off. I think others have argued well to enable it by default, while you've simply dismissed them with your holier-than-thou attitude and lack of any solid rationale. -g On May 31, 2010 8:06 PM, Eric Covener cove...@gmail.com wrote: On Mon, May 31, 2010 at 8:30 PM, Bryan McQuade bmcqu...@google.com wrote: I propose providing an... An additional httpd.conf doesn't sound valuable to me. What slice of non-savvy users would scrutinize an alternate config file, can replace the config file of their webserver, isn't using a webserver packaged by their OS, and wouldn't have just gotten the same information today from the manual and 400,000 other websites? There's currently no ifModule bloat in the default conf, but you're welcome to submit a patch that adds one for deflate or expires (latter seems more unwise to me). See the supplemental configuration section of the generated config. This doesn't address mass-vhost companies failing to allow deflate because it's not in the no-args HTTPD ./configure , which sounds far-fetched to me. I can't recall a users@ or #httpd user implying being subjected to such a thing with their own build or with cheap hosting. -- Eric Covener cove...@gmail.com
Re: Fast by default
web sites are loading too slow for pipes and web-server power that we have. The key phrase there is 'that WE have'. YOU need to tune YOUR configs to match what YOU have. ANYONE who uses Apache can/should/must do that. That's how that works. The discussion at this moment is what 'default' configs should ship with Apache. It is NOT POSSIBLE to accomodate EVERYONE. The default httpd.conf for Apache is simply JFW ( Just Feckin Works )... and for a product as complicated as Apache I tend to agree with those who think that is all that it needs to 'ship' with. Kevin Kiley -Original Message- From: Sergey Chernyshev sergey.chernys...@gmail.com To: dev@httpd.apache.org Sent: Tue, Jun 1, 2010 5:30 pm Subject: Re: Fast by default It's not 'broken'. Why change it? Please don't think that old configurations and practices are not broken - web sites are loading too slow for pipes and web-server power that we have. And situation is getting worse year after year - here's analysis by Patrick Meanan of WebPageTest.org's one year history: http://blog.patrickmeenan.com/2010/05/are-pages-getting-faster.html Sergey Kevin Kiley [snip] -Original Message- From: Greg Stein gst...@gmail.com To: dev@httpd.apache.org Sent: Tue, Jun 1, 2010 7:40 am Subject: Re: Fast by default Geez, Eric. No wonder people don't want to contribute to httpd, when they run into an attitude like yours. That dismissiveness makes me embarressed for our community. There is zero reason for us to avoid putting deflate into the default configuration. It is also very arguable that we should leave it off. I think others have argued well to enable it by default, while you've simply dismissed them with your holier-than-thou attitude and lack of any solid rationale. -g On May 31, 2010 8:06 PM, Eric Covener cove...@gmail.com wrote: On Mon, May 31, 2010 at 8:30 PM, Bryan McQuade bmcqu...@google.com wrote: I propose providing an... An additional httpd.conf doesn't sound valuable to me. What slice of non-savvy users would scrutinize an alternate config file, can replace the config file of their webserver, isn't using a webserver packaged by their OS, and wouldn't have just gotten the same information today from the manual and 400,000 other websites? There's currently no ifModule bloat in the default conf, but you're welcome to submit a patch that adds one for deflate or expires (latter seems more unwise to me). See the supplemental configuration section of the generated config. This doesn't address mass-vhost companies failing to allow deflate because it's not in the no-args HTTPD ./configure , which sounds far-fetched to me. I can't recall a users@ or #httpd user implying being subjected to such a thing with their own build or with cheap hosting. -- Eric Covener cove...@gmail.com
Re: Fast by default
On Tue, Jun 1, 2010 at 16:25, Sergey Chernyshev sergey.chernys...@gmail.com wrote: This sounds scary! How do large companies enable gzip then? How many hoops do they jump through? sounds like those hoops are in thousands! And I don't understand how one company's setup would be different from another still, even if situation is that bad as you describe it. What kind of trade-offs do large companies go for when they enable gzip? more overall traffic? no cache? Sergey On Tue, Jun 1, 2010 at 6:17 PM, toki...@aol.com wrote: There is zero reason for us to avoid putting deflate into the default configuration. Sorry. There ARE (good) reasons to avoid doing so. I'm the one who wrote the FIRST mod_gzip module for Apache 1.x series so you would think I'd be a strong advocate of 'auto-enablement' by default, but I am NOT. There is HOMEWORK involved here and most users will get into deep tapioca unless they understand all the (ongoing) issues. it is also very arguable that we should leave it off. Yes, it is. I think others have argued well to enable it by default. Disagree. I haven't seen the 'good' argument for 'auto-enablement' yet. Some of the reasons to NOT 'go there' are coming out in other similar threads right now... Here's a clip from the (concurrent) message thread entitled... 'Canned deflate conf in manual - time to drop the NS4/Vary' [snip] Don't forget the ongoing issue that if you ONLY vary on 'Accept-Encoding' then almost ALL browsers will then refuse to cache a response entity LOCALLY and the pain factor moves directly to the Proxy/Content Server(s). If you vary on 'User-Agent' ( No longer reasonable because of the abuse of that header 'out there'? ) then the browsers WILL cache responses locally and the pain is reduced at the Proxy/Content server level, but pie is not free at a truck stop and there are then OTHER issues to deal with. The OTHER 'ongoing issue' regarding compression is that, to this day, it still ONLY works for a limited set of MIME types. The 'Accept-Encoding: gzip,deflate' header coming from ALL major browser is still mostly a LIE. It would seem to indicate that the MIME type doesn't matter and it will 'decode' for ANY MIME type but nothing could be further from the truth. There is no browser on the planet that will 'Accept-Encoding' for ANY/ALL mime type(s). If you are going to turn compression ON by default, without the user having to make any decisions for their particular environment, then part of the decision for the default config has to be 'Which MIME types?' text/plain and/or text/html only? SOME browsers can 'Accept-Encoding' on the ever-increasing .js Javascript backloads but some CANNOT. These 2 issues alone are probably enough to justify keeping compression OFF by default. A lot of people that use Apache won't even be able to get their heads around either one of these 'issues' and they really SHOULD do a little homework before turning it ON. Someone already quoted that... 'people expect the default config to just WORK without major issues'. That's exactly what you have now. It's not 'broken'. Why change it? Kevin Kiley [snip] -Original Message- From: Greg Stein gst...@gmail.com To: dev@httpd.apache.org Sent: Tue, Jun 1, 2010 7:40 am Subject: Re: Fast by default Geez, Eric. No wonder people don't want to contribute to httpd, when they run into an attitude like yours. That dismissiveness makes me embarressed for our community. There is zero reason for us to avoid putting deflate into the default configuration. It is also very arguable that we should leave it off. I think others have argued well to enable it by default, while you've simply dismissed them with your holier-than-thou attitude and lack of any solid rationale. -g On May 31, 2010 8:06 PM, Eric Covener cove...@gmail.com wrote: On Mon, May 31, 2010 at 8:30 PM, Bryan McQuade bmcqu...@google.com wrote: I propose providing an... An additional httpd.conf doesn't sound valuable to me. What slice of non-savvy users would scrutinize an alternate config file, can replace the config file of their webserver, isn't using a webserver packaged by their OS, and wouldn't have just gotten the same information today from the manual and 400,000 other websites? There's currently no ifModule bloat in the default conf, but you're welcome to submit a patch that adds one for deflate or expires (latter seems more unwise to me). See the supplemental configuration section of the generated config. This doesn't address mass-vhost companies failing to allow deflate because it's not in the no-args HTTPD ./configure , which sounds far-fetched to me. I can't recall a users@ or #httpd user implying being subjected to such a thing with their own build or with cheap hosting. -- Eric Covener cove...@gmail.com You seem to think large corporations are competent at web design/administration. -- Sent
Re: Fast by default
Let me preface ALL the remarks below with TWO statements... 1. I haven't done any research on these HTTP based Client/Server compression topics in quite some time. It is all, essentially, 'ancient history' for me but it still amazes me that some of the issues are, so many years later, still being 'guessed about' and no one has all the answers. 2. Let's not lose the TOPIC of this thread. The thread is about whether or not it's time to just turn mod_deflate ON by default in the 'safe' httpd.conf that ships with Apache. Regardless of disagreement on some of the internals and the remaining 'questions' I think it's clear to some now that this is NOT just an 'easy decision'. It's complicated. It WILL cause 'problems' for some installations and some environments. Bryan McQuade wrote... Kevin Kiley wrote... Don't forget the ongoing issue that if you ONLY vary on 'Accept-Encoding' then almost ALL browsers will then refuse to cache a response entity LOCALLY and the pain factor moves directly to the Proxy/Content Server(s). I don't think this is true for browsers in use today. Well, it's certainly true of the MSIE 6 I have 'in use today' on almost all of my Windows XP Virtual Machines that I use for testing. Also 'I don't think this is true' is certainly not 'I'm SURE this is not true'. Firefox will certainly cache responses with Vary: Accept-Encoding. I haven't done any testing with Firefox. Firefox wasn't even around when this first became an issue years ago. I'll take your word for it unless/until you can provide some Firefox specific test results page(s)that prove this to be true. The Mozilla/Firefox family has ALWAYS had a 'different' approach to how the client-end decompression gets done. That browser lineage chose to always use the local 'cache' as the place where the decompression takes place. That's why, when you use those browsers and you receive a compressed entity, you always get TWO cache files. One is simply the browser opening up a cache file to store the COMPRESSED version of the entity and the other is the DECOMPRESSED version. This is true even if the response is labeled 'Cache-control: private'. It will STILL 'cache' the response and ignore all 'Cache-Control:' directives but that's another 'bug' story altogether. They are simply doing all the decompression 'on disk' and using plain old file-based GUNZIP to get the job done so there HAS to be a 'cached copy' of the response regardless of any 'Cache-Control:' directives. It will also keep BOTH copies of the file around. The MSIE browser line will also use a cache-file ( sometimes ) for the decompression but, unlike the Mozilla lineage, MSIE will DELETE the initial compressed copy to avoid confusion. There used to be this weird-ass bug with Mozilla that would only show up if you tried to PRINT a decompressed page. The browser would forget that it had TWO disk files representing the compressed/uncompressed response and it would accidentally try to PRINT the COMPRESSED version. I certainly hope the Firefox branch of this source code line worked that little bug out. Eric Lawrence of the Internet Explorer team has a nice blog post that explains that in IE6 and later, Vary: Accept-Encoding are cached: http://blogs.msdn.com/b/ieinternals/archive/2009/06/17/vary-header-prevents-caching-in-ie.aspx. The 'EricLaw' MSDN Blog link you mention only says this about MSIE 6... [snip] Internet Explorer 6 Internet Explorer 6 will treat a response with a Vary header as completely uncacheable, unless the Vary header contains only the token User-Agent or Accept-Encoding. Hence, a subsequent request will be made unconditionally, resulting in a full re-delivery of the unchanged response. This results in a significant performance problem when Internet Explorer 6 encounters Vary headers. [/snip] This does NOT match my own research with MSIE 6 so the guy must be talking about some very specific BUILD version or 'hotpatched' version of MSIE 6. In case you missed it... here is the link to one of the discussions about this on the Apache mod_gzip forum which contains complete test results obtained using a kernel debugger with MSIE 4, 5 and 6... http://marc.info/?l=apache-modgzipm=103958533520502w=2 Eric also fails to mention that if you include MULTIPLE Vary: headers and/or multiple conditionals on the same 'Vary:' line ( as RFC 2616 says you are supposed to be able to do ) then MSIE 6 stops caching and treats those inbounds as the infamous 'Vary: *'. ( Vary: STAR ) I believe that last part is STILL TRUE even to this day which means it is STILL 'Non-RFC compliant'. This 'other issue' was also covered in my own MSIE 6 testing at the links above. Other variants of Vary do prevent caching in IE but Vary: Accept-Encoding is safe. According to EricLaw, yes... but see links and test results above. That is NOT what my own MSIE 6 testing showed. My testing on MSIE 6 only showed that ANY presence of ANY 'Vary:' header OTHER
Re: Fast by default
All, I was once offered money to provide a high-performance Apache configuration file for a website. When I pointed out that I would need to come in, analyze their app and its performance, and then iteratively tune the config accordingly, I was given to understand that this was not necessary. Just send us the config, please. They were highly miffed when I didn't lay that particular flavor of golden egg for them. I actually got fired from that gig. On Jun 1, 2010, at 5:50 AM, Plüm, Rüdiger, VF-Group wrote: And others have argued well to leave it off by default. My personal opinion is that we should leave it disabled by default for the reasons (CPU, Proxies, Browser behaviour, ETAG problem) mentioned by others. I thought it isn't in the default build because it requires an external library that may not be on the system. If this is not of concern, we might as well turn it on in the default build. Same for mod_ssl. Generally, I think we see the build and runtime configuration as a starting point from which to create your own environment, not a canonical default that is right for all (or even most) users. Distributors (Red Hat et. al.) should (and do) build httpd according to the capabilities of their environment. If that environment includes libz and openssl, no reason why packagers shouldn't build those modules. Including those features is good for their customers. As others have pointed out, a lot of performance tuning is highly site and situation specific. Once again the default configuration file cannot be expected to cover all possible situations. Deflate, caching, load balancing, proxying, content segregation, etc. can only be optimally configured only in the context of the individual deployment. If there were a silver bullet to making the web server fast, don't you think we would have fired it some time in the past 15 years? There is no such thing. The only way to get a handle on it is to read the books, figure it out, or hire a consultant. But don't expect him to lay any golden performance eggs. S. -- Sander Temme scte...@apache.org PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF
Re: Fast by default
On Tue, Jun 1, 2010 at 9:04 AM, William A. Rowe Jr. wr...@rowe-clan.net wrote: [...] Plus deflate may provide no benefit, and degrade performance, if the CPU utilization is a greater concern than bandwidth utilization. The CPU utilization is an interesting topic for me because I've been working on a related type of transform (minify of dynamic content) and I've gotten into the habit of measuring nanoseconds of processing time per byte of content. With that in mind, I just added in some instrumentation around the zlib calls in mod_deflate and found that the compression takes 30-60 ns/byte for buffers of a few KB. (This is with Apache 2.2.14 and zlib 1.2.3 on a 2.8GHz x64 processor.) To put that number in perspective, if you were to devote the equivalent of one CPU core worth of processing power on a web server host to compression, 30ns/byte means the host would be able to do real-time compression of ~266Mb/s of outbound traffic. Assume that your monthly bandwidth pricing is $10 per 95th percentile peak Mbps. Assume further that by turning on deflate you can reduce outbound bandwidth usage by 75% (i.e., you're getting 4:1 compression). Thus the CPU core that you've devoted completely to deflate processing will save you ~$2000 per month in bandwidth pricing. If the CPU core costs less than $24000 per year (amortized capital cost plus power, cooling, support, data center space, marginal cost of additional sysadmins needed to support each additional server, etc, etc), then you still come out ahead by turning on deflate. A few additional thoughts: 1. Speeding up the deflate implementation would be an unqualified win. Supposedly the recently-released zlib 1.2.5 is faster, but I don't have any data on it. 2. The best practice I've found for implementing compression in a web server is to do the compression in the server closest to the network edge. E.g., if you have a web server fronted by a proxy cache, do the compression dynamically in the proxy cache rather than the web server. That way the cache doesn't have to store multiple variants of each object. Similarly, if you're using a CDN for content that can benefit from gzip, ask your CDN if they can do conditional compression for non-problematic user-agents on-the-fly. -Brian
Re: Fast by default
Thanks to Covener for pointing out this thread to me. My first thought when reading it wasn't exactly what I then found. When thinking about ``fast by default'', I'm thinking about httpd making smart decisions for MPM default values, based on the number of CPU cores and the amount of memory available. The idea isn't exactly new: IIS and most JEE servers for instance, spawn $number-of-CPU-cores threads for garbage collection. My two cents. -- Igor Galić Tel: +43 (0) 699 122 96 338 Fax: +43(0) 1 91 333 41 Mail: i.ga...@brainsware.org URL: http://brainsware.org/
Re: Fast by default
On Sun, May 30, 2010 at 9:22 AM, Eric Covener cove...@gmail.com wrote: On Sun, May 30, 2010 at 1:40 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Wouldn't you agree that deflate has all reasons to be on a default configuration? Again, I don't agree. I think it should be configured. There are two parts to the problem - enabling gzip module by default in configure so when people compile Apache, they have it available - enabling the module in Apache config and configuring gzip for specific content I think having gzip available is a must, otherwise people will have to go through compiling it in which is almost impossible for most of the people. I checked and it seems that http://httpd.apache.org/ is using gzip compression - why you think other people (with less control over their servers) shouldn't get it by default?! Our infrastructure has configured it, just like everyone else can. First of all they had to compile it in and in case of Apache Foundation, I have no doubts they new what they were doing and has no hesitation in that regard. In case of a regular internet provider or enterprise IT or Linux distribution packager, I think this is very different and they have hard time understanding this and I believe it's important for a team maintaining most popular web server in the world to make such decisions for them as you did with other modules that are compiled in by default. If you're forbidden from enabling mod_deflate, it's probably for a good reason (CPU, support) Let me address these two separately - first, CPU. I think CPU whatever it is more important then extra time spent for apache child uploading the data over the network should be really tested. I'll be happy to ask people in the industry to provide their data whatever CPU limit was more important then slowed download times. I know that many people who can afford it, use SSL and GZIP-enabling (and even caching) reverse proxies in front of their Apache farms and some of them even use lightweight servers like nginx or lighttpd. But we're not talking about teams that can configure sophisticated server setups, but about regular infrastructure teams. Now, the second part - Support: I think browser support is very hight for gzip and in any case, we're not enabling gzipping blindly - mod_deflate honors Accept-encoding headers and sends data only in those cases. I'm well aware of bugs in some older browsers and it should be taken into account when default rules are created for ignoring some known bad behavior. The biggest point here is that leaving people to figure those details out is dangerous and there is no better place to provide support to people is in default configuration - documentation already lists some edge cases and I think people would benefit from such configuration: http://httpd.apache.org/docs/2.2/mod/mod_deflate.html#recommended I have a feeling that default configuration can be quite simple - all is needed is to identify most common content types that are compressible and white-list them for compression (instead of not very reliable everything but images rule). Second step is to identify those old browsers that have broken support for gzip (while advertising it through Accept-encoding) and disable them. Same is true for Expires as well: - Yahoo: http://developer.yahoo.com/performance/rules.html#expires Smartly considering your content expiration is wise, having the default conf guess is unwise. Agree completely - it was wrong for me to start the conversation with directives that I have in drop-in htaccess because those require other things to be done on application layer as I mentioned. I think, again, we should separate two issues - configuration and availability of the module. People who don't have access to recompiling Apache or enabling the modules don't have ability to configure expiration right now. And all I'm asking is to give them ability to configure their sites. Exact configuration is probably a question of their apps, although some best practices are available there as well. Here's my article explaining why expiration is solving caching problem people are having and helps load static content once without re-requesting it from the server waisting connections for pointless 304s: Setting infinite Expires headers is a great solution to the problem I know what the headers do, I disagree that httpd should send them with arbitrary values by default. There are numerous books written about performance and all of them talk about these two aspects in one way or another. That should help people make smart decisions, but these are not exactly obscure topics. Not sure I got your point about the obscurity of the topics - my point was that it's not just my whim to have gzip and expiration, but an industry practice instead and it's strange that multitude of books are being written to help people understand, but software doesn't do the job. I don't
Re: Fast by default
In case of a regular internet provider or enterprise IT or Linux distribution packager, I think this is very different and they have hard time understanding this and I believe it's important for a team maintaining most popular web server in the world to make such decisions for them as you did with other modules that are compiled in by default. nonsense -- Eric Covener cove...@gmail.com
Re: Fast by default
Hi All I would like to offer some information just to help decision making in this area; 1. Host virtualisation often means you do not get what you think you are getting in terms resources; 2. Not all proxy servers or server versions support compression perfectly, older versions of Squid, for an example; 3. Most of my users are on computers with not that much power; 4. Compression makes it more difficult to trouble-shoot. I have a large user base, and have had experience recently of having to trouble-shoot IEs with the same configuration but with success/failure of accessing a site with compressed content. I did not enjoy that. Just my 2c. Regards, Jie * Igor Gali?? i.ga...@brainsware.org wrote: Date: Mon, 31 May 2010 11:29:17 + (UTC) From: Igor Gali?? i.ga...@brainsware.org To: dev@httpd.apache.org Subject: Re: Fast by default X-Mailer: Zimbra 6.0.5_GA_2213.DEBIAN5_64 (ZimbraWebClient - FF3.0 ([unknown])/6.0.5_GA_2213.DEBIAN5_64) Thanks to Covener for pointing out this thread to me. My first thought when reading it wasn't exactly what I then found. When thinking about ``fast by default'', I'm thinking about httpd making smart decisions for MPM default values, based on the number of CPU cores and the amount of memory available. The idea isn't exactly new: IIS and most JEE servers for instance, spawn $number-of-CPU-cores threads for garbage collection. My two cents. -- Igor Gali?? Tel: +43 (0) 699 122 96 338 Fax: +43(0) 1 91 333 41 Mail: i.ga...@brainsware.org URL: http://brainsware.org/
Re: Fast by default
I had a conversation with a well known hosting provider recently and they told me they use the default Apache configuration for their shared hosting service. When I asked if they provide gzip as an option for their users, they said no, since it was not enabled by default. When I explained to them that enabling gzip has significant benefits for end users they were very interested in turning on gzip. This company just used the default Apache config, assuming that it was reasonably well tuned by default. You can claim that they're making bad, uninformed decisions, or whatever you want to, but the fact remains: some Apache users assume that the default config is a reasonably good config, and use it as-is. There are two classes of users out there: power users that want to tune every knob manually, and typical users that just want Apache to work out of the box. Apache developers on this list are part of the former group, which I assume is why the default Apache config is very simple. It's designed by power users, for power users. But there are non-power users out there. It would be nice if the Apache community provided configuration hints for these users. In 2010, IMO there is no good reason to have gzip disabled by default. Almost all websites enable it. There are a handful of prominent websites that do not. I've had conversations with a few of these sites. Most of them have not turned it on because they don't understand what it does, not because they don't have enough CPU. gzip has been used on the web now for well over 10 years. Only *very* old browsers, proxies, etc don't have perfect support for gzip. I can respect that the default httpd.conf is designed to be simple and minimalist. But it would be helpful to have an additional example configuration in svn trunk and as part of Apache releases, that enable things like mod_deflate. The current comments in httpd.conf explain that there are additional directives and you have been warned but IMO this is not very helpful or specific. We can do better. I propose providing an additional httpd.conf in the svn trunk and as part of future Apache releases that enables modules and directives that are commonly recommended on Apache performance tuning websites. This includes mod_deflate, mod_expires, etc. This will allow power users to continue to start with the current httpd.conf while typical users can just use the well optimized configuration. Hopefully this suggestion isn't too controversial. If there are concerns about some of the specific directives suggested in this thread, I'm sure we can work those out through discussion. Can we agree that it would be useful to provide an additional configuration file for non-power users that enables commonly recommended modules by default? On Mon, May 31, 2010 at 5:24 PM, Eric Covener cove...@gmail.com wrote: In case of a regular internet provider or enterprise IT or Linux distribution packager, I think this is very different and they have hard time understanding this and I believe it's important for a team maintaining most popular web server in the world to make such decisions for them as you did with other modules that are compiled in by default. nonsense -- Eric Covener cove...@gmail.com
Re: Fast by default
On Mon, May 31, 2010 at 8:30 PM, Bryan McQuade bmcqu...@google.com wrote: I propose providing an additional httpd.conf in the svn trunk and as part of future Apache releases that enables modules and directives that are commonly recommended on Apache performance tuning websites. This includes mod_deflate, mod_expires, etc. This will allow power users to continue to start with the current httpd.conf while typical users can just use the well optimized configuration. An additional httpd.conf doesn't sound valuable to me. What slice of non-savvy users would scrutinize an alternate config file, can replace the config file of their webserver, isn't using a webserver packaged by their OS, and wouldn't have just gotten the same information today from the manual and 400,000 other websites? There's currently no ifModule bloat in the default conf, but you're welcome to submit a patch that adds one for deflate or expires (latter seems more unwise to me). See the supplemental configuration section of the generated config. This doesn't address mass-vhost companies failing to allow deflate because it's not in the no-args HTTPD ./configure , which sounds far-fetched to me. I can't recall a users@ or #httpd user implying being subjected to such a thing with their own build or with cheap hosting. -- Eric Covener cove...@gmail.com
Re: Fast by default
Hi Bryan, On Tue, Jun 1, 2010 at 2:30 AM, Bryan McQuade bmcqu...@google.com wrote: I had a conversation with a well known hosting provider recently and they told me they use the default Apache configuration for their shared hosting service. When I asked if they provide gzip as an option for their users, they said no, since it was not enabled by default. When I explained to them that enabling gzip has significant benefits for end users they were very interested in turning on gzip. This company just used the default Apache config, assuming that it was reasonably well tuned by default. You can claim that they're making bad, uninformed decisions, or whatever you want to, but the fact remains: some Apache users assume that the default config is a reasonably good config, and use it as-is. I already got that kind of discussion with either hosting providers or people in the hosting team of some big websites. The answer is the same as you described. They just install the Apache package for their distribution and that's it, they do not even think about tuning httpd.conf a bit because they think it just works fine by default. There are two classes of users out there: power users that want to tune every knob manually, and typical users that just want Apache to work out of the box. Apache developers on this list are part of the former group, which I assume is why the default Apache config is very simple. It's designed by power users, for power users. But there are non-power users out there. It would be nice if the Apache community provided configuration hints for these users. In 2010, IMO there is no good reason to have gzip disabled by default. Almost all websites enable it. There are a handful of prominent websites that do not. I've had conversations with a few of these sites. Most of them have not turned it on because they don't understand what it does, not because they don't have enough CPU. gzip has been used on the web now for well over 10 years. Only *very* old browsers, proxies, etc don't have perfect support for gzip. I remember having a lot of troubles with some combinations of Squid + IE (6 or so) where the compressed content was just never gunziped. Was I the only one to get such problems ? I can respect that the default httpd.conf is designed to be simple and minimalist. But it would be helpful to have an additional example configuration in svn trunk and as part of Apache releases, that enable things like mod_deflate. The current comments in httpd.conf explain that there are additional directives and you have been warned but IMO this is not very helpful or specific. We can do better. I propose providing an additional httpd.conf in the svn trunk and as part of future Apache releases that enables modules and directives that are commonly recommended on Apache performance tuning websites. This includes mod_deflate, mod_expires, etc. This will allow power users to continue to start with the current httpd.conf while typical users can just use the well optimized configuration. I love the idea. A httpd.conf-RECOMMENDED file would be really helpful to a lot of users. Another idea would be to provide a script that analyzes the current httpd.conf and could propose suggestions on how to improve it like The value for directive XXX is too high/low you should consider doing bla bla bla bla. Since providing auto generated solutions can be bad sometimes such a script should be link to a documentation page explaining advantages and drawbacks of each advice so people can make decision that is best for them. What do you think ? Best Regards -- Jérôme
Re: Fast by default
On Sun, May 30, 2010 at 1:40 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Wouldn't you agree that deflate has all reasons to be on a default configuration? Again, I don't agree. I think it should be configured. I checked and it seems that http://httpd.apache.org/ is using gzip compression - why you think other people (with less control over their servers) shouldn't get it by default?! Our infrastructure has configured it, just like everyone else can. If you're forbidden from enabling mod_deflate, it's probably for a good reason (CPU, support) Same is true for Expires as well: - Yahoo: http://developer.yahoo.com/performance/rules.html#expires Smartly considering your content expiration is wise, having the default conf guess is unwise. Here's my article explaining why expiration is solving caching problem people are having and helps load static content once without re-requesting it from the server waisting connections for pointless 304s: Setting infinite Expires headers is a great solution to the problem I know what the headers do, I disagree that httpd should send them with arbitrary values by default. There are numerous books written about performance and all of them talk about these two aspects in one way or another. That should help people make smart decisions, but these are not exactly obscure topics. BTW, I always build Apaches from the source and before writing to the list, I downloaded the latest versions and checked - none of the three modules I mentioned are enabled by default in the default build configuration file. If I'm wrong, just let me know and I'll go and celebrate ;) No clue here about starting with a default build configuration file -- maybe you're starting with httpd.spec. When you ask for deflate or expires and build from source, they end up loaded in the generated config. -- Eric Covener cove...@gmail.com
Re: Fast by default
-- Forwarded message -- From: Eric Covener cove...@gmail.com To: dev@httpd.apache.org Date: Fri, 28 May 2010 07:31:46 -0400 Subject: Re: Fast by default On Fri, May 28, 2010 at 12:08 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: I don't think anything in the htaccess (compression, arbitrary guesses at expiration, per-directory rewrite rules) in your site is reasonable for the default configuration. Note that expires and deflate are loaded in the default conf if you build them, although not necessarily when third parties distribute a default configuration. Eric, thanks for your reply. I think my htaccess definitely does not represent ideal configuration and that's why I'm trying to bring it up here so together we can come up with a good solution to help people have faster sites. Wouldn't you agree that deflate has all reasons to be on a default configuration? it saves people lots of traffic and is among first in every best practices list out there: - Yahoo: http://developer.yahoo.com/performance/rules.html#gzip - Google: http://code.google.com/speed/page-speed/docs/payload.html#GzipCompression Here's the article about the compression and how it affects the performance: http://googlecode.blogspot.com/2009/11/use-compression-to-make-web-faster.html I checked and it seems that http://httpd.apache.org/ is using gzip compression - why you think other people (with less control over their servers) shouldn't get it by default?! Same is true for Expires as well: - Yahoo: http://developer.yahoo.com/performance/rules.html#expires - Google: http://code.google.com/speed/page-speed/docs/caching.html#LeverageBrowserCaching Here's my article explaining why expiration is solving caching problem people are having and helps load static content once without re-requesting it from the server waisting connections for pointless 304s: http://www.sergeychernyshev.com/blog/caching-problem-no-more-svn-assets/ There are numerous books written about performance and all of them talk about these two aspects in one way or another. BTW, I always build Apaches from the source and before writing to the list, I downloaded the latest versions and checked - none of the three modules I mentioned are enabled by default in the default build configuration file. If I'm wrong, just let me know and I'll go and celebrate ;) -- Eric Covener cove...@gmail.com -- Forwarded message -- From: Tom Evans tevans...@googlemail.com To: dev@httpd.apache.org Date: Fri, 28 May 2010 12:57:12 +0100 Subject: Re: Fast by default On Fri, May 28, 2010 at 12:31 PM, Eric Covener cove...@gmail.com wrote: On Fri, May 28, 2010 at 12:08 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: I thought 'Fast by default' and '.htaccess' could only be used in the same sentence like this: 'Your site will never be fast by default if you litter it with .htaccess'. I absolutely agree that .htaccess is not the right way to go and affects backend performance, let's replace that with functionality in the core - people are forced to use .htaccess and create things like drop-in .htaccess file because core does not support fundamental functionality by default! The problem with caching is that whole world is downloading content over and over again while it didn't change - useful download rate is terrible. And the problem right now is that people have very little control over caching process, they don't know the lifetime of their assets and so on. There is a lot to do in application to manage the assets, but without having ability to control expiration, there is not much people can do - php-based handlers that output cache control and expiration headers are definitely not a good replacement for a core web server functionality. Let's figure out how to help people with their Apaches and not make fun of each other. Thank you, Sergey Cheers Tom
Re: Fast by default
On Fri, May 28, 2010 at 12:08 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Hi, I'm working on helping people to have their sites fast by default and was wondering if Apache HTTPD team can consider enabling a few modules by default and maybe adding a few directive to default httpd.conf file that comes with distribution? I did have some preliminary look at this problem when working on drop-in .htaccess file for performance, but then realized that it's probably better to fix it at the source. I don't think anything in the htaccess (compression, arbitrary guesses at expiration, per-directory rewrite rules) in your site is reasonable for the default configuration. Not that expires and deflate are loaded in the default conf if you build them, although not necessarily when third parties distribute a default configuration. -- Eric Covener cove...@gmail.com
Re: Fast by default
On Fri, May 28, 2010 at 12:31 PM, Eric Covener cove...@gmail.com wrote: On Fri, May 28, 2010 at 12:08 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Hi, I'm working on helping people to have their sites fast by default and was wondering if Apache HTTPD team can consider enabling a few modules by default and maybe adding a few directive to default httpd.conf file that comes with distribution? I did have some preliminary look at this problem when working on drop-in .htaccess file for performance, but then realized that it's probably better to fix it at the source. I don't think anything in the htaccess (compression, arbitrary guesses at expiration, per-directory rewrite rules) in your site is reasonable for the default configuration. Not that expires and deflate are loaded in the default conf if you build them, although not necessarily when third parties distribute a default configuration. -- Eric Covener cove...@gmail.com I thought 'Fast by default' and '.htaccess' could only be used in the same sentence like this: 'Your site will never be fast by default if you litter it with .htaccess'. Cheers Tom
Re: Fast by default
On 28.05.2010 13:31, Eric Covener wrote: On Fri, May 28, 2010 at 12:08 AM, Sergey Chernyshev sergey.chernys...@gmail.com wrote: Hi, I'm working on helping people to have their sites fast by default and was wondering if Apache HTTPD team can consider enabling a few modules by default and maybe adding a few directive to default httpd.conf file that comes with distribution? I did have some preliminary look at this problem when working on drop-in .htaccess file for performance, but then realized that it's probably better to fix it at the source. I don't think anything in the htaccess (compression, arbitrary guesses at expiration, per-directory rewrite rules) in your site is reasonable for the default configuration. Not that expires and deflate are loaded in the default conf if you build them, although not necessarily when third parties distribute a default configuration. Important typo: Not that - Note that. Regards, Rainer
Re: Fast by default
On Fri, May 28, 2010 at 8:24 AM, Rainer Jung rainer.j...@kippdata.de wrote: Not that expires and deflate are loaded in the default conf if you build them, although not necessarily when third parties distribute a default configuration. Important typo: Not that - Note that. whoops, thanks for catching that -- Eric Covener cove...@gmail.com
