Re: [naviserver-devel] compression
the version on the tip handles now identity and q-values.
The logic sketched below does not handle cases, where e.g.
identiy or * is
higher than gzip qvalue, or explicit forbidding of gzip. The
values are doubles,
so one has to be careful with comparisons. I am not sure
about the logic for
http/1.1, if we really should assume gzip by default.
-gustaf
On 27.10.12 15:03, Stephen Deasey wrote:
> On Sat, Oct 27, 2012 at 1:13 PM, Gustaf Neumann wrote:
>> On 26.10.12 15:47, Stephen Deasey wrote:
>>> I think the spec says that for HTTP/1.1, if the client doesn't
>>> explicitly say to NOT send the body gzipped, say by using a q value of
>>> 0 (which we don't actually check for, woops...), then the server can
>>> choose the encoding, although it should prefer the identity, unless
>>> that's unavailable. So we're being very pushy...
>> now we do check for the qvalues used in connection with
>> gzip. So, a client can now specify explicitly, that gzip is
>> NOT wanted via "gzip; q=0". What the code still does not do
>> is comparing the identity-qvalue with the gzip-qvalue or
>> combination with wildcards "*;q=..."
> Jeff had a go at this as well:
>
>https://bitbucket.org/jeffr/naviserver-queues/changesets
>
> Here's the feedback I tacked on to the end of a mercurial question via email:
>
>
>
> It bothers me a bit that a fresh Ns_Set has to be allocated, and also
> the parsing code is pretty gnarly and hard to verify. It looks about
> right, but I'd have to resort to pencil and paper to be more sure, and
> the fact I haven't done that reminds me that people tend to not look
> too closely at these things and that's where bugs can fester. I wonder
> if there's another way of doing this...
>
> You'll have to double check the details, but IIRC q values go from
> 0-999. If 'gzip' is present without a q value then it's as if it had
> q=1 -- that's the default. If it's not present, it's as if it were but
> with q=0. So, how about a function like Ns_HeaderQ(header, attr) which
> returns the integer q value for the specified atttribute. It simply
> uses strstr to find gzip. If it doesn't, return 0. If it does, then
> check for a q=. If it's the character '0', return 0. If there's no q,
> return 1. If it's something else, parse the int and return that. Then
> you can use it something like:
>
>
> if (!(connPtr->flags & NS_CONN_SENTHDRS)
> && !(connPtr->flags & NS_CONN_SKIPBODY)) {
>
> contentEncoding = Ns_SetIGet(Ns_ConnHeaders(conn), "Accept-Encoding");
>
> if (Ns_HeaderQ(contentEncoding, "gzip") > 0 ||
> Ns_HeaderQ(contentEncoding, "*") > 0) {
> gzip = 1;
>
>
> I'm not sure how eager the server should be sending gzipped content.
> An alternative to the above would be to keep the existing eager use of
> gzip, but respect the client's negative assertion:
>
>
> if (!(connPtr->flags & NS_CONN_SENTHDRS)
> && !(connPtr->flags & NS_CONN_SKIPBODY)) {
>
> contentEncoding = Ns_SetIGet(Ns_ConnHeaders(conn), "Accept-Encoding");
>
> if (connPtr->request->version >= 1.1) {
> if ((!Ns_HeaderQ(contentEncoding, "gzip", &q) || q > 0)
> && (!Ns_HeaderQ(contentEncoding, "*", &q) || q > 0)) {
> gzip = 1;
> }
> } else if ((Ns_HeaderQ(contentEncoding, "gzip", &q) && q > 0)
> || (Ns_HeaderQ(contentEncoding, "*", &q) && q > 0)) {
> gzip = 1;
> }
>
>
> Here Ns_HeaderQ is modified to return NS_TRUE or NS_FALSE depending on
> whether the attribute is present, and the q value is returned into the
> given variable. HTTP 1.0 clients have to explicitly ask for gzip, 1.1
> clients get it unless the say they don't want it.
>
> --
> WINDOWS 8 is here.
> Millions of people. Your app in 30 days.
> Visit The Windows 8 Center at Sourceforge for all your go to resources.
> http://windows8center.sourceforge.net/
> join-generation-app-and-make-money-coding-fast/
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
--
WINDOWS 8 is here.
Millions of people. Your app in 30 days.
Visit The Windows 8 Center at Sourceforge for all your go to resources.
http://windows8center.sourceforge.net/
join-generation-app-and-make-money-coding-fast/
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] compression
On Sat, Oct 27, 2012 at 1:13 PM, Gustaf Neumann wrote:
> On 26.10.12 15:47, Stephen Deasey wrote:
>> I think the spec says that for HTTP/1.1, if the client doesn't
>> explicitly say to NOT send the body gzipped, say by using a q value of
>> 0 (which we don't actually check for, woops...), then the server can
>> choose the encoding, although it should prefer the identity, unless
>> that's unavailable. So we're being very pushy...
>
> now we do check for the qvalues used in connection with
> gzip. So, a client can now specify explicitly, that gzip is
> NOT wanted via "gzip; q=0". What the code still does not do
> is comparing the identity-qvalue with the gzip-qvalue or
> combination with wildcards "*;q=..."
Jeff had a go at this as well:
https://bitbucket.org/jeffr/naviserver-queues/changesets
Here's the feedback I tacked on to the end of a mercurial question via email:
It bothers me a bit that a fresh Ns_Set has to be allocated, and also
the parsing code is pretty gnarly and hard to verify. It looks about
right, but I'd have to resort to pencil and paper to be more sure, and
the fact I haven't done that reminds me that people tend to not look
too closely at these things and that's where bugs can fester. I wonder
if there's another way of doing this...
You'll have to double check the details, but IIRC q values go from
0-999. If 'gzip' is present without a q value then it's as if it had
q=1 -- that's the default. If it's not present, it's as if it were but
with q=0. So, how about a function like Ns_HeaderQ(header, attr) which
returns the integer q value for the specified atttribute. It simply
uses strstr to find gzip. If it doesn't, return 0. If it does, then
check for a q=. If it's the character '0', return 0. If there's no q,
return 1. If it's something else, parse the int and return that. Then
you can use it something like:
if (!(connPtr->flags & NS_CONN_SENTHDRS)
&& !(connPtr->flags & NS_CONN_SKIPBODY)) {
contentEncoding = Ns_SetIGet(Ns_ConnHeaders(conn), "Accept-Encoding");
if (Ns_HeaderQ(contentEncoding, "gzip") > 0 ||
Ns_HeaderQ(contentEncoding, "*") > 0) {
gzip = 1;
I'm not sure how eager the server should be sending gzipped content.
An alternative to the above would be to keep the existing eager use of
gzip, but respect the client's negative assertion:
if (!(connPtr->flags & NS_CONN_SENTHDRS)
&& !(connPtr->flags & NS_CONN_SKIPBODY)) {
contentEncoding = Ns_SetIGet(Ns_ConnHeaders(conn), "Accept-Encoding");
if (connPtr->request->version >= 1.1) {
if ((!Ns_HeaderQ(contentEncoding, "gzip", &q) || q > 0)
&& (!Ns_HeaderQ(contentEncoding, "*", &q) || q > 0)) {
gzip = 1;
}
} else if ((Ns_HeaderQ(contentEncoding, "gzip", &q) && q > 0)
|| (Ns_HeaderQ(contentEncoding, "*", &q) && q > 0)) {
gzip = 1;
}
Here Ns_HeaderQ is modified to return NS_TRUE or NS_FALSE depending on
whether the attribute is present, and the q value is returned into the
given variable. HTTP 1.0 clients have to explicitly ask for gzip, 1.1
clients get it unless the say they don't want it.
--
WINDOWS 8 is here.
Millions of people. Your app in 30 days.
Visit The Windows 8 Center at Sourceforge for all your go to resources.
http://windows8center.sourceforge.net/
join-generation-app-and-make-money-coding-fast/
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] compression
On 26.10.12 15:47, Stephen Deasey wrote: > I think the spec says that for HTTP/1.1, if the client doesn't > explicitly say to NOT send the body gzipped, say by using a q value of > 0 (which we don't actually check for, woops...), then the server can > choose the encoding, although it should prefer the identity, unless > that's unavailable. So we're being very pushy... now we do check for the qvalues used in connection with gzip. So, a client can now specify explicitly, that gzip is NOT wanted via "gzip; q=0". What the code still does not do is comparing the identity-qvalue with the gzip-qvalue or combination with wildcards "*;q=..." >> Also on compression, a separate compression stream is pre-allocated and >> initialized for each pre-allocated Conn, whether or not compression is >> even enabled for the server. The causes a fairly large initial memory >> footprint. I think this pre-initialization could be made optional, and >> bypassed entirely when compression isn't enabled. Any thoughts? > Seems reasonable. the initialization is on the server level during startup for every connection structure, the actual need comes up in a connection thread. One could defer the Ns_CompressInit until a connection thread needs it (by checking, whether the compress stream is available/initialized. -gustaf neumann -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/ ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] compression
On Thu, Oct 25, 2012 at 9:20 PM, Jeff Rogers wrote: > It looks like we're enabling compression for all http/1.1 requests > regardless of whether it was specified in the request header, or even > specifically disallowed. This seems incorrect, but the code has been in > place for several years (connio.c:CheckCompress) ; is there a reason > not to change that? I think the spec says that for HTTP/1.1, if the client doesn't explicitly say to NOT send the body gzipped, say by using a q value of 0 (which we don't actually check for, woops...), then the server can choose the encoding, although it should prefer the identity, unless that's unavailable. So we're being very pushy... There's a bunch of tests for this in tests/ns_adp_compress.test, including with and without the Accept-Encoding header and q values, but many of them are disabled with -constraints knownBug. I guess it's something someone thought about but no one got round to fixing it. This page describes how to run these particular tests: http://wiki.tcl.tk/21659 > Also on compression, a separate compression stream is pre-allocated and > initialized for each pre-allocated Conn, whether or not compression is > even enabled for the server. The causes a fairly large initial memory > footprint. I think this pre-initialization could be made optional, and > bypassed entirely when compression isn't enabled. Any thoughts? Seems reasonable. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] compression
It looks like we're enabling compression for all http/1.1 requests regardless of whether it was specified in the request header, or even specifically disallowed. This seems incorrect, but the code has been in place for several years (connio.c:CheckCompress) ; is there a reason not to change that? Also on compression, a separate compression stream is pre-allocated and initialized for each pre-allocated Conn, whether or not compression is even enabled for the server. The causes a fairly large initial memory footprint. I think this pre-initialization could be made optional, and bypassed entirely when compression isn't enabled. Any thoughts? -J -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
