Do you like the CLI standards?
January 6, 2005 quote Snip I am a member of ECMA, an international standards body, where I am chair of the committee responsible for standardizing the CLI (Common Language Infrastructure) and C#. Snip So here is my initial question to ponder: What is your feeling about the standardization of the CLI within ECMA and ISO? Snip What are your thoughts here? Is standardization a good thing? Bad thing? Doesn't matter? We are currently in the midst of developing edition 3 of the CLI specification and hopefully should be complete soon. /quote Joel Marcey http://spaces.msn.com/members/jimarcey/ Jeff
Re: svn commit: r123867 - /httpd/mod_aspdotnet/trunk/Apache.Web/WorkerRequest.h
From: [EMAIL PROTECTED]
Handle GetProtocol correctly
(and consistant with ASP.NET conventions.)
A very, very good idea! :)
virtual String* GetProtocol(void)
quote
When overridden in a derived
class, returns the HTTP protocol
(HTTP or HTTPS).
/quote
.NET Framework Class Library
HttpWorkerRequest.GetProtocol Method
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemWebHttpWorkerRequestClassGetProtocolTopic.asp
{
-// TODO: HTTPS when appropriate
+#if MODULE_MAGIC_NUMBER_MAJOR = 20050101
+// Fixes in httpd-2.1-dev
+const char __nogc *method = ap_http_scheme(rr);
+#else
+const char __nogc *method = ap_http_method(rr);
+#endif
Seems to me, that some other developers
are the ones who cannot make up their
minds, as to just what all this really
means. :)
+String *scheme(method);
Really confusing me now, and
using only three words. :)
+
+// For some silly reason, the ASP.NET team thought that
schemes
+// are upper case entities;
I disagree with your reading of
most of the .NET documentation.
Protocol when used in most tech
papers today, is usually always
upper cased.
quote
In information technology,
a protocol (pronounced PROH-tuh-cahl,
from the Greek protocollon,
which was a leaf of paper
glued to a manuscript volume,
describing its contents) is
the special set of rules that
end points in a telecommunication
connection use when they communicate.
/quote
protocol
http://searchnetworking.techtarget.com/sDefinition/0,,sid7_gci212839,00.html
Notice in the above link
that the usage for all
named protocols
is always upper cased.
Also see the
Protocol Directory
Index of WAN, LAN, and
ATM Protocols
http://www.protocols.com/pbook/index.htm
+scheme = scheme-ToUpper(nullCulture);
#ifdef _DEBUG
-LogRequestError(LGetProtocol: returns HTTP,
APLOG_DEBUG, 0);
+String *schemeMsg = String::Concat(LGetProtocol:
returns , scheme);
+LogRequestError(schemeMsg, APLOG_DEBUG, 0);
#endif
-return new String(LHTTP);
+return scheme;
}
quote
A string containing the
scheme of the specified URI.
/quote
.NET Framework Class Library
Uri.Scheme Property
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemUriClassSchemeTopic.asp
Now please try the sample
included with the above
link (about URI/Scheme)
but use all uppercase chars.
For example:
newUri(http://www
change to
new Uri(HTTP://www
What do you get back?
Notice the below:
quote
Provides an object representation
of a uniform resource identifier
(URI) and easy access to the parts
of the URI.
SNIP
To put the URI in canonical form,
the Uri constructor performs the
following steps.
Converts the URI scheme to lower case.
Converts the host name to lower case.
/quote
.NET Framework Class Library
Uri Class
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemUriClassTopic.asp
I believe your source code comment
quote
+
+// For some silly reason, the ASP.NET team thought that
schemes
+// are upper case entities;
/quote
should be removed.
If you disagree with me, please
provide some sort of other
information (.NET docs, links,
and etc.) with your reply.
Jeff
Re: Working on some load balancing methods
Jim Jagielski wrote: I'm currently working on code that extended the lb method within the 2.1/2.2 proxy from what is basically a weighted request count to also be a weighted traffic count (as measured by bytes transferred) and a weighted load count (as measured by response time). Sure, the general idea was to allow different lb methods. I've started to collect the transferred for various protocols to be able to do a traffic balancing. You have that params already present in the shared memory, so making sure we collect the real values transferred is a first step thought. I'm also working on extending the load balancer to a domain clustering model for grouping cluster nodes in groups to lower the session replication transfer for backends that offer the session replication. The former is further along and the methods will be selectable at runtime... This is definitely a scratch I'm itching, but before I spend too much (additional) time on it, I'd like some feedback on whether the concept is one we can all get behind. +1. I am also toying with the idea of supporting a CPU load method when the origin servers are Apache via a custom response header... Well, that'll be awesome to achieve, not only to dynamically update the balancer node member, but to safely remove it from the cluster in a two phase process if the backend 'decides' so. Right now this is possible using 'balancer-handler', so the dynamic logic is there already. The header processing should also allow to discover the topology dynamically (there is already PROXY_DYNAMIC_BALANCER_LIMIT meant to be used for that). I'm not sure if this will be possible on all protocols but we have a long-standing AJP13 extension proposal that if implemented will allow that kind of operation. http://jakarta.apache.org/tomcat/connectors-doc/common/ajpv13ext.html Regards, Mladen.
Re: Dumb APR_BUCKET_BUFF_SIZE question
On Fri, 7 Jan 2005, Rasmus Lerdorf wrote: Why is it hardcoded to be 8000? It would seem like you could easily be unlucky and just miss the cutoff and end up with a 6000 byte heap bucket followed by a 3000 byte transient bucket, for example, as a result of 3 3000 byte ap_rwrites. For that particular case it might be quite beneficial to increase APR_BUCKET_BUFF_SIZE to 9000 which would suggest that it might be something that should be configurable. It was at some point I think :-) But at 8k it happens to fit on some intelish architectures in a common page of 8k (8192 bytes). I would not mind to see if much more configurable run-time -and- auto detecting compile time; we've got great improvements on some arch's with 16k pages -and- by fiddling with it so that it also happens to fit in a nice DMA multiple of the ethernetcard. Dw.
Re: Working on some load balancing methods
On Fri, 7 Jan 2005, Jim Jagielski wrote: I'm currently working on code that extended the lb method within the 2.1/2.2 proxy from what is basically a weighted request count to also be a weighted traffic count (as measured by bytes transferred) and a weighted load count (as measured by response time). The former is further along and the methods will be selectable at runtime... This is definitely a scratch I'm itching, but before I spend too much (additional) time on it, I'd like some feedback on whether the concept is one we can all get behind. We scratched this one at some point at Covalent as well - using the scoreboard and some mod_snmp stuff. I am also toying with the idea of supporting a CPU load method when the origin servers are Apache via a custom response header... What you'd ideally want is a pluggable 'metric' mechanism - which has two layers (this is more or less in line with what we dit at Covalent and what you can do inside some of the Cisco LD's): layer one: get_current_SRV(for some container) which returns a list of id, IP, weight, priority for a given service, uri subspace, etc. The result is a function of the callee. So this is not a de-facto statement server wide. This works much the same way as a true SRV rercod (as in DNS, as use by various services such as an active directory service, etc) - the priority allows for fail over and the weight allows for loadbalancing. and secondly a handoff_to(id) to confirm to the plugin that a certain backend is used for statistics, N-path, firewall/nat reprogramming, whatever. The second layer is also pluggable anbd simply returns state get_load(given backend, for some container) load: value between -1 (off-line) and 0..255. this is used by layer one to get the list on a case by case basis. On top of this you also want a 'ConfigBackEnd type [ opaquate values]' which carries the config info to the right plugin. And finally (and this is where I got stuck last time and used a horrible hack (private scoreboard) some neat way of expending the scoreboard. What I never did - but which you really do want ultimately is something which can snarf timing and data out of the returned header to detect 500 error's, time-to-server and other things, such as cookies - to record these and use in the next round as an indication of reliability. The reason why I never got this done properly is that in a lot of setups you do NOT want the frontend apache to properly process the data going back - but you want to route that TCP traffic around the server. Which is what that handoff() is for. But in those cases such return sniffing is of no use. As it never sees the data. The reason for the two layers is that - on layer one you can do three simple implementations: static: round robin on a list, fail over, etc DNS: based off SRV records in DNS real: based on a backend which uses layer 2 - on layer two you can do simple things; like just pinging - and augment as needed with things like SNMP later. - the reason you have container info (uri space, vhost, cookies, ssl session info) is to do all sort of stickyness when you feel like it. end of braindump and itch sratching. Dw
Re: Dumb APR_BUCKET_BUFF_SIZE question
On Fri, 7 Jan 2005, Rasmus Lerdorf wrote: I still think it would be worthwhile to make it configurable. Linux or FreeBSD5 on IA64 with 16k pages, for example, might show some decent gains by setting that to 15000. Or do a getpagesize() call on startup to determine it dynamically. It might be. We've considered having it be configurable before. There are just a lot of implications in changing the value; for example, it affects the memory footprint of the server, it affects how much data gets read in to memory per read() call on a file bucket (which might alter the decision of whether read() or mmap() is preferable, for example), it affects how well the data fits into memory pages, and it affects how much data is buffered in one block for the scenarios you described. There are probably others I'm not thinking of right now, too. Lots of different axes that must be considered when trying to pick an optimal value. But still, if you want to make it configurable, go right ahead. :)
Re: Dumb APR_BUCKET_BUFF_SIZE question
Cliff Woolley wrote: It might be. We've considered having it be configurable before. There are just a lot of implications in changing the value; for example, it affects the memory footprint of the server, it affects how much data gets read in to memory per read() call on a file bucket (which might alter the decision of whether read() or mmap() is preferable, for example), it affects how well the data fits into memory pages, and it affects how much data is buffered in one block for the scenarios you described. There are probably others I'm not thinking of right now, too. Lots of different axes that must be considered when trying to pick an optimal value. But still, if you want to make it configurable, go right ahead. :) Right, I saw all the things it affected which was precisely why I wondered why it wasn't made configurable. I can see perhaps protecting users from themselves, but other parts of Apache2 doesn't really follow that. The WindowSize config option in mod_deflate would be a very bad idea to change away from the default of 15, for example. Anyway, I will go play a bit with the bucket size on 64-bit boxes. -Rasmus
Re: Working on some load balancing methods
On Jan 7, 2005, at 4:14 PM, Sander Striker wrote: From: Jim Jagielski [mailto:[EMAIL PROTECTED] Sent: Friday, January 07, 2005 8:52 PM To: dev@httpd.apache.org Subject: Working on some load balancing methods I'm currently working on code that extended the lb method within the 2.1/2.2 proxy from what is basically a weighted request count to also be a weighted traffic count (as measured by bytes transferred) and a weighted load count (as measured by response time). The former is further along and the methods will be selectable at runtime... This is definitely a scratch I'm itching, I'm sure you are not the only one with that itch. You are welcome to harvest any plumbing you like from mod_backhand. It does exactly what you want only in apache 1.3.x but before I spend too much (additional) time on it, I'd like some feedback on whether the concept is one we can all get behind. FWIW, I like it. I am also toying with the idea of supporting a CPU load method when the origin servers are Apache via a custom response header... CPU load is tricky. It has observational bias. System load adjusts two slowly. The number of concurrent connections to each machine actually works pretty well as it suggests that the there are that many Apache children (adapt working as needed for Apache 2) working on the box and it lends itself to a current length of the run queue. All of these methods require total knowledge, otherwise you have contention issues and suffer from inaccuracies due to stale information. There are several nice randomized approaches that work well to smooth our spikes due to stale data. mod_backhand uses a simple stackable selection mechanism called candidacy functions that implement all this stuff (optionally as loadable modules) so they cleverness of the load balancing decisions can be decided later. // Theo Schlossnagle // Principal Engineer -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/ // Ecelerity: fastest MTA on Earth
Re: Working on some load balancing methods
Jim Jagielski wrote: I'm currently working on code that extended the lb method within the 2.1/2.2 proxy from what is basically a weighted request count to also be a weighted traffic count (as measured by bytes transferred) and a weighted load count (as measured by response time). The former is further along and the methods will be selectable at runtime... This is definitely a scratch I'm itching, but before I spend too much (additional) time on it, I'd like some feedback on whether the concept is one we can all get behind. I am also toying with the idea of supporting a CPU load method when the origin servers are Apache via a custom response header... Errr... mod_backhand? -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff
