Sync to CVS problem
Title: Sync to CVS problem Hi, I have previously downloaded a copy of the Apache server code from the cvs repository using "anoncvs" login and CVSROOT=:pserver:[EMAIL PROTECTED]:/home/cvspublic. However, I am unable to connect to the server for the past couple of days. I get the following error: (Logging in to [EMAIL PROTECTED]) CVS password: cvs [login aborted]: recv() from server cvs.apache.org: EOF Also, if I use the other server (CVSROOT= :pserver:[EMAIL PROTECTED]:/cvs/apache) I can connect to the server. However I cannot export the code using the command "cvs export -d httpd-2.0 -r APACHE_2_0_44 httpd-2.0". I get the following error: cvs export: cannot open /root/.cvsignore: Permission denied cvs export: warning: cannot write to history file /cvs/apache/CVSROOT/history: Permission denied cvs export: Updating httpd-2.0 cvs export: failed to create lock directory for `/cvs/apache/httpd-2.0' (/cvs/apache/httpd-2.0/#cvs.lock): Permission denied cvs export: failed to obtain dir lock in repository `/cvs/apache/httpd-2.0' cvs [export aborted]: read lock failed - giving up Any ideas what may be wrong? Thanks Juan
ErrorDocument and ProxyPass rules
Title: ErrorDocument and ProxyPass rules I'm trying to figure out how you can configure ErrorDocument when using ProxyPass / http://webserver/ The problem is that if I use a URL like ErrorDocument 502 /error/502.html it will be proxied instead of picking the file locally. Any suggestions on how you can configure this? Juan
Scoreboard access rights on Windows
Title: Scoreboard access rights on Windows Does Apache set access rights in the scoreboard so that no other processes can write to it? I refer to named scoreboards using ScoreBoardFile. Juan
RE: Message: winnt_accept: Asynchronous AcceptEx failed.
Title: RE: Message: winnt_accept: Asynchronous AcceptEx failed. What are the circumstances for this to happen? -Original Message- From: Bill Stoddard [mailto:[EMAIL PROTECTED]] Sent: Sunday, July 27, 2003 10:30 PM To: [EMAIL PROTECTED] Subject: Re: Message: winnt_accept: Asynchronous AcceptEx failed. Alex Bereznyi wrote: Hi, We see a lot of WARNING messages in Windows Event Log: * * *winnt_accept**: Asynchronous AcceptEx failed. * The actual error is always 995: ERROR_OPERATION_ABORTED - The I/O operation has been aborted because of either a thread exit or an application request. It's coming from *server\mpm\winnt\child.c* The call: GetOverlappedResult((HANDLE)context-accept_socket, context-Overlapped, BytesRead, FALSE) GetLastError() is not called there, instead: closesocket(context-accept_socket); context-accept_socket = INVALID_SOCKET; It all seems like normal condition to me - why log WARNING ? Is it OK to turn it off? I'll turn this into an info message. I agree it should not be a warning. Bill
APR_BUCKET_BUFF_SIZE question
Title: APR_BUCKET_BUFF_SIZE question So currently Apache reads from the socket in APR_BUCKET_BUFF_SIZE chunks (8000 bytes by default) This value is a compiled value and can not be changed or configure at runtime. I was wondering if there is a particular reason not to make this configurable in some way. I was looking at the code and was thinking about a very simple way where we can read in larger or smaller chunks. I does not seems like a lot of code to change but I may be missing something. So the way I would deal with this is first: Make the apr_recv len parameter an input/output parameter instead of an output parameter so we can tell it how much data we would like to read. So if I set the value of len to something greater than 0 it would use that value. If len is zero or less it would use APR_BUCKET_BUFF_SIZE. In the code input filter, set the len value to readbytes or a configure value, or if not use APR_BUCKET_BUFF_SIZE. So we could have something like: ReadBufferSize 8000 Or ReadBufferSize 2000 Or ReadBufferSize Dynamic # This will use the readbytes value instead. That way a filter could ask for more or less bytes than the 8K. This seems awfully simple and I'm sure I'm missing something to why we don't do this already. Can somebody shed some light on this one? Juan
RE: APR_BUCKET_BUFF_SIZE question
Title: RE: APR_BUCKET_BUFF_SIZE question I figured it has to do something with the way pools work. Would it make a difference if the buffer size used is always a multiple of APR_BUCKET_BUFF_SIZE? Like 16, 32, 64? Do you know where the 8K number come from? Is it an empirical number or a theoretical one? -Original Message- From: Cliff Woolley [mailto:[EMAIL PROTECTED]] Sent: Wednesday, July 09, 2003 6:25 PM To: '[EMAIL PROTECTED]' Subject: Re: APR_BUCKET_BUFF_SIZE question On Wed, 9 Jul 2003, Juan Rivera wrote: That way a filter could ask for more or less bytes than the 8K. That's what apr_brigade_partition() is for. Does that not get you what you want? And anyway, having a runtime-configurable buffer size would make the bucket allocator WAY more complicated and less efficient... --Cliff
Pre shutdown hook
Has anybody consider providing a hook to be called right before shutting down the server? This is the scenario: A module creates a thread during initialization and then you want to close the thread during shutdown. I figured this could be done by registering a destroy function in a pool, but I'm not sure if this is the right way to do it (architecturally speaking). If there are initialization hooks why don't we have clean up hooks? Or are the pool destroy functions the designed way to do this? Juan
[PATCH] EventLog display name patch
Title: [PATCH] EventLog display name patch The current nt_eventlog implementation has the hardcoded Apache Service as the event log application name. This can be problematic if you have multiple instances of Apache running on the same box. If, for instance, you register multiple apache services using different display names (-k install -n My display name), it is necessary that the events in the event log map to the actual service that generated it, not Apache Service. To fix this, I made midifications on two files: The service.c.patch simply adds the display name used during -k install into the actual service command line. For example if you run Apache2.exe -k install -n web server 1 The service command like would be: Apache2.exe -k runservice -n web server 1 Instead of Apache2.exe -k runservice This is important because the display_name variable will set when the service is executed. This variable is used by the nt_eventlog.c The nt_eventlog.c patch replaces the hard-coded registry path SYSTEM\\CurrentControlSet\\Services\\EventLog\\Application\\Apache Service with SYSTEM\\CurrentControlSet\\Services\\EventLog\\Application\\ + display_name Now your messages will be loged with the right application name. So you can identify which service actually generated the error. Juan service.c.path Description: Binary data nt_eventlog.c.patch Description: Binary data
RE: [PATCH] EventLog display name patch
Title: [PATCH] EventLog display name patch BTW, the service.c patch should also take care of the following bug. If your configuration file has a syntax error, you will get something like this in your event log: The Apache service named reported the following error: Syntax error on line 32 of C:/Apache2/conf/httpd.conf: What is missing is the actual service display name between the words named and reported. With the service.c patch it will show it like this: The Apache service named My Service Name reported the following error: Syntax error on line 32 of C:/Apache2/conf/httpd.conf: -Original Message- From: Juan Rivera [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 18, 2003 10:59 AM To: [EMAIL PROTECTED] Subject: [PATCH] EventLog display name patch The current nt_eventlog implementation has the hardcoded Apache Service as the event log application name. This can be problematic if you have multiple instances of Apache running on the same box. If, for instance, you register multiple apache services using different display names (-k install -n My display name), it is necessary that the events in the event log map to the actual service that generated it, not Apache Service. To fix this, I made midifications on two files: The service.c.patch simply adds the display name used during -k install into the actual service command line. For example if you run Apache2.exe -k install -n web server 1 The service command like would be: Apache2.exe -k runservice -n web server 1 Instead of Apache2.exe -k runservice This is important because the display_name variable will set when the service is executed. This variable is used by the nt_eventlog.c The nt_eventlog.c patch replaces the hard-coded registry path SYSTEM\\CurrentControlSet\\Services\\EventLog\\Application\\Apache Service with SYSTEM\\CurrentControlSet\\Services\\EventLog\\Application\\ + display_name Now your messages will be loged with the right application name. So you can identify which service actually generated the error. Juan
RE: [PATCH] EventLog display name patch
Title: RE: [PATCH] EventLog display name patch Is there any other way to get the display name? I look for other ways but couldn't figure out another approach. Juan -Original Message- From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] Sent: Wednesday, June 18, 2003 3:43 PM To: [EMAIL PROTECTED] Subject: Re: [PATCH] EventLog display name patch At 09:59 AM 6/18/2003, Juan Rivera wrote: The current nt_eventlog implementation has the hardcoded Apache Service as the event log application name. This can be problematic if you have multiple instances of Apache running on the same box. To fix this, I made midifications on two files: The service.c.patch simply adds the display name used during -k install into the actual service command line. This part of the patch isn't acceptable because; The service command like would be: Apache2.exe -k runservice -n web server 1 this requirement above breaks exiting installed services, which is not good, IMHO. This is important because the display_name variable will set when the service is executed. This variable is used by the nt_eventlog.c I believe we can go a little further to assure that is filled in or retrieved as soon as it is needed. Let me look for the underlying bug. The nt_eventlog.c patch replaces the hard-coded registry path SYSTEM\\CurrentControlSet\\Services\\EventLog\\Application\\Apache Service with SYSTEM\\CurrentControlSet\\Services\\EventLog\\Application\\ + display_name Now your messages will be loged with the right application name. So you can identify which service actually generated the error. You must also register the event provider, which I will verify in your patch or code up if it's missing before I commit. I was originally hesitant to waste the additional space to create event loggers for each copy of Apache. You've sold me, and I understand your concern about trying to review an event log of many active instances of Apache. Most of my httpd time has been stuck in ssl issues, I hope now to dig out and get back to your patch and port the Win98/ME support from 1.3 as well. Thanks for all of your efforts, Juan! Bill
Socket Read Buffer Size
Have you guys ever considered making the size of the socket read buffer configurable (default is 8K)? Juan
RE: [PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets
Title: RE: [PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets
Cliff,
Is your code assuming that b is empty? If so, I'm not sure we can make that assumption.
-Original Message-
From: Cliff Woolley [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 12, 2003 5:08 PM
To: [EMAIL PROTECTED]
Subject: Re: [PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets
On Thu, 12 Jun 2003, Justin Erenkrantz wrote:
for (bucket = APR_BUCKET_FIRST(ctx-b);
bucket != e bucket != APR_BRIGADE_LAST(ctx-b);
bucket = APR_BUCKET_NEXT(bucket)) {
apr_bucket_remove(bucket);
APR_BRIGADE_INSERT_TAIL(b, bucket);
}
No! Bad!! The whole beauty of the ring data structure is that all of
these operations can be done in constant time, no loops. Bear with me,
and I'll explain.
If you understand the type safety checks it is attempting, you are a far
more intelligent person than I. =)
Well, I dunno about that... I do understand them but then again I've
focused on them since literally the day I started contributing to this lil
ole web server.
The following code assumes that ctx-b has at least one bucket in it,
namely e.
if (APR_BRIGADE_FIRST(ctx-b) != e)
{
/* move the sequence into the new brigade */
APR_RING_SPLICE_TAIL(b-list,
APR_BRIGADE_FIRST(ctx-b),
APR_BUCKET_PREV(e),
apr_bucket, link);
/* fixup the dangling pointers in ctx-b */
APR_BRIGADE_FIRST(ctx-b) = e;
APR_BUCKET_PREV(e) = APR_BRIGADE_SENTINEL(ctx-b);
}
Lovely, eh? I didn't actually test this to make sure it's 100% right, but
conceptually I believe it should do the trick.
--Cliff
RE: [PATCH] Avoid unnecessary brigade splits on core input and ou tput filters. WAS: EOS or FLUSH buckets
Title: RE: [PATCH] Avoid unnecessary brigade splits on core input and output
filters. WAS: EOS or FLUSH buckets
My bad, never mind. You are fixing up the
ctx-b pointers. I'm going to try it out.
-Original Message-
From: Juan Rivera
[mailto:[EMAIL PROTECTED]
Sent: Thursday, June 12, 2003 5:25
PM
To: '[EMAIL PROTECTED]'
Subject: RE: [PATCH] Avoid
unnecessary brigade splits on core input and ou tput filters. WAS: EOS or FLUSH
buckets
Cliff,
Is your code assuming that b is empty? If so, I'm not
sure we can make that assumption.
-Original Message-
From: Cliff Woolley [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 12, 2003 5:08
PM
To: [EMAIL PROTECTED]
Subject: Re: [PATCH] Avoid
unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH
buckets
On Thu, 12 Jun 2003, Justin Erenkrantz wrote:
for (bucket = APR_BUCKET_FIRST(ctx-b);
bucket != e bucket != APR_BRIGADE_LAST(ctx-b);
bucket = APR_BUCKET_NEXT(bucket)) {
apr_bucket_remove(bucket);
APR_BRIGADE_INSERT_TAIL(b, bucket);
}
No! Bad!! The whole beauty of the ring
data structure is that all of
these operations can be done in
constant time, no loops. Bear with me,
and I'll explain.
If you understand the type safety checks it is
attempting, you are a far
more intelligent person than
I. =)
Well, I dunno about that... I do understand them but
then again I've
focused on them since literally the
day I started contributing to this lil
ole web server.
The following code assumes that ctx-b has at least
one bucket in it,
namely e.
if (APR_BRIGADE_FIRST(ctx-b) != e)
{
/* move the
sequence into the new brigade */
APR_RING_SPLICE_TAIL(b-list,
APR_BRIGADE_FIRST(ctx-b),
APR_BUCKET_PREV(e),
apr_bucket, link);
/* fixup the dangling pointers in
ctx-b */
APR_BRIGADE_FIRST(ctx-b) = e;
APR_BUCKET_PREV(e) = APR_BRIGADE_SENTINEL(ctx-b);
}
Lovely, eh? I didn't actually test this to make
sure it's 100% right, but
conceptually I believe it should do
the trick.
--Cliff
RE: [PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets
Title: RE: [PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets Cliff, How would the code look like? I tried unsplicing from the first bucket to the bucket previous to e and then splicing it at the tail of b. But didn't work. I don't understand well how the ring data structure works. Juan -Original Message- From: Cliff Woolley [mailto:[EMAIL PROTECTED]] Sent: Wednesday, June 11, 2003 5:14 PM To: [EMAIL PROTECTED] Subject: Re: [PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets On Tue, 10 Jun 2003, Justin Erenkrantz wrote: What you are really wanting to do is a partial concatenation of the brigade. Something like (not there, but it could be easily added): APR_BRIGADE_CONCAT_UNTIL(b, ctx-b, e) I'm not sure I see exactly where you're going, but you might be able to get what you want by doing an APR_RING_SPLICE(). --Cliff
RE: EOS or FLUSH buckets
Title: RE: EOS or FLUSH buckets
Greg,
Right, my module leaks memory because the core input and output filters split the bucket brigades. So it keeps creating more and more bucket brigades that are not released until the connection is gone.
First of all, I think the split in the core input filter (READBYTES) should be optimized because all it is doing is splitting the brigade to concatenate it into another brigade. Wouldn't be more efficient to do a move buckets from brigade ctx-b to b and avoid creating a temporary brigade?
So for the output side, when I send a flush, it splits the brigade. If the flush is the last bucket, this might not be necessary, what do you think?
On the topic of EOS, I think that if the last bucket is an EOS and is not a keep alive connection it should not hold the data but it currently does.
Juan
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, June 10, 2003 10:38 AM
To: [EMAIL PROTECTED]
Subject: Re: EOS or FLUSH buckets
Juan Rivera wrote:
Here is why I'm asking. I wrote a SOCKS proxy module. It has two
connection records, one for the client and one for the backend server.
When I received data I pass it to the other side with a flush at the
end. It works fine with one problem. The core output filter splits the
brigade after the flush bucket creating a new bucket brigade. This
bucket brigade is never destroyed, consuming 16 bytes of memory
(apr_brigade.c: line 84). This may not be a problem for short lived
connection but it the connection is long lived the pool keeps getting
bigger and bigger. Is there any way around this?
So I though, I should use an EOS bucket instead (maybe not a good idea)
but I found that the core output filter was setting aside my buckets.
This section in core.c looks bogus to me:
Core.c line 3884:
if (nbytes + flen AP_MIN_BYTES_TO_WRITE
((!fd !more !APR_BUCKET_IS_FLUSH(last_e))
|| (APR_BUCKET_IS_EOS(last_e)
c-keepalive == AP_CONN_KEEPALIVE))) {
/* set aside the buckets */
}
What is weird about this code is that if the last bucket in the
bridade is an EOS, this part: ((!fd !more
!APR_BUCKET_IS_FLUSH(last_e)) will return true as long as you are not
serving a file.
But it seems that you want that to happen only if the connection
is a keep-alive connection. Right?
Juan,
Congratulations! You have homed right in on one of the trickiest new parts of
Apache 2.0. I believe the intention here is to hang on to the data across HTTP
connections, so we can take advantage of keepalives and pipelining and
potentially pack more data into fewer TCP segments. Usually what will happen is
that at the end of a request, we look for more inbound data (the next request),
don't find any, and send a FLUSH bucket which causes the network write.
Any suggestions how to deal with this?
Deal with what? Are you saying that we leak memory with your SOCKS module?
Greg
[PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets
Title: [PATCH] Avoid unnecessary brigade splits on core input and output filters. WAS: EOS or FLUSH buckets I'm seen this problem with a SOCKS protocol module I wrote. I'm including a patch that fixes this problem. It does what I mentioned below. In the input filter, it moves the buckets rather than creating a new brigade and then concatenate. In the output filter it splits the brigade after a flush bucket only if there are buckets after the flush. Juan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 10, 2003 3:41 PM To: [EMAIL PROTECTED] Subject: Re: EOS or FLUSH buckets Juan Rivera wrote: Right, my module leaks memory because the core input and output filters split the bucket brigades. So it keeps creating more and more bucket brigades that are not released until the connection is gone. When you see this, are we talking about a lot of HTTP requests pipelined on a single connection, or a single HTTP request that lasts a long time? First of all, I think the split in the core input filter (READBYTES) should be optimized because all it is doing is splitting the brigade to concatenate it into another brigade. Wouldn't be more efficient to do a move buckets from brigade ctx-b to b and avoid creating a temporary brigade? So for the output side, when I send a flush, it splits the brigade. If the flush is the last bucket, this might not be necessary, what do you think? I'll defer these two questions to our Bucketmeister and/or efficiency experts. (Cliff? Brian?) On the topic of EOS, I think that if the last bucket is an EOS and is not a keep alive connection it should not hold the data but it currently does. Maybe. But if it's not a keepalive connection, we should be sending a FLUSH bucket within microseconds, no? OK, maybe that path could be optimized. But we'd have to be careful because keepalive connections are very common. We wouldn't want to penalize the hot path by optimizing for the less common case. Greg core.c.patch Description: Binary data
EOS or FLUSH buckets
When should one use an EOS bucket vs. a FLUSH bucket? Juan
RE: EOS or FLUSH buckets
Title: RE: EOS or FLUSH buckets
Here is why I'm asking. I wrote a SOCKS proxy module. It has two connection records, one for the client and one for the backend server.
When I received data I pass it to the other side with a flush at the end. It works fine with one problem. The core output filter splits the brigade after the flush bucket creating a new bucket brigade. This bucket brigade is never destroyed, consuming 16 bytes of memory (apr_brigade.c: line 84). This may not be a problem for short lived connection but it the connection is long lived the pool keeps getting bigger and bigger. Is there any way around this?
So I though, I should use an EOS bucket instead (maybe not a good idea) but I found that the core output filter was setting aside my buckets. This section in core.c looks bogus to me:
Core.c line 3884:
if (nbytes + flen AP_MIN_BYTES_TO_WRITE
((!fd !more !APR_BUCKET_IS_FLUSH(last_e))
|| (APR_BUCKET_IS_EOS(last_e)
c-keepalive == AP_CONN_KEEPALIVE))) {
/* set aside the buckets */
}
What is weird about this code is that if the last bucket in the bridade is an EOS, this part: ((!fd !more !APR_BUCKET_IS_FLUSH(last_e)) will return true as long as you are not serving a file.
But it seems that you want that to happen only if the connection is a keep-alive connection. Right?
Any suggestions how to deal with this?
Juan
-Original Message-
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]]
Sent: Monday, June 09, 2003 5:42 PM
To: [EMAIL PROTECTED]
Cc: '[EMAIL PROTECTED]'
Subject: Re: EOS or FLUSH buckets
At 04:29 PM 6/9/2003, Juan Rivera wrote:
When should one use an EOS bucket vs. a FLUSH bucket?
At EOS, that's all she wrote; body or connection finished, nothing
more, don't call me, and I won't call you.
So a flush bucket is a request; please get rid of any lingering data
and push it on to the client.
I meant it's a request, consider a filter that does codepage translation,
and we have half of a multibyte character sequence. There is no way
to flush that half a character because we don't know what it will become.
So flush should be respected as far as possible, but you cannot trust
that everything is flushed. At EOS, such an incomplete translation
would have to be performed (or discarded) because there is nothing
more to come.
Bill
[PATCH] Winnt MPM missing my_pid in some ap_log_error calls
Two ap_log_error calls had the "Child %d" but no my_pid in the argument list. This patch adds my_pid so that the messages are displayed correctly. Juan child.c.patch Description: Binary data
Reusing buffers when reading from socket
In the socket_bucket_read function (apr_bucket_socket.c) it reads from the socket using an 8K buffer. Now, if you only get 100 bytes, the rest of the buffer is wasted. Right? I guess HTTP typically gets large chucks of data at a time but when implementing other protocols this 8K buffer might be inefficient. I wonder if there could be a generic way to solve this problem. I was thinking that when the socket bucket gets small chunks of data (100 bytes), it will hold that bucket and it will reuse it next time., creating a refcount bucket pointing to the same buffer but with an offset start. This way, memory will be used more efficiently when you are getting small chunks of data at a time. When you get large chunks of data it will behave just like it does today. Has anybody looked into this before? Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
Sending multiple buffers using writev and WSASend
When sending multiple buckets on a socket does writev and WSASend create a packet per buffer, or a single packet for all buffers? I would assume the answer might depend on the Operating system or maybe the size of the buffers. I'm interested in Windows 2000 and linux. Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
Apache UTF-8
I'm starting to look into what would it takes to support UTF-8 in configuration files. This means modules must have to handle parameters in UTF-8 format. Anybody has any pointers that can help me? I think making Apache UTF-8 compliant will be very helpful. Do you guys agree? Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
Apache i18n support
Has anybody looked into internationalizing Apache 2? There are a couple of areas like: Messages Error pages Multibyte/Unicode support Anything else? Is this in the roadmap for Apache? What are the caveats of doing it? Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
RE: Artificial delay on traffic
When I create the backend socket, I call ap_sock_disable_nagle(). Is there something else I have to do? Best regards, Juan C. Rivera Citrix Systems, Inc. -Original Message- From: Aaron Bannert [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 13, 2003 5:02 PM To: [EMAIL PROTECTED] Subject: Re: Artificial delay on traffic Could it be TCP_CORK? -aaron On Thursday, February 13, 2003, at 12:14 PM, Juan Rivera wrote: I wrote a SOCKS module for Apache 2. I started off with the proxy_connect code and added the SOCKS handshake plus the filter chain on the client socket. I use it to pass ICA traffic. The ICA protocol is a very chatty protocol; it sends very small packets back and forth. At first I was experiencing very large latency but it was because the socket created to the backend had nagling enabled. I disabled that and latency drop to almost zero. On Linux, I still experience some latency but I can not identify the problem. The httpd process is not saturating at all. I'm wondering if there is another type of hidden buffering that is causing that delay.
Artificial delay on traffic
Title: Artificial delay on traffic I wrote a SOCKS module for Apache 2. I started off with the proxy_connect code and added the SOCKS handshake plus the filter chain on the client socket. I use it to pass ICA traffic. The ICA protocol is a very chatty protocol; it sends very small packets back and forth. At first I was experiencing very large latency but it was because the socket created to the backend had nagling enabled. I disabled that and latency drop to almost zero. On Linux, I still experience some latency but I can not identify the problem. The httpd process is not saturating at all. I'm wondering if there is another type of hidden buffering that is causing that delay. On Windows, the problem is a lot less noticeable but still there is a delay. I'm using the worker mpm on Linux. Do you guys know about anything in particular specific to Linux or the Linux implementation of Apache that might be causing the artificial delay? Or is it simply the overhead of going through the filter chain for a bunch of 4 bytes packets? Best regards, Juan C. Rivera Citrix Systems, Inc.
Request for comments: Managing max connections per protocol
One of the missing pieces in Apache 2 to support multiple protocols is the ability to limit the number of connections per protocol. Right now you specify the max number of connections but there is no way to manage how many connections a particular protocol can manage. Anyways, the problem seems like that by the time the protocol handler decides it will process the request, it is too late for the core to do anything. Now, if we add an extra optional hook to the protocol handlers that manages the max connections for the protocol bypassing the core functionality. It will be up to the protocol handler to write the code that enforces a maximum number of connections. I though that the core could do that if the extra hook just tells the core that the module will manage the request. The problem I see with this approach is that if protocol handlers need some complex way to manage how many concurrent connections are handled, this model will not satisfy that. Comments? Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
RE: Custom Winnt MPM
Title: RE: Custom Winnt MPM Thanks, that's what I have been doing so far. I though the functionality of loading MPM dynamically was there. Best regards, Juan C. Rivera Citrix Systems, Inc. -Original Message- From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]] Sent: Monday, November 25, 2002 6:11 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: Custom Winnt MPM At 04:54 PM 11/25/2002, Brian Pane wrote: On Mon, 2002-11-25 at 14:03, Juan Rivera wrote: I've been working on an async i/o mpm for Windows using IO completion ports. I used the current winnt mpm as a starting point but I rather have it in it's own dll instead of linked directly in libhttpd. My question is how do I configure Apache to use my mpm? Is it just using LoadModule? How do I disable the default winnt mpm? I'm not sure how it works on Win32, but here's the Unix answer, in case it's useful as a starting point: 1. update server/mpm/config.m4 so that it knows about your new MPM. Check out the code in there for the leader, threadpool, and perchild MPMs; they should be a good starting point. 2. make clean 3. ./configure --with-mpm=[new MPM] Very similar on win32... modify libhttpd.dsp ... you will notice a section for the mpm files. Just replace the sources you don't want with the sources you are creating. Note that you can't have a 'loadable' MPM, since the mpm defines how processes are created and how the server initializes (mpms start running before we start processing the config file.) An early design hoped for 'loadable' mpms. It didn't happen, I'm afraid, because it proved very difficult to do. Bill
RE: request for comments: multiple-connections-per-thread MPM design
Title: RE: request for comments: multiple-connections-per-thread MPM design Brian, Does your model assume that connections are short lived (HTTP)? One problem with the current model is that if you implement, let's say mod_socks, it holds a connection per thread. Is that something your model addresses? I'm looking into a pure async i/o model which addresses this problem but has bigger compatibility issues, yours may not. Best regards, Juan C. Rivera Citrix Systems, Inc. -Original Message- From: Brian Pane [mailto:[EMAIL PROTECTED]] Sent: Saturday, November 23, 2002 9:41 PM To: [EMAIL PROTECTED] Subject: request for comments: multiple-connections-per-thread MPM design Here's an outline of my latest thinking on how to build a multiple-connections-per-thread MPM for Apache 2.2. I'm eager to hear feedback from others who have been researching this topic. Thanks, Brian Overview The design described here is a hybrid sync/async architecture: * Do the slow part of request processing--network reads and writes--in an event loop for scalability. * Do the fast part of request processing--everything other than network I/O--in a one-request-per-thread mode so that module developers don't have to rewrite all their code as reentrant state machines. Basic structure --- Each httpd child process has four thread pools: 1. Listener thread A Listener thread accept(2)s a connection, creates a conn_rec for it, and sends it to the Reader thread. 2. Reader thread A Reader thread runs a poll loop to watch for incoming data on all connections that have been passed to it by a Listener or Writer. It reads the next request from each connection, builds a request_rec, and passes the conn_rec and the request_rec on to the Request Processor thread pool. 3. Request Processor threads Each Request Processor thread handles one request_rec at a time. When it receives a request from the Reader thread, the Request Processor runs all the request processing hooks (auth, map to storage, handler, etc) except the logger, plus the output filter stack except the core_output_filter. As the Request Processor produces output brigades, it sends them to the Writer thread pool. Once the Request processor has finished handling the request, it sends the last of the output data, plus the request_rec, to the Writer. 4. Writer thread The Writer thread runs a poll loop to output the data for all connections that have been passed to it. When it finishes writing the response for a request, the Writer calls the logger, destroys the request_rec, and either executes the lingering_close on the connection or sends the connection back to the Reader, depending on whether the connection is a keep-alive. Component details - * Listener thread: This thread will need to use an accept_mutex to serialize the accept, just like 2.0 does. * Passing connections from Listener to Reader: When the Listener creates a new connection, it adds it to a global queue and writes one byte to a pipe. The other end of the pipe is in the Reader's pollset. When the poll(2) in the Reader completes, the Reader detects the data available on the pipe, reads and discards the byte, and retrieves all the new connections in the queue. * Passing connections from Reader to Request Processor: When the Reader has consumed all the data in a connection, it adds the connection and the newly created request_rec to a global queue and signals a condition variable. The idle Request Processor threads take turns waiting on the condition variable (leader/followers model). * Passing output brigades from Request Processor to Writer: Same model as the Listener-to-Reader handoff: add to a queue, and write a byte to a pipe. * Bucket management: Implicit in this design is the idea that the Writer thread can be writing part of an HTTP response while a Request Processor thread is still generating more buckets for that request. This is a good thing because it means that the Request Processor thread won't ever find itself blocked on a network write, so it can produce all its output quickly and move on to another request (which is the key to keeping the number of threads low). However, it does mean that we need a thread-safe solution for allocating and destroying buckets and brigades. * request_rec lifetime: When a Request Processor thread has produced all of the output for a response, it adds a metadata bucket to the last output brigade. This bucket points to the request_rec. Upon sending the last of the request's output, the Writer thread is responsible for calling the logger and the destroying the request and its pool. This would be a major change from how 1.x and 2.0 work. The rationale for it is twofold: - Eliminate the need to set aside buckets from the request pool into the connection pool in the core_output_filter, which has been a source of many bugs in 2.0. -
Custom Winnt MPM
Title: Custom Winnt MPM I've been working on an async i/o mpm for Windows using IO completion ports. I used the current winnt mpm as a starting point but I rather have it in it's own dll instead of linked directly in libhttpd. My question is how do I configure Apache to use my mpm? Is it just using LoadModule? How do I disable the default winnt mpm? Best regards, Juan C. Rivera Citrix Systems, Inc.
RE: WELCOME to dev@httpd.apache.org
Title: RE: WELCOME to [EMAIL PROTECTED] You can write an output filter and configure it like this VirtualHost * Proxy Entries Here Proxy * SetOutputFilter MyFilterName /Proxy /VirtualHost Your filter will have a chance to look and modify the content that goes to the client coming from the backend web server. Best regards, Juan C. Rivera Citrix Systems, Inc. -Original Message- From: Prajakt Deolasee [mailto:[EMAIL PROTECTED]] Sent: Tuesday, November 19, 2002 1:07 AM To: [EMAIL PROTECTED] Subject: RE: WELCOME to [EMAIL PROTECTED] Hi Everybody, I have a very simple question. I am using the Apache HTTP Server as a loadbalancer. I could get it ruuning and it is smartly acting a loadbalancer. In addition to this I have a requirement, that the response which is send back to the client, I want to trap that response in the loadbalancer and make some changes and then send it across. How do I do it? Can somabody give me some pointers as to in which module exacly I should look in? I thought it should be in mod_proxy or mod_proxy_http are the module I should be looking at. I could see one function in mod_proxy_http module ap_proxy_http_process_response but the this function is never called when I run it in debugger. I am using VC++ 6.0. And I am using mod_proxy, mod_rewrite and mod_proxy_http modules. Please see if anybody can help. -Prajakt
RE: [PATCH] mod_deflate extensions
Title: RE: [PATCH] mod_deflate extensions I agree that mod_deflate should be part of the distribution. Are there any issues with apache including zlib in the distribution? Best regards, Juan C. Rivera Citrix Systems, Inc. -Original Message- From: Jeff Trawick [mailto:[EMAIL PROTECTED]] Sent: Tuesday, November 19, 2002 7:33 AM To: [EMAIL PROTECTED] Subject: Re: [PATCH] mod_deflate extensions Henri Gomez [EMAIL PROTECTED] writes: When you drop the network bandwith by 30 to 70% factor, you make your IT managers happy since they save money and you make end-users very happy since they feel you application is faster. when you drop the web server throughput by x% factor you may make yourself sad :) So adding mod_deflate to the default distribution, under control of configure which will verify if zlib is available on the system to enable it, shouldn't hurt. I wish it were so simple as finding a zlib, but static zlib distributed with some OSs vs dynamic zlib distributed with others seems to be the difference between success and failure. More users will use mod_deflate, more chance to see remaining bugs discovered and fixed. I definitely agree that it would be nice to turn on mod_deflate in the build automagically, I just don't want to do it at the expense of more problems encountered by users. I suspect that we would need to ship a subset of zlib ourselves in order to have a fool-proof build of it. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
[PATCH]util_debug.c
The declarations of the debug versions for ap_strXXX may differ from the header file if the default calling convention is not the same as the one define for AP_DECLARE. Those functions are declared without the AP_DECLARE. This patch adds the AP_DECLARE to those functions. Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391 util_debug.patch Description: Binary data
Async I/O
I have been modifying the Winnt MPM to support IO Completion ports to avoid client per thread model. I also refactored mod_echo for testing purposes. I'm almost done but I think the async i/o is going to introduce all sorts of compatibility problems with filters that block while reading or writing. But if the modules are aware of this, this is a workable solution. I have no problem contributing the code but I think there is some planning and discussion that need to happen before Apache can move to Async I/O. Since you guys went through the pain of a new model for modules in 2.0, I think if the model for async i/o is different, this will cause a lot of complains. Do you guys see async I/O as a 3.0 feature rather than a 2.0. Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
RE: Apache and persistent connections
Title: RE: Apache and persistent connections Is async I/O something you guys have in your plans? How can I contribute? Do you think protocol handlers (HTTP one) must need to be modified to use async I/O? Juan -Original Message- From: Jeff Trawick [mailto:[EMAIL PROTECTED]] Sent: Saturday, November 16, 2002 8:27 AM To: [EMAIL PROTECTED] Subject: Re: Apache and persistent connections Juan Rivera [EMAIL PROTECTED] writes: I looked at how Apache 2 works and it seems like a thread holds a connection for the lifetime of it. Is this right? yes If I wanted to optimize this so that connections that have no data are move to a pool so that threads can move to the next connection and when there is data, go back and deal with the other connection. It sounds, I'll have to write my own MPM with IO completion ports to do something like this, right? at least... core code is not going to do the right thing as-is or hang around until folks really start working on async I/O and join in the fun -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
mod_proxy question
I have a question about the way mod_proxy_http works. Does the thread block until the response from the back end server is completed? Is this MPM specific? Any document available about the basic architecture of mod_proxy? Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
XML-based configuration
Probably this issue has been brought many times. I wonder if there are any effort to support xml-based configuration files instead of the current format? Any links or pointers would be helpful. Since, Apache 2 does not support it probably is for a good reason(s). Anybody can point me to those? The reason I interested in xml-based configuration is to simplify the development of an MMC-based configuration tool. Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
RE: mod_blank dev
If you do: SetOutputFilter BLANK;DEFLATE should work with no problems. Best regards, Juan C. Rivera Citrix Systems, Inc. -Original Message- From: Justin Erenkrantz [mailto:jerenkrantz;apache.org] Sent: Sunday, November 03, 2002 2:04 PM To: [EMAIL PROTECTED] Subject: Re: mod_blank dev --On Sunday, November 3, 2002 11:08 AM +0100 fabio rohrich [EMAIL PROTECTED] wrote: Have I to fill the blank_cmds if I have a directive? And I need a directive? Or more than one? And which? I would strongly suggest reading how other modules do this. Since your module should be implemented as a filter, I would recommend looking at mod_deflate in modules/filters/mod_deflate.c. You will see examples on how to do multiple directives there (see deflate_filter_cmds). I'll be curious to see how mod_deflate and mod_blank would work together. Good luck! -- justin
mod_proxy and AddOutputFilterByType
Question on Apache 2.0.43: I'm trying to use AddOutputFilterByType on a reverse proxy vhost. Looking at the code in core.c line 2633, it checks if the request is proxied and if so, they don't add the filters specified in the AddOutputFilterByType entries. Why? Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391
request_rec-cookies
Is there any easy way to get a list of the cookies (name/value pairs) from a client's request? Best regards, Juan C. Rivera Citrix Systems, Inc.
RE: Compiling Apache modules for windows
Igor, You have to put the replace entries like this: Proxy /someloc Replace colour color text/html SetOutputFilter REPLACE /Proxy ProxyPass /someloc http://Server/someloc ProxyPassReverse /someloc http://Server/someloc Best regards, Juan C. Rivera Citrix Systems, Inc. -Original Message- From: Igor Leturia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 15, 2002 6:06 AM To: [EMAIL PROTECTED] Subject: Re: Compiling Apache modules for windows Hi all! Günter sent me a binary version of mod_replace. I started trying it and I couldn't get it to work. At last I found that there is an error in the example configuration file. Instead of being 'Replace colour color text/html', it should be 'Replace colour color text/html'. At least this is the only way it worked for me. I say this in case anyone is interested. Now I've got some questions. I've been able to make replacements in the files if they have .html extension, but not in the rest (.js, etc.). In the example cfg you sent me, this is what you said (with my correction already done): Replace colour color text/html AddOutputFilter REPLACE html I've tried writing js instead of html in the second line, but I don't know what to write in the first line instead of text/html. I've tried lots of things but none work. What should I do to apply the replacements to all the files served, independent of their extension? I tried *, but it doesn't work. Another question: do you know how should I write the httpd.conf to make the replacements to files proxied from another server? I wrote this, but it doesn't work: Location /someloc Replace colour color text/html AddOutputFilter REPLACE html ProxyPass http://someserver/someloc ProxyPassReverse http://someserver/someloc /Location I've tried different orders of the lines, but none worked. Thanks in advance, Igor Leturia
[PATCH] Apache -k restart for Winnt fix
There is a problem when calling apache -k restart. The issue is with permissions set with the service control manager. It happens that Apache sends a user define control code for restarting, signal 128. But when getting the handle of the service it is not passing the flag to allow user define control code causing the ControlService call to return ERROR_ACCESS_DENIED. The following patch for the file server/mpm/winnt/service.c adds the flag to the OpenService call to allow user define control codes. This patch should fix bug 12011. Juan C. Rivera Citrix Systems, Inc Tel: (954)229-6391 service.c.patch Description: Binary data
