apache module registry is working?

2011-11-16 Thread MATSUMOTO Ryosuke
Hi, all

I submitted my module to apache module registry on 5 days ago, but response is 
nothing yet.

The site is woking now?

Submitted module name is mod_process_security.

regards,





Re: Improving SSL config

2011-11-16 Thread Igor Galić


- Original Message -
 On 14.11.2011 15:46, William A. Rowe Jr. wrote:
  Isn't it similarly time to deploy SSLProtocol -SSLv2 by default?

 Oh yes, definitely. I didn't realize that all is still the default
 for
 SSLProtocol... for trunk and 2.4, I would suggest to change the
 defaults
 in the code. In decreasing order of preference:

 - completely drop SSLv2 support

 - change the default (in modssl_ctx_init) to
   SSL_PROTOCOL_ALL  ~SSL_PROTOCOL_SSLV2

+1

 The first option also means that we would comply with RFC 6176 (in
 case someone complains about mod_ssl dropping support for a clearly
 outdated and insecure protocol).

 Kaspar


--
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.ga...@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515  2EA5 4B1D 9E08 A097 C9AE



[DRAFT] Wanted: Patch Manager

2011-11-16 Thread Igor Galić
Stolen from

http://subversion.apache.org/docs/community-guide/roles.html#patch-manager

  Patch Manager

  $project usually has a Patch Manager, whose job is to watch the
  dev@ mailing list and make sure that no patches slip through the
  cracks.

  This means watching every thread containing [PATCH] mails, and
  taking appropriate action based on the progress of the thread. If
  the thread resolves on its own (because the patch gets committed,
  or because there is consensus that the patch doesn't need to be
  applied, or whatever) then no further action need be taken. But if
  the thread fades out without any clear decision, then the patch
  needs to be saved in the issue tracker. This means that a summary
  of any discussion threads around that patch, and links to relevant
  mailing list archives, will be added to some issue in the tracker.
  For a patch which addresses an existing issue tracker item, the
  patch is saved to that item. Otherwise, a new issue of the correct
  type — '$DEFECT', '$FEATURE', or '$ENHANCEMENT' (not '$PATCH') —
  is filed, the patch is saved to that new issue, and the patch
  keyword is recorded on the issue.

  The Patch Manager needs a basic technical understanding of
  $project, and the ability to skim a thread and get a rough
  understanding of whether consensus has been reached, and if so, of
  what kind. It does not require actual $project development
  experience or commit access. Expertise in using one's mail reading
  software is optional, but recommended :-).

  The current Patch Manager is: $person $contact

I really like the basic outline, but what I feel is missing, is the
notion of watching $issuetracker. Often we have patches getting lost
in there. So our patch manager should also look out for $PATCH issues
and make sure they are followed up, or assigned, or whatever.

Thoughts? Comments? Ideas?
I happily welcome all of these.

So long,
i

--
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.ga...@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515  2EA5 4B1D 9E08 A097 C9AE


Re: Windows build bot

2011-11-16 Thread Igor Galić


- Original Message -
 Hi,

 we were told by pgollucci that there is already a Windows build bot
 at the
 ASF. It would be awesome if someone knowledgable in Windows could
 work
 with the Infra guys to set it up. The mail address is
 bui...@apache.org
 according to Philip.

+++1! If all else fails, open a Jira, and/or bug Gavin directly.
Or buy him a beer. That should help :)

 Cheers,
 Stefan

i

--
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.ga...@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515  2EA5 4B1D 9E08 A097 C9AE



Fwd: svn commit: r1202257 - in /httpd/httpd/trunk/server/mpm/event: config3.m4 equeue.c equeue.h event.c

2011-11-16 Thread Rüdiger Plüm



 Original-Nachricht 
Betreff:svn commit: r1202257 - in /httpd/httpd/trunk/server/mpm/event: 
config3.m4 equeue.c equeue.h event.c
Datum:  Tue, 15 Nov 2011 15:51:04 GMT
Von:pque...@apache.org



Author: pquerna
Date: Tue Nov 15 15:51:03 2011
New Revision: 1202257

URL: http://svn.apache.org/viewvc?rev=1202257view=rev
Log:
Create a new lock free circular queue, and use it in the EventMPM to remove the 
timeout mutex
that was wrapping both timeout queue operations and pollset operations.

Added:
httpd/httpd/trunk/server/mpm/event/equeue.c   (with props)
httpd/httpd/trunk/server/mpm/event/equeue.h   (with props)
Modified:
httpd/httpd/trunk/server/mpm/event/config3.m4
httpd/httpd/trunk/server/mpm/event/event.c

Modified: httpd/httpd/trunk/server/mpm/event/config3.m4
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/server/mpm/event/equeue.c?rev=1202257view=auto
==
--- httpd/httpd/trunk/server/mpm/event/equeue.c (added)
+++ httpd/httpd/trunk/server/mpm/event/equeue.c Tue Nov 15 15:51:03 2011
@@ -0,0 +1,125 @@
+/* Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include equeue.h
+
+#includeapr_atomic.h
+#includesched.h
+
+struct ap_equeue_t {
+apr_uint32_t nelem;
+apr_size_t elem_size;
+uint8_t *bytes;
+volatile apr_uint32_t writeCount;
+volatile apr_uint32_t readCount;
+};
+
+
+static APR_INLINE apr_uint32_t count_to_index(ap_equeue_t *eq, apr_uint32_t 
count)
+{
+return (count  (eq-nelem - 1));
+}
+
+static APR_INLINE void* index_to_bytes(ap_equeue_t *eq, apr_uint32_t idx)
+{
+apr_size_t offset = idx * eq-elem_size;
+return (void*)eq-bytes[offset];
+}
+
+static APR_INLINE apr_uint32_t nearest_power(apr_uint32_t num)
+{
+apr_uint32_t n = 1;
+while (n  num) {
+n= 1;
+}
+
+return n;
+}
+
+#if 0
+static void dump_queue(ap_equeue_t *eq)
+{
+apr_uint32_t i;
+
+fprintf(stderr, dumping %p\n, eq);
+fprintf(stderr,   nelem:   %u\n, eq-nelem);
+fprintf(stderr,   esize:   %APR_SIZE_T_FMT\n, eq-elem_size);
+fprintf(stderr,   wcnt:%u\n, eq-writeCount);
+fprintf(stderr,   rcnt:%u\n, eq-writeCount);
+fprintf(stderr,   bytes:   %p\n, eq-bytes);
+for (i = 0; i  eq-nelem; i++) {
+fprintf(stderr, [%u] = %p\n, i, index_to_bytes(eq, i));
+}
+
+fprintf(stderr, \n);
+fflush(stderr);
+}
+#endif
+
+apr_status_t
+ap_equeue_create(apr_pool_t *p, apr_uint32_t nelem, apr_size_t elem_size, 
ap_equeue_t **eqout)
+{
+ap_equeue_t *eq;
+
+*eqout = NULL;
+
+eq = apr_palloc(p, sizeof(ap_equeue_t));
+eq-bytes = apr_palloc(p, (1 + nelem) * elem_size);
+eq-nelem = nearest_power(nelem);
+eq-elem_size = elem_size;
+eq-writeCount = 0;
+eq-readCount = 0;
+*eqout = eq;
+
+return APR_SUCCESS;
+}
+
+void *
+ap_equeue_reader_next(ap_equeue_t *eq)
+{
+if (apr_atomic_read32(eq-writeCount) == eq-readCount) {
+return NULL;
+}
+else {
+apr_uint32_t idx = count_to_index(eq, 
apr_atomic_inc32(eq-readCount));
+return index_to_bytes(eq, idx);
+}
+}
+
+void *
+ap_equeue_writer_value(ap_equeue_t *eq)
+{
+apr_uint32_t idx;
+
+while (1) {
+apr_uint32_t readCount = apr_atomic_read32(eq-readCount);
+
+if (count_to_index(eq, eq-writeCount + 1) != count_to_index(eq, 
readCount)) {
+break;
+}
+/* TODO: research if sched_yield is even worth doing  */
+sched_yield();
+}
+
+idx = count_to_index(eq, eq-writeCount);
+return index_to_bytes(eq, idx);
+}

I assume that only a single thread tries write operations on this queue, 
correct?
Otherwise it seems unsafe if another thread calls
ap_equeue_writer_value in parallel as it would return the same slot until
ap_equeue_writer_onward was called.





Re: setting TZ env var

2011-11-16 Thread Rainer Jung

On 15.11.2011 17:06, Paul Querna wrote:

So, I was looking at all the system calls we make in a single request,
and comparing it to nginx.

We were actually pretty close, baring supporting our features like
htaccess, there was only one thing that stood out.

Glibc is opening, calling fstat twice, and then reading /etc/localtime
for every request:

[pid 31496]  0.51 open(/etc/localtime, O_RDONLY) = 80.14
[pid 31496]  0.41 fstat(8, {st_mode=S_IFREG|0644, st_size=118,
...}) = 00.11
[pid 31496]  0.48 fstat(8, {st_mode=S_IFREG|0644, st_size=118,
...}) = 00.10
[pid 31496]  0.48 mmap(NULL, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd87efa.13
[pid 31496]  0.40 read(8,
TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0...,
4096) = 1180.15
[pid 31496]  0.51 lseek(8, -62, SEEK_CUR) = 560.11
[pid 31496]  0.34 read(8,
TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0...,
4096) = 620.11
[pid 31496]  0.48 close(8)  = 00.12


One way to fix this, is to set the TZ environment variable.

For example:
   
https://github.com/pquerna/httpd/commit/37e5815a70e88a733cd088398d016803146b545f

This specific patch has some issues, but is there any objections to
the concept of setting the timezone on process startup?


Fine for me.


This single change gives about a 2% performance boost in my testing.

To merge to trunk I'd like to have it detect your active timezone,
instead of forcing UTC+0, but if you changed timezones on your
machine, you will need to restart httpd.


I'd say it is OK. I assume a apachectl restart would do?
It should be expected for a long running daemon to signal it to adopt to 
environmental changes like the timezone.


Regards,

Rainer


Re: [DRAFT] Wanted: Patch Manager

2011-11-16 Thread Nick Kew

On 16 Nov 2011, at 08:28, Igor Galić wrote:

 Stolen from

Stolen from the reply I just posted to another dev list ;)

 Patch Manager

Sounds like a role for an Igor.

(as any Pratchett fan will instantly understand :D )

Seriously, though, while it's a Good Thing to ensure patches
don't slip through the cracks, it's not clear that specifying such
a role really adds anything.  If a explicit [PATCH] is posted to
this list I think our natural reaction would already be to suggest
the poster opens a ticket in the issue tracker.

But if you'd like to take explicit responsibility that's fine!

-- 
Nick Kew

Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Rainer Jung

On 15.11.2011 20:57, Jeff Trawick wrote:

On Tue, Nov 15, 2011 at 2:32 PM, William A. Rowe Jr.
wr...@rowe-clan.net  wrote:

On 11/15/2011 12:33 PM, Stefan Fritsch wrote:


On Tuesday 15 November 2011, Paul Querna wrote:


On Tue, Nov 15, 2011 at 9:17 AM, Stefan Fritschs...@sfritsch.de


wrote:


On Tue, 15 Nov 2011, pque...@apache.org wrote:


Author: pquerna
Date: Tue Nov 15 15:49:19 2011
New Revision: 1202255

URL: http://svn.apache.org/viewvc?rev=1202255view=rev
Log:
disable mod_reqtimeout if not configured


Why that? We have just changed the default to be enabled in
r1199447 and several developers at the hackathon agreed to this
change.


Didn't know it was discussed in depth at the hackathon, and there
wasn't any discussion on the list

It showed up quite quickly in my profiling of the Event MPM,
because every pull/push on the filters would cause a
apr_time_now() call.

I don't really like that just by loading the module, it changes the
behavior and performance of the server so drastically.


It only acts on reads from the client. Normal non-POST requests arrive
in one or two packets, which would mean approx. 3 additional
apr_time_now calls per request. I haven't done benchmarks, but I can't
imagine that this has a drastic impact on performance. And if it costs
1-2%, then that's a small cost compared to the impact of slowloris
type attacks which eat lots of memory.

The general intention of the recent changes in default configs and
module selection/loading was to make it easier to only load those
modules that are really needed, have a reasonable default config, and
have the compiled-in default values be the same as those in the
example config files.


Which means, build by default, disable by default.  I think that keeps
everyone happy.  When abuse arrives, it's trivial to load.


Timeout 60 isn't nearly as bad as the old Timeout 300 that is probably
still in wide use, but mod_reqtimeout can provide a much more
reasonable out of the box configuration.  I think we should keep it in
place by default.


+1

Rainer



Re: [DRAFT] Wanted: Patch Manager

2011-11-16 Thread Daniel Ruggeri
On 11/16/2011 2:28 AM, Igor Galić wrote:
 I really like the basic outline, but what I feel is missing, is the
 notion of watching $issuetracker. Often we have patches getting lost
 in there. So our patch manager should also look out for $PATCH issues
 and make sure they are followed up, or assigned, or whatever.

Yes, agreed - we always do ask for an issue to be opened up, but
sometimes those can also fall through the cracks, too. The first part of
the email seems pretty straight forward since we don't seem to really
get many emails on dev@ containing patches that don't end up with an
associated issue. Having a bugzilla cop to identify duplicate issues,
pester the dev list about lingering issues, provide gentle reminders of
issues in progress if they stall, close out junk and how to tickets
and even bring up issues opened in the tracker that never had a
corresponding dev discussion would be pretty great.

I'm more than happy to help in this role, but don't always consistently
have the time available to keep as sharp an eye on the tracker as I
would like.

-- 
Daniel Ruggeri



Re: 2.3.15 RewriteRule P

2011-11-16 Thread Steffen

Mailed the error log to Stefan.

What I noticed, it is connecting to a port by a formerly used proxied 
connection (port 7080 instead of 81);


Summary log:

[proxy:debug] [pid 8680:tid 2668] proxy_util.c(2140): proxy: HTTP: has 
acquired connection for (*)
[proxy:debug] [pid 8680:tid 2668] proxy_util.c(2193): proxy: connecting 
http://127.0.0.1:81/sysadmin to 127.0.0.1:81
[proxy:debug] [pid 8680:tid 2668] proxy_util.c(2319): proxy: connected 
/sysadmin to 127.0.0.1:7080
[proxy:debug] [pid 8680:tid 2668] proxy_util.c(2510): proxy: HTTP: backend 
socket is disconnected.
[proxy:trace2] [pid 8680:tid 2668] proxy_util.c(2572): proxy: HTTP: fam 2 
socket created to connect to *
[proxy:debug] [pid 8680:tid 2668] proxy_util.c(2724): proxy: HTTP: 
connection complete to 127.0.0.1:7080 (127.0.0.1)
[proxy:debug] [pid 8680:tid 2668] proxy_util.c(2155): proxy: *: has released 
connection for (*)
[deflate:debug] [pid 8680:tid 2668] mod_deflate.c(739): [client 
127.0.0.1:53303] Zlib: Compressed 94 to 92 : URL /sysadmin


-Original Message- 
From: Stefan Fritsch

Sent: Tuesday, November 15, 2011 2:57 PM Newsgroups: gmane.comp.apache.devel
To: Steffen
Cc: dev@httpd.apache.org
Subject: Re: 2.3.15 RewriteRule P

Hi Steffen,

On Mon, 14 Nov 2011, Steffen wrote:


The issue below seems not to be related to SSL.

Tested in different non-SSL configs with eg.
RewriteRule /sysadmin(.*) http://%{HTTP_HOST}:81/sysadmin$1 [P,L]

And different servers as the back, Sambar, Surge, DManager and Apache.

Sometimes it works but mostly it is not passing (good) URL info to the 
server.

Get errors like: requested URL not found, requested file: ()


Does the 'sometimes' depend on the URL? I.e. do some URLs work and some
don't?

Can you post errorlog output with loglevel debug proxy:trace6
rewrite:trace6? Thanks.

Cheers,
Stefan


It is a minimal config with a commonly used rewrite:

Listen 443
SSLEngine on
DocumentRoot f:/web/unknown
RewriteEngine on
RewriteRule /(.*) http://%{HTTP_HOST}/$1 [P,L]

In the log no clue, only
[ssl:info] [pid 6836:tid 2588] (70014)End of file found: [client
85.223.52.177:38857] SSL input filter read failed.
But that I see also with 2.2.21

Looks like more errors when I have AcceptFilter https none instead of
leaving this out.

Steffen 




Report a Bug In truck and 2.4.x at winnt mpm, I think very important

2011-11-16 Thread zhiguo zhao
request_rec-connection-current_thread is a new feature, and
every request_rec-connection-current_thread should have different pool,
but now with a same pool,
This is need to fix.



Index: mpm/winnt/child.c
===
--- mpm/winnt/child.c   (版本 1202642)
+++ mpm/winnt/child.c   (工作副本)
@@ -754,9 +754,11 @@
 int rc;
 conn_rec *c;
 apr_int32_t disconnected;
+apr_pool_t* self;

 osthd = apr_os_thread_current();
-apr_os_thread_put(thd, osthd, pchild);
+apr_pool_create(self,pchild);
+apr_os_thread_put(thd, osthd, self);

 while (1) {

@@ -858,6 +860,7 @@

 ap_update_child_status_from_indexes(0, thread_num, SERVER_DEAD,
 (request_rec *) NULL);
+apr_pool_destroy(self);

 return 0;
 }


Re: Windows build bot

2011-11-16 Thread Jim Jagielski
I've posted something on the ooo-dev@incubator list asking for
volunteers to help us with httpd-2.4 builds on Windows, btw.

On Nov 16, 2011, at 3:31 AM, Igor Galić wrote:

 
 
 - Original Message -
 Hi,
 
 we were told by pgollucci that there is already a Windows build bot
 at the
 ASF. It would be awesome if someone knowledgable in Windows could
 work
 with the Infra guys to set it up. The mail address is
 bui...@apache.org
 according to Philip.
 
 +++1! If all else fails, open a Jira, and/or bug Gavin directly.
 Or buy him a beer. That should help :)
 
 Cheers,
 Stefan
 
 i
 
 --
 Igor Galić
 
 Tel: +43 (0) 664 886 22 883
 Mail: i.ga...@brainsware.org
 URL: http://brainsware.org/
 GPG: 6880 4155 74BD FD7C B515  2EA5 4B1D 9E08 A097 C9AE
 



Re: Report a Bug In truck and 2.4.x at winnt mpm, I think very important

2011-11-16 Thread zhiguo zhao
Give chance to other module save thread-scope information, Please.

在 2011年11月16日 下午8:20,zhiguo zhao zha...@gmail.com写道:

 request_rec-connection-current_thread is a new feature, and
 every request_rec-connection-current_thread should have different pool,
 but now with a same pool,
 This is need to fix.



 Index: mpm/winnt/child.c
 ===
 --- mpm/winnt/child.c   (版本 1202642)
 +++ mpm/winnt/child.c   (工作副本)
 @@ -754,9 +754,11 @@
  int rc;
  conn_rec *c;
  apr_int32_t disconnected;
 +apr_pool_t* self;

  osthd = apr_os_thread_current();
 -apr_os_thread_put(thd, osthd, pchild);
 +apr_pool_create(self,pchild);
 +apr_os_thread_put(thd, osthd, self);

  while (1) {

 @@ -858,6 +860,7 @@

  ap_update_child_status_from_indexes(0, thread_num, SERVER_DEAD,
  (request_rec *) NULL);
 +apr_pool_destroy(self);

  return 0;
  }



Re: svn commit: r1202257 - in /httpd/httpd/trunk/server/mpm/event: config3.m4 equeue.c equeue.h event.c

2011-11-16 Thread Paul Querna
On Wed, Nov 16, 2011 at 1:20 AM, Rüdiger Plüm
ruediger.pl...@vodafone.com wrote:


  Original-Nachricht 
 Betreff: svn commit: r1202257 - in /httpd/httpd/trunk/server/mpm/event: 
 config3.m4 equeue.c equeue.h event.c
 Datum: Tue, 15 Nov 2011 15:51:04 GMT
 Von: pque...@apache.org

 Author: pquerna
 Date: Tue Nov 15 15:51:03 2011
 New Revision: 1202257

 URL: http://svn.apache.org/viewvc?rev=1202257view=rev
 Log:
 Create a new lock free circular queue, and use it in the EventMPM to remove 
 the timeout mutex
 that was wrapping both timeout queue operations and pollset operations.

 Added:
 httpd/httpd/trunk/server/mpm/event/equeue.c   (with props)
 httpd/httpd/trunk/server/mpm/event/equeue.h   (with props)
 Modified:
 httpd/httpd/trunk/server/mpm/event/config3.m4
 httpd/httpd/trunk/server/mpm/
 event/event.c

 Modified: httpd/httpd/trunk/server/mpm/event/config3.m4
 URL: 
 http://svn.apache.org/viewvc/httpd/httpd/trunk/server/mpm/event/equeue.c?rev=1202257view=auto
 ==
 --- httpd/httpd/trunk/server/mpm/event/equeue.c (added)
 +++ httpd/httpd/trunk/server/mpm/event/equeue.c Tue Nov 15 15:51:03 2011
 @@ -0,0 +1,125 @@
 +/* Licensed to the Apache Software Foundation (ASF) under one or more
 + * contributor license agreements.  See the NOTICE file distributed with
 + * this work for additional information regarding copyright ownership.
 + * The ASF licenses this file to You under the Apache License, Version 2.0
 + * (the License); you may not use this file except in compliance with
 + * the License.  You may obtain a copy of the License at
 + *
 + * http://www.apache.org/licenses/LICENSE-2.0
 + *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an AS IS BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
 + */
 +
 +#include equeue.h
 +
 +#include apr_atomic.h
 +#include sched.h
 +
 +struct ap_equeue_t {
 +apr_uint32_t nelem;
 +apr_size_t elem_size;
 +uint8_t *bytes;
 +volatile apr_uint32_t writeCount;
 +volatile apr_uint32_t readCount;
 +};
 +
 +
 +static APR_INLINE apr_uint32_t count_to_index(ap_equeue_t *eq, apr_uint32_t 
 count)
 +{
 +return (count  (eq-nelem - 1));
 +}
 +
 +static APR_INLINE void* index_to_bytes(ap_equeue_t *eq, apr_uint32_t idx)
 +{
 +apr_size_t offset = idx * eq-elem_size;
 +return (void*)eq-bytes[offset];
 +}
 +
 +static APR_INLINE apr_uint32_t nearest_power(apr_uint32_t num)
 +{
 +apr_uint32_t n = 1;
 +while (n  num) {
 +n = 1;
 +}
 +
 +return n;
 +}
 +
 +#if 0
 +static void dump_queue(ap_equeue_t *eq)
 +{
 +apr_uint32_t i;
 +
 +fprintf(stderr, dumping %p\n, eq);
 +fprintf(stderr,   nelem:   %u\n, eq-nelem);
 +fprintf(stderr,   esize:   %APR_SIZE_T_FMT\n, eq-elem_size);
 +fprintf(stderr,   wcnt:%u\n, eq-writeCount);
 +fprintf(stderr,   rcnt:%u\n, eq-writeCount);
 +fprintf(stderr,   bytes:   %p\n, eq-bytes);
 +for (i = 0; i  eq-nelem; i++) {
 +fprintf(stderr, [%u] = %p\n, i, index_to_bytes(eq, i));
 +}
 +
 +fprintf(stderr, \n);
 +fflush(stderr);
 +}
 +#endif
 +
 +apr_status_t
 +ap_equeue_create(apr_pool_t *p, apr_uint32_t nelem, apr_size_t elem_size, 
 ap_equeue_t **eqout)
 +{
 +ap_equeue_t *eq;
 +
 +*eqout = NULL;
 +
 +eq = apr_palloc(p, sizeof(ap_equeue_t));
 +eq-bytes = apr_palloc(p, (1 + nelem) * elem_size);
 +eq-nelem = nearest_power(nelem);
 +eq-elem_size = elem_size;
 +eq-writeCount = 0;
 +eq-readCount = 0;
 +*eqout = eq;
 +
 +return APR_SUCCESS;
 +}
 +
 +void *
 +ap_equeue_reader_next(ap_equeue_t *eq)
 +{
 +if (apr_atomic_read32(eq-writeCount) == eq-readCount) {
 +return NULL;
 +}
 +else {
 +apr_uint32_t idx = count_to_index(eq, 
 apr_atomic_inc32(eq-readCount));
 +return index_to_bytes(eq, idx);
 +}
 +}
 +
 +void *
 +ap_equeue_writer_value(ap_equeue_t *eq)
 +{
 +apr_uint32_t idx;
 +
 +while (1) {
 +apr_uint32_t readCount = apr_atomic_read32(eq-readCount);
 +
 +if (count_to_index(eq, eq-writeCount + 1) != count_to_index(eq, 
 readCount)) {
 +break;
 +}
 +/* TODO: research if sched_yield is even worth doing  */
 +sched_yield();
 +}
 +
 +idx = count_to_index(eq, eq-writeCount);
 +return index_to_bytes(eq, idx);
 +}

 I assume that only a single thread tries write operations on this queue, 
 correct?
 Otherwise it seems unsafe if another thread calls
 ap_equeue_writer_value in parallel as it would return the same slot until
 ap_equeue_writer_onward was called.

Correct, it only supports a single writer and  a single reader, which
lets you get away with only a few atomic operations and no 

Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Paul Querna
On Wed, Nov 16, 2011 at 2:44 AM, Rainer Jung rainer.j...@kippdata.de wrote:
 On 15.11.2011 20:57, Jeff Trawick wrote:

 On Tue, Nov 15, 2011 at 2:32 PM, William A. Rowe Jr.
 wr...@rowe-clan.net  wrote:

 On 11/15/2011 12:33 PM, Stefan Fritsch wrote:

 On Tuesday 15 November 2011, Paul Querna wrote:

 On Tue, Nov 15, 2011 at 9:17 AM, Stefan Fritschs...@sfritsch.de

 wrote:

 On Tue, 15 Nov 2011, pque...@apache.org wrote:

 Author: pquerna
 Date: Tue Nov 15 15:49:19 2011
 New Revision: 1202255

 URL: http://svn.apache.org/viewvc?rev=1202255view=rev
 Log:
 disable mod_reqtimeout if not configured

 Why that? We have just changed the default to be enabled in
 r1199447 and several developers at the hackathon agreed to this
 change.

 Didn't know it was discussed in depth at the hackathon, and there
 wasn't any discussion on the list

 It showed up quite quickly in my profiling of the Event MPM,
 because every pull/push on the filters would cause a
 apr_time_now() call.

 I don't really like that just by loading the module, it changes the
 behavior and performance of the server so drastically.

 It only acts on reads from the client. Normal non-POST requests arrive
 in one or two packets, which would mean approx. 3 additional
 apr_time_now calls per request. I haven't done benchmarks, but I can't
 imagine that this has a drastic impact on performance. And if it costs
 1-2%, then that's a small cost compared to the impact of slowloris
 type attacks which eat lots of memory.

 The general intention of the recent changes in default configs and
 module selection/loading was to make it easier to only load those
 modules that are really needed, have a reasonable default config, and
 have the compiled-in default values be the same as those in the
 example config files.

 Which means, build by default, disable by default.  I think that keeps
 everyone happy.  When abuse arrives, it's trivial to load.

 Timeout 60 isn't nearly as bad as the old Timeout 300 that is probably
 still in wide use, but mod_reqtimeout can provide a much more
 reasonable out of the box configuration.  I think we should keep it in
 place by default.

 +1


What I am really opposed to is that the LoadModule causes such a
degradation in performance.

I am 100% +1 to adding conf commands to the default configuration in
the httpd.conf, but what I do not like is that having just a
LoadModule with nothing else causes reqtimeout to do work.  It is too
trivial for people to have accidental load modules in many distros.


Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Jeff Trawick
On Wed, Nov 16, 2011 at 11:20 AM, Paul Querna p...@querna.org wrote:
 On Wed, Nov 16, 2011 at 2:44 AM, Rainer Jung rainer.j...@kippdata.de wrote:
 On 15.11.2011 20:57, Jeff Trawick wrote:

 On Tue, Nov 15, 2011 at 2:32 PM, William A. Rowe Jr.
 wr...@rowe-clan.net  wrote:

 On 11/15/2011 12:33 PM, Stefan Fritsch wrote:

 On Tuesday 15 November 2011, Paul Querna wrote:

 On Tue, Nov 15, 2011 at 9:17 AM, Stefan Fritschs...@sfritsch.de

 wrote:

 On Tue, 15 Nov 2011, pque...@apache.org wrote:

 Author: pquerna
 Date: Tue Nov 15 15:49:19 2011
 New Revision: 1202255

 URL: http://svn.apache.org/viewvc?rev=1202255view=rev
 Log:
 disable mod_reqtimeout if not configured

 Why that? We have just changed the default to be enabled in
 r1199447 and several developers at the hackathon agreed to this
 change.

 Didn't know it was discussed in depth at the hackathon, and there
 wasn't any discussion on the list

 It showed up quite quickly in my profiling of the Event MPM,
 because every pull/push on the filters would cause a
 apr_time_now() call.

 I don't really like that just by loading the module, it changes the
 behavior and performance of the server so drastically.

 It only acts on reads from the client. Normal non-POST requests arrive
 in one or two packets, which would mean approx. 3 additional
 apr_time_now calls per request. I haven't done benchmarks, but I can't
 imagine that this has a drastic impact on performance. And if it costs
 1-2%, then that's a small cost compared to the impact of slowloris
 type attacks which eat lots of memory.

 The general intention of the recent changes in default configs and
 module selection/loading was to make it easier to only load those
 modules that are really needed, have a reasonable default config, and
 have the compiled-in default values be the same as those in the
 example config files.

 Which means, build by default, disable by default.  I think that keeps
 everyone happy.  When abuse arrives, it's trivial to load.

 Timeout 60 isn't nearly as bad as the old Timeout 300 that is probably
 still in wide use, but mod_reqtimeout can provide a much more
 reasonable out of the box configuration.  I think we should keep it in
 place by default.

 +1


 What I am really opposed to is that the LoadModule causes such a
 degradation in performance.

 I am 100% +1 to adding conf commands to the default configuration in
 the httpd.conf, but what I do not like is that having just a
 LoadModule with nothing else causes reqtimeout to do work.  It is too
 trivial for people to have accidental load modules in many distros.

How many distinct concerns are there?  (Cut through the detail that
the implementation of request phase-specific timeouts is in a module,
and hence conflicts with thoughts for what it means to have code
loaded by LoadModule but not explicitly configured.)  I see at least
two raised.

1) performance degradation out of the box with no explicit configuration*
2) any processing at all out of the box with no explicit configuration
3) relatively complex-to-understand timeout mechanism out of the box
with no explicit configuration (independent of whether it is core or
mod_)
4) disagreement that the mod_reqtimeout defaults are generally
better across arbitrary situations than Timeout somenumber
5) ???

(I assume that the performance issue related to the apr_time_now()
calls in mod_reqtimeout is addressable, and probably in a way that
helps other modules that don't need to know the time with high
precision.)


Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Jim Jagielski

On Nov 16, 2011, at 11:20 AM, Paul Querna wrote:
 
 What I am really opposed to is that the LoadModule causes such a
 degradation in performance.
 
 I am 100% +1 to adding conf commands to the default configuration in
 the httpd.conf, but what I do not like is that having just a
 LoadModule with nothing else causes reqtimeout to do work.  It is too
 trivial for people to have accidental load modules in many distros.
 

+1


Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Jeff Trawick
On Wed, Nov 16, 2011 at 11:51 AM, Jeff Trawick traw...@gmail.com wrote:
 On Wed, Nov 16, 2011 at 11:20 AM, Paul Querna p...@querna.org wrote:
 On Wed, Nov 16, 2011 at 2:44 AM, Rainer Jung rainer.j...@kippdata.de wrote:
 On 15.11.2011 20:57, Jeff Trawick wrote:

 On Tue, Nov 15, 2011 at 2:32 PM, William A. Rowe Jr.
 wr...@rowe-clan.net  wrote:

 On 11/15/2011 12:33 PM, Stefan Fritsch wrote:

 On Tuesday 15 November 2011, Paul Querna wrote:

 On Tue, Nov 15, 2011 at 9:17 AM, Stefan Fritschs...@sfritsch.de

 wrote:

 On Tue, 15 Nov 2011, pque...@apache.org wrote:

 Author: pquerna
 Date: Tue Nov 15 15:49:19 2011
 New Revision: 1202255

 URL: http://svn.apache.org/viewvc?rev=1202255view=rev
 Log:
 disable mod_reqtimeout if not configured

 Why that? We have just changed the default to be enabled in
 r1199447 and several developers at the hackathon agreed to this
 change.

 Didn't know it was discussed in depth at the hackathon, and there
 wasn't any discussion on the list

 It showed up quite quickly in my profiling of the Event MPM,
 because every pull/push on the filters would cause a
 apr_time_now() call.

 I don't really like that just by loading the module, it changes the
 behavior and performance of the server so drastically.

 It only acts on reads from the client. Normal non-POST requests arrive
 in one or two packets, which would mean approx. 3 additional
 apr_time_now calls per request. I haven't done benchmarks, but I can't
 imagine that this has a drastic impact on performance. And if it costs
 1-2%, then that's a small cost compared to the impact of slowloris
 type attacks which eat lots of memory.

 The general intention of the recent changes in default configs and
 module selection/loading was to make it easier to only load those
 modules that are really needed, have a reasonable default config, and
 have the compiled-in default values be the same as those in the
 example config files.

 Which means, build by default, disable by default.  I think that keeps
 everyone happy.  When abuse arrives, it's trivial to load.

 Timeout 60 isn't nearly as bad as the old Timeout 300 that is probably
 still in wide use, but mod_reqtimeout can provide a much more
 reasonable out of the box configuration.  I think we should keep it in
 place by default.

 +1


 What I am really opposed to is that the LoadModule causes such a
 degradation in performance.

 I am 100% +1 to adding conf commands to the default configuration in
 the httpd.conf, but what I do not like is that having just a
 LoadModule with nothing else causes reqtimeout to do work.  It is too
 trivial for people to have accidental load modules in many distros.

 How many distinct concerns are there?  (Cut through the detail that
 the implementation of request phase-specific timeouts is in a module,
 and hence conflicts with thoughts for what it means to have code
 loaded by LoadModule but not explicitly configured.)  I see at least
 two raised.

 1) performance degradation out of the box with no explicit configuration*
 2) any processing at all out of the box with no explicit configuration
 3) relatively complex-to-understand timeout mechanism out of the box
 with no explicit configuration (independent of whether it is core or
 mod_)
 4) disagreement that the mod_reqtimeout defaults are generally
 better across arbitrary situations than Timeout somenumber
 5) ???

 (I assume that the performance issue related to the apr_time_now()
 calls in mod_reqtimeout is addressable, and probably in a way that
 helps other modules that don't need to know the time with high
 precision.)

See attached patch for one possible fix (completely untested :) ).  It
should be determined (even if by changing code) that event's listener
thread can update the timestamp in the child at least every 200ms.
ap_time_around_now(), ap_roughly_time_now(), etc. might be better API
names.  Over the long term it might be helpful to have a couple of
versions with different requirements, as some MPMs will simply punt to
apr_time_now() if asked to provide something within 200ms.
Index: server/mpm/event/event.c
===
--- server/mpm/event/event.c(revision 1202829)
+++ server/mpm/event/event.c(working copy)
@@ -1381,6 +1381,18 @@
 }
 }
 
+static apr_time_t roughly_now; /* set only by listener thread */
+
+static int event_get_time_200(apr_time_t *now)
+{
+if (roughly_now == 0) {
+return DECLINED;
+}
+
+*now = roughly_now;
+return OK;
+}
+
 static void * APR_THREAD_FUNC listener_thread(apr_thread_t * thd, void *dummy)
 {
 timer_event_t *ep;
@@ -1397,7 +1409,7 @@
 const apr_pollfd_t *out_pfd;
 apr_int32_t num = 0;
 apr_interval_time_t timeout_interval;
-apr_time_t timeout_time = 0, now, last_log;
+apr_time_t timeout_time = 0, last_log;
 listener_poll_type *pt;
 int closed = 0, listeners_disabled = 0;
 
@@ -1440,11 +1452,11 @@
 

Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Jeff Trawick
On Wed, Nov 16, 2011 at 2:23 PM, Jeff Trawick traw...@gmail.com wrote:
 On Wed, Nov 16, 2011 at 11:51 AM, Jeff Trawick traw...@gmail.com wrote:
 On Wed, Nov 16, 2011 at 11:20 AM, Paul Querna p...@querna.org wrote:
 On Wed, Nov 16, 2011 at 2:44 AM, Rainer Jung rainer.j...@kippdata.de 
 wrote:
 On 15.11.2011 20:57, Jeff Trawick wrote:

 On Tue, Nov 15, 2011 at 2:32 PM, William A. Rowe Jr.
 wr...@rowe-clan.net  wrote:

 On 11/15/2011 12:33 PM, Stefan Fritsch wrote:

 On Tuesday 15 November 2011, Paul Querna wrote:

 On Tue, Nov 15, 2011 at 9:17 AM, Stefan Fritschs...@sfritsch.de

 wrote:

 On Tue, 15 Nov 2011, pque...@apache.org wrote:

 Author: pquerna
 Date: Tue Nov 15 15:49:19 2011
 New Revision: 1202255

 URL: http://svn.apache.org/viewvc?rev=1202255view=rev
 Log:
 disable mod_reqtimeout if not configured

 Why that? We have just changed the default to be enabled in
 r1199447 and several developers at the hackathon agreed to this
 change.

 Didn't know it was discussed in depth at the hackathon, and there
 wasn't any discussion on the list

 It showed up quite quickly in my profiling of the Event MPM,
 because every pull/push on the filters would cause a
 apr_time_now() call.

 I don't really like that just by loading the module, it changes the
 behavior and performance of the server so drastically.

 It only acts on reads from the client. Normal non-POST requests arrive
 in one or two packets, which would mean approx. 3 additional
 apr_time_now calls per request. I haven't done benchmarks, but I can't
 imagine that this has a drastic impact on performance. And if it costs
 1-2%, then that's a small cost compared to the impact of slowloris
 type attacks which eat lots of memory.

 The general intention of the recent changes in default configs and
 module selection/loading was to make it easier to only load those
 modules that are really needed, have a reasonable default config, and
 have the compiled-in default values be the same as those in the
 example config files.

 Which means, build by default, disable by default.  I think that keeps
 everyone happy.  When abuse arrives, it's trivial to load.

 Timeout 60 isn't nearly as bad as the old Timeout 300 that is probably
 still in wide use, but mod_reqtimeout can provide a much more
 reasonable out of the box configuration.  I think we should keep it in
 place by default.

 +1


 What I am really opposed to is that the LoadModule causes such a
 degradation in performance.

 I am 100% +1 to adding conf commands to the default configuration in
 the httpd.conf, but what I do not like is that having just a
 LoadModule with nothing else causes reqtimeout to do work.  It is too
 trivial for people to have accidental load modules in many distros.

 How many distinct concerns are there?  (Cut through the detail that
 the implementation of request phase-specific timeouts is in a module,
 and hence conflicts with thoughts for what it means to have code
 loaded by LoadModule but not explicitly configured.)  I see at least
 two raised.

 1) performance degradation out of the box with no explicit configuration*
 2) any processing at all out of the box with no explicit configuration
 3) relatively complex-to-understand timeout mechanism out of the box
 with no explicit configuration (independent of whether it is core or
 mod_)
 4) disagreement that the mod_reqtimeout defaults are generally
 better across arbitrary situations than Timeout somenumber
 5) ???

 (I assume that the performance issue related to the apr_time_now()
 calls in mod_reqtimeout is addressable, and probably in a way that
 helps other modules that don't need to know the time with high
 precision.)

 See attached patch for one possible fix (completely untested :) ).  It
 should be determined (even if by changing code) that event's listener
 thread can update the timestamp in the child at least every 200ms.
 ap_time_around_now(), ap_roughly_time_now(), etc. might be better API
 names.  Over the long term it might be helpful to have a couple of
 versions with different requirements, as some MPMs will simply punt to
 apr_time_now() if asked to provide something within 200ms.

oops, I forgot to 0 out roughly_now during special situations (maybe
just exiting?); its just a concept patch ;)


Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Stefan Fritsch
On Wednesday 16 November 2011, Paul Querna wrote:
 On Wed, Nov 16, 2011 at 2:44 AM, Rainer Jung 
rainer.j...@kippdata.de wrote:
  On 15.11.2011 20:57, Jeff Trawick wrote:
  On Tue, Nov 15, 2011 at 2:32 PM, William A. Rowe Jr.
  
  wr...@rowe-clan.net  wrote:
  On 11/15/2011 12:33 PM, Stefan Fritsch wrote:
  On Tuesday 15 November 2011, Paul Querna wrote:
  On Tue, Nov 15, 2011 at 9:17 AM, Stefan
  Fritschs...@sfritsch.de
  
  wrote:
  On Tue, 15 Nov 2011, pque...@apache.org wrote:
  Author: pquerna
  Date: Tue Nov 15 15:49:19 2011
  New Revision: 1202255
  
  URL: http://svn.apache.org/viewvc?rev=1202255view=rev
  Log:
  disable mod_reqtimeout if not configured
  
  Why that? We have just changed the default to be enabled in
  r1199447 and several developers at the hackathon agreed to
  this change.
  
  Didn't know it was discussed in depth at the hackathon, and
  there wasn't any discussion on the list
  
  It showed up quite quickly in my profiling of the Event MPM,
  because every pull/push on the filters would cause a
  apr_time_now() call.
  
  I don't really like that just by loading the module, it
  changes the behavior and performance of the server so
  drastically.
  
  It only acts on reads from the client. Normal non-POST
  requests arrive in one or two packets, which would mean
  approx. 3 additional apr_time_now calls per request. I
  haven't done benchmarks, but I can't imagine that this has a
  drastic impact on performance. And if it costs 1-2%, then
  that's a small cost compared to the impact of slowloris type
  attacks which eat lots of memory.
  
  The general intention of the recent changes in default configs
  and module selection/loading was to make it easier to only
  load those modules that are really needed, have a reasonable
  default config, and have the compiled-in default values be
  the same as those in the example config files.
  
  Which means, build by default, disable by default.  I think
  that keeps everyone happy.  When abuse arrives, it's trivial
  to load.
  
  Timeout 60 isn't nearly as bad as the old Timeout 300 that is
  probably still in wide use, but mod_reqtimeout can provide a
  much more reasonable out of the box configuration.  I think we
  should keep it in place by default.
  
  +1
 
 What I am really opposed to is that the LoadModule causes such a
 degradation in performance.

In my quick (and maybe not that accurate) tests, the penalty caused by 
mod_reqtimeout (1.4%) was smaller than the penalty of not having 
BufferedLogs on (2.1%). But there is no question that the current 
default 'BufferedLogs off' is better for the  majority of users, just 
the same as having reqtimeout enabled is better for the majority of 
users. Actually, RequestReadTimeout should probably be into the core, 
but none got around to moving it.

 I am 100% +1 to adding conf commands to the default configuration
 in the httpd.conf, but what I do not like is that having just a
 LoadModule with nothing else causes reqtimeout to do work.  It is
 too trivial for people to have accidental load modules in many
 distros.

There are other cases where just loading a module can have much larger 
influence on the performance (look at AP_AUTH_INTERNAL_PER_URI versus 
AP_AUTH_INTERNAL_PER_CONF in the auth hooks). We need to educate users 
that they should disable modules that they don't need.


Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Stefan Fritsch
On Wednesday 16 November 2011, Stefan Fritsch wrote:
  What I am really opposed to is that the LoadModule causes such a
  degradation in performance.
 
 In my quick (and maybe not that accurate) tests, the penalty caused
 by  mod_reqtimeout (1.4%) was smaller than the penalty of not
 having BufferedLogs on (2.1%). But there is no question that the
 current default 'BufferedLogs off' is better for the  majority of
 users, just the same as having reqtimeout enabled is better for
 the majority of users. Actually, RequestReadTimeout should
 probably be into the core, but none got around to moving it.

I did some more accurate benchmarks on a dual core AMD E350, which is 
slow enough that it can be saturated with a simple ab run from a fast 
machine over a 1GB/s link. The numbers are reproducible over several 
runs with less than 0.5% variation.

Requesting a zero-size file
ab -c 100 -n 10 http://10.1.1.20:8081/zero
Trunk r1202537
Default configure, but CFLAGS=-O3
Default config, but
StartServers 2
MinSpareThreads 20
MaxSpareThreads100
ThreadLimit200
ThreadsPerChild 50
MaxRequestWorkers  600

The reqtimeout config was
RequestReadTimeout header=20-40,MinRate=500 body=20,MinRate=500
if enabled

Results:

reqtimeout disabled, buffered logs off 4060 100%
mod_reqtimeout not loaded, buffered logs off   4090 100.7%
reqtimeout disabled, buffered logs on  4378 107.8%
reqtimeout enabled, buffered logs on   4299 105.9%
reqtimeout enabled, buffered logs off  3934  96.9%
reqtimeout enabled+r1202886, buffered logs off 3996  98.5%

Even without r1202886, I think the degradation would have been 
acceptable. With r1202886 it's even less of a problem.

To put the number into perspective, if I request the default 45 byte 
index.html instead, the request rate drops by approx. 16%.



Re: setting TZ env var

2011-11-16 Thread William A. Rowe Jr.

On 11/16/2011 4:39 AM, Rainer Jung wrote:


I'd say it is OK. I assume a apachectl restart would do?
It should be expected for a long running daemon to signal it to adopt to 
environmental
changes like the timezone.


Yup.  Not that the times are wrong, they carry zone info within
the access logs.

However, our error log timestamps appear to be bullshit.  We don't
carry the timezone delta.  Ergo the DST-ST shift can't be delineated
anyways.  So the existing log time logic is broken.

+1 to the whole concept provided that error logging adopts access.log
time representation.



Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread William A. Rowe Jr.

On 11/16/2011 10:20 AM, Paul Querna wrote:


I am 100% +1 to adding conf commands to the default configuration in
the httpd.conf, but what I do not like is that having just a
LoadModule with nothing else causes reqtimeout to do work.  It is too
trivial for people to have accidental load modules in many distros.


+1; in this case it will be tricky.  This would be a per listener
or per phys-vhost option.  Although it could be a per-named-vhost
option, it makes next to no sense during the request headers input
phase when no host: will be parsed.

Will people be confused by a directive that does nothing when placed
in a named vhost?  Or should this just be global-only and forget
about it?  It would be nice to cripple this module, say, for the
intranet side of a server, and enable the overhead only on the
external side.

There's also nothing wrong with having it default to enabled, provided
the admin can disable it most of the time if they choose, for speed.



Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread William A. Rowe Jr.

Were these results all based on 100% cpu pegged?



Re: Apache shared memory implementation

2011-11-16 Thread William A. Rowe Jr.

On 11/16/2011 2:03 PM, Pranesh Vadhirajan wrote:


I am trying to understand how to implement sharing of objects in memory between 
various
processes in apache. I have been using the routines defined in sys/mman.h to 
implement
the shared memory mapping functionality. However, I am not seeing any actual 
sharing of
the objects in my output.


Hi Pranesh; you are more likely to get help with this on a linux
developer forum, since there is nothing Apache specific here.

You might want to look at the httpd scoreboard implementation,
or better yet, mod_socache_shm.c in the new betas, none of which
use the linux/unix shm conventions.  These are all based on the
portability layer in APR, so that the shared objects work on
additional platforms.


Death of 2.3-HEAD, 2.1-HEAD (bugzilla tags)

2011-11-16 Thread William A. Rowe Jr.

All bugs categorized as 2.3-HEAD apply to 2.5-HEAD, so I have
simply renamed the tag.

There is no such entity 2.1-HEAD, as it became 2.2.0... but that
category is a mishmash of 2.0 specific bugs, 2.2 specific bugs,
and those that still apply to 2.5-HEAD.

Anyone who is willing to help out, I'd love to delete this tag
as well, just as soon as this version bug list is empty;

https://issues.apache.org/bugzilla/buglist.cgi?version=2.1-HEAD;product=Apache%20httpd-2



Re: Apache shared memory implementation

2011-11-16 Thread Sorin Manolache
On Wed, Nov 16, 2011 at 21:03, Pranesh Vadhirajan
vadhira...@teralogics.com wrote:
 Hello Everyone,

  I am trying to understand how to implement sharing of objects in memory
 between various processes in apache.  I have been using the routines defined
 in sys/mman.h to implement the shared memory mapping functionality.
 However, I am not seeing any actual sharing of the objects in my output.



 I have attached the code segment of my module containing the request
 handler:



 #include httpd.h

 #include http_core.h

 #include http_config.h

 #include stdlib.h

 #include time.h

 #include sys/time.h

 #include sys/types.h

 #include sessions.h

 #include regex.h

 #include stdio.h

 #include sys/mman.h

 #include sys/stat.h

 #include fcntl.h

 #include unistd.h





 void set_shared_region(int *shared_int)

 {

    int fd;



    /* Create shared memory object and set its size */

    fd = shm_open(/myregion, O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);

    if(fd == -1)

   fprintf(stderr,shm_open error);



    if(ftruncate(fd, sizeof(int)) == -1)

    fprintf(stderr,ftruncate error);



     /* Map shared memory object */

    shared_int = mmap(NULL, sizeof(int),PROT_READ | PROT_WRITE, MAP_SHARED,
 fd, 0);

    if(shared_int == MAP_FAILED)

   fprintf(stderr,mmap error);

 }



 int start_mutex()

 {

    if(initial_mutex_value == 0)

    {

   if(sem_init(session_mutex,1,1)  0)

   {

  fprintf(stderr,error initializing semaphore);

  return 0;

   }

   initial_mutex_value = 1;

    }

    return 1;

 }



 static int counter_handler(request_rec* r)

 {

    char time_buffer[30];

    char entry[1024];

    long int tid,pid;

    apr_ctime(time_buffer,r-request_time);

    start_mutex();

    sem_wait(session_mutex);

    set_shared_region(ctr);

    ctr++;

    sem_post(session_mutex);

    //tid = (long int)getthreadid();

    pid = (long int)getpid();

    sprintf(entry,counter: %i, thread: %ld, process: %ld, request: %s,
 time: %s;,ctr,(long int)0,pid,r-the_request,time_buffer);

    tempToDB(entry);



    return DECLINED;

 }



 static void counter_register_hooks (apr_pool_t *p)

 {

    ap_hook_handler(counter_handler, NULL, NULL, APR_HOOK_REALLY_FIRST);

 }



 module AP_MODULE_DECLARE_DATA counter_module =

 {

   STANDARD20_MODULE_STUFF,

   NULL,//uvds_metrics_dir_conf,  /* Per-Directory Configuration */

   NULL,//uvds_metrics_dir_merge,   /* Directory Config Merger */

   NULL,//uvds_metrics_server_conf, /* Per-Server Configuration */

   NULL,//uvds_metrics_server_merge,  /* Server Config Merger */

   NULL,//uvds_metrics_cmds,    /* Command Table (Directives) */

   counter_register_hooks /* Registering Hooks */

 };



 The handler code uses the variable ctr which is defined in the header file
 “sessions.h” .  Here’s the relevant segment in sessions.h:



         #include stdio.h

 #include stdlib.h

 #include string.h

 #include libpq-fe.h

 #include semaphore.h



 sem_t session_mutex;

 int initial_mutex_value = 0;

 int ctr = 0;



 void tempToDB(char *entry);







 Here’s the output of my code:



     64201;counter: 1, thread:
 0, process: 18194, request: POST /login?destination=login HTTP/1.1,  time:
 Wed Nov 16 19:56:04 2011;

 64202;counter: 2, thread: 0, process: 18194, request: POST
 /login?destination=login HTTP/1.1,  time: Wed Nov 16 19:56:04 2011;

 64203;counter: 3, thread: 0, process: 18194, request: GET /login HTTP/1.1,
 time: Wed Nov 16 19:56:05 2011;

 64204;counter: 4, thread: 0, process: 18194, request: GET /login HTTP/1.1,
 time: Wed Nov 16 19:56:05 2011;

 64205;counter: 5, thread: 0, process: 18194, request: GET /dashboard
 HTTP/1.1,  time: Wed Nov 16 19:56:06 2011;

 64206;counter: 6, thread: 0, process: 18194, request: GET /dashboard
 HTTP/1.1,  time: Wed Nov 16 19:56:06 2011;

 64207;counter: 7, thread: 0, process: 18194, request: GET /viewfeeds
 HTTP/1.1,  time: Wed Nov 16 19:56:07 2011;

 64208;counter: 8, thread: 0, process: 18194, request: GET /viewfeeds
 HTTP/1.1,  time: Wed Nov 16 19:56:07 2011;

 64209;counter: 9, thread: 0, process: 18194, request: GET
 /viewfeeds/dosortsortby=none HTTP/1.1,  time: Wed Nov 16 19:56:07 2011;

 64210;counter: 10, thread: 0, process: 18194, request: GET
 /viewfeeds/dosortsortby=none HTTP/1.1,  time: Wed Nov 16 19:56:07 2011;

 64211;counter: 1, thread: 0, process: 18201, request: GET
 /misc/viewfeeds/images/sidebar_05.png HTTP/1.1,  time: Wed Nov 16 19:56:07
 2011;

 64212;counter: 2, thread: 0, process: 18201, request: GET
 /misc/viewfeeds/images/sidebar_05.png HTTP/1.1,  time: Wed Nov 16 19:56:07
 2011;

 64213;counter: 11, thread: 0, process: 18194, request: GET /recordings
 HTTP/1.1,  time: Wed Nov 16 19:56:08 2011;

 64214;counter: 12, thread: 0, process: 18194, request: GET /recordings
 HTTP/1.1,  time: Wed Nov 16 19:56:08 2011;

 

Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Stefan Fritsch
On Wednesday 16 November 2011, William A. Rowe Jr. wrote:
 Were these results all based on 100% cpu pegged?

Yes, the machine running httpd had both cores at 100%. The machine 
running ab had a single core at ~ 70%.



Re: svn commit: r428374 - /httpd/httpd/branches/httpd-proxy-scoreboard/modules/mem/sharedmem_util.c

2011-11-16 Thread William A. Rowe Jr.

On 8/3/2006 7:11 AM, jfcl...@apache.org wrote:

Author: jfclere
Date: Thu Aug  3 05:11:14 2006
New Revision: 428374

URL: http://svn.apache.org/viewvc?rev=428374view=rev
Log:
A presistence to the shared memory slotmem.


Yes, that's from the wayback machine but applies as a roadblock
to shipping 2.4.0...


+static const char *store_filename(apr_pool_t *pool, const char *slotmemname)
+{
+const char *storename;
+const char *fname;
+if (strcmp(slotmemname, anonymous) == 0)
+fname = ap_server_root_relative(pool, logs/anonymous);


anonymous this is not.  An anonymous shm has no name.  On some platforms
it is backed by swap, on NO PLATFORM is it addressable without being
inherited.  So if we have four 'anonymous' shm's they are going to collide
with each other?  The defect persists to this day.

I rather suspect this design failure might be the root of the whole
win32 issue, because win32 is create-process, not fork()ed, and yet
perhaps we have collided by-name, but I haven't had cycles to research.
Maybe this turkey holiday is going to give me cycles to work on the
parts of httpd I need to stick my fingers into.  And pie.


Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread Stefan Fritsch
On Wednesday 16 November 2011, William A. Rowe Jr. wrote:
 On 11/16/2011 10:20 AM, Paul Querna wrote:
  I am 100% +1 to adding conf commands to the default configuration
  in the httpd.conf, but what I do not like is that having just a
  LoadModule with nothing else causes reqtimeout to do work.  It
  is too trivial for people to have accidental load modules in
  many distros.
 
 +1; in this case it will be tricky.  This would be a per listener
 or per phys-vhost option.  Although it could be a per-named-vhost
 option, it makes next to no sense during the request headers input
 phase when no host: will be parsed.
 
 Will people be confused by a directive that does nothing when
 placed in a named vhost?  Or should this just be global-only and
 forget about it?  It would be nice to cripple this module, say,
 for the intranet side of a server, and enable the overhead only on
 the external side.

Currently, it's per phys-vhost. Many SSL directives also have no 
effect in non-default named-vhost, therefore I don't think that 
RequestReadTimeout makes things worse.

 There's also nothing wrong with having it default to enabled,
 provided the admin can disable it most of the time if they choose,
 for speed.


Re: svn commit: r1202255 - /httpd/httpd/trunk/modules/filters/mod_reqtimeout.c

2011-11-16 Thread William A. Rowe Jr.

On 11/16/2011 5:06 PM, Stefan Fritsch wrote:

On Wednesday 16 November 2011, William A. Rowe Jr. wrote:


Will people be confused by a directive that does nothing when
placed in a named vhost?  Or should this just be global-only and
forget about it?  It would be nice to cripple this module, say,
for the intranet side of a server, and enable the overhead only on
the external side.


Currently, it's per phys-vhost. Many SSL directives also have no
effect in non-default named-vhost, therefore I don't think that
RequestReadTimeout makes things worse.


It would really be amazing if we could distinguish the current
vhost context as physical (al la first-of-many named) or a named
host (being at least the second instance).