Re: svn commit: r1742822 - /httpd/httpd/trunk/modules/http/http_core.c

2016-05-13 Thread Yann Ylavic
On Fri, May 13, 2016 at 7:25 PM, Yann Ylavic  wrote:
> On Fri, May 13, 2016 at 6:57 PM, Yann Ylavic  wrote:
>>
>> But for the above cases or an error while reading/validating the
>> headers or running post_read_request(), we finally call ap_die() or
>> ap_send_error_response().
>> I think we should report SERVER_BUSY_WRITE with r before those calls,
>> and use NULL for SERVER_BUSY_LOG.
>
> For the REQUEST_TIME_OUT case, we may need something like the below too.
> The goal would be to answer something to the client (408) if the
> timeout occurs with some request line byte(s) received (that an
> invalid request but still a request), and report the r (even "NULL"
> for the first empty request on the connection).

Just to be clear, full proposed patch (including yours) attached.
Index: modules/http/http_core.c
===
--- modules/http/http_core.c	(revision 1743265)
+++ modules/http/http_core.c	(working copy)
@@ -148,10 +148,9 @@ static int ap_process_http_async_connection(conn_r
 c->keepalive = AP_CONN_UNKNOWN;
 /* process the request if it was read without error */
 
-ap_update_child_status(c->sbh, SERVER_BUSY_WRITE,
-   r->the_request ? r : NULL);
 if (r->status == HTTP_OK) {
 cs->state = CONN_STATE_HANDLER;
+ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
 ap_process_async_request(r);
 /* After the call to ap_process_request, the
  * request pool may have been deleted.  We set
@@ -204,11 +203,10 @@ static int ap_process_http_sync_connection(conn_re
 c->keepalive = AP_CONN_UNKNOWN;
 /* process the request if it was read without error */
 
-ap_update_child_status(c->sbh, SERVER_BUSY_WRITE,
-   r->the_request ? r : NULL);
 if (r->status == HTTP_OK) {
 if (cs)
 cs->state = CONN_STATE_HANDLER;
+ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
 ap_process_request(r);
 /* After the call to ap_process_request, the
  * request pool will have been deleted.  We set
Index: server/protocol.c
===
--- server/protocol.c	(revision 1743265)
+++ server/protocol.c	(working copy)
@@ -1090,20 +1090,34 @@ request_rec *ap_read_request(conn_rec *conn)
 ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO(00566)
   "request failed: invalid characters in URI");
 }
+ap_update_child_status(conn->sbh, SERVER_BUSY_WRITE, r);
 access_status = r->status;
 r->status = HTTP_OK;
 ap_die(access_status, r);
-ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r);
+ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, NULL);
 ap_run_log_transaction(r);
 r = NULL;
 apr_brigade_destroy(tmp_bb);
 goto traceout;
 case HTTP_REQUEST_TIME_OUT:
-ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, NULL);
-if (!r->connection->keepalives)
+if (r->the_request || !r->connection->keepalives) {
+request_rec *log_r = r;
+ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO()
+  "request failed: client's request-line timed out");
+if (r->the_request) {
+ap_update_child_status(conn->sbh, SERVER_BUSY_WRITE, r);
+access_status = r->status;
+r->status = HTTP_OK;
+ap_die(access_status, r);
+log_r = NULL;
+}
+ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, log_r);
 ap_run_log_transaction(r);
-apr_brigade_destroy(tmp_bb);
-goto traceout;
+r = NULL;
+apr_brigade_destroy(tmp_bb);
+goto traceout;
+}
+/* fall through */
 default:
 apr_brigade_destroy(tmp_bb);
 r = NULL;
@@ -1129,8 +1143,9 @@ request_rec *ap_read_request(conn_rec *conn)
 if (r->status != HTTP_OK) {
 ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO(00567)
   "request failed: error reading the headers");
+ap_update_child_status(conn->sbh, SERVER_BUSY_WRITE, r);
 ap_send_error_response(r, 0);
-ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r);
+ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, NULL);
 ap_run_log_transaction(r);
 apr_brigade_destroy(tmp_bb);
 goto traceout;
@@ -1151,8 +1166,9 @@ request_rec *ap_read_request(conn_rec *conn)
   "(%s): %s", tenc, r->uri);
 

Re: svn commit: r1742822 - /httpd/httpd/trunk/modules/http/http_core.c

2016-05-13 Thread Yann Ylavic
On Fri, May 13, 2016 at 7:25 PM, Yann Ylavic  wrote:
> +if (r->the_request) {
> +access_status = r->status;
> +r->status = HTTP_OK;
> +ap_update_child_status(conn->sbh, SERVER_BUSY_WRITE, r);
> +ap_die(access_status, r);
> +r = NULL;
> +}

With an "else" here instead of r = NULL above...

> +ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r);

... so to not segfault below :)

>  ap_run_log_transaction(r);


Re: svn commit: r1742822 - /httpd/httpd/trunk/modules/http/http_core.c

2016-05-13 Thread Yann Ylavic
On Fri, May 13, 2016 at 6:57 PM, Yann Ylavic  wrote:
> On Fri, May 13, 2016 at 6:49 PM, William A Rowe Jr  
> wrote:
>> On Fri, May 13, 2016 at 11:41 AM, Yann Ylavic  wrote:
>>>
>>> On Thu, May 12, 2016 at 4:55 PM, William A Rowe Jr 
>>> wrote:
>>> >
>>> > I'd explained in another thread this week why this patch is invalid,
>>> > and I've gone ahead and reverted.
>>> >
>>> > We agreed there is a defect here, what about the attached fix?
>>>
>>> Looks good, if something bad happened in ap_read_request() we already
>>> have responded with ap_send_error_response().
>>>
>>> What may be missing is reporting SERVER_BUSY_WRITE (with the
>>> partial/bad request) in ap_send_error_response(), and then we'd
>>> probably don't need to pass r for SERVER_BUSY_LOG in the error paths
>>> of ap_read_request().
>>
>>
>> AIUI, at that point in the code, if there was an error response it was
>> already
>> sent, the other paths appear to be simple disconnection states with no
>> further
>> logging or socket activity, other than forcing lingering close... perhaps.
>
> That's the case where reading the request line fails for anything but
> URI_TOO_LARGE, BAD_REQUEST or REQUEST_TIMEOUT.
>
> But for the above cases or an error while reading/validating the
> headers or running post_read_request(), we finally call ap_die() or
> ap_send_error_response().
> I think we should report SERVER_BUSY_WRITE with r before those calls,
> and use NULL for SERVER_BUSY_LOG.

For the REQUEST_TIME_OUT case, we may need something like the below too.
The goal would be to answer something to the client (408) if the
timeout occurs with some request line byte(s) received (that an
invalid request but still a request), and report the r (even "NULL"
for the first empty request on the connection).

Index: server/protocol.c
===
--- server/protocol.c(revision 1743265)
+++ server/protocol.c(working copy)
@@ -1099,11 +1099,23 @@ request_rec *ap_read_request(conn_rec *conn)
 apr_brigade_destroy(tmp_bb);
 goto traceout;
 case HTTP_REQUEST_TIME_OUT:
-ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, NULL);
-if (!r->connection->keepalives)
+if (r->the_request || !r->connection->keepalives) {
+ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO()
+  "request failed: client's request-line
timed out");
+if (r->the_request) {
+access_status = r->status;
+r->status = HTTP_OK;
+ap_update_child_status(conn->sbh, SERVER_BUSY_WRITE, r);
+ap_die(access_status, r);
+r = NULL;
+}
+ap_update_child_status(conn->sbh, SERVER_BUSY_LOG, r);
 ap_run_log_transaction(r);
-apr_brigade_destroy(tmp_bb);
-goto traceout;
+r = NULL;
+apr_brigade_destroy(tmp_bb);
+goto traceout;
+}
+/* fall through */
 default:
 apr_brigade_destroy(tmp_bb);
 r = NULL;
_


Re: svn commit: r1742822 - /httpd/httpd/trunk/modules/http/http_core.c

2016-05-13 Thread Yann Ylavic
On Fri, May 13, 2016 at 6:49 PM, William A Rowe Jr  wrote:
> On Fri, May 13, 2016 at 11:41 AM, Yann Ylavic  wrote:
>>
>> On Thu, May 12, 2016 at 4:55 PM, William A Rowe Jr 
>> wrote:
>> >
>> > I'd explained in another thread this week why this patch is invalid,
>> > and I've gone ahead and reverted.
>> >
>> > We agreed there is a defect here, what about the attached fix?
>>
>> Looks good, if something bad happened in ap_read_request() we already
>> have responded with ap_send_error_response().
>>
>> What may be missing is reporting SERVER_BUSY_WRITE (with the
>> partial/bad request) in ap_send_error_response(), and then we'd
>> probably don't need to pass r for SERVER_BUSY_LOG in the error paths
>> of ap_read_request().
>
>
> AIUI, at that point in the code, if there was an error response it was
> already
> sent, the other paths appear to be simple disconnection states with no
> further
> logging or socket activity, other than forcing lingering close... perhaps.

That's the case where reading the request line fails for anything but
URI_TOO_LARGE, BAD_REQUEST or REQUEST_TIMEOUT.

But for the above cases or an error while reading/validating the
headers or running post_read_request(), we finally call ap_die() or
ap_send_error_response().
I think we should report SERVER_BUSY_WRITE with r before those calls,
and use NULL for SERVER_BUSY_LOG.


Re: svn commit: r1742822 - /httpd/httpd/trunk/modules/http/http_core.c

2016-05-13 Thread William A Rowe Jr
On Fri, May 13, 2016 at 11:41 AM, Yann Ylavic  wrote:

> On Thu, May 12, 2016 at 4:55 PM, William A Rowe Jr 
> wrote:
> >
> > I'd explained in another thread this week why this patch is invalid,
> > and I've gone ahead and reverted.
> >
> > We agreed there is a defect here, what about the attached fix?
>
> Looks good, if something bad happened in ap_read_request() we already
> have responded with ap_send_error_response().
>
> What may be missing is reporting SERVER_BUSY_WRITE (with the
> partial/bad request) in ap_send_error_response(), and then we'd
> probably don't need to pass r for SERVER_BUSY_LOG in the error paths
> of ap_read_request().
>

AIUI, at that point in the code, if there was an error response it was
already
sent, the other paths appear to be simple disconnection states with no
further
logging or socket activity, other than forcing lingering close... perhaps.

In the lingering close phase, we do update the score status, so I don't
think
that needs to be done in these alternate code paths here.


Re: svn commit: r1742822 - /httpd/httpd/trunk/modules/http/http_core.c

2016-05-13 Thread Yann Ylavic
On Thu, May 12, 2016 at 4:55 PM, William A Rowe Jr  wrote:
>
> I'd explained in another thread this week why this patch is invalid,
> and I've gone ahead and reverted.
>
> We agreed there is a defect here, what about the attached fix?

Looks good, if something bad happened in ap_read_request() we already
have responded with ap_send_error_response().

What may be missing is reporting SERVER_BUSY_WRITE (with the
partial/bad request) in ap_send_error_response(), and then we'd
probably don't need to pass r for SERVER_BUSY_LOG in the error paths
of ap_read_request().


Re: svn commit: r1742822 - /httpd/httpd/trunk/modules/http/http_core.c

2016-05-13 Thread William A Rowe Jr
On Thu, May 12, 2016 at 9:55 AM, William A Rowe Jr 
wrote:

> On Sun, May 8, 2016 at 8:53 AM,  wrote:
>
>> Author: rjung
>> Date: Sun May  8 13:53:37 2016
>> New Revision: 1742822
>>
>> URL: http://svn.apache.org/viewvc?rev=1742822&view=rev
>> Log:
>> Fix yet another case where we clobber the
>> server-status request info when a timeout happens.
>>
>> Modified:
>> httpd/httpd/trunk/modules/http/http_core.c
>>
>> Modified: httpd/httpd/trunk/modules/http/http_core.c
>> URL:
>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/http_core.c?rev=1742822&r1=1742821&r2=1742822&view=diff
>>
>> ==
>> --- httpd/httpd/trunk/modules/http/http_core.c (original)
>> +++ httpd/httpd/trunk/modules/http/http_core.c Sun May  8 13:53:37 2016
>> @@ -148,7 +148,8 @@ static int ap_process_http_async_connect
>>  c->keepalive = AP_CONN_UNKNOWN;
>>  /* process the request if it was read without error */
>>
>> -ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
>> +ap_update_child_status(c->sbh, SERVER_BUSY_WRITE,
>> +   r->the_request ? r : NULL);
>>  if (r->status == HTTP_OK) {
>>  cs->state = CONN_STATE_HANDLER;
>>  ap_process_async_request(r);
>> @@ -203,7 +204,8 @@ static int ap_process_http_sync_connecti
>>  c->keepalive = AP_CONN_UNKNOWN;
>>  /* process the request if it was read without error */
>>
>> -ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
>> +ap_update_child_status(c->sbh, SERVER_BUSY_WRITE,
>> +   r->the_request ? r : NULL);
>>  if (r->status == HTTP_OK) {
>>  if (cs)
>>  cs->state = CONN_STATE_HANDLER;
>>
>
> I'd explained in another thread this week why this patch is invalid,
> and I've gone ahead and reverted.
>
> We agreed there is a defect here, what about the attached fix?
>

Reposting, since gmail likes to base64 encode attached patches...

--- modules/http/http_core.c (revision 1743511)
+++ modules/http/http_core.c (working copy)
@@ -148,9 +148,9 @@
 c->keepalive = AP_CONN_UNKNOWN;
 /* process the request if it was read without error */

-ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
 if (r->status == HTTP_OK) {
 cs->state = CONN_STATE_HANDLER;
+ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
 ap_process_async_request(r);
 /* After the call to ap_process_request, the
  * request pool may have been deleted.  We set
@@ -203,10 +203,10 @@
 c->keepalive = AP_CONN_UNKNOWN;
 /* process the request if it was read without error */

-ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
 if (r->status == HTTP_OK) {
 if (cs)
 cs->state = CONN_STATE_HANDLER;
+ap_update_child_status(c->sbh, SERVER_BUSY_WRITE, r);
 ap_process_request(r);
 /* After the call to ap_process_request, the
  * request pool will have been deleted.  We set


>
>
>
>


Re: Scoreboard 3.0

2016-05-13 Thread William A Rowe Jr
On Fri, May 13, 2016 at 8:34 AM, Eric Covener  wrote:

> On Wed, May 11, 2016 at 6:39 PM, William A Rowe Jr 
> wrote:
> > In the next iteration for trunk I see a value in having both the basic
> > activity flag per worker thread so we can see those states at a glance
> as we
> > can today, as well as a summary table per process of the number of
> > recognized workers and inactive Conn/req per process by state.
>
> I believe we have (most of?) this data in 2.4 server-status for event.
>

Yup, that's certainly a great start to what we want.

My open question is still, whether administrators would want to see
both the per-connection and the per-thread detailed score records
on the same server.

Once we are in agreement on the backports of scoreboard.c to 2.4.x
branch, it would be a good time to introduce a new state between
reading and writing, perhaps called loading or processing, which
reflects the time between the start of the request handler, and the
beginning of writing to the client.  This would reflect both processing
time, and glomming of any input body.


Re: Timers and mpm-event

2016-05-13 Thread Stefan Eissing

> Am 13.05.2016 um 16:11 schrieb Eric Covener :
> 
> On Fri, May 13, 2016 at 7:02 AM, Stefan Eissing
>  wrote:
>> That would allow HTTP/2 processing to become fully async and it would no 
>> longer need its own worker thread pool, at least with mpm_event.
>> 
>> Thoughts?
> 
> One bit I am still ignorant of is all of the beam-ish stuff (the
> problem and the solution) and how moving the main connection from
> thread-to-thread might impact that.

The bucket beams have no thread affinity. What I describe in the comments as 
'red side' and 'green side' is just a name for the thread that is *currently* 
handling the red/green pool.

So, the red pool and green pools are fixed during the lifetime of a beam. The 
threads may vary. Whichever thread is the one owning operations on the red 
pool, I call the red thread.

The beam just manages the lifetimes of buckets and their pool affinity without 
unnecessary copying. The bucket being sent, the red buckets, are only ever 
manipulated in calls from the sending side. The buckets received, the green 
buckets, are separate instances, so they can be safely manipulated by the 
receiver (split/read/destroy).

The trick is that a green bucket may expose red data when read. That means the 
red bucket must stay around at least as long as the green one. So, a green 
bucket calls its beam when it's last share gets destroyed. The beam then knows 
that the corresponding red bucket is no longer needed.

Those no longer needed red buckets are placed on a 'purge' list. This list gets 
cleared and the buckets destroyed on the next call from the sending side, the 
red side.

The fragile thing is the cleanup of the red pool. That must not happen while 
the beam has outstanding green buckets. For this, the beam has a shutdown 
method that may block until this is true.

> Maybe  you could have a pipe()  with a writing end in each slave,  and
> read by the master,  that the event loop watches to re-schedule the
> master?

Hmm, I do not see the need. Those pipe()s will only generate events when 
another part of the process wants to. Unless we are talking about spreading 
HTTP/2 processing across multiple processes and using the pipes to transfer the 
actual data. And I am not convinced that this is a good idea.

And signaling would also need to go the other direction: from master to slave. 
Which would then require 2(4?) file handles per active requests, I assume?

-Stefan

Re: Timers and mpm-event

2016-05-13 Thread Eric Covener
On Fri, May 13, 2016 at 7:02 AM, Stefan Eissing
 wrote:
> That would allow HTTP/2 processing to become fully async and it would no 
> longer need its own worker thread pool, at least with mpm_event.
>
> Thoughts?

One bit I am still ignorant of is all of the beam-ish stuff (the
problem and the solution) and how moving the main connection from
thread-to-thread might impact that.

Maybe  you could have a pipe()  with a writing end in each slave,  and
read by the master,  that the event loop watches to re-schedule the
master?
-- 
Eric Covener
cove...@gmail.com


Re: Timers and mpm-event

2016-05-13 Thread Eric Covener
On Fri, May 13, 2016 at 7:02 AM, Stefan Eissing
 wrote:
> 1. Is this ever intended to work on a socket that is a main connection? Since 
> event itself will add this socket to its pollset  now and then, I see a 
> potential conflict. But that can be resolved, since mod_http2 is only 
> interested in the callback while *inside* process_connection. If could 
> unregister before returning or whatever is helpful.

Yes, but the only user so far is pretty unique as it is basically a
handler that drops down into TCP forwarding mode so there are really
no HTTP or filters around anymore.

But I would think that by the time you can register such a callback,
the socket is not in the pollset -- it only sits in there when we're
waiting on it (to pop from keepalive, to become writable, or to close
out)


> 2. I assume the callback gets invoked on whatever worker thread is currently 
> available? it probably should return rather immediately, I assume?

yes, it gets sent down to the queue of normal event worker threads --
the same ones that would otherwise handle e.g. a new connection or a
keepalive request request showing up on an old connection.  When
healthy this is meant to happen immediately.


-- 
Eric Covener
cove...@gmail.com


Re: Timers and mpm-event

2016-05-13 Thread Stefan Eissing
Hijacking this thread...

> Am 13.05.2016 um 15:29 schrieb Eric Covener :
> 
> On Fri, May 13, 2016 at 4:33 AM, Luca Toscano  wrote:
>> - What does PT_USER represents and how it is used?
> 
> PT_USER is what event tracks when you call
> event_register_poll_callback().  This callback
> allows a module to run some code when either of a pair of sockets
> becomes readable or writable.
> 
> It was written to allow mod_proxy_wstunnel to not tie up a thread when
> both ends of the connection
> are idle.
> 
> Note that it is still trunk-only.

Funny that you mention that...

I made a quick and dirty attempt to use this in mod_http2 yesterday. I want to 
get rid of the BUSY polling using timeouts that happens when the main 
connection is waiting for workers to come back with responses. That can block 
on a conditional, however it also needs to react when new data is arriving from 
the client. So, in its current form, it makes a timed wait on the conditional 
and checks the main connection again.

So, I registered on POLLIN on the main connection and signalled the conditional 
in that callback. Sort of worked, however was not very stable. Before I put in 
more work, there are some things I need to know too. Maybe that helps everyone 
with understanding this new feature.

1. Is this ever intended to work on a socket that is a main connection? Since 
event itself will add this socket to its pollset  now and then, I see a 
potential conflict. But that can be resolved, since mod_http2 is only 
interested in the callback while *inside* process_connection. If could 
unregister before returning or whatever is helpful.

2. I assume the callback gets invoked on whatever worker thread is currently 
available? it probably should return rather immediately, I assume?


Ideally, however, mod_http2 would use another mechanism:

   a) return from process_connection immediately when there is nothing to do
   b) have outgoing data sitting in output filters for streaming out event based
   c) be called via process_connection again if signaled by someone else*)
  *) someone else would be a slave connection that has produced new data
   d) slave connections could also leave their process_connection and go to 
"sleep".
  They have no socket, but can be signaled by others that data is available
  or that they can "write" more output.

In this way, slave connections and a master connections are both handled by 
MPM. While the latter have a socket that generates POLLIN/OUT/HUP events, the 
slave ones get these events generated by something else. This "something else" 
is in case of HTTP/2 the interworking between h2 session and stream requests. 

So, in short:
- POLLIN/POLLOUT/HUP event handling can be triggered by other threads via a new 
MPM API
- Slave connections can be started via a new MPM API. They will not have a 
socket, but take part in event handling

That would allow HTTP/2 processing to become fully async and it would no longer 
need its own worker thread pool, at least with mpm_event.

Thoughts?

-Stefan

Re: Scoreboard 3.0

2016-05-13 Thread Eric Covener
On Wed, May 11, 2016 at 6:39 PM, William A Rowe Jr  wrote:
> In the next iteration for trunk I see a value in having both the basic
> activity flag per worker thread so we can see those states at a glance as we
> can today, as well as a summary table per process of the number of
> recognized workers and inactive Conn/req per process by state.

I believe we have (most of?) this data in 2.4 server-status for event.


Re: Timers and mpm-event

2016-05-13 Thread Eric Covener
On Fri, May 13, 2016 at 4:33 AM, Luca Toscano  wrote:
> - What does PT_USER represents and how it is used?

PT_USER is what event tracks when you call
event_register_poll_callback().  This callback
allows a module to run some code when either of a pair of sockets
becomes readable or writable.

It was written to allow mod_proxy_wstunnel to not tie up a thread when
both ends of the connection
are idle.

Note that it is still trunk-only.

> - How is a new timer inserted in the skiplist? I followed the code and the
> only "insert" actions that I can see are triggered by event_get_timer_event,
> that is used for PT_USER events and by the function hooked to
> mpm_register_timed_callback (I can see it declared in mpm_common but no idea
> about how/when it runs). There are also some peek/pop actions executed
> before apr_pollset_poll but no trace of inserts.

It is via the insert in event_get_timer_event()

PT_USER uses the timer part to implement a timeout callback on waiting for the
sockets to become usable.it was added as a proof of concept, IIUC,
for mod_dialup to
help demonstrate async handlers (give the thread back by returning
SUSPENDED and run again later
by being called back after some time)
>
> I know that those are very generic questions so even some hints would be
> really appreciated. I have some goals in mind:
>
> 1) Add an overview in https://httpd.apache.org/docs/current/mod/event.html
> (maybe adding a definitive answer to
> https://bz.apache.org/bugzilla/show_bug.cgi?id=57399)

> 2) Create infographics (or even simple images) about prefork/worker/event
> (and motorz?) to compare them in a "under the hood" section of the
> documentation.

It might be good to separate design details from the reference manual
so users are not overwhelmed

> 3) Complete http://httpd.apache.org/docs/current/misc/perf-tuning.html.

There is a PR or other kind of complaint about the hard-coded 100ms
sleep time even when no timers may be in use.  Since timers
are not used by common modules, it should be possible to improve this.


-- 
Eric Covener
cove...@gmail.com


Timers and mpm-event

2016-05-13 Thread Luca Toscano
Hi Apache devs,

I have some questions about how mpm-event uses timers. If I understood
correctly the code, there are two main things that the listener thread
cares about:

- timeout_queue(s), that represents connections waiting for completion,
keep alive or in lingering close.
- timer_skiplist, that sorts timers for the above queues in an efficient
way (this is of course a very inaccurate and simplistic description).

My understanding is that the listener thread, using apr_pollset_poll, will
react on new connection requests plus all events related to the
timeout_queues. It also sleeps a maximum of 0.1s anyway to be able to check
the skiplist and update sockets that have an expired timeout (for example,
when a connection doesn't send any new data for more than KeepAliveTimeout).

Now the questions (if what I've said is vaguely true):

- What does PT_USER represents and how it is used?
- How is a new timer inserted in the skiplist? I followed the code and the
only "insert" actions that I can see are triggered by
event_get_timer_event, that is used for PT_USER events and by the function
hooked to mpm_register_timed_callback (I can see it declared in mpm_common
but no idea about how/when it runs). There are also some peek/pop actions
executed before apr_pollset_poll but no trace of inserts.

I know that those are very generic questions so even some hints would be
really appreciated. I have some goals in mind:

1) Add an overview in https://httpd.apache.org/docs/current/mod/event.html
(maybe adding a definitive answer to
https://bz.apache.org/bugzilla/show_bug.cgi?id=57399)
2) Create infographics (or even simple images) about prefork/worker/event
(and motorz?) to compare them in a "under the hood" section of the
documentation.
3) Complete http://httpd.apache.org/docs/current/misc/perf-tuning.html.

Thanks in advance!

Regards,

Luca


Re: End of the road of 2.2.x maintenance?

2016-05-13 Thread Noel Butler

On 12/05/2016 05:13, Ruediger Pluem wrote:

On 05/10/2016 10:38 PM, William A Rowe Jr wrote:

It's been a year, and seems to be a good time to revisit this topic
while those folks who are present at ApacheCon can discuss f2f
the merits of bringing the 2.2.x chapter to a close, and share their
thoughts back here on-list.

According to 
http://w3techs.com/technologies/history_details/ws-apache/2

the inflection point of a majority of 2.4 instances over 2.2 appears
to occur about 9 months from now.

OpenSUSE 13.1 adopted 2.4 way back in Nov of '13.

Ubuntu - 14.04 LTS, and Debian 8 (Jessie) switched to 2.4 in April 
'14.


RHEL / CentOS 7 are well over a year old, adopted 2.4 in June '14.
Fedora 18 shipped 2.4 way back in Jan '13.

E.g. every user of the broadly distributed Linux releases will have 
had

three full years to adopt 2.4 by June of 2017.  I expect the BSD world
looks similar (modulo any Apache License 2.0 stupidity we are all
too familiar with.)  If someone in the BSD, Solaris and other spheres
wants to chime in here with those milestones, that would be great.

I am prepared to RM a final bug-fix release of 2.2 in conjunction with
the next 2.4 release effort, to gather in any final requests for fixes
before we move to a 12-month, security-fixes-only window on that 
branch.

Once those 12 months expire, as we've done with 1.3 and 2.0, there's
the possibility that relatively few committers would collect some 
critical
patches/apply-to-2.2.xx final security fixes, but no further releases 
would

occur.

Are we ready to start the 12 month countdown as of the next/final bug
fix release of 2.2, and highlight this in both the 2.2 and 2.4 
announce

broadcasts?

I'm hoping we conclude some fixes of 2.4 scoreboard regressions and
get to the point of releasing 2.4 with mod_proxy_http2 sometime within
the next month or so, and that we can reach a consensus about how
we will proceed on the 2.2 branch, before we get to that release.

Feedback desired,


Sounds like a sensible plan.

Regards

RĂ¼diger



Agreed

(Slackware moved to 2.4 in Sept 2012)


--
If you have the urge to reply to all rather than reply to list, you best
first read  http://members.ausics.net/qwerty/


Re: [PATCH] mod_rewrite: support proxying via mod_proxy_http2

2016-05-13 Thread Evgeny Kotkov
Stefan Eissing  writes:

> Hmm, can someone with more brains than me on mod_rewrite look at the
> first patch if we want to add handling for h2: and h2c: uri schemes
> here or if there is a better way? Thanks.

In case this will help the review, here are some of the existing changes
that target similar problems (mod_rewrite not picking up a new scheme
for proxy URLs).  I examined them prior to preparing the patch:

  https://svn.apache.org/r1528556
  https://svn.apache.org/r405625
  https://svn.apache.org/r124584


Regards,
Evgeny Kotkov