On the subject of r->uri [was: Post 2.4.25]

[William A Rowe Jr]

On Wed, Dec 28, 2016 at 6:42 PM, Yann Ylavic  wrote:

> [Bill, you definitely should do something with your email client, e.g.
> using plain text only, replying to your messages breaks indentation
> level (like the number of '>' preceding/according to the initial
> message)].
>

(Again, it's gmail, /shrug. I can attempt to undecorate but doubt I'm
moving to a local client/mail store again. If anyone has good gmail
formatting tips for their default settings, I'd love a pointer.)


> On Thu, Dec 29, 2016 at 12:28 AM, William A Rowe Jr 
> wrote:
> >
> > On Dec 24, 2016 07:57, "Jim Jagielski"  wrote:
> >
> > Well as breaking changes go, changing URI to remain an encoded value and
> to
> > mandate module authors accept that req_rec is free threaded are breaking
> > changes.
>
> Not sure what the second point means, but preserving r->uri while also
> allowing to (and internally work with) an escaped form of the URI does
> not necessarily require an API change.
> (We could possibly have an opaque struct to manipulate the URI in its
> different forms and some helper(s) be compatible with the API changes'
> requirements, e.g. 2.4.x).
>

To be clear, this isn't possible.

There are multiple meanings of every path segment character which is
in the reserved set. There is no way to preserve these multiple meanings
in a decoded context. The parallel entities may exist in any undecoded
string. So r->uri, if it still exists, will be subsumed by some variable
like
r->uri_path_unencoded and be retrievable into a decoded form.

Functions such as ap_hook_map_to_storage, in the filesystem backend,
will only be interested in the decoded form. Functions such as the http
proxy module will only be interested in passing a never-mangled version
of the encoded uri.

Even if r->uri is available as a read-only input, there is no simple way for
httpd to resolve r->uri manipulations if changed in place (it isn't const)
and whether an r->uri_path_unencoded mismatch which is canonical,
and what mishmash the legacy abuser of r->uri did with these parallel
reserved characters in their encoded and unnencoded forms. We are
stuck with the current mess of various %-escape workarounds until
we replace the core assumption.

This deserves a long discussion which already exists in the security@
list, but needs to be pushed outward on dev@, preferably by the original
authors of these thoughts. That includes the r->uri preserving flavor
that you mention above, as well as the various discussions about the
% entity encoding, and my concerns about canonicalization. With some
first-level triage already complete, there is no reason for uri discussion
to remain 'behind the curtain.'

HTTP/2 frame prioritization not honored

[Kyriakos Zarifis]

I am experimenting with contention between lower/higher priority HTTP/2
streams, and I think I am noticing that high-priority frames are not given
high priority (quickly enough)

The process is: Download dummy page A, which, after onLoad, Prefetches 8
objects (5MB each)  at low priority. While the first few of those are
downloaded, click on link to page B (high priority stream), and measure how
long it takes to receive.

The client-server RTT is 20ms, and ideally because of higher prioritization
we would want the second HTML to be received at a timeframe as close to
that RTT as possible. However, I am noticing that the second HTML is
delayed (although not completely HoL-blocked) by objects that are being
Prefetched at lower priority.

This  plots
the client-side trace (Chrome log): HTML A is stream_id 1, the 8 prefetched
objects are streams 3-17, and HTML B (the object that is delayed) is
stream_id 19.  Dots below the x axis are the GET requests for the
respectively colored streams. Dots above the x axis are received DATA
frames. GETs for the prefetched objects are sent after onLoad, and that of
stream_id 19 is sent when I click on the link, around t=1900.  As this
shows, the reply for the high-priority stream (19) arrives ~1.5s later.

Apache's log shows that it sent the reply to stream_id 19 (HTML B) ~500ms
after it got its request. During those 500ms, it wrote another ~350 frames
(~5MB) from other streams. So it honors the higher priority in the sense
that it interleaves the HTML between the currently written object, but it
continues to write many lower-priority frames before it does so, delaying
that high-prio HTML. In contrast, when I wait until all prefetched objects
are served and then click on the link, the logs shows that the reply is
written within a few ms from the request arrival, as expected, and the
second page loads observably faster.

This seems to break any default prioritization or prioritization imposed by
the client, and leads to unexpected performance in different
prioritization-dependent scenarios that I have tried.

Has anyone experience this before, and do you know what causes it and what
the right way to fix it would be?

Thanks

Re: Post 2.4.25

[Yann Ylavic]

[Bill, you definitely should do something with your email client, e.g.
using plain text only, replying to your messages breaks indentation
level (like the number of '>' preceding/according to the initial
message)].

On Thu, Dec 29, 2016 at 12:28 AM, William A Rowe Jr  wrote:
>
> On Dec 24, 2016 07:57, "Jim Jagielski"  wrote:
>
[For example, here I had to add a '>' for Jim's original text to be
associated with the above "Jim ... wrote:"]
>>
>> My point is that even having a 6 month minimal (and that
>> is, IMO, widely optimistic and unrealistic) of "no new
>> features for 2.4" means that we are basically giving people
>> reasons to drop httpd. It would be a widely different story
>> if (1) trunk was ready to release and (2) we "committed" to
>> releasing trunk quickly by focusing on low-hanging fruit
>> which would make lives happier and better for our end-users.
>> As I said, my fear is that we will not be able to "control"
>> ourselves in limiting what is in 2.6, which will push the
>> actual release far past the point where it is even relevant.
>
> Well as breaking changes go, changing URI to remain an encoded value and to
> mandate module authors accept that req_rec is free threaded are breaking
> changes.

Not sure what the second point means, but preserving r->uri while also
allowing to (and internally work with) an escaped form of the URI does
not necessarily require an API change.
(We could possibly have an opaque struct to manipulate the URI in its
different forms and some helper(s) be compatible with the API changes'
requirements, e.g. 2.4.x).

Regarding the changes in configuration/behaviours, I don't think we
should break things anyway (even accross majors, if possible/relevant
of course), but rather provide options to have the one or the other
behaviour (trunk having the now considered better behaviour, stable(s)
the compatible one).

My point is mainly that rather than focusing on version numbers, we
probably should focus on:
1. what we have now (in trunk), and want to backport
2. what we don't have now, do it (the better wrt trunk), and go to 1.

There's (almost) always a way to backport things, though it should not
prevent us from doing the *necessary* changes (in trunk) for new
improvements/features.

Yet, first step is the "inventory" of what we have/want, each/all of
us involved and constructive...

Once this is done, let's see what is compatible or not (and if yes at
which cost).
If we are going to introduce incompatible features, let's do 3.x.
If we are going to introduce many features at once, let's do 2.6.x
(that's an announce/marketing "value", the user does not care about
the version, (s)he does about the features).
Otherwise let's continue improving trunk with 2.4.x.

When we start implementing new features, first discuss/specify them,
then implement, and see if it's compatible/backportable.
For now, I don't see many (if any) incompatibilities in trunk (I
surely don't have an exhaustive view), but many improvements to
backport.

Just my 2 cts...

Regards,
Yann.

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[Yehuda Katz]

On Wed, Dec 28, 2016 at 12:35 AM, William A Rowe Jr 
wrote:

> Our adoption is *broadly* based on the OS distributions
> from vendors, not from people picking up our sources.
> Yes - some integrate directly from source, and others
> use a non-OS distribution.
>

I think a significant number of users of nginx add the official nginx
yum/apt sources and keep up to date that way (
http://nginx.org/en/linux_packages.html#mainline).
This is particularly true because the vendor-supplied version are so old.
You can see this in the w3techs data: nginx 1.10.2 came out in October and
already makes up 75% of all nginx 1.10 users. nginx 1.11.8 usage has
similar trends.

A possible solution to this would be to start publishing binaries in a
package-manager-accessible format.
I am confident it would see a much higher rate of adoption.

- Y

Re: Post 2.4.25

[William A Rowe Jr]

On Dec 24, 2016 08:32, "Eric Covener"  wrote:

> I'm not saying we don't do one so we can do the other; I'm
> saying we do both, at the same time, in parallel. I still
> don't understand why that concept is such an anathema to some
> people.

I also worry about our ability to deliver a 3.0 with enough
re-architecture for us and and function for users, vs a more
continuous delivery (apologies for bringing buzzaords to dev@httpd)
cadence on 2.4 as we've been in.


Here is the confusion (see the versioning thread.)

2.6 is a break in ABI compatibility.

3.0 is a break in API compatibility.

Size in this case doesn't matter. Any break at all merits these changes.

We are not a commercial product. We are httpd. Nobody cares what the
version no is other than us, they very largely install and forget, OS
vendors grab new at one point in their distribution gathering phase and
don't revisit.

Adoption outside of OS distros is largely irrelevant. Talk about
do-nothing, PCRE2 has been out a very long time with all the activity and
no adoption, PCRE 8.x is on life support with little pulse and is the
defacto standard.

Your assumptions don't reflect the actual adoption behaviors.

Re: Post 2.4.25

[William A Rowe Jr]

On Dec 24, 2016 07:57, "Jim Jagielski"  wrote:


> On Dec 24, 2016, at 8:29 AM, Rich Bowen  wrote:
>
> On 12/23/2016 03:52 PM, Jim Jagielski wrote:
>> Personally, I don't think that backporting stuff to
>> 2.4 prevents or disallows development on 2.6/3.0. In
>> fact, I think it helps. We can easily do both...
>> after all, we are still "working" on 2.2.
>>
>> As I have also stated, my personal belief is that
>> 2.4 is finally reaching some traction, and if we
>> "turn off" development/enhancement of 2.4, we will
>> stop the uptake of 2.4 in its track. We need to keep
>> 2.4 viable and worthwhile we, at the same time, work
>> on 2.6/3.0. I think we all understand that getting
>> 2.6/3.0 out will not be a quick and/or painless
>> action.
>
> From my perspective, watching Nginx gain traction through superior
> marketing, and channeling Dilbert's Pointy Haired Boss in assuming that
> everything which I have never done must be simple, I, for one, would
> like to see us release a 2.6, and more generally, to release a 2.x every
> 2 years, or less, rather than every 4 years, or more.
>
> My opinion on this, I would emphasize, is 100% marketing, and 0%
> technical. I realize we "don't do" marketing, but if we want to still ve
> having the fun of doing this in another 20 years, it may be necessary to
> get our name out there a little more frequently in terms of doing new
> things. We are frankly not great at telling the world about the cool new
> things we're doing.
>

Yeah, right now the way we are "doing marketing" is by
continually adding features and enhancements to 2.4... It
is what keeps 2.4 relevant and is what either keeps current
httpd users using httpd or maybe help those on the fence decide
on httpd instead of nginx/whatever.


And again to play devil's advocate, how has that worked out in the four
years of httpd 2.4?

My point is that even having a 6 month minimal (and that
is, IMO, widely optimistic and unrealistic) of "no new
features for 2.4" means that we are basically giving people
reasons to drop httpd. It would be a widely different story
if (1) trunk was ready to release and (2) we "committed" to
releasing trunk quickly by focusing on low-hanging fruit
which would make lives happier and better for our end-users.
As I said, my fear is that we will not be able to "control"
ourselves in limiting what is in 2.6, which will push the
actual release far past the point where it is even relevant.


Well as breaking changes go, changing URI to remain an encoded value and to
mandate module authors accept that req_rec is free threaded are breaking
changes. We did that on conn_rec post-2.2. But I suspect that we have now
done this to req_rec with the event mpm and sf's http2 enhancements already
on 2.4?

To be clear, if our goal was "Fork trunk as 2.5 NOW, polish
and tune 2.5 'as-is' with minimal major refactoring with
the goal of getting 2.6 out ASAP" then yeah, sure, the idea
of "no new features in 2.4" would have some merit. But based
on current conversation, it's obvious that, at least to me,
that won't happen and we will be continually refactoring 2.6
to make it a 3.0.


Two mistakes. If we commit to a new contract on these two things, there is
never 2.6.

Because fixing r->uri is such a priority, trust that I'll be voting every
2.6 candidate a -1 until it exists. I don't know why the original httpd
founders are so hung up on version number conservation, they are cheap, but
we are breaking a key field of a core request structure and no other OSS
project would be stupid enough to call that n.m+1.

So you can presume there is no such thing as 2.6.

Error 2 and you ignored the first reply, there is no branch until 3.0
occurs.

I'll save that detail for the next post.

Again, you don't "stop" development on a
current release until the next release is ready or, at least,
"this close" to being ready. You don't sacrifice the present,
nor do you leave your users in that limbo.


Users had been in limbo 10 months for security fixes and far longer for bug
fixes PatchAvailable in bugzilla, and you are worried about feature
development on an maintenance branch. Sigh...

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Dec 28, 2016 10:34, "William A Rowe Jr"  wrote:


Specific
Revision
Of all Most
Recent
Of m.m Of all
Apache/1.3.x 391898 3.33% 1.3.42 42392 10.82% 0.36%
Apache/2.0.x 551117 4.68% 2.0.64 36944 6.70% 0.31%
Apache/2.2.x 7129391 60.49% 2.2.31 1332448 18.78% 11.31%
Apache/2.4.x 3713364 31.51% 2.4.17+ 1502061 42.90% 12.74%

11785770
2.4.23 754385 21.54% 6.40%


Since this table is illegible to some, please see the second tab of

https://docs.google.com/spreadsheets/d/1aOxBRZ2IHsUJJcQNXu-oe6la4wMRIHN2mOlJCQGRy0k/edit?usp=drivesdk

The first tab is a crossref of many OS distribution components used by
httpd, as well as httpd itself.

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

Hi Jim,

Talk to Google and the OpenOffice Team, that was a paste from OpenOffice
Calc.

I'll be happy to start summarizing as a shared Google sheet.

Cheers,

Bill


On Dec 28, 2016 14:22, "Jim Jagielski"  wrote:

> Bill, I don't know if it's just my Email client or not (doesn't
> look like it) but could you fix your Email client? It's impossible to
> reply and have the quoted parts parsed out correctly. I think
> it's to do w/ your messages being RTF or something.
>
> Thx!
>
> Included is an example of how a Reply misses quote levels...
>
> > On Dec 28, 2016, at 1:34 PM, William A Rowe Jr 
> wrote:
> >
> > On Wed, Dec 28, 2016 at 9:13 AM, Jim Jagielski  wrote:
> > cPanel too... They are moving to EA4 which is Apache 2.4.
> >
> > If not moved yet, that example wouldn't be helpful, it reinforces my
> point
> > four years later. But EA itself seems to track pretty closely to the most
> > contemperanious versions, looks like within a month.
> >
> >
> > So the idea that supplemental (ie: 2.4.x->2.4.y) patches don't
> > have the reach or range of larger ones (2.4.x->2.6/3.0) isn't
> > quite accurate.
> >
> > It's entirely accurate. It isn't all-encompassing. We have that data too,
> > let's tear down SecuritySpace's Nov '16 dataset;
> > http://www.securityspace.com/s_survey/data/201611/servers.html
> >
>
>

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[Jim Jagielski]

Bill, I don't know if it's just my Email client or not (doesn't
look like it) but could you fix your Email client? It's impossible to
reply and have the quoted parts parsed out correctly. I think
it's to do w/ your messages being RTF or something.

Thx!

Included is an example of how a Reply misses quote levels...

> On Dec 28, 2016, at 1:34 PM, William A Rowe Jr  wrote:
> 
> On Wed, Dec 28, 2016 at 9:13 AM, Jim Jagielski  wrote:
> cPanel too... They are moving to EA4 which is Apache 2.4.
>  
> If not moved yet, that example wouldn't be helpful, it reinforces my point
> four years later. But EA itself seems to track pretty closely to the most
> contemperanious versions, looks like within a month.
> 
> 
> So the idea that supplemental (ie: 2.4.x->2.4.y) patches don't
> have the reach or range of larger ones (2.4.x->2.6/3.0) isn't
> quite accurate.
> 
> It's entirely accurate. It isn't all-encompassing. We have that data too,
> let's tear down SecuritySpace's Nov '16 dataset;
> http://www.securityspace.com/s_survey/data/201611/servers.html 
> 

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[Jan Ehrhardt]

William A Rowe Jr in gmane.comp.apache.devel (Wed, 28 Dec 2016 10:46:51
-0600):
>On Wed, Dec 28, 2016 at 9:05 AM, Jan Ehrhardt  wrote:
>
>> Do not underestimate the influence of control panels. On all my Centos
>> servers I am running Directadmin. DA always offers to upgrade to the
>> latest release within a day after the release. Hence, I am running
>> Apache 2.4.25 everywhere at the moment.
>
> Excellent pointer. Thanks Jan.

BTW: I would be more hesitant to directly install a new release if I
would not have tested the dev/dist release candidates on my Windows
dev-server. But I am quite sure a lot of Directadmin users follow suit
soon.
-- 
Jan

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Wed, Dec 28, 2016 at 9:13 AM, Jim Jagielski  wrote:

> cPanel too... They are moving to EA4 which is Apache 2.4.
>

If not moved yet, that example wouldn't be helpful, it reinforces my point
four years later. But EA itself seems to track pretty closely to the most
contemperanious versions, looks like within a month.


So the idea that supplemental (ie: 2.4.x->2.4.y) patches don't
> have the reach or range of larger ones (2.4.x->2.6/3.0) isn't
> quite accurate.
>

It's entirely accurate. It isn't all-encompassing. We have that data too,
let's tear down SecuritySpace's Nov '16 dataset;
http://www.securityspace.com/s_survey/data/201611/servers.html

First off, if you follow that link, you'll find much larger numbers
associated
to those specific revisions shipped with the likes of RHEL or CentOS, Ubuntu
(particularly -LTS flavors), etc etc etc. That was my contention in the top
post. But let's quantify 'accuracy' as you defined it in the reply...

Specific
Revision
Of all Most
Recent
Of m.m Of all
Apache/1.3.x 391898 3.33% 1.3.42 42392 10.82% 0.36%
Apache/2.0.x 551117 4.68% 2.0.64 36944 6.70% 0.31%
Apache/2.2.x 7129391 60.49% 2.2.31 1332448 18.78% 11.31%
Apache/2.4.x 3713364 31.51% 2.4.17+ 1502061 42.90% 12.74%

11785770
2.4.23 754385 21.54% 6.40%

The applicable data are 37.47% of all 'Apache[/n[.n[.n]]]' items, meaning
that some 2/3rds of users drop the ServerTokens down to product only
or major version only, and we can't derive anything useful from them, so
we will ignore the Apache and Apache/2 references for our % evaluations,
'Of all' refers to those with at least Apache/2.x designations.

I included 2.4.17-2.4.23 as an item, because that group are the versions
that released within the past year of this particular survey data (that does
include the then-current 2.4.23.)

The 'Of m.m' - same major.minor - backs out that Apache/2.x (without a
known subversion) from the calculation because we can't tell whether they
are the corresponding or a different subversion.

Of httpd users we can quantify, 6.4% updated within months of the 2.4.23
release (your 'power users' classification.) That minority doesn't move the
needle much on total adoption of httpd vs. others.

Only 11.3% bothered to pick up the final 2.2.31 that has been out
over a year, and combined with 12.74% running some 2.4.17...2.4.23,
*** only 24% *** run a version that had been a current release within
the preceding year.  E.g. of those running a somewhat-current version,
more than 1/4 are running the July 2.4.23 release by the end of November.
Note that Fedora 25 didn't move the needle much on this, it shipped GA
in December.

aren't the ones we are talking about in the 1st place. We are
> talking about real, "power" users, who want/need the latest
> and greatest.
>

Not if you are talking overall adoption rate. As illustrated, those
users adopting 2.4.23 already are an nearly accidental minority,
after 5 mos half of the 'current' 2.4 users are running 2.4.23, the
other half are running a flavor between 12 and 6 mos old. That
looks like overall random distribution by deployment date, with
no particular effort expended on 'staying current'.

Re: svn commit: r1776076 - in /httpd/httpd/trunk: docs/manual/mod/mod_proxy_protocol.xml modules/filters/mod_proxy_protocol.c

[Jim Jagielski]

I would simply add a note to the header of mod_remoteip

> On Dec 28, 2016, at 12:23 PM, Daniel Ruggeri  wrote:
> 
> 
> On 12/28/2016 7:49 AM, Jim Jagielski wrote:
>> Sounds good... We could simply move the filter aspects over to
>> mod_remoteip and save us a module :)
> 
> This may be a naive question, but how would the copyright for such mixed
> copyright sources be handled in the file for such a case?
> 
> -- 
> Daniel Ruggeri
> 

Re: svn commit: r1776076 - in /httpd/httpd/trunk: docs/manual/mod/mod_proxy_protocol.xml modules/filters/mod_proxy_protocol.c

[Daniel Ruggeri]

On 12/28/2016 7:49 AM, Jim Jagielski wrote:
> Sounds good... We could simply move the filter aspects over to
> mod_remoteip and save us a module :)

This may be a naive question, but how would the copyright for such mixed
copyright sources be handled in the file for such a case?

-- 
Daniel Ruggeri

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Wed, Dec 28, 2016 at 9:05 AM, Jan Ehrhardt  wrote:

> William A Rowe Jr in gmane.comp.apache.devel (Tue, 27 Dec 2016 23:35:50
> -0600):
> >But the vast majority of httpd, nginx, and yes - even IIS
> >users are all running what they were handed from their
> >OS distribution.
>
> Do not underestimate the influence of control panels. On all my Centos
> servers I am running Directadmin. DA always offers to upgrade to the
> latest release within a day after the release. Hence, I am running
> Apache 2.4.25 everywhere at the moment.
>
> Excellent pointer. Thanks Jan.

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[Jim Jagielski]

cPanel too... They are moving to EA4 which is Apache 2.4.

So the idea that supplemental (ie: 2.4.x->2.4.y) patches don't
have the reach or range of larger ones (2.4.x->2.6/3.0) isn't
quite accurate.

IMO, people who are comfortable with "whatever the OS provides"
aren't the ones we are talking about in the 1st place. We are
talking about real, "power" users, who want/need the latest
and greatest.

just my 2c

> On Dec 28, 2016, at 10:05 AM, Jan Ehrhardt  wrote:
> 
> William A Rowe Jr in gmane.comp.apache.devel (Tue, 27 Dec 2016 23:35:50
> -0600):
>> But the vast majority of httpd, nginx, and yes - even IIS
>> users are all running what they were handed from their
>> OS distribution.
> 
> Do not underestimate the influence of control panels. On all my Centos
> servers I am running Directadmin. DA always offers to upgrade to the
> latest release within a day after the release. Hence, I am running
> Apache 2.4.25 everywhere at the moment.
> -- 
> Jan
> 

Re: [users@httpd] Web sockets & proxypass - No protocol handler was valid for the URL

[Yann Ylavic]

On Wed, Dec 28, 2016 at 3:56 PM, Eric Covener  wrote:
> On Wed, Dec 28, 2016 at 9:54 AM, Yann Ylavic  wrote:
>>> I guess we just got unlucky when this overlap was "fixed" before since
>>> the order is not deterministic. I don't think we'll break anyone by
>>> making wstunnell try first.
>>
>> I'm not sure the different schemes ("https" vs "wss") would allow both
>> handlers to run successively.
>>
>> r->filename is built once for all (proxy_trans), so mod_proxy_http
>> would DECLINE "wss://" and same for mod_proxy_wstunnel with
>> "https://";.
>>
>> I proposed Upgrade handling in mod_proxy_http (a while ago, see [1]),
>> which is IMHO a better way to deal with HTTP Upgrading in httpd...
>
> Doh, I guess at least having the order be deterministic is not harmful
> so I will leave in r1776290

Yes, makes sense to try WebSocket first.
Maybe we should 426 (Upgrade Required) instead of DECLINing at AH02900, though.

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[Jan Ehrhardt]

William A Rowe Jr in gmane.comp.apache.devel (Tue, 27 Dec 2016 23:35:50
-0600):
>But the vast majority of httpd, nginx, and yes - even IIS
>users are all running what they were handed from their
>OS distribution.

Do not underestimate the influence of control panels. On all my Centos
servers I am running Directadmin. DA always offers to upgrade to the
latest release within a day after the release. Hence, I am running
Apache 2.4.25 everywhere at the moment.
-- 
Jan

Re: [users@httpd] Web sockets & proxypass - No protocol handler was valid for the URL

[Jim Jagielski]

+1

> On Dec 28, 2016, at 9:56 AM, Eric Covener  wrote:
> 
> On Wed, Dec 28, 2016 at 9:54 AM, Yann Ylavic  wrote:
>>> I guess we just got unlucky when this overlap was "fixed" before since
>>> the order is not deterministic. I don't think we'll break anyone by
>>> making wstunnell try first.
>> 
>> I'm not sure the different schemes ("https" vs "wss") would allow both
>> handlers to run successively.
>> 
>> r->filename is built once for all (proxy_trans), so mod_proxy_http
>> would DECLINE "wss://" and same for mod_proxy_wstunnel with
>> "https://";.
>> 
>> I proposed Upgrade handling in mod_proxy_http (a while ago, see [1]),
>> which is IMHO a better way to deal with HTTP Upgrading in httpd...
> 
> Doh, I guess at least having the order be deterministic is not harmful
> so I will leave in r1776290
> 
> Thanks for the hint.
> 
> -- 
> Eric Covener
> cove...@gmail.com

Re: [users@httpd] Web sockets & proxypass - No protocol handler was valid for the URL

[Eric Covener]

On Wed, Dec 28, 2016 at 9:54 AM, Yann Ylavic  wrote:
>> I guess we just got unlucky when this overlap was "fixed" before since
>> the order is not deterministic. I don't think we'll break anyone by
>> making wstunnell try first.
>
> I'm not sure the different schemes ("https" vs "wss") would allow both
> handlers to run successively.
>
> r->filename is built once for all (proxy_trans), so mod_proxy_http
> would DECLINE "wss://" and same for mod_proxy_wstunnel with
> "https://";.
>
> I proposed Upgrade handling in mod_proxy_http (a while ago, see [1]),
> which is IMHO a better way to deal with HTTP Upgrading in httpd...

Doh, I guess at least having the order be deterministic is not harmful
so I will leave in r1776290

Thanks for the hint.

-- 
Eric Covener
cove...@gmail.com

Re: [users@httpd] Web sockets & proxypass - No protocol handler was valid for the URL

[Yann Ylavic]

On Wed, Dec 28, 2016 at 3:40 PM, Eric Covener  wrote:
> I guess we just got unlucky when this overlap was "fixed" before since
> the order is not deterministic. I don't think we'll break anyone by
> making wstunnell try first.

I'm not sure the different schemes ("https" vs "wss") would allow both
handlers to run successively.

r->filename is built once for all (proxy_trans), so mod_proxy_http
would DECLINE "wss://" and same for mod_proxy_wstunnel with
"https://";.

I proposed Upgrade handling in mod_proxy_http (a while ago, see [1]),
which is IMHO a better way to deal with HTTP Upgrading in httpd...


[1] 
https://lists.apache.org/thread.html/e441f75818d07112e8122232440752bd8c27d5539b199bcd115e3b4e@%3Cdev.httpd.apache.org%3E

Fwd: [users@httpd] Web sockets & proxypass - No protocol handler was valid for the URL

[Eric Covener]

I guess we just got unlucky when this overlap was "fixed" before since
the order is not deterministic. I don't think we'll break anyone by
making wstunnell try first.
-- Forwarded message --
From: Eric Covener 
Date: Wed, Dec 28, 2016 at 9:36 AM
Subject: Re: [users@httpd] Web sockets & proxypass - No protocol
handler was valid for the URL
To: us...@httpd.apache.org


On Wed, Dec 28, 2016 at 9:21 AM, Yann Ylavic  wrote:
> On Wed, Dec 28, 2016 at 1:32 PM, Adam Teale  wrote:
>> more detail from the error log:
>>
>> [Wed Dec 28 09:31:05.974744 2016] [proxy:debug] [pid 27844]
>> mod_proxy.c(1198): [client 127.0.0.1:54002] AH01143: Running scheme wss
>> handler (attempt 0)
>
> Looks like the request is missing the "Upgrade: WebSocket" header,
> which mod_proxy_wstunnel requires.
>
> From your configuration:
> ProxyPass /chat/stream/ wss://localhost:8000/chat/stream/
> it seems that /chat/stream/ is used for "normal" HTTP traffic (before
> the upgrade to WebSocket, possibly).
>
> Couldn't the pure WebSocket things be handled by another (inner) URL
> instead, so that the following could work:
> ProxyPass /chat/stream/ https://localhost:8000/chat/stream/
> ProxyPass /chat/stream/wss wss://localhost:8000/chat/stream/wss
>

Seems like we should setup wstunnel to go before http, so you could
then just configure both proxypasses and wstunnell would defer.


-- 
Eric Covener
cove...@gmail.com

Re: svn commit: r1776076 - in /httpd/httpd/trunk: docs/manual/mod/mod_proxy_protocol.xml modules/filters/mod_proxy_protocol.c

[Jim Jagielski]

> On Dec 26, 2016, at 10:30 PM, Daniel Ruggeri  wrote:
> 
> 
> On 12/26/2016 3:42 PM, j...@apache.org wrote:
>> Author: jim
>> Date: Mon Dec 26 21:42:26 2016
>> New Revision: 1776076
>> 
>> URL: http://svn.apache.org/viewvc?rev=1776076&view=rev
>> Log:
>> revert back... no conflict w/ name
>> 
>> Modified:
>>httpd/httpd/trunk/docs/manual/mod/mod_proxy_protocol.xml
>>httpd/httpd/trunk/modules/filters/mod_proxy_protocol.c
>> ...
> 
> I feel a little dissonance toward this being a separate module versus
> being folded into mod_remoteip. This is mostly since mod_remoteip
> already has a bit more of a comprehensive suite of access checks for
> internal/trusted proxies and is (what I would expect) users would look
> toward first to enable this kind of functionality. The other part is
> that it could also help avoid namespace confusion with the many
> server-side proxy modules since they their module names with mod_proxy_*
> and their directives with Proxy*.
> 
> Thoughts?
> 

Sounds good... We could simply move the filter aspects over to
mod_remoteip and save us a module :)