Okay, the log has lines such as these:
2020-12-17 14:01:57.397341 - - debug http2 2003:d5:72f:...: AH03348:
h2_task(70-15): open output to GET dotforward.de /
Then 2 seconds later when I opened the subdomain:
2020-12-17 14:01:59.413740 - - debug http2 2003:d5:72f:...: AH03082:
h2_stream(70-31,IDLE): created
2020-12-17 14:01:59.413754 - - debug http2 2003:d5:72f:...: AH03078:
h2_session(70,BUSY,1): transit [IDLE] -- stream change --> [BUSY]
2020-12-17 14:01:59.413783 - - debug http2 2003:d5:72f:...: AH03066:
h2_session(70,BUSY,1): recv FRAME[HEADERS[length=590, hend=1, stream=31,
eos=1]], frames=31/99 (r/s)
2020-12-17 14:01:59.413789 - - debug http2 2003:d5:72f:...: AH03066:
h2_session(70,BUSY,1): recv FRAME[WINDOW_UPDATE[stream=31, incr=12451840]],
frames=32/99 (r/s)
2020-12-17 14:01:59.413913 - - debug http2 2003:d5:72f:...: AH03348:
h2_task(83-31): open output to GET control.dotforward.de /
2020-12-17 14:01:59.413946 - - debug http2 2003:d5:72f:...: AH03073:
h2_stream(70-31,HALF_CLOSED_REMOTE): submit response 301,
REMOTE_WINDOW_SIZE=12582912
2020-12-17 14:01:59.413958 - - debug http2 2003:d5:72f:...: AH02936:
h2_stream(70-31,HALF_CLOSED_REMOTE): resumed
2020-12-17 14:01:59.413967 - - debug http2 2003:d5:72f:...: AH03068:
h2_session(70,BUSY,1): sent FRAME[HEADERS[length=76, hend=1, stream=31,
eos=0]], frames=33/100 (r/s)
2020-12-17 14:01:59.413974 - - debug http2 2003:d5:72f:...: AH03068:
h2_session(70,BUSY,1): sent FRAME[DATA[length=230, flags=1, stream=31,
padlen=0]], frames=33/101 (r/s)
2020-12-17 14:01:59.414017 - - debug http2 2003:d5:72f:...:
beam(83-31,output,closed=1,aborted=1,empty=1,buf=0): AH03385: h2_task_destroy,
reuse secondary
2020-12-17 14:01:59.414056 - - debug http2 2003:d5:72f:...: AH03078:
h2_session(70,IDLE,0): transit [BUSY] -- no io (keepalive) --> [IDLE]
2020-12-17 14:01:59.447946 - - debug http2 2003:d5:72f:...: AH03082:
h2_stream(70-33,IDLE): created
2020-12-17 14:01:59.447959 - - debug http2 2003:d5:72f:...: AH03078:
h2_session(70,BUSY,1): transit [IDLE] -- stream change --> [BUSY]
2020-12-17 14:01:59.447975 - - debug http2 2003:d5:72f:...: AH03066:
h2_session(70,BUSY,1): recv FRAME[HEADERS[length=22, hend=1, stream=33,
eos=1]], frames=33/101 (r/s)
2020-12-17 14:01:59.447981 - - debug http2 2003:d5:72f:...: AH03066:
h2_session(70,BUSY,1): recv FRAME[WINDOW_UPDATE[stream=33, incr=12451840]],
frames=34/101 (r/s)
2020-12-17 14:01:59.449109 - - debug http2 2003:d5:72f:...: AH03348:
h2_task(87-33): open output to GET dotforward.de /
2020-12-17 14:01:59.449134 - - debug http2 2003:d5:72f:...: AH03073:
h2_stream(70-33,HALF_CLOSED_REMOTE): submit response 200,
REMOTE_WINDOW_SIZE=12582912
2020-12-17 14:01:59.449147 - - debug http2 2003:d5:72f:...: AH02936:
h2_stream(70-33,HALF_CLOSED_REMOTE): resumed
2020-12-17 14:01:59.449153 - - debug http2 2003:d5:72f:...: AH03068:
h2_session(70,BUSY,1): sent FRAME[HEADERS[length=6, hend=1, stream=33, eos=0]],
frames=35/102 (r/s)
2020-12-17 14:01:59.449160 - - debug http2 2003:d5:72f:...: AH03068:
h2_session(70,BUSY,1): sent FRAME[DATA[length=1291, flags=0, stream=33,
padlen=0]], frames=35/103 (r/s)
2020-12-17 14:01:59.449164 - - debug http2 2003:d5:72f:...: AH03068:
h2_session(70,BUSY,1): sent FRAME[DATA[length=1291, flags=0, stream=33,
padlen=0]], frames=35/104 (r/s)
2020-12-17 14:01:59.449169 - - debug http2 2003:d5:72f:...: AH03068:
h2_session(70,BUSY,1): sent FRAME[DATA[length=359, flags=1, stream=33,
padlen=0]], frames=35/105 (r/s)
2020-12-17 14:01:59.449215 - - debug http2 2003:d5:72f:...:
beam(87-33,output,closed=1,aborted=1,empty=1,buf=0): AH03385: h2_task_destroy,
reuse secondary
2020-12-17 14:01:59.449241 - - debug http2 2003:d5:72f:...: AH03078:
h2_session(70,IDLE,0): transit [BUSY] -- no io (keepalive) --> [IDLE]
2020-12-17 14:02:04.449869 - - debug http2 2003:d5:72f:...: AH03068:
h2_session(70,IDLE,0): sent FRAME[GOAWAY[error=0, reason='timeout',
last_stream=33]], frames=35/106 (r/s)
2020-12-17 14:02:04.449936 - - debug http2 2003:d5:72f:...: AH03069:
h2_session(70,IDLE,0): sent GOAWAY, err=0, msg=timeout
2020-12-17 14:02:04.449942 - - debug http2 2003:d5:72f:...: AH03078:
h2_session(70,DONE,0): transit [IDLE] -- local goaway --> [DONE]
2020-12-17 14:02:04.449945 - - debug http2 2003:d5:72f:...: AH03078:
h2_session(70,CLEANUP,0): transit [DONE] -- pre_close --> [CLEANUP]
So Apache sees that I want to get something else.
I've erased my browser cache several times during these tests. I first
suspected that Firefox has a bug and loads different domains on its own,
as clearing the cache didn't help often. But it seems it's HTTP2's fault.
When the wrong site loads, Firefox shows a 301 redirect to the previous
domain. This is pretty nasty in the browser cache so I have to erase it
after such a situation.
The proxied apps do not have an access log. I can't see what requests
they get. But none of the apps know about their public domain so how
should they somehow act on it?
This is the VirtualHost config of the proxied app: (Port 80 left out here)
Listen [2a01:4f8:251:54f::bf96:8d02]:443
<VirtualHost 168.119.141.48:443 [2a01:4f8:251:54f::bf96:8d02]:443>
ServerName control.dotforward.de
ServerAlias www.control.dotforward.de
DocumentRoot /var/www/...
RewriteEngine on
# Redirection
RewriteCond %{HTTP_HOST} !^control\.dotforward\.de(:443)?$ [NC]
RewriteCond %{HTTP_HOST} !=""
RewriteRule ^/(.*) https://control.dotforward.de/$1 [L,R=301]
<Directory "/var/www/web001">
Options +IncludesNOEXEC
</Directory>
# ASP.NET app
ProxyPass "/wwwroot/app_offline.htm" !
ErrorDocument 503 "/wwwroot/app_offline.htm"
ProxyPass "/" "http://127.0.0.1:8000/"; retry=5
ProxyPassReverse "/" "http://127.0.0.1:8000/";
ProxyPreserveHost on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteRule .* ws://127.0.0.1:8000%{REQUEST_URI} [P]
RequestHeader set X-Forwarded-Proto "https"
SSLEngine on
SSLCertificateFile /etc/ssl/.../dotforward.de
SSLCertificateKeyFile /etc/ssl/.../dotforward.de
</VirtualHost>
There are the following settings:
* Redirect from a www subdomain to the domain without www
* Proxy to the app
* Also proxy WebSockets
* Tell the app this is https on the other side of the proxy
(probably useless)
* Use the wildcard certificate of the parent domain
Do I have to implement the foreign domain check myself? Does Apache not
know where to turn and requires me to tell it when it went wrong? How
would that work if we go down that route? I don't know how to configure
something like this:
if domain != {,www.}control.dotforward.de then
// Dear Apache, look again closely at the ServerName above!
respond with status 421
terminate request
end if
-Yves
PS: It's still confusing that my messages do not show up on the list.
Can't this be set up somewhere? Mailing lists are so complicated and
hard to use.
-------- Ursprüngliche Nachricht --------
Von: Stefan Eissing <stefan.eiss...@greenbytes.de>
Gesendet: Donnerstag, 17. Dezember 2020, 14:41 MEZ
Betreff: [users@httpd] Disable HTTP2 connection coalescing for different
virtual hosts/domains
Am 17.12.2020 um 14:05 schrieb Yves Goergen <nospam.l...@unclassified.de>:
I found out I cannot use a test environment because it doesn't have
wildcard certificates. So I had to quickly run this on the live server.
Now I have a bunch of log lines about http2. What should I look for and
how can I understand them? Please advise.
You should see log lines of the pattern:
[http2:debug] ... h2_task.c(83): [...] AH03348: h2_task(130-1): open
output to GET <hostname>:<port> <path>
where hostname, port and path specify what resource your browser
requested, irregardless on where the connection was started. If those
host names look correct, I would advise to look at the access log of
your proxied server to see which requests it sees.
Also, just for completeness, make sure your browser cache is empty and
does not carry resources from the time your server had an older,
different config. You can always use "curl" to get an honest opinion and
with "-v" also some good output of what actually happens on the client side.
Best regards,
Stefan
-Yves
-------- Ursprüngliche Nachricht --------
Von: Stefan Eissing <stefan.eiss...@greenbytes.de>
Gesendet: Dienstag, 15. Dezember 2020, 15:24 MEZ
Betreff: [users@httpd] Disable HTTP2 connection coalescing for different
virtual hosts/domains
Hi Yves,
there is no "intentional" misdirecting by the spec or the server. Let's
sort out where the problem lies and how to fix it.
1. You are correct that the browser will see your wildcard cert, see
that it applies to another host and use the already open connection to
make the request.
2. But the request should carry the hostname and thus forward it to your
backend proxy, just like with http/1.1. And since you have
"ProxyPreserveHost on" this should select the correct resources.
How can we find out where things go wrong?
- You could publish a different resource directly, without proxying in
your vhosts and check that the correct one is seen in your browser. That
would prove that the requests are made with the correct hostname.
- If this is not the case, a log with "LogLevel http2:debug" would help
to see what is wrong here.
- But if this works, then the mixup happens somewhere in the proxy
handling. What requests do you see incoming in your proxy logs in that case?
Best regards, Stefan
Am 15.12.2020 um 14:33 schrieb Yves Goergen <nospam.l...@unclassified.de>:
Hello,
I just found out the hard way that HTTP2 has a great new feature that
intentionally misdirects requests to the wrong domain. I'm using Apache
on Ubuntu 20.04 with Virtual Hosts, a single shared IPv4 address (what
else can you do these days), HTTP2 and HTTPS. Some of these domains use
the same wildcard certificate for the main domain and subdomains. Some
of these virtual hosts also use a reverse proxy to a backend application
server.
When I open both these sites after another in Firefox, I always get the
same content, even redirecting the second called domain back to the
first. So that HTTP2 connection coalescing thing is clearly a critical
bug in the spec or somewhere else that is expected to be worked around
by each and every webserver admin. How sad. They did say they wanted to
make it quicker. No word on safer or more reliable. Every optimisation
is a tradeoff, this time it broke things.
How should I do this now? I have the option to disable HTTP2 and deny
the progress. It immediately resolves the issue. Or I could somehow
somewhere make Apache respond with that 421 status code that teaches the
browsers that this feature is bad and they should not use it. How could
this be done? I wasn't able to find any resources about that. All sites'
config files look similar to this:
Listen [...IPv6...]:80
<VirtualHost ...SharedIPv4...:80 [...IPv6...]:80>
ServerName example.com
ServerAlias www.example.com
DocumentRoot /var/www/example/path
RewriteEngine on
# Redirection
RewriteRule ^/(.*) https://example.com/$1 [L,R=301]
<Directory "/var/www/example/path">
Options +IncludesNOEXEC
</Directory>
# CGI/PHP (optional)
SuexecUserGroup example webusers
FcgidWrapper /var/www/php-bin/example/php-fcgi .php
AddHandler fcgid-script .php
# ASP.NET app (optional)
ProxyPass "/" "http://127.0.0.1:7001/"; retry=5
ProxyPassReverse "/" "http://127.0.0.1:7001/";
ProxyPreserveHost on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteRule .* ws://127.0.0.1:7001%{REQUEST_URI} [P]
RequestHeader set X-Forwarded-Proto "http"
</VirtualHost>
Listen [...IPv6...]:443
<VirtualHost ...SharedIPv4...:443 [...IPv6...]:443>
ServerName example.com
ServerAlias www.example.com
DocumentRoot /var/www/example/path
RewriteEngine on
# Redirection
RewriteCond %{HTTP_HOST} !^example\.com(:443)?$ [NC]
RewriteCond %{HTTP_HOST} !=""
RewriteRule ^/(.*) https://example.com/$1 [L,R=301]
<Directory "/var/www/example/path">
Options +IncludesNOEXEC
</Directory>
# CGI/PHP (optional)
SuexecUserGroup example webusers
FcgidWrapper /var/www/php-bin/example/php-fcgi .php
AddHandler fcgid-script .php
# ASP.NET app (optional)
ProxyPass "/" "http://127.0.0.1:7001/"; retry=5
ProxyPassReverse "/" "http://127.0.0.1:7001/";
ProxyPreserveHost on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteRule .* ws://127.0.0.1:7001%{REQUEST_URI} [P]
RequestHeader set X-Forwarded-Proto "https"
SSLEngine on
SSLCertificateFile /etc/ssl/private/example.com
SSLCertificateKeyFile /etc/ssl/private/example.com
</VirtualHost>
-Yves
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
