Re: [squid-users] Empty transfer-encoding header causes 502 response

2022-10-26 Thread Alex Rousskov 

On 10/25/22 20:55, Matthew H wrote:


I have included the requested output from tcpdump below:


Thank you! This raw output is sufficient to determine that no transfer 
encoding was used by this buggy origin server. I have updated the GitHub 
comment/summary accordingly.


N.B. In the future, please consider sharing libpcap packet captures 
instead of raw tcpdump console output. It is not necessary to re-share 
anything now.


FWIW, I am not aware of any official Squid workarounds for this origin 
server bug. Some of the features Factory is currently working on will be 
useful here, but they are not yet ready for the official submission. One 
can remove the corresponding check from Squid source code, of course, 
but doing so will open modified Squid (and other HTTP agents) to serious 
security vulnerabilities, so I cannot recommend such a blunt workaround.


Alex.


  tcpdump -A -s 0 -ni enp4s0 "host 159.203.14.9 and (((ip[2:2] - 
((ip[0]&0xf)<<2)) - ((tcp[12]&

0xf0)>>2)) != 0)"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp4s0, link-type EN10MB (Ethernet), snapshot length 262144 
bytes


01:40:17.310479 IP 10.0.160.10.43426 > 159.203.14.9.1996: Flags [P.], 
seq 2955630477:2955630939, ack 2382737005, win 502, options [nop,nop,TS 
val 3000375654 ecr 1932743995], length 462

E:@.?.7.
..
...     .+WmY..
...fs3U;GET http://nintendo.com/  HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) 
Gecko/20100101 Firefox/106.0
Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8

Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Upgrade-Insecure-Requests: 1
Host: nintendo.com 
Via: 1.1 dce3749b9671 (squid/5.6)
X-Forwarded-For: 10.0.130.210
Cache-Control: max-age=259200
Connection: keep-alive


01:40:18.957475 IP 159.203.14.9.1996 > 10.0.160.10.43426: Flags [P.], 
seq 1:1466, ack 462, win 114, options [nop,nop,TS val 1932744407 ecr 
3000375654], length 1465

E@@.3..
..
...m.+Y[...r]..
s3VfHTTP/1.1 200 OK
x-powered-by: Express
content-type: text/html; charset=iso-8859-1
transfer-encoding:
date: Wed, 26 Oct 2022 00:40:20 GMT
connection: close



http://nintendo.com//./hallway/index.html 
">

Nintendo Power Source



On Tue, Oct 25, 2022 at 2:08 PM Alex Rousskov 
> wrote:


On 10/23/22 20:36, Matthew H wrote:
 > Hi,
 >
 > I'm using Squid to proxy HTTP requests to another proxy. I can
see squid
 > sending the request to the parent and getting a response, but it
sends
 > the client that initiated the request a 502 Bad Gateway response.
 >
 > On closer inspection it appears the parent proxy is sending an
 > empty transfer-encoding header, and this is causing Squid to send
a 502.

Do you know whether the response body was using chunked (or any other
non-identity) encoding? I have already added your case to the list of
known rejected responses[1], but it would be good to update that with
the information on the actual response encoding.

[1]
https://github.com/squid-cache/squid/pull/702#issuecomment-762459132


If the very first bytes of the response are " 2022/10/24 00:23:59.106| ctx: enter level  0:
'http://nintendo.com/ 
 > >'
 > 2022/10/24 00:23:59.106| 11,3| http.cc(666) processReplyHeader:
 > processReplyHeader: key '19010C00'
 > 2022/10/24 00:23:59.106| 11,2| http.cc(720) processReplyHeader: HTTP
 > Server conn294 local=172.25.0.3:57802 
 > > remote=159.203.14.9:1996

 > > FIRSTUP_PARENT FD 26 flags=1
 > 2022/10/24 00:23:59.106| 11,2| http.cc(721) processReplyHeader: HTTP
 > Server RESPONSE:
 > -
 > HTTP/1.1 200 OK
 > x-powered-by: Express
 > content-type: text/html; charset=iso-8859-1
 > transfer-encoding:
 > date: Mon, 24 Oct 2022 00:23:57 GMT
 > connection: close
 >
 > --
 > 2022/10/24 00:23:59.106| 55,3| HttpHeader.cc(882) getList: empty
list
 > header: Transfer-Encoding(Transfer-Encoding[63])
 > 2022/10/24 00:23:59.106| 55,2| HttpHeader.cc(559) parse: WARNING:
 > unsupported Transfer-Encoding used by client:
 > 2022/10/24 00:23:59.106| ctx: exit level  0
 > 2022/10/24 00:23:59.106| 20,3| store.cc(1673) reset:
 > http://nintendo.com/  >
 > 2022/10/24 00:23:59.107| 17,3| FwdState.cc(492) fail:
ERR_INVALID_RESP
 > "Bad Gateway"

Re: [squid-users] Empty transfer-encoding header causes 502 response

2022-10-25 Thread Matthew H 
Hi all,

Thanks for the replies.

I have included the requested output from tcpdump below:

 tcpdump -A -s 0 -ni enp4s0 "host 159.203.14.9 and (((ip[2:2] -
((ip[0]&0xf)<<2)) - ((tcp[12]&
0xf0)>>2)) != 0)"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp4s0, link-type EN10MB (Ethernet), snapshot length 262144
bytes

01:40:17.310479 IP 10.0.160.10.43426 > 159.203.14.9.1996: Flags [P.], seq
2955630477:2955630939, ack 2382737005, win 502, options [nop,nop,TS val
3000375654 ecr 1932743995], length 462
E:@.?.7.
..
... .+WmY..
...fs3U;GET http://nintendo.com/ HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0)
Gecko/20100101 Firefox/106.0
Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Upgrade-Insecure-Requests: 1
Host: nintendo.com
Via: 1.1 dce3749b9671 (squid/5.6)
X-Forwarded-For: 10.0.130.210
Cache-Control: max-age=259200
Connection: keep-alive


01:40:18.957475 IP 159.203.14.9.1996 > 10.0.160.10.43426: Flags [P.], seq
1:1466, ack 462, win 114, options [nop,nop,TS val 1932744407 ecr
3000375654], length 1465
E@@.3..
..
...m.+Y[...r]..
s3VfHTTP/1.1 200 OK
x-powered-by: Express
content-type: text/html; charset=iso-8859-1
transfer-encoding:
date: Wed, 26 Oct 2022 00:40:20 GMT
connection: close



http://nintendo.com//./hallway/index.html;>
Nintendo Power Source



On Tue, Oct 25, 2022 at 2:08 PM Alex Rousskov <
rouss...@measurement-factory.com> wrote:

> On 10/23/22 20:36, Matthew H wrote:
> > Hi,
> >
> > I'm using Squid to proxy HTTP requests to another proxy. I can see squid
> > sending the request to the parent and getting a response, but it sends
> > the client that initiated the request a 502 Bad Gateway response.
> >
> > On closer inspection it appears the parent proxy is sending an
> > empty transfer-encoding header, and this is causing Squid to send a 502.
>
> Do you know whether the response body was using chunked (or any other
> non-identity) encoding? I have already added your case to the list of
> known rejected responses[1], but it would be good to update that with
> the information on the actual response encoding.
>
> [1] https://github.com/squid-cache/squid/pull/702#issuecomment-762459132
>
> If the very first bytes of the response are " encoding was probably applied. If you see what can be interpreted as a
> small hex number followed by a new line, then chunked encoding was
> probably applied (at least). If you cannot tell, or are not sure, feel
> free to share the response packet in libpcap format, captured with
> wireshark or "tcpdump -s0".
>
>
> Thank you,
>
> Alex.
>
>
>
> > 2022/10/24 00:23:59.106| ctx: enter level  0: 'http://nintendo.com/
> > '
> > 2022/10/24 00:23:59.106| 11,3| http.cc(666) processReplyHeader:
> > processReplyHeader: key '19010C00'
> > 2022/10/24 00:23:59.106| 11,2| http.cc(720) processReplyHeader: HTTP
> > Server conn294 local=172.25.0.3:57802
> >  remote=159.203.14.9:1996
> >  FIRSTUP_PARENT FD 26 flags=1
> > 2022/10/24 00:23:59.106| 11,2| http.cc(721) processReplyHeader: HTTP
> > Server RESPONSE:
> > -
> > HTTP/1.1 200 OK
> > x-powered-by: Express
> > content-type: text/html; charset=iso-8859-1
> > transfer-encoding:
> > date: Mon, 24 Oct 2022 00:23:57 GMT
> > connection: close
> >
> > --
> > 2022/10/24 00:23:59.106| 55,3| HttpHeader.cc(882) getList: empty list
> > header: Transfer-Encoding(Transfer-Encoding[63])
> > 2022/10/24 00:23:59.106| 55,2| HttpHeader.cc(559) parse: WARNING:
> > unsupported Transfer-Encoding used by client:
> > 2022/10/24 00:23:59.106| ctx: exit level  0
> > 2022/10/24 00:23:59.106| 20,3| store.cc(1673) reset:
> > http://nintendo.com/ 
> > 2022/10/24 00:23:59.107| 17,3| FwdState.cc(492) fail: ERR_INVALID_RESP
> > "Bad Gateway"
> > http://nintendo.com/ 
> > 2022/10/24 00:23:59.107| 17,3| FwdState.cc(533) unregister:
> > http://nintendo.com/ 
> >
> > ___
> > squid-users mailing list
> > squid-users@lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Empty transfer-encoding header causes 502 response

2022-10-25 Thread Alex Rousskov 

On 10/23/22 20:36, Matthew H wrote:

Hi,

I'm using Squid to proxy HTTP requests to another proxy. I can see squid 
sending the request to the parent and getting a response, but it sends 
the client that initiated the request a 502 Bad Gateway response.


On closer inspection it appears the parent proxy is sending an 
empty transfer-encoding header, and this is causing Squid to send a 502. 


Do you know whether the response body was using chunked (or any other 
non-identity) encoding? I have already added your case to the list of 
known rejected responses[1], but it would be good to update that with 
the information on the actual response encoding.


[1] https://github.com/squid-cache/squid/pull/702#issuecomment-762459132

If the very first bytes of the response are "encoding was probably applied. If you see what can be interpreted as a 
small hex number followed by a new line, then chunked encoding was 
probably applied (at least). If you cannot tell, or are not sure, feel 
free to share the response packet in libpcap format, captured with 
wireshark or "tcpdump -s0".



Thank you,

Alex.



2022/10/24 00:23:59.106| ctx: enter level  0: 'http://nintendo.com/ 
'
2022/10/24 00:23:59.106| 11,3| http.cc(666) processReplyHeader: 
processReplyHeader: key '19010C00'
2022/10/24 00:23:59.106| 11,2| http.cc(720) processReplyHeader: HTTP 
Server conn294 local=172.25.0.3:57802 
 remote=159.203.14.9:1996 
 FIRSTUP_PARENT FD 26 flags=1
2022/10/24 00:23:59.106| 11,2| http.cc(721) processReplyHeader: HTTP 
Server RESPONSE:

-
HTTP/1.1 200 OK
x-powered-by: Express
content-type: text/html; charset=iso-8859-1
transfer-encoding:
date: Mon, 24 Oct 2022 00:23:57 GMT
connection: close

--
2022/10/24 00:23:59.106| 55,3| HttpHeader.cc(882) getList: empty list 
header: Transfer-Encoding(Transfer-Encoding[63])
2022/10/24 00:23:59.106| 55,2| HttpHeader.cc(559) parse: WARNING: 
unsupported Transfer-Encoding used by client:

2022/10/24 00:23:59.106| ctx: exit level  0
2022/10/24 00:23:59.106| 20,3| store.cc(1673) reset: 
http://nintendo.com/ 
2022/10/24 00:23:59.107| 17,3| FwdState.cc(492) fail: ERR_INVALID_RESP 
"Bad Gateway"

http://nintendo.com/ 
2022/10/24 00:23:59.107| 17,3| FwdState.cc(533) unregister: 
http://nintendo.com/ 


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Empty transfer-encoding header causes 502 response

2022-10-25 Thread squid3 

On 2022-10-24 13:36, Matthew H wrote:

Hi,

I'm using Squid to proxy HTTP requests to another proxy. I can see 
squid
sending the request to the parent and getting a response, but it sends 
the

client that initiated the request a 502 Bad Gateway response.


That is correct behaviour. Squid does not know how to decode the content 
for delivery.




On closer inspection it appears the parent proxy is sending an
empty transfer-encoding header, and this is causing Squid to send a 
502. Is

there any way to ignore this?



This MUST NOT be ignored. The server has explicitly indicated that the 
response content area is encoded, but not how. Squid cannot tell where 
the boundaries of the message content are, nor how to transform it for 
delivery to the client.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Empty transfer-encoding header causes 502 response

2022-10-23 Thread Matthew H 
Hi,

I'm using Squid to proxy HTTP requests to another proxy. I can see squid
sending the request to the parent and getting a response, but it sends the
client that initiated the request a 502 Bad Gateway response.

On closer inspection it appears the parent proxy is sending an
empty transfer-encoding header, and this is causing Squid to send a 502. Is
there any way to ignore this? It looks like its enforced in
HttpHeader.cc#L510


I'm using Squid 5.6, or more specifically the ubuntu/squid:5.6-22.10_edge
docker image.

I've included the logs below.

Thanks
Matthew

2022/10/24 00:23:59.106| ctx: enter level  0: 'http://nintendo.com/'
2022/10/24 00:23:59.106| 11,3| http.cc(666) processReplyHeader:
processReplyHeader: key '19010C00'
2022/10/24 00:23:59.106| 11,2| http.cc(720) processReplyHeader: HTTP Server
conn294 local=172.25.0.3:57802 remote=159.203.14.9:1996 FIRSTUP_PARENT FD
26 flags=1
2022/10/24 00:23:59.106| 11,2| http.cc(721) processReplyHeader: HTTP Server
RESPONSE:
-
HTTP/1.1 200 OK
x-powered-by: Express
content-type: text/html; charset=iso-8859-1
transfer-encoding:
date: Mon, 24 Oct 2022 00:23:57 GMT
connection: close

--
2022/10/24 00:23:59.106| 55,3| HttpHeader.cc(882) getList: empty list
header: Transfer-Encoding(Transfer-Encoding[63])
2022/10/24 00:23:59.106| 55,2| HttpHeader.cc(559) parse: WARNING:
unsupported Transfer-Encoding used by client:
2022/10/24 00:23:59.106| ctx: exit level  0
2022/10/24 00:23:59.106| 20,3| store.cc(1673) reset: http://nintendo.com/
2022/10/24 00:23:59.107| 17,3| FwdState.cc(492) fail: ERR_INVALID_RESP "Bad
Gateway"
http://nintendo.com/
2022/10/24 00:23:59.107| 17,3| FwdState.cc(533) unregister:
http://nintendo.com/
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users