Re: TCP: tcp_parse_options: Illegal window scaling value 15 > 14 received

[Casey Deccio]

> On Sep 7, 2022, at 12:41 PM, Jim Popovitch  wrote:
> 
> On Wed, 2022-09-07 at 12:37 -0600, Casey Deccio wrote:
>> 
>>> On Sep 7, 2022, at 11:46 AM, Jim Popovitch >> > wrote:
>>> 
>>> I saw some much of the verbose '15 > 14' logs that I just decided to
>>> net.ipv4.tcp_window_scaling=0 and be done with it. Cleared up the
>>> noise, haven't noticed any problems since. ymmv.
>> 
>> Sounds like you've seen a non-trivial amount of this?
>> 
>> Disabling window scaling with net.ipv4.tcp_window_scaling=0 will "fix" the 
>> logs, but of course, it will also disable window scaling, which means that 
>> you are limiting the size of your congestion window to 64KB.  This 
>> effectively limits the throughput of TCP sessions over "long, fat pipes".
> 
> Yep.  I don't own/run/maintain anything on fat pipes, just destinations such 
> as webservers, email  servers, and dns servers for mailinglists.  If the 
> bandwidth for them is now capped at 2MBs, that's ok in my book.   The notion 
> that everything needs to support 10Gb interfaces and terabyte sized hardware 
> is just not realistic. 

Fair enough/  Just trying to interpret "ymmv" :)

Cheers,
Casey

Re: TCP: tcp_parse_options: Illegal window scaling value 15 > 14 received

[Jim Popovitch]

On Wed, 2022-09-07 at 12:37 -0600, Casey Deccio wrote:
> 
> > On Sep 7, 2022, at 11:46 AM, Jim Popovitch  wrote:
> > 
> > I saw some much of the verbose '15 > 14' logs that I just decided to
> > net.ipv4.tcp_window_scaling=0 and be done with it. Cleared up the
> > noise, haven't noticed any problems since. ymmv.
> 
> Sounds like you've seen a non-trivial amount of this?
> 
> Disabling window scaling with net.ipv4.tcp_window_scaling=0 will "fix"
> the logs, but of course, it will also disable window scaling, which
> means that you are limiting the size of your congestion window to
> 64KB.  This effectively limits the throughput of TCP sessions over
> "long, fat pipes".


Yep.  I don't own/run/maintain anything on fat pipes, just destinations
such as webservers, email  servers, and dns servers for mailinglists.
 If the bandwidth for them is now capped at 2MBs, that's ok in my book.
  The notion that everything needs to support 10Gb interfaces and
terabyte sized hardware is just not realistic. 

-Jim P.

Re: TCP: tcp_parse_options: Illegal window scaling value 15 > 14 received

[Casey Deccio]

> On Sep 7, 2022, at 11:46 AM, Jim Popovitch  wrote:
> 
> I saw some much of the verbose '15 > 14' logs that I just decided to
> net.ipv4.tcp_window_scaling=0 and be done with it. Cleared up the
> noise, haven't noticed any problems since. ymmv.

Sounds like you've seen a non-trivial amount of this?

Disabling window scaling with net.ipv4.tcp_window_scaling=0 will "fix" the 
logs, but of course, it will also disable window scaling, which means that you 
are limiting the size of your congestion window to 64KB.  This effectively 
limits the throughput of TCP sessions over "long, fat pipes".

Casey

Re: TCP: tcp_parse_options: Illegal window scaling value 15 > 14 received

[Jim Popovitch]

On Wed, 2022-09-07 at 11:09 -0600, Casey Deccio wrote:
> Hi Michael,
> 
> > On Sep 7, 2022, at 5:49 AM, Michael Grant  wrote:
> > 
> > I'm seeing this error over and over in /var/log/messages:
> > 
> > Sep  6 05:02:42 hostname kernel: [408794.655182] TCP: tcp_parse_options: 
> > Illegal window scaling value 15 > 14 received
> > Sep  6 05:02:43 hostname kernel: [408794.830639] TCP: tcp_parse_options: 
> > Illegal window scaling value 15 > 14 received
> > Sep  6 05:02:43 hostname kernel: [408794.960811] TCP: tcp_parse_options: 
> > Illegal window scaling value 15 > 14 received
> > Sep  6 05:02:43 hostname kernel: [408795.180464] TCP: tcp_parse_options: 
> > Illegal window scaling value 15 > 14 received
> > 
> > I've not been able to find much about these messages by searching,
> > nothing useful is coming up.  Is anyone else seeing something like
> > this?
> 
> This is consistent with RFC 7323, Section 2.3 [1], which states:
> 
>    "If a
>    Window Scale option is received with a shift.cnt value larger than
>    14, the TCP SHOULD log the error but MUST use 14 instead of the
>    specified value."
> 
> >  Is this some sort of attack?
> 
> I am not sure.  But the purpose of keeping the window scale below 15 is to 
> "insure that new data is never mistakenly considered old and vice versa" [1]. 
>  In any case, it seems to me that 1) your kernel appears to be handling it 
> properly (hence the logs) and 2) even if it weren't, it doesn't *seem* like a 
> problem for the server as much as for the entity that wanted the data.  Just 
> my $0.02.
> 
> Interestingly, I happen to have some software using different window scale 
> values in its interactions with Internet servers.  I just yesterday 
> discovered a bug which was occasionally allowing 15 to be used as a window 
> scale value, and I have corrected that. I don't know if my software was 
> responsible for the log messages that Michael observed, but I have reached 
> out off-list to investigate.
> 

I saw some much of the verbose '15 > 14' logs that I just decided to
net.ipv4.tcp_window_scaling=0 and be done with it.  Cleared up the
noise, haven't noticed any problems since.  ymmv.

-Jim P.

Re: TCP: tcp_parse_options: Illegal window scaling value 15 > 14 received

[Casey Deccio]

Hi Michael,

> On Sep 7, 2022, at 5:49 AM, Michael Grant  wrote:
> 
> I'm seeing this error over and over in /var/log/messages:
> 
> Sep  6 05:02:42 hostname kernel: [408794.655182] TCP: tcp_parse_options: 
> Illegal window scaling value 15 > 14 received
> Sep  6 05:02:43 hostname kernel: [408794.830639] TCP: tcp_parse_options: 
> Illegal window scaling value 15 > 14 received
> Sep  6 05:02:43 hostname kernel: [408794.960811] TCP: tcp_parse_options: 
> Illegal window scaling value 15 > 14 received
> Sep  6 05:02:43 hostname kernel: [408795.180464] TCP: tcp_parse_options: 
> Illegal window scaling value 15 > 14 received
> 
> I've not been able to find much about these messages by searching,
> nothing useful is coming up.  Is anyone else seeing something like
> this?

This is consistent with RFC 7323, Section 2.3 [1], which states:

   "If a
   Window Scale option is received with a shift.cnt value larger than
   14, the TCP SHOULD log the error but MUST use 14 instead of the
   specified value."

>  Is this some sort of attack?

I am not sure.  But the purpose of keeping the window scale below 15 is to 
"insure that new data is never mistakenly considered old and vice versa" [1].  
In any case, it seems to me that 1) your kernel appears to be handling it 
properly (hence the logs) and 2) even if it weren't, it doesn't *seem* like a 
problem for the server as much as for the entity that wanted the data.  Just my 
$0.02.

Interestingly, I happen to have some software using different window scale 
values in its interactions with Internet servers.  I just yesterday discovered 
a bug which was occasionally allowing 15 to be used as a window scale value, 
and I have corrected that. I don't know if my software was responsible for the 
log messages that Michael observed, but I have reached out off-list to 
investigate.

Casey

[1] https://www.rfc-editor.org/rfc/rfc7323.html

TCP: tcp_parse_options: Illegal window scaling value 15 > 14 received

[Michael Grant]

I'm seeing this error over and over in /var/log/messages:

Sep  6 05:02:42 hostname kernel: [408794.655182] TCP: tcp_parse_options: 
Illegal window scaling value 15 > 14 received
Sep  6 05:02:43 hostname kernel: [408794.830639] TCP: tcp_parse_options: 
Illegal window scaling value 15 > 14 received
Sep  6 05:02:43 hostname kernel: [408794.960811] TCP: tcp_parse_options: 
Illegal window scaling value 15 > 14 received
Sep  6 05:02:43 hostname kernel: [408795.180464] TCP: tcp_parse_options: 
Illegal window scaling value 15 > 14 received

I've not been able to find much about these messages by searching,
nothing useful is coming up.  Is anyone else seeing something like
this?  Is this some sort of attack?

Please cc me on replies, thanks.

Michael Grant


signature.asc
Description: PGP signature