Re: pf+TSO patch

[Kristof Provost]

A quick update:
Sean Bruno tested the patch and found a problem with rdr rules.
I've managed to reproduce and fix that. The current version on
https://reviews.freebsd.org/D3779 has the fix.

I believe that version to be working correctly (to the point that I
trust it with my own e-mail), but I'd appreciate further testing and/or
review.

Regards,
Kristof

On 2015-10-02 12:08:05 (+0200), Kristof Provost  wrote:
> Hi,
> 
> I've found a little time to look at the pf TSO issue (which made pf
> unusable on Xen VMs, like Amazon EC2).
> 
> I've posted the patch here:
> https://reviews.freebsd.org/D3779
> 
> It still needs a bit more testing, but so far it looks good.
> 
> I'd be very grateful for any brave souls who want to give this a try.
> 
> This work was very kindly sponsored by RootBSD (rootbsd.net).
> 
> Regards,
> Kristof
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: pf+TSO patch

[Sossi Andrej]

Kristof Provost je 02. 10. 2015 ob 12:08 napisal:
> Hi,
> 
> I've found a little time to look at the pf TSO issue (which made pf
> unusable on Xen VMs, like Amazon EC2).
> 
> I've posted the patch here:
> https://reviews.freebsd.org/D3779
> [...]

Thank you for your patch.

I have a similar problem but using ipfw. My issue is described in detail
in the attached message. You thing it is the same case?

Thank you in advance for your reply.
--- Begin Message ---
Hello,

I have a weird network problem which I believe may be caused by the
FreeBSD igb driver or perhaps even the network adapter.

Let me try to explain the scenario in brief:
I have a FreeBSD 10.0-RELEASE-p10  server with a public IP address, in
which N virtual machines are installed through JAIL; the machines hold
private IP addresses on the loopback1 adapter. The VMs access the
internet through NATting on the public IP via ipfw:

nat 1 config ip X.Y.Z.W if igb0 unreg_only same_ports

add 6 nat 1 ip from 192.168.250.0/24 to any out xmit igb0 keep-state
add 60001 nat 1 ip from any to X.Y.Z.W in recv igb0

In addition, port forwarding is configured on the real machine towards
the VMs in order to support public services (Apache httpd, database, etc.)

The network adapter is:
igb0: flags=8843 metric 0 mtu 1500

options=403bb
ether 00:45:80:dd:32:30
inet X.Y.Z.W netmask 0xff00 broadcast X.Y.Z.W
inet6 XX::YY:ZZ:WWW:VVV%igb0 prefixlen 64 scopeid 0x1
inet6 XX:YY:ZZ:WWW::1 prefixlen 64
nd6 options=23
media: Ethernet autoselect (1000baseT )
status: active

The loopback1 adapter, where the VMs' IPs are assigned, too, has MTU 1500.

So far so good, in the sense that everything works as expected, almost.
Occasionally there are requests originated by the VMs towards internet
servers which end in timeout (http, sftp, etc.). The very same requests,
if executed by the real machine, end correctly with a response.
After countless experiments I have managed to reproduce the problem
deterministically.

Through a tcpdump executed on the request's recipient I have noticed
that all TCP packets with a payload between 101 e 106  (inclusive)
bytes in size arrive with a wrong TCP checksum and as such are rejected.
Subsequent retransmissions of the same packet continue to bear a wrong
checksum
and this continues until the connection timeout is reached. The IP
checksum, instead, is always correct. Packets smaller than 101 bytes are
transmitted and received with the correct checksum, as the same happens
to packets with a payload in excess of 116 bytes in size.

If TXCSUM is disabled, the problem disappears.

The same problem I have on second server with same configuration and
hardware bat with FreeBSD 10.0-RELEASE-p1 .

I believe the above behavior is something error with the driver, as on
a third machine, with identical configuration with jail machines NATting
but with an em driver, the checksum problem didn't appear.

-- 
Cordiali saluti
Sossi Andrej
-
DOTCOM Information technology

Via Machiavelli, 28
34132 - Trieste (TS)
Italy

tel: +39 040 9828090
fax: +39 040 0641954
E-mail: aso...@dotcom.ts.it


Ai sensi del D.lgs n. 196 del 30.06.03 (Codice Privacy) si precisa che
le informazioni contenute in questo messaggio sono riservate e ad uso
esclusivo del destinatario. Qualora il messaggio in parola Le fosse
pervenuto per errore, La preghiamo di eliminarlo senza copiarlo e di non
inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie

This message, for the D.lgs n. 196 / 30.06.03 (Privacy Code), may
contain confidential and/or privileged information. If you are not the
addressee or authorized to receive this for the addressee, you must not
use, copy, disclose or take any action based on this message or any
information herein. If you have received this message in error, please
advise the sender immediately by reply e-mail and delete this message.
Thank you for your cooperation.
--- End Message ---
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: pf+TSO patch

[Kristof Provost]

On 2015-10-02 12:28:12 (+0200), Sossi Andrej  wrote:
> I have a similar problem but using ipfw. My issue is described in detail
> in the attached message. You thing it is the same case?
> 
> Thank you in advance for your reply.

I think it might be a different problem. The issue with PF mostly occurs
with large packets (because it's triggered by TSO).

This seems to only happen with very specific packet sizes, which is
quite interesting. It'd be very useful to know if this also happens
with other network cards.

Can you create a bug report on https://bugs.freebsd.org/bugzilla/ ?

Regards,
Kristof
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: pf+TSO patch

[Eric Joyner]

I don't know where that email goes, either. You should really open a bug
report; it'll be something to start off of in investigating the issue.

On Fri, Oct 2, 2015 at 5:01 AM Sossi Andrej  wrote:

> Kristof Provost je 02. 10. 2015 ob 13:24 napisal:
> > On 2015-10-02 12:28:12 (+0200), Sossi Andrej 
> wrote:
> >> I have a similar problem but using ipfw. My issue is described in detail
> >> in the attached message. You thing it is the same case?
> >>
> >> Thank you in advance for your reply.
> >
> > I think it might be a different problem. The issue with PF mostly occurs
> > with large packets (because it's triggered by TSO).
> >
> > This seems to only happen with very specific packet sizes, which is
> > quite interesting. It'd be very useful to know if this also happens
> > with other network cards.
> >
> > Can you create a bug report on https://bugs.freebsd.org/bugzilla/ ?
>
> No, I no open a bug. I' don't know if it is a bug of driver or ipfw or
> jail virtualization. I wrote only a message on the net-list and to
> original writer (free...@intel.com), but i don't receive any reply.
>
> I want to do other tests to determinate more precisely nature of bug,
> but servers who I reproduce the problem is a production server (!) and I
> have no time (!!!) to do this. Sorry.
> ___
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"