Re: AW: AW: AW: IPoIB GRO

2013-11-05 Thread Erez Shitrit
I see. This didn't happen on our setups here since we tests with newer cards (ConnectX2/3/3-pro). For ConnectX1 (A0) and this firmware that you are using smells like something goes wrong. If possible, I would change to newish card. No problem with that. My journey up to here was hard but very

Re: AW: AW: AW: IPoIB GRO

2013-11-05 Thread Or Gerlitz
On 05/11/2013 10:25, Markus Stockhausen wrote: Are the TCP Ids in an TSO setup generated through firmware or in the software stack? in HW And if in firmware: How does the card know how to increase them? I would expect that it only works with IB packets and does not know of the IP

Re: AW: AW: AW: IPoIB GRO

2013-11-05 Thread Jason Gunthorpe
On Tue, Nov 05, 2013 at 10:48:10AM +0200, Erez Shitrit wrote: so, to summarize: The HW does the work (truncates the big ip packet to series of ip packets, each with the relevant mtu size and increases the ip-id for each) The FW enables that work on the HW the FW in A0 card doesn't enable

AW: AW: AW: AW: IPoIB GRO

2013-11-05 Thread Markus Stockhausen
so, to summarize: The HW does the work (truncates the big ip packet to series of ip packets, each with the relevant mtu size and increases the ip-id for each) The FW enables that work on the HW the FW in A0 card doesn't enable that option for the HW. Sounds like this bug causes a

AW: IPoIB GRO

2013-11-05 Thread Markus Stockhausen
Von: Or Gerlitz [ogerl...@mellanox.com] Gesendet: Mittwoch, 6. November 2013 08:50 An: Markus Stockhausen; Jason Gunthorpe; Erez Shitrit Cc: linux-rdma@vger.kernel.org; Wendy Cheng Betreff: Re: IPoIB GRO On 05/11/2013 20:08, Markus Stockhausen wrote: Incredible how a card that does not

AW: IPoIB GRO

2013-11-04 Thread Markus Stockhausen
Thats why the flush flag is always set and the GRO stack does not work at all. I'm willing to dig deeper into this but I'm unsure if those fields are filled on sender or receiver side and especially where in the IPoIB stack. Maybe I got the reason for that strange ack behaviour during large

AW: IPoIB GRO

2013-11-04 Thread Markus Stockhausen
Hi Erez, don't you see that behaviour in tcpdump? what kernel are you using? On server side we have a 3.5 on client side a 3.11 kernel each of them with kernel standard drivers/modules. I can see the same pattern of GRO aggregation on the client that you mention but only if I disable TSO for

Re: AW: IPoIB GRO

2013-11-04 Thread Erez Shitrit
Hi Markus, Can you please tell me what is the FW version you have on your ConnectX cards? Thanks, Erez Hi Erez, don't you see that behaviour in tcpdump? what kernel are you using? On server side we have a 3.5 on client side a 3.11 kernel each of them with kernel standard drivers/modules.

AW: AW: IPoIB GRO

2013-11-04 Thread Markus Stockhausen
Hi Markus, Can you please tell me what is the FW version you have on your ConnectX cards? of course. the server has: root@client:~# ibstat CA 'mlx4_0' CA type: MT26418 Number of ports: 1 Firmware version: 2.9.1000 Hardware version: a0 Node GUID:

Re: AW: IPoIB GRO

2013-11-04 Thread Wendy Cheng
I looked at TSO code earlier this year. IIRC, if TSO is on, the upper layer (e.g. IP) would just send the super-packet down (to IPOIB) w/out segmentation (for send); if off, it then does the segmentation (to match the MTU size) before calling device's send. For GSO, I would imagine it needs some