Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 16-Apr-19 17:39, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has obtained
such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has been added
to the mlx5e_priv structure to track its usage. Because the send tag resides
in the mlx5e_channel structure, there is no need to wait for refcounts to
reach
zero until the mlx4en(4) driver is detached. The channels structure is
persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be
accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because there are now
two different tag types which end up in the driver which need to be
distinguished.
Submitted by: hselasky@
Approved by:hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+* Tell upper layers about route
+* change and to re-transmit this
+* packet:
+*/
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi Andrey,
Thanks for letting us know about this regression.
I would like to try to reproduce this issue in house.
Can you please share the exact steps to reproduce it?
- Can I reproduce the issue with B2B setup?
- What is the route command you used to make the route between the VLANs?
- What app are you using to generate the traffic?
Slava
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 16.04.2019 18:26, Slava Shwartsman wrote: > Thanks for letting us know about this regression. > I would like to try to reproduce this issue in house. > > Can you please share the exact steps to reproduce it? > - Can I reproduce the issue with B2B setup? > - What is the route command you used to make the route between the VLANs? > - What app are you using to generate the traffic? > I think this can be reproduced on simple router, where single mce(4) interface is used as parent for several vlan(4) interfaces. E.g. [host1] vlan100 <--> mce0.100 [gateway] mce0.200 <--> vlan200 [host2] 10.0.0.110.0.0.254 192.168.0.254192.168.0.1 gateway: sysctl net.inet.ip.forwarding=1 host1: route add 192.168.0.0/24 10.0.0.254 host2: route add 10.0.0.0/24 192.168.0.254 ping 10.0.0.1 I.e. you need to make setup, where ingress and egress interface is the same - mce0. -- WBR, Andrey V. Elsukov signature.asc Description: OpenPGP digital signature
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
> On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
>> On 05.12.2018 17:25, Slava Shwartsman wrote:
>>> Author: slavash
>>> Date: Wed Dec 5 14:25:03 2018
>>> New Revision: 341586
>>> URL: https://svnweb.freebsd.org/changeset/base/341586
>>>
>>> Log:
>>>mlx5en: Implement backpressure indication.
>>>
>>>The backpressure indication is implemented using an unlimited rate type
>>> of
>>>mbuf send tag. When the upper layers typically the socket layer has
>>> obtained such
>>>a tag, it can then query the destination driver queue for the current
>>>amount of space available in the send queue.
>>>
>>>A single mbuf send tag may be referenced multiple times and a refcount
>>> has been added
>>>to the mlx5e_priv structure to track its usage. Because the send tag
>>> resides
>>>in the mlx5e_channel structure, there is no need to wait for refcounts
>>> to reach
>>>zero until the mlx4en(4) driver is detached. The channels structure is
>>> persistant
>>>during the lifetime of the mlx5en(4) driver it belongs to and can so be
>>> accessed
>>>without any need of synchronization.
>>>
>>>The mlx5e_snd_tag structure was extended to contain a type field,
>>> because there are now
>>>two different tag types which end up in the driver which need to be
>>> distinguished.
>>>
>>>Submitted by: hselasky@
>>>Approved by:hselasky (mentor)
>>>MFC after: 1 week
>>>Sponsored by: Mellanox Technologies
>>> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
>>> struct mlx5e_sq *sq;
>>> int ret;
>>>
>>> - sq = mlx5e_select_queue(ifp, mb);
>>> - if (unlikely(sq == NULL)) {
>>> -#ifdef RATELIMIT
>>> - /* Check for route change */
>>> - if (mb->m_pkthdr.snd_tag != NULL &&
>>> - mb->m_pkthdr.snd_tag->ifp != ifp) {
>>> + if (mb->m_pkthdr.snd_tag != NULL) {
>>> + sq = mlx5e_select_queue_by_send_tag(ifp, mb);
>>> + if (unlikely(sq == NULL)) {
>>> + /* Check for route change */
>>> + if (mb->m_pkthdr.snd_tag->ifp != ifp) {
>>> + /* Free mbuf */
>>> + m_freem(mb);
>>> +
>>> + /*
>>> +* Tell upper layers about route
>>> +* change and to re-transmit this
>>> +* packet:
>>> +*/
>>> + return (EAGAIN);
>>> + }
>>
>> Hi,
>>
>> I just discovered something strange and found that this commit is the
>> cause.
>> The test system has mlx5en 100G interface. It has two vlans: vlan500 and
>> vlan100.
>> Via vlan500 it receives some packets flows. Then it routes these packets
>> into vlan100.
>> But packets are dropped in mlx5e_xmit() with EAGAIN error code.
>>
>> # dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
>> dtrace: description 'fbt::ip6_output:return ' matched 1 probe
>> CPU IDFUNCTION:NAME
>> 23 54338ip6_output:return 35
>> 16 54338ip6_output:return 35
>> 21 54338ip6_output:return 35
>> 22 54338ip6_output:return 35
>> 24 54338ip6_output:return 35
>> 23 54338ip6_output:return 35
>> 14 54338ip6_output:return 35
>> ^C
>>
>> # dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
>> dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
>> CPU IDFUNCTION:NAME
>> 16 69030mlx5e_xmit:return 35
>> 23 69030mlx5e_xmit:return 35
>> 26 69030mlx5e_xmit:return 35
>> 25 69030mlx5e_xmit:return 35
>> 24 69030mlx5e_xmit:return 35
>> 21 69030mlx5e_xmit:return 35
>> 26 69030mlx5e_xmit:return 35
>> ^C
>>
>> The kernel config is GENERIC.
>> 13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
>>
>
> Hi,
>
> This might be a case where rcvif in the mbuf's pktheader is not cleared
> before the packet is fed back on the wire.
>
> John Baldwin is working on the send tags implementation, to eliminate
> the EAGAIN handling in the network drivers.
I will try to push this branch sooner then since it affects more than just
TLS. Part of the change includes a new flag we can use to assert that we
aren't just getting a stale rcvif (though there are also now assertions in
ip_output that should catch this case I think).
--
John Baldwin
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has obtained
such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has been added
to the mlx5e_priv structure to track its usage. Because the send tag resides
in the mlx5e_channel structure, there is no need to wait for refcounts to
reach
zero until the mlx4en(4) driver is detached. The channels structure is
persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be
accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because there are now
two different tag types which end up in the driver which need to be
distinguished.
Submitted by: hselasky@
Approved by:hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+* Tell upper layers about route
+* change and to re-transmit this
+* packet:
+*/
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi,
This might be a case where rcvif in the mbuf's pktheader is not cleared
before the packet is fed back on the wire.
John Baldwin is working on the send tags implementation, to eliminate
the EAGAIN handling in the network drivers.
--HPS
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 05.12.2018 17:25, Slava Shwartsman wrote:
> Author: slavash
> Date: Wed Dec 5 14:25:03 2018
> New Revision: 341586
> URL: https://svnweb.freebsd.org/changeset/base/341586
>
> Log:
> mlx5en: Implement backpressure indication.
>
> The backpressure indication is implemented using an unlimited rate type of
> mbuf send tag. When the upper layers typically the socket layer has
> obtained such
> a tag, it can then query the destination driver queue for the current
> amount of space available in the send queue.
>
> A single mbuf send tag may be referenced multiple times and a refcount has
> been added
> to the mlx5e_priv structure to track its usage. Because the send tag resides
> in the mlx5e_channel structure, there is no need to wait for refcounts to
> reach
> zero until the mlx4en(4) driver is detached. The channels structure is
> persistant
> during the lifetime of the mlx5en(4) driver it belongs to and can so be
> accessed
> without any need of synchronization.
>
> The mlx5e_snd_tag structure was extended to contain a type field, because
> there are now
> two different tag types which end up in the driver which need to be
> distinguished.
>
> Submitted by: hselasky@
> Approved by:hselasky (mentor)
> MFC after: 1 week
> Sponsored by: Mellanox Technologies
> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
> struct mlx5e_sq *sq;
> int ret;
>
> - sq = mlx5e_select_queue(ifp, mb);
> - if (unlikely(sq == NULL)) {
> -#ifdef RATELIMIT
> - /* Check for route change */
> - if (mb->m_pkthdr.snd_tag != NULL &&
> - mb->m_pkthdr.snd_tag->ifp != ifp) {
> + if (mb->m_pkthdr.snd_tag != NULL) {
> + sq = mlx5e_select_queue_by_send_tag(ifp, mb);
> + if (unlikely(sq == NULL)) {
> + /* Check for route change */
> + if (mb->m_pkthdr.snd_tag->ifp != ifp) {
> + /* Free mbuf */
> + m_freem(mb);
> +
> + /*
> + * Tell upper layers about route
> + * change and to re-transmit this
> + * packet:
> + */
> + return (EAGAIN);
> + }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
--
WBR, Andrey V. Elsukov
signature.asc
Description: OpenPGP digital signature
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 01-May-19 19:45, John Baldwin wrote:
On 5/1/19 1:05 AM, Slava Shwartsman wrote:
On 01-May-19 10:28, Slava Shwartsman wrote:
On 01-May-19 10:09, Slava Shwartsman wrote:
On 30-Apr-19 00:14, John Baldwin wrote:
On 4/25/19 12:10 AM, Slava Shwartsman wrote:
On 17-Apr-19 00:28, John Baldwin wrote:
On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an
unlimited rate type of
mbuf send tag. When the upper layers typically the socket
layer has obtained such
a tag, it can then query the destination driver queue for
the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and
a refcount has been added
to the mlx5e_priv structure to track its usage. Because the
send tag resides
in the mlx5e_channel structure, there is no need to wait
for refcounts to reach
zero until the mlx4en(4) driver is detached. The channels
structure is persistant
during the lifetime of the mlx5en(4) driver it belongs to
and can so be accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type
field, because there are now
two different tag types which end up in the driver which
need to be distinguished.
Submitted by: hselasky@
Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf
*mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+ * Tell upper layers about route
+ * change and to re-transmit this
+ * packet:
+ */
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is
the
cause.
The test system has mlx5en 100G interface. It has two vlans:
vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these
packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU ID FUNCTION:NAME
23 54338 ip6_output:return 35
16 54338 ip6_output:return 35
21 54338 ip6_output:return 35
22 54338 ip6_output:return 35
24 54338 ip6_output:return 35
23 54338 ip6_output:return 35
14 54338 ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU ID FUNCTION:NAME
16 69030 mlx5e_xmit:return 35
23 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
25 69030 mlx5e_xmit:return 35
24 69030 mlx5e_xmit:return 35
21 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi,
This might be a case where rcvif in the mbuf's pktheader is not
cleared
before the packet is fed back on the wire.
John Baldwin is working on the send tags implementation, to eliminate
the EAGAIN handling in the network drivers.
I will try to push this branch sooner then since it affects more
than just
TLS. Part of the change includes a new flag we can use to assert
that we
Thanks John!
aren't just getting a stale rcvif (though there are also now
assertions in
ip_output that should catch this case I think).
Hi Andrey,
Yes, we were able to reproduce this issue in house. If you don't
mind, I
prefer to wait for John's update - where he eliminates the EAGAIN
handling in the network drivers.
I have rebased the branch for this, but for now it will just panic
sooner
I believe by tripping an assertion. Can you grab the diff (or just
the branch)
from the 'send_tags' branch at github/bsdjhb/freebsd and reproduce
under a
kernel with INVARIA
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 5/1/19 1:05 AM, Slava Shwartsman wrote:
>
>
> On 01-May-19 10:28, Slava Shwartsman wrote:
>>
>>
>> On 01-May-19 10:09, Slava Shwartsman wrote:
>>>
>>>
>>> On 30-Apr-19 00:14, John Baldwin wrote:
On 4/25/19 12:10 AM, Slava Shwartsman wrote:
>
>
> On 17-Apr-19 00:28, John Baldwin wrote:
>> On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
>>> On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
> Author: slavash
> Date: Wed Dec 5 14:25:03 2018
> New Revision: 341586
> URL: https://svnweb.freebsd.org/changeset/base/341586
>
> Log:
> mlx5en: Implement backpressure indication.
> The backpressure indication is implemented using an
> unlimited rate type of
> mbuf send tag. When the upper layers typically the socket
> layer has obtained such
> a tag, it can then query the destination driver queue for
> the current
> amount of space available in the send queue.
> A single mbuf send tag may be referenced multiple times and
> a refcount has been added
> to the mlx5e_priv structure to track its usage. Because the
> send tag resides
> in the mlx5e_channel structure, there is no need to wait
> for refcounts to reach
> zero until the mlx4en(4) driver is detached. The channels
> structure is persistant
> during the lifetime of the mlx5en(4) driver it belongs to
> and can so be accessed
> without any need of synchronization.
> The mlx5e_snd_tag structure was extended to contain a type
> field, because there are now
> two different tag types which end up in the driver which
> need to be distinguished.
> Submitted by: hselasky@
> Approved by: hselasky (mentor)
> MFC after: 1 week
> Sponsored by: Mellanox Technologies
> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf
> *mb)
> struct mlx5e_sq *sq;
> int ret;
> - sq = mlx5e_select_queue(ifp, mb);
> - if (unlikely(sq == NULL)) {
> -#ifdef RATELIMIT
> - /* Check for route change */
> - if (mb->m_pkthdr.snd_tag != NULL &&
> - mb->m_pkthdr.snd_tag->ifp != ifp) {
> + if (mb->m_pkthdr.snd_tag != NULL) {
> + sq = mlx5e_select_queue_by_send_tag(ifp, mb);
> + if (unlikely(sq == NULL)) {
> + /* Check for route change */
> + if (mb->m_pkthdr.snd_tag->ifp != ifp) {
> + /* Free mbuf */
> + m_freem(mb);
> +
> + /*
> + * Tell upper layers about route
> + * change and to re-transmit this
> + * packet:
> + */
> + return (EAGAIN);
> + }
Hi,
I just discovered something strange and found that this commit is
the
cause.
The test system has mlx5en 100G interface. It has two vlans:
vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these
packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU ID FUNCTION:NAME
23 54338 ip6_output:return 35
16 54338 ip6_output:return 35
21 54338 ip6_output:return 35
22 54338 ip6_output:return 35
24 54338 ip6_output:return 35
23 54338 ip6_output:return 35
14 54338 ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU ID FUNCTION:NAME
16 69030 mlx5e_xmit:return 35
23 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
25 69030 mlx5e_xmit:return 35
24 69030 mlx5e_xmit:return 35
21 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
>>>
>>> Hi,
>>>
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 01-May-19 10:28, Slava Shwartsman wrote:
On 01-May-19 10:09, Slava Shwartsman wrote:
On 30-Apr-19 00:14, John Baldwin wrote:
On 4/25/19 12:10 AM, Slava Shwartsman wrote:
On 17-Apr-19 00:28, John Baldwin wrote:
On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an
unlimited rate type of
mbuf send tag. When the upper layers typically the socket
layer has obtained such
a tag, it can then query the destination driver queue for
the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and
a refcount has been added
to the mlx5e_priv structure to track its usage. Because the
send tag resides
in the mlx5e_channel structure, there is no need to wait
for refcounts to reach
zero until the mlx4en(4) driver is detached. The channels
structure is persistant
during the lifetime of the mlx5en(4) driver it belongs to
and can so be accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type
field, because there are now
two different tag types which end up in the driver which
need to be distinguished.
Submitted by: hselasky@
Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf
*mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+ * Tell upper layers about route
+ * change and to re-transmit this
+ * packet:
+ */
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is
the
cause.
The test system has mlx5en 100G interface. It has two vlans:
vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these
packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU ID FUNCTION:NAME
23 54338 ip6_output:return 35
16 54338 ip6_output:return 35
21 54338 ip6_output:return 35
22 54338 ip6_output:return 35
24 54338 ip6_output:return 35
23 54338 ip6_output:return 35
14 54338 ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU ID FUNCTION:NAME
16 69030 mlx5e_xmit:return 35
23 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
25 69030 mlx5e_xmit:return 35
24 69030 mlx5e_xmit:return 35
21 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi,
This might be a case where rcvif in the mbuf's pktheader is not
cleared
before the packet is fed back on the wire.
John Baldwin is working on the send tags implementation, to eliminate
the EAGAIN handling in the network drivers.
I will try to push this branch sooner then since it affects more
than just
TLS. Part of the change includes a new flag we can use to assert
that we
Thanks John!
aren't just getting a stale rcvif (though there are also now
assertions in
ip_output that should catch this case I think).
Hi Andrey,
Yes, we were able to reproduce this issue in house. If you don't
mind, I
prefer to wait for John's update - where he eliminates the EAGAIN
handling in the network drivers.
I have rebased the branch for this, but for now it will just panic
sooner
I believe by tripping an assertion. Can you grab the diff (or just
the branch)
from the 'send_tags' branch at github/bsdjhb/freebsd and reproduce
under a
kernel with INVARIANTS? I think we will have to explicitly clear the
'rcvif'
pointer somewhere, but I want
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
Hi, Try this first: https://reviews.freebsd.org/D20117 --HPS ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 30-Apr-19 00:14, John Baldwin wrote:
On 4/25/19 12:10 AM, Slava Shwartsman wrote:
On 17-Apr-19 00:28, John Baldwin wrote:
On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has
obtained such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has been added
to the mlx5e_priv structure to track its usage. Because the send tag
resides
in the mlx5e_channel structure, there is no need to wait for refcounts to
reach
zero until the mlx4en(4) driver is detached. The channels structure is
persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be
accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because there are now
two different tag types which end up in the driver which need to be
distinguished.
Submitted by: hselasky@
Approved by:hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+* Tell upper layers about route
+* change and to re-transmit this
+* packet:
+*/
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi,
This might be a case where rcvif in the mbuf's pktheader is not cleared
before the packet is fed back on the wire.
John Baldwin is working on the send tags implementation, to eliminate
the EAGAIN handling in the network drivers.
I will try to push this branch sooner then since it affects more than just
TLS. Part of the change includes a new flag we can use to assert that we
Thanks John!
aren't just getting a stale rcvif (though there are also now assertions in
ip_output that should catch this case I think).
Hi Andrey,
Yes, we were able to reproduce this issue in house. If you don't mind, I
prefer to wait for John's update - where he eliminates the EAGAIN
handling in the network drivers.
I have rebased the branch for this, but for now it will just panic sooner
I believe by tripping an assertion. Can you grab the diff (or just the branch)
from the 'send_tags' branch at github/bsdjhb/freebsd and reproduce under a
k
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 01-May-19 10:09, Slava Shwartsman wrote:
On 30-Apr-19 00:14, John Baldwin wrote:
On 4/25/19 12:10 AM, Slava Shwartsman wrote:
On 17-Apr-19 00:28, John Baldwin wrote:
On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an
unlimited rate type of
mbuf send tag. When the upper layers typically the socket
layer has obtained such
a tag, it can then query the destination driver queue for
the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and
a refcount has been added
to the mlx5e_priv structure to track its usage. Because the
send tag resides
in the mlx5e_channel structure, there is no need to wait for
refcounts to reach
zero until the mlx4en(4) driver is detached. The channels
structure is persistant
during the lifetime of the mlx5en(4) driver it belongs to
and can so be accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type
field, because there are now
two different tag types which end up in the driver which
need to be distinguished.
Submitted by: hselasky@
Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+ * Tell upper layers about route
+ * change and to re-transmit this
+ * packet:
+ */
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans:
vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these
packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU ID FUNCTION:NAME
23 54338 ip6_output:return 35
16 54338 ip6_output:return 35
21 54338 ip6_output:return 35
22 54338 ip6_output:return 35
24 54338 ip6_output:return 35
23 54338 ip6_output:return 35
14 54338 ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU ID FUNCTION:NAME
16 69030 mlx5e_xmit:return 35
23 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
25 69030 mlx5e_xmit:return 35
24 69030 mlx5e_xmit:return 35
21 69030 mlx5e_xmit:return 35
26 69030 mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi,
This might be a case where rcvif in the mbuf's pktheader is not
cleared
before the packet is fed back on the wire.
John Baldwin is working on the send tags implementation, to eliminate
the EAGAIN handling in the network drivers.
I will try to push this branch sooner then since it affects more
than just
TLS. Part of the change includes a new flag we can use to assert
that we
Thanks John!
aren't just getting a stale rcvif (though there are also now
assertions in
ip_output that should catch this case I think).
Hi Andrey,
Yes, we were able to reproduce this issue in house. If you don't mind, I
prefer to wait for John's update - where he eliminates the EAGAIN
handling in the network drivers.
I have rebased the branch for this, but for now it will just panic sooner
I believe by tripping an assertion. Can you grab the diff (or just
the branch)
from the 'send_tags' branch at github/bsdjhb/freebsd and reproduce
under a
kernel with INVARIANTS? I think we will have to explicitly clear the
'rcvif'
pointer somewhere, but I want to see what the stack trace looks like
so I can
thi
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 30.04.2019 00:14, John Baldwin wrote: >> Yes, we were able to reproduce this issue in house. If you don't mind, I >> prefer to wait for John's update - where he eliminates the EAGAIN >> handling in the network drivers. > > I have rebased the branch for this, but for now it will just panic sooner > I believe by tripping an assertion. Can you grab the diff (or just the > branch) > from the 'send_tags' branch at github/bsdjhb/freebsd and reproduce under a > kernel with INVARIANTS? I think we will have to explicitly clear the 'rcvif' > pointer somewhere, but I want to see what the stack trace looks like so I can > think about the "right" place to clear it. Hi, please note, that rcvif is used by firewall to track inbound interface and clearing it can be unexpected in some cases, and can break firewall rules. -- WBR, Andrey V. Elsukov ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 4/25/19 12:10 AM, Slava Shwartsman wrote:
>
>
> On 17-Apr-19 00:28, John Baldwin wrote:
>> On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
>>> On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
> Author: slavash
> Date: Wed Dec 5 14:25:03 2018
> New Revision: 341586
> URL: https://svnweb.freebsd.org/changeset/base/341586
>
> Log:
> mlx5en: Implement backpressure indication.
>
> The backpressure indication is implemented using an unlimited rate
> type of
> mbuf send tag. When the upper layers typically the socket layer has
> obtained such
> a tag, it can then query the destination driver queue for the current
> amount of space available in the send queue.
>
> A single mbuf send tag may be referenced multiple times and a
> refcount has been added
> to the mlx5e_priv structure to track its usage. Because the send tag
> resides
> in the mlx5e_channel structure, there is no need to wait for
> refcounts to reach
> zero until the mlx4en(4) driver is detached. The channels structure
> is persistant
> during the lifetime of the mlx5en(4) driver it belongs to and can so
> be accessed
> without any need of synchronization.
>
> The mlx5e_snd_tag structure was extended to contain a type field,
> because there are now
> two different tag types which end up in the driver which need to be
> distinguished.
>
> Submitted by: hselasky@
> Approved by:hselasky (mentor)
> MFC after: 1 week
> Sponsored by: Mellanox Technologies
> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
> struct mlx5e_sq *sq;
> int ret;
>
> - sq = mlx5e_select_queue(ifp, mb);
> - if (unlikely(sq == NULL)) {
> -#ifdef RATELIMIT
> - /* Check for route change */
> - if (mb->m_pkthdr.snd_tag != NULL &&
> - mb->m_pkthdr.snd_tag->ifp != ifp) {
> + if (mb->m_pkthdr.snd_tag != NULL) {
> + sq = mlx5e_select_queue_by_send_tag(ifp, mb);
> + if (unlikely(sq == NULL)) {
> + /* Check for route change */
> + if (mb->m_pkthdr.snd_tag->ifp != ifp) {
> + /* Free mbuf */
> + m_freem(mb);
> +
> + /*
> + * Tell upper layers about route
> + * change and to re-transmit this
> + * packet:
> + */
> + return (EAGAIN);
> + }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
>>>
>>> Hi,
>>>
>>> This might be a case where rcvif in the mbuf's pktheader is not cleared
>>> before the packet is fed back on the wire.
>>>
>>> John Baldwin is working on the send tags implementation, to eliminate
>>> the EAGAIN handling in the network drivers.
>>
>> I will try to push this branch sooner then since it affects more than just
>> TLS. Part of the change includes a new flag we can use to assert that we
> Thanks John!
>> aren't just getting a stale rcvif (though there are also now assertions in
>> ip_output that should catch this case I think).
>>
>
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 17-Apr-19 00:28, John Baldwin wrote:
On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has
obtained such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has been added
to the mlx5e_priv structure to track its usage. Because the send tag resides
in the mlx5e_channel structure, there is no need to wait for refcounts to
reach
zero until the mlx4en(4) driver is detached. The channels structure is
persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be
accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because there are now
two different tag types which end up in the driver which need to be
distinguished.
Submitted by: hselasky@
Approved by:hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+* Tell upper layers about route
+* change and to re-transmit this
+* packet:
+*/
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi,
This might be a case where rcvif in the mbuf's pktheader is not cleared
before the packet is fed back on the wire.
John Baldwin is working on the send tags implementation, to eliminate
the EAGAIN handling in the network drivers.
I will try to push this branch sooner then since it affects more than just
TLS. Part of the change includes a new flag we can use to assert that we
Thanks John!
aren't just getting a stale rcvif (though there are also now assertions in
ip_output that should catch this case I think).
Hi Andrey,
Yes, we were able to reproduce this issue in house. If you don't mind, I
prefer to wait for John's update - where he eliminates the EAGAIN
handling in the network drivers.
Slava
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 4/16/19 8:32 AM, Hans Petter Selasky wrote:
> On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
>> On 05.12.2018 17:25, Slava Shwartsman wrote:
>>> Author: slavash
>>> Date: Wed Dec 5 14:25:03 2018
>>> New Revision: 341586
>>> URL: https://svnweb.freebsd.org/changeset/base/341586
>>>
>>> Log:
>>>mlx5en: Implement backpressure indication.
>>>
>>>The backpressure indication is implemented using an unlimited rate type
>>> of
>>>mbuf send tag. When the upper layers typically the socket layer has
>>> obtained such
>>>a tag, it can then query the destination driver queue for the current
>>>amount of space available in the send queue.
>>>
>>>A single mbuf send tag may be referenced multiple times and a refcount
>>> has been added
>>>to the mlx5e_priv structure to track its usage. Because the send tag
>>> resides
>>>in the mlx5e_channel structure, there is no need to wait for refcounts
>>> to reach
>>>zero until the mlx4en(4) driver is detached. The channels structure is
>>> persistant
>>>during the lifetime of the mlx5en(4) driver it belongs to and can so be
>>> accessed
>>>without any need of synchronization.
>>>
>>>The mlx5e_snd_tag structure was extended to contain a type field,
>>> because there are now
>>>two different tag types which end up in the driver which need to be
>>> distinguished.
>>>
>>>Submitted by: hselasky@
>>>Approved by:hselasky (mentor)
>>>MFC after: 1 week
>>>Sponsored by: Mellanox Technologies
>>> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
>>> struct mlx5e_sq *sq;
>>> int ret;
>>>
>>> - sq = mlx5e_select_queue(ifp, mb);
>>> - if (unlikely(sq == NULL)) {
>>> -#ifdef RATELIMIT
>>> - /* Check for route change */
>>> - if (mb->m_pkthdr.snd_tag != NULL &&
>>> - mb->m_pkthdr.snd_tag->ifp != ifp) {
>>> + if (mb->m_pkthdr.snd_tag != NULL) {
>>> + sq = mlx5e_select_queue_by_send_tag(ifp, mb);
>>> + if (unlikely(sq == NULL)) {
>>> + /* Check for route change */
>>> + if (mb->m_pkthdr.snd_tag->ifp != ifp) {
>>> + /* Free mbuf */
>>> + m_freem(mb);
>>> +
>>> + /*
>>> +* Tell upper layers about route
>>> +* change and to re-transmit this
>>> +* packet:
>>> +*/
>>> + return (EAGAIN);
>>> + }
>>
>> Hi,
>>
>> I just discovered something strange and found that this commit is the
>> cause.
>> The test system has mlx5en 100G interface. It has two vlans: vlan500 and
>> vlan100.
>> Via vlan500 it receives some packets flows. Then it routes these packets
>> into vlan100.
>> But packets are dropped in mlx5e_xmit() with EAGAIN error code.
>>
>> # dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
>> dtrace: description 'fbt::ip6_output:return ' matched 1 probe
>> CPU IDFUNCTION:NAME
>> 23 54338ip6_output:return 35
>> 16 54338ip6_output:return 35
>> 21 54338ip6_output:return 35
>> 22 54338ip6_output:return 35
>> 24 54338ip6_output:return 35
>> 23 54338ip6_output:return 35
>> 14 54338ip6_output:return 35
>> ^C
>>
>> # dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
>> dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
>> CPU IDFUNCTION:NAME
>> 16 69030mlx5e_xmit:return 35
>> 23 69030mlx5e_xmit:return 35
>> 26 69030mlx5e_xmit:return 35
>> 25 69030mlx5e_xmit:return 35
>> 24 69030mlx5e_xmit:return 35
>> 21 69030mlx5e_xmit:return 35
>> 26 69030mlx5e_xmit:return 35
>> ^C
>>
>> The kernel config is GENERIC.
>> 13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
>>
>
> Hi,
>
> This might be a case where rcvif in the mbuf's pktheader is not cleared
> before the packet is fed back on the wire.
>
> John Baldwin is working on the send tags implementation, to eliminate
> the EAGAIN handling in the network drivers.
I will try to push this branch sooner then since it affects more than just
TLS. Part of the change includes a new flag we can use to assert that we
aren't just getting a stale rcvif (though there are also now assertions in
ip_output that should catch this case I think).
--
John Baldwin
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 16-Apr-19 17:39, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has obtained
such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has been added
to the mlx5e_priv structure to track its usage. Because the send tag resides
in the mlx5e_channel structure, there is no need to wait for refcounts to
reach
zero until the mlx4en(4) driver is detached. The channels structure is
persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be
accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because there are now
two different tag types which end up in the driver which need to be
distinguished.
Submitted by: hselasky@
Approved by:hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+* Tell upper layers about route
+* change and to re-transmit this
+* packet:
+*/
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi Andrey,
Thanks for letting us know about this regression.
I would like to try to reproduce this issue in house.
Can you please share the exact steps to reproduce it?
- Can I reproduce the issue with B2B setup?
- What is the route command you used to make the route between the VLANs?
- What app are you using to generate the traffic?
Slava
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 16.04.2019 18:26, Slava Shwartsman wrote: > Thanks for letting us know about this regression. > I would like to try to reproduce this issue in house. > > Can you please share the exact steps to reproduce it? > - Can I reproduce the issue with B2B setup? > - What is the route command you used to make the route between the VLANs? > - What app are you using to generate the traffic? > I think this can be reproduced on simple router, where single mce(4) interface is used as parent for several vlan(4) interfaces. E.g. [host1] vlan100 <--> mce0.100 [gateway] mce0.200 <--> vlan200 [host2] 10.0.0.110.0.0.254 192.168.0.254192.168.0.1 gateway: sysctl net.inet.ip.forwarding=1 host1: route add 192.168.0.0/24 10.0.0.254 host2: route add 10.0.0.0/24 192.168.0.254 ping 10.0.0.1 I.e. you need to make setup, where ingress and egress interface is the same - mce0. -- WBR, Andrey V. Elsukov signature.asc Description: OpenPGP digital signature
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 4/16/19 4:39 PM, Andrey V. Elsukov wrote:
On 05.12.2018 17:25, Slava Shwartsman wrote:
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has obtained
such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has been added
to the mlx5e_priv structure to track its usage. Because the send tag resides
in the mlx5e_channel structure, there is no need to wait for refcounts to
reach
zero until the mlx4en(4) driver is detached. The channels structure is
persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be
accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because there are now
two different tag types which end up in the driver which need to be
distinguished.
Submitted by: hselasky@
Approved by:hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
@@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
struct mlx5e_sq *sq;
int ret;
- sq = mlx5e_select_queue(ifp, mb);
- if (unlikely(sq == NULL)) {
-#ifdef RATELIMIT
- /* Check for route change */
- if (mb->m_pkthdr.snd_tag != NULL &&
- mb->m_pkthdr.snd_tag->ifp != ifp) {
+ if (mb->m_pkthdr.snd_tag != NULL) {
+ sq = mlx5e_select_queue_by_send_tag(ifp, mb);
+ if (unlikely(sq == NULL)) {
+ /* Check for route change */
+ if (mb->m_pkthdr.snd_tag->ifp != ifp) {
+ /* Free mbuf */
+ m_freem(mb);
+
+ /*
+* Tell upper layers about route
+* change and to re-transmit this
+* packet:
+*/
+ return (EAGAIN);
+ }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
Hi,
This might be a case where rcvif in the mbuf's pktheader is not cleared
before the packet is fed back on the wire.
John Baldwin is working on the send tags implementation, to eliminate
the EAGAIN handling in the network drivers.
--HPS
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
On 05.12.2018 17:25, Slava Shwartsman wrote:
> Author: slavash
> Date: Wed Dec 5 14:25:03 2018
> New Revision: 341586
> URL: https://svnweb.freebsd.org/changeset/base/341586
>
> Log:
> mlx5en: Implement backpressure indication.
>
> The backpressure indication is implemented using an unlimited rate type of
> mbuf send tag. When the upper layers typically the socket layer has
> obtained such
> a tag, it can then query the destination driver queue for the current
> amount of space available in the send queue.
>
> A single mbuf send tag may be referenced multiple times and a refcount has
> been added
> to the mlx5e_priv structure to track its usage. Because the send tag resides
> in the mlx5e_channel structure, there is no need to wait for refcounts to
> reach
> zero until the mlx4en(4) driver is detached. The channels structure is
> persistant
> during the lifetime of the mlx5en(4) driver it belongs to and can so be
> accessed
> without any need of synchronization.
>
> The mlx5e_snd_tag structure was extended to contain a type field, because
> there are now
> two different tag types which end up in the driver which need to be
> distinguished.
>
> Submitted by: hselasky@
> Approved by:hselasky (mentor)
> MFC after: 1 week
> Sponsored by: Mellanox Technologies
> @@ -587,27 +609,33 @@ mlx5e_xmit(struct ifnet *ifp, struct mbuf *mb)
> struct mlx5e_sq *sq;
> int ret;
>
> - sq = mlx5e_select_queue(ifp, mb);
> - if (unlikely(sq == NULL)) {
> -#ifdef RATELIMIT
> - /* Check for route change */
> - if (mb->m_pkthdr.snd_tag != NULL &&
> - mb->m_pkthdr.snd_tag->ifp != ifp) {
> + if (mb->m_pkthdr.snd_tag != NULL) {
> + sq = mlx5e_select_queue_by_send_tag(ifp, mb);
> + if (unlikely(sq == NULL)) {
> + /* Check for route change */
> + if (mb->m_pkthdr.snd_tag->ifp != ifp) {
> + /* Free mbuf */
> + m_freem(mb);
> +
> + /*
> + * Tell upper layers about route
> + * change and to re-transmit this
> + * packet:
> + */
> + return (EAGAIN);
> + }
Hi,
I just discovered something strange and found that this commit is the
cause.
The test system has mlx5en 100G interface. It has two vlans: vlan500 and
vlan100.
Via vlan500 it receives some packets flows. Then it routes these packets
into vlan100.
But packets are dropped in mlx5e_xmit() with EAGAIN error code.
# dtrace -n 'fbt::ip6_output:return {printf("%d", arg1);}'
dtrace: description 'fbt::ip6_output:return ' matched 1 probe
CPU IDFUNCTION:NAME
23 54338ip6_output:return 35
16 54338ip6_output:return 35
21 54338ip6_output:return 35
22 54338ip6_output:return 35
24 54338ip6_output:return 35
23 54338ip6_output:return 35
14 54338ip6_output:return 35
^C
# dtrace -n 'fbt::mlx5e_xmit:return {printf("%d", arg1);}'
dtrace: description 'fbt::mlx5e_xmit:return ' matched 1 probe
CPU IDFUNCTION:NAME
16 69030mlx5e_xmit:return 35
23 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
25 69030mlx5e_xmit:return 35
24 69030mlx5e_xmit:return 35
21 69030mlx5e_xmit:return 35
26 69030mlx5e_xmit:return 35
^C
The kernel config is GENERIC.
13.0-CURRENT #9 r345758+82f3d57(svn_head)-dirty
--
WBR, Andrey V. Elsukov
signature.asc
Description: OpenPGP digital signature
svn commit: r341586 - head/sys/dev/mlx5/mlx5_en
Author: slavash
Date: Wed Dec 5 14:25:03 2018
New Revision: 341586
URL: https://svnweb.freebsd.org/changeset/base/341586
Log:
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has obtained
such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has
been added
to the mlx5e_priv structure to track its usage. Because the send tag resides
in the mlx5e_channel structure, there is no need to wait for refcounts to
reach
zero until the mlx4en(4) driver is detached. The channels structure is
persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be
accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because
there are now
two different tag types which end up in the driver which need to be
distinguished.
Submitted by: hselasky@
Approved by:hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies
Modified:
head/sys/dev/mlx5/mlx5_en/en.h
head/sys/dev/mlx5/mlx5_en/en_rl.h
head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c
head/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c
head/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c
Modified: head/sys/dev/mlx5/mlx5_en/en.h
==
--- head/sys/dev/mlx5/mlx5_en/en.h Wed Dec 5 14:24:33 2018
(r341585)
+++ head/sys/dev/mlx5/mlx5_en/en.h Wed Dec 5 14:25:03 2018
(r341586)
@@ -580,6 +580,11 @@ enum {
MLX5E_SQ_FULL
};
+struct mlx5e_snd_tag {
+ struct m_snd_tag m_snd_tag; /* send tag */
+ u32 type; /* tag type */
+};
+
struct mlx5e_sq {
/* data path */
struct mtx lock;
@@ -640,11 +645,27 @@ mlx5e_sq_has_room_for(struct mlx5e_sq *sq, u16 n)
return ((sq->wq.sz_m1 & (cc - pc)) >= n || cc == pc);
}
+static inline u32
+mlx5e_sq_queue_level(struct mlx5e_sq *sq)
+{
+ u16 cc;
+ u16 pc;
+
+ if (sq == NULL)
+ return (0);
+
+ cc = sq->cc;
+ pc = sq->pc;
+
+ return (((sq->wq.sz_m1 & (pc - cc)) *
+ IF_SND_QUEUE_LEVEL_MAX) / sq->wq.sz_m1);
+}
+
struct mlx5e_channel {
/* data path */
struct mlx5e_rq rq;
+ struct mlx5e_snd_tag tag;
struct mlx5e_sq sq[MLX5E_MAX_TX_NUM_TC];
- struct ifnet *ifp;
u32 mkey_be;
u8 num_tc;
@@ -770,6 +791,7 @@ struct mlx5e_priv {
u32 pdn;
u32 tdn;
struct mlx5_core_mr mr;
+ volatile unsigned int channel_refs;
u32 tisn[MLX5E_MAX_TX_NUM_TC];
u32 rqtn;
@@ -907,6 +929,24 @@ mlx5e_cq_arm(struct mlx5e_cq *cq, spinlock_t *dblock)
mcq = &cq->mcq;
mlx5_cq_arm(mcq, MLX5_CQ_DB_REQ_NOT, mcq->uar->map, dblock, cq->wq.cc);
+}
+
+static inline void
+mlx5e_ref_channel(struct mlx5e_priv *priv)
+{
+
+ KASSERT(priv->channel_refs < INT_MAX,
+ ("Channel refs will overflow"));
+ atomic_fetchadd_int(&priv->channel_refs, 1);
+}
+
+static inline void
+mlx5e_unref_channel(struct mlx5e_priv *priv)
+{
+
+ KASSERT(priv->channel_refs > 0,
+ ("Channel refs is not greater than zero"));
+ atomic_fetchadd_int(&priv->channel_refs, -1);
}
extern const struct ethtool_ops mlx5e_ethtool_ops;
Modified: head/sys/dev/mlx5/mlx5_en/en_rl.h
==
--- head/sys/dev/mlx5/mlx5_en/en_rl.h Wed Dec 5 14:24:33 2018
(r341585)
+++ head/sys/dev/mlx5/mlx5_en/en_rl.h Wed Dec 5 14:25:03 2018
(r341586)
@@ -129,7 +129,7 @@ struct mlx5e_rl_channel_param {
};
struct mlx5e_rl_channel {
- struct m_snd_tag m_snd_tag;
+ struct mlx5e_snd_tag tag;
STAILQ_ENTRY(mlx5e_rl_channel) entry;
struct mlx5e_sq * volatile sq;
struct mlx5e_rl_worker *worker;
Modified: head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c
==
--- head/sys/dev/mlx5/mlx5_en/mlx5_en_main.cWed Dec 5 14:24:33 2018
(r341585)
+++ head/sys/dev/mlx5/mlx5_en/mlx5_en_main.cWed Dec 5 14:25:03 2018
(r341586)
@@ -886,7 +886,7 @@ mlx5e_create_rq(struct mlx5e_channel *c,
wq_sz = mlx5_wq_ll_get_size(&rq->wq);
- err = -tcp_lro_init_args(&rq->lro, c->ifp, TCP_LRO_ENTRIES, wq_sz);
+ err = -tcp_lro_init_args(&rq->lro, c->tag.m_snd_tag.ifp,
TCP_LRO_ENTRIES, wq_sz);
if (err)
goto err_rq_wq_destroy;
@@ -916,7 +916,7 @@ mlx5e_create_rq(struct mlx5e_channel *c,
#endif
}
- rq->ifp = c->ifp;
+ rq->ifp = c->tag.m_snd_tag.ifp;
rq->channel = c;
rq->ix = c->ix;
@@ -1778,7 +1778,9 @@ mlx
