Re: How to forward traffic out via a non-default interface?

[Jack Vogel]

You got your answer on the other site :)  You do know you can change the
default address
of the router so its not 192.168.1 don't you, it even persists over a
reset...


On Sat, Oct 29, 2016 at 1:21 PM, Mikhail T. 
wrote:

> Hello! I asked this question online already:
>
>http://superuser.com/q/1138996/247471?sem=2
>
> if you have an answer, please, share it wherever you feel appropriate...
>
> have a fairly vanilla home LAN with an Internet provider-supplied router
> providing DHCP. The router uses many of its default settings unmodified.
> The home network is 192.168.1.0/24 and the router itself is known
> internally as 192.168.1.1.
>
> I need to configure several similar routers to change their default
> settings. One of my computers (running FreeBSD) has a spare Ethernet
> interface (|bce0|) and I connected one of these additional routers to it.
> As expected, this router /also/ uses 192.168.1.1. The machine now has two
> "configured" interfaces:
>
>  * |bce1| -- 192.168.1.8, which is connected to the "real" router,
>connecting it to the Internet and the rest of the home LAN
>  * |bce0| -- 192.168.1.5, which is connected to the second router,
>which I need to change to use different network settings
>
> How can I reach this second router without breaking the machine's
> connectivity with the LAN and the Internet? Simply trying to reach
> 192.168.1.1 goes through |bce1| and reaches my normal router...
>
> Can |ipfw(8)| somehow force certain packets to go out using |bce0| instead
> of |bce1|? Preferably, without the use of "fib", because adding one
> requires a reboot...
>
> Thanks! Yours,
>
>-mi
>
>
> ___
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: "igb" interfaces aren't picking up media on boot

[Jack Vogel]

So, time to ditch the DLink I'd say :)

On Wed, Sep 28, 2016 at 9:29 AM, Andrew Savchenko 
wrote:

> I've been using this particular NIC under FreeBSD 10.0-REL connected via
> cross cable to another Intel NIC / two different routers / TP-Link switch
> without any similar issues. The problem shows only with the mentioned DLink
> switch.
>
> On Wed, Sep 28, 2016 at 7:20 PM, Jack Vogel  wrote:
>
>> So, I would check into the link partner, often times it can be the
>> culprit, try the nic
>> in a simple back-to-back setup and see if it makes a diff, etc. etc...
>>
>> Jack
>>
>>
>> On Wed, Sep 28, 2016 at 9:03 AM, Andrew Savchenko > > wrote:
>>
>>> Nope, unfortunately they stay dead (aka "no carrier") until `netif
>>> restart`.
>>>
>>> On Wed, Sep 28, 2016 at 6:12 PM, Freddie Cash  wrote:
>>>
>>> > On Wed, Sep 28, 2016 at 8:02 AM, Andrew Savchenko <
>>> gradiomet...@gmail.com>
>>> > wrote:
>>> >
>>> >> MAC's and IP's are abviously changed.
>>> >> For the moment being I have "solved" this by a tiny script in
>>> /etc/rc.d/
>>> >> that restarts networking & routing after the system boot:
>>> >>
>>> >
>>> > ​Do the NICs eventually pick up carrier status if you leave it after
>>> > boot?  Could it just be slow picking up status changes?  We had a
>>> couple
>>> > em(4) or igb(4) (forget now) NICs that were like that.  Took a good
>>> minute
>>> > after boot before the NIC was finished initialising and detected the
>>> link.
>>> >
>>> > If that's the case, and you don't mind the wait (and want to make sure
>>> > networking is working once the boot is complete), you can use the
>>> netwait_*
>>> > options in /etc/rc.conf.  That will pause the boot process until
>>> either the
>>> > network link status changes to up, and/or you can ping the default
>>> gateway
>>> > or any other IP on the network.
>>> >
>>> > Worked for us.  Might work for you.​
>>> >
>>> >
>>> > --
>>> > Freddie Cash
>>> > fjwc...@gmail.com
>>> >
>>> ___
>>> freebsd-net@freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>>
>>
>>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: "igb" interfaces aren't picking up media on boot

[Jack Vogel]

So, I would check into the link partner, often times it can be the culprit,
try the nic
in a simple back-to-back setup and see if it makes a diff, etc. etc...

Jack


On Wed, Sep 28, 2016 at 9:03 AM, Andrew Savchenko 
wrote:

> Nope, unfortunately they stay dead (aka "no carrier") until `netif
> restart`.
>
> On Wed, Sep 28, 2016 at 6:12 PM, Freddie Cash  wrote:
>
> > On Wed, Sep 28, 2016 at 8:02 AM, Andrew Savchenko <
> gradiomet...@gmail.com>
> > wrote:
> >
> >> MAC's and IP's are abviously changed.
> >> For the moment being I have "solved" this by a tiny script in /etc/rc.d/
> >> that restarts networking & routing after the system boot:
> >>
> >
> > ​Do the NICs eventually pick up carrier status if you leave it after
> > boot?  Could it just be slow picking up status changes?  We had a couple
> > em(4) or igb(4) (forget now) NICs that were like that.  Took a good
> minute
> > after boot before the NIC was finished initialising and detected the
> link.
> >
> > If that's the case, and you don't mind the wait (and want to make sure
> > networking is working once the boot is complete), you can use the
> netwait_*
> > options in /etc/rc.conf.  That will pause the boot process until either
> the
> > network link status changes to up, and/or you can ping the default
> gateway
> > or any other IP on the network.
> >
> > Worked for us.  Might work for you.​
> >
> >
> > --
> > Freddie Cash
> > fjwc...@gmail.com
> >
> ___
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Will this card (probably) work?

[Jack Vogel]

I think I should let Intel speak to this, they do have quad port adapters
in their offerings, and
as far as I know they are fine. I was speaking of times past when the
bridge device would be
the source of problems, and just saying if this was a non-Intel design it
could be an issue, but
since you are being brave don't let me ruin your adventure :)

Jack



On Tue, Sep 27, 2016 at 12:32 PM, Lee Brown  wrote:

>
>
> On Tue, Sep 27, 2016 at 12:24 PM, Jack Vogel  wrote:
>
>> Its not that they, whoever they is :). "use" the Intel PCID, no, it
>> really IS the Intel device,
>> but it's silicon, not an assembled adapter, its that part which is being
>> done by someone
>> else, and btw, the components to support 4 port type adapters have been
>> notoriously
>> problematic in the past, so having it done by someone besides Intel is at
>> least a potential
>> source of concern.
>>
>> Do you know if 2-port adapters are generally more reliable?  I could
> revise my server purchase so I can stuff a couple of cards in there,
> instead of one.
>
>
>
>> I wish you the best,
>>
> Thanks :)
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Will this card (probably) work?

[Jack Vogel]

Its not that they, whoever they is :). "use" the Intel PCID, no, it really
IS the Intel device,
but it's silicon, not an assembled adapter, its that part which is being
done by someone
else, and btw, the components to support 4 port type adapters have been
notoriously
problematic in the past, so having it done by someone besides Intel is at
least a potential
source of concern.

I wish you the best,

Jack


On Tue, Sep 27, 2016 at 9:02 AM, Lee Brown  wrote:

>
>
> On Mon, Sep 26, 2016 at 4:44 PM, Jack Vogel  wrote:
>
>> Yes, "should be", but just remember now, if this adapter does not look
>> like a
>> genuine Intel made part (and from some comment from either Jeff or Eric
>> it
>> sounded like it wasn't), then IF you have a problem you are going to get
>> no
>> support from them.  Of course, in most worlds you'd have no problem, but
>> there's
>> always some possible one in which you know :)
>>
>> Just being the voice of caution here...
>>
>> Jack
>>
>>
>> On Mon, Sep 26, 2016 at 4:10 PM, Sean Bruno  wrote:
>>
>>> >
>>> > OK the vendor got back to me:
>>> >
>>> > 8086:1522
>>>
>>> That appears to be governed by the the igb(4) driver.  You should be
>>> good to go.
>>>
>>> fbsd_head/sys/dev/e1000 % grep -r E1000_DEV_ID_I350_FIBER *
>>> e1000_api.c:case E1000_DEV_ID_I350_FIBER:
>>> e1000_hw.h:#define E1000_DEV_ID_I350_FIBER  0x1522
>>> if_igb.c:   {IGB_INTEL_VENDOR_ID, E1000_DEV_ID_I350_FIBER,  0, 0, 0},
>>>
>>> sean
>>>
>>>
>> Much appreciated both of you.  For $168 I'm willing to risk it.
> I assume they use Intel's PCID instead of their own because they use the
> Intel driver (which is encouraging).  However Intel make nothing like this
> card (maybe they used to), so yes, it is a foray into the unknown.
> If I'm unable to get it working or working well, at least I have
> alternatives (meda converters at the far end and SFP into an existing fiber
> switch).
> And if I do, well we've got one more card to add to the compatibility list.
> -- lee
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Will this card (probably) work?

[Jack Vogel]

Yes, "should be", but just remember now, if this adapter does not look like
a
genuine Intel made part (and from some comment from either Jeff or Eric it
sounded like it wasn't), then IF you have a problem you are going to get no
support from them.  Of course, in most worlds you'd have no problem, but
there's
always some possible one in which you know :)

Just being the voice of caution here...

Jack


On Mon, Sep 26, 2016 at 4:10 PM, Sean Bruno  wrote:

> >
> > OK the vendor got back to me:
> >
> > 8086:1522
>
> That appears to be governed by the the igb(4) driver.  You should be
> good to go.
>
> fbsd_head/sys/dev/e1000 % grep -r E1000_DEV_ID_I350_FIBER *
> e1000_api.c:case E1000_DEV_ID_I350_FIBER:
> e1000_hw.h:#define E1000_DEV_ID_I350_FIBER  0x1522
> if_igb.c:   {IGB_INTEL_VENDOR_ID, E1000_DEV_ID_I350_FIBER,  0, 0, 0},
>
> sean
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Will this card (probably) work?

[Jack Vogel]

On Tue, Sep 20, 2016 at 8:47 AM, Sean Bruno  wrote:

>
>
> On 09/20/16 03:42, Lee Brown wrote:
> >
> >
> > On Mon, Sep 19, 2016 at 1:48 PM, Sean Bruno  > > wrote:
> >
> >
> >
> > On 09/19/16 14:17, Lee Brown wrote:
> > > I am looking to purchase this
> >  > > card
> > > (LR-LINK I350AM4 Chip Quad SFP Port 1000Base-X Server Adapter
> > I350-F4),
> > > into which I intend to use 1000-LX SFP's.
> > >
> > > If I'm understanding correctly, the igb(4) driver supports the
> > i350 (not
> > > sure if i350AM4 is the same) and altq(4) states the igb(4) driver
> is
> > > supported (which I'll be needing).
> > >
> > > Does this look like a viable PCIe card for me to get?
> > >
> > > Thanks in advance -- lee
> > > ___
> > > freebsd-net@freebsd.org  mailing
> list
> > > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > 
> > > To unsubscribe, send any mail to
> > "freebsd-net-unsubscr...@freebsd.org
> > "
> > >
> >
> > In order to really know, I'd need to know the pci-id that this thing
> > shows up as.
> >
> > sean
> >
> > Thanks, I've reached out for that info.
> > Otherwise, if you happen to know of a 4-port or even 2-port PCIe card
> > that supports 1000Base-LX or SFP's that will, I'm all ears.  I'd
> > consider a single port card as last resort.
> >
>
> Pretty sure that the i350 is what you're in the market for.  Just not
> sure if it will show up as an igb(4) nic.  If it doesn't, it'd be
> trivial to add the pci-ids if it is just a variant.
>
> sean
>
>
What else would it show up as :) ?
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Slow performance with Intel X540-T2 10Gb NIC

[Jack Vogel]

NUMA issues maybe? They have been a problem on some recent system
architectures.

On Thu, Jul 21, 2016 at 1:53 PM, Chris Dunbar  wrote:

> Eric, et al:
>
> I haven't tried netperf yet, but I do have some new information to share.
> I have two systems that I am using for testing: the new server and an older
> (not too old) desktop PC. I installed CentOS on the new server again
> because I know it can achieve >9 GB/s with the X540. I replaced Windows on
> the desktop PC with FreeBSD 10.3 (it also has an X540) and ran iperf3
> again. I was able to achieve >9 GB/s so I know the problem isn't the X540
> and I know the problem isn't anything with the default installation of
> FreeBSD 10.3. So, what in the world might be nutty in my BIOS settings (or
> elsewhere) that would cause the new server + FreeBSD 10.3 + X540 to equal
> slow performance?
>
> Regards,
> Chris
>
>
> From: "Eric Joyner" 
> To: "chris" , "freebsd-net" 
> Sent: Thursday, July 21, 2016 1:27:10 PM
> Subject: Re: Slow performance with Intel X540-T2 10Gb NIC
>
> (Replying-all this time)
>
> Did you try to set these settings that ESnet recommends?
> https://fasterdata.es.net/host-tuning/freebsd/
>
> We don't use iperf3 here at Intel (we use netperf instead), so I'm not
> sure I can be much help diagnosing what's wrong.
>
> On Wed, Jul 20, 2016 at 5:39 PM Chris Dunbar < ch...@dunbar.net > wrote:
>
>
> Hello,
>
> I am new to FreeBSD and recently built a file server out of new components
> running FreeBSD 10.3. I installed an Intel X540-T2 10 Gb NIC and am
> experiencing what I consider to be slow transfer speeds. I am using iperf3
> to measure the speed and test the results of modifications. So far nothing
> I have done has made a noticeable difference. If I run iperf3 -s on the
> FreeBSD server, I see transfer speeds of approximately 1.6 Gb/s. If I run
> iperf3 in client mode, the speed improves to ~2.75 Gb/s. However, if I
> replace FreeBSD with CentOS 7 on the same hardware, I see iperf3 speeds
> surpassing 8 GB/s. The other end of my iperf3 test is a Windows 10 box that
> also has an Intel X540-T2 installed.
>
> I did notice that FreeBSD 10.3 (and 11.0 alpha 6 for that matter) includes
> a slightly older Intel driver (v3.1.13-k). I managed to build a custom
> kernel that removed the Intel PRO/10GbE PCIE NIC drivers. That allowed me
> to manually load the latest 3.1.14 driver downloaded from Intel's web site.
> Unfortunately that did not produce any improvements. I am working my way
> through man tuning() and some other articles on network performance. So far
> nothing I tweak makes a noticeable difference. I'm increasingly skeptical
> that I am going to find a setting or two that more than doubles the speed I
> am currently experiencing.
>
> I am open to any and all suggestions at this point.
>
> Thank you!
> Chris
> ___
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to " freebsd-net-unsubscr...@freebsd.org "
>
>
>
>
> ___
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Possible transmit/stats problem in igb driver.

[Jack Vogel]

That's an interesting theory, you could add a check into the tx path
looking for a zero m_len and see, seems unlikely though :)

Jack



On Fri, Jun 3, 2016 at 1:15 PM, Sreekanth Rupavatharam  wrote:

> Wondering if this can happen if somehow the mbuf->m_len is not
> correct(e.g., 0) and thus causing the dma to fail silently. The only way
> this is happening if the arp request is larger than 64 bytes and the arp
> response code is reusing the packet to send a 64 byte response.
>
> Thanks,
>
> -Sreekanth
>
>
> On 6/2/16, 2:41 PM, "hiren panchasara"  wrote:
>
> >+ Sean, Eric
> >
> >On 06/02/16 at 09:11P, Sreekanth Rupavatharam wrote:
> >> Inline
> >>
> >> >Apart from stats, do you see anything else going wrong? i.e. do you
> >> >actually see less packets (arp replies??) than expected?
> >>
> >> [SR] The packets are not going out on the wire. The tool doesn?t
> receive the packets. That?s how I started noticing the issue.
> >>
> >> >Taking your example, tx_packets is something we count in the drivers
> and
> >> >total_pkts_txd is calculated in the card and we just read it off of it
> >> >to report (E1000_TPT).
> >>
> >> [SR] Correct. My main question would be under what circumstance would
> the packet handed off to hardware will *not* be transmitted?. Especially
> considering there are no transmit errors or pause frames received. There
> are no dma tx failures either. That?s the baffling part. I tried another
> exercise where I used ping of various sizes going out, but that doesn?t
> seem to trigger the problem.
> >>
> >>
> >> >To understand your setup better, ixia is the sender and your box with
> >> >igb(4) is the receiver and your are sending arp requests to it.
> >>
> >> Yes, correct.
> >>
> >> >Can you post following for working (size <= 64bytes) and non-working
> >> >(size > 64bytes) cases for before/after?
> >> >
> >> >sysctl dev.igb | grep tx_packets
> >> >sysctl dev.igb | grep total_pkts_txd
> >> >sysctl dev.igb | grep rx_packets
> >> >sysctl dev.igb | grep total_pkts_recvd
> >>
> >>
> >> Before(not working):
> >> dev.igb.1.queue0.tx_packets: 24907933
> >> dev.igb.1.queue0.rx_packets: 18086575
> >> dev.igb.1.mac_stats.total_pkts_recvd: 25057359
> >> dev.igb.1.mac_stats.total_pkts_txd: 16647169
> >>
> >> After(not working):
> >> dev.igb.1.queue0.tx_packets: 24913324
> >> dev.igb.1.queue0.rx_packets: 18091832
> >> dev.igb.1.mac_stats.total_pkts_recvd: 25062618
> >> dev.igb.1.mac_stats.total_pkts_txd: 16647545
> >> >netstat -sp arp
> >>
> >> The difference is  5391 for queue0.tx_packets but for
> mac_stats.total_pkts_txd  is 376
> >> Everything else is matching up.
> >>
> >> Before (working)
> >> dev.igb.1.queue0.tx_packets: 25359165
> >> dev.igb.1.queue0.rx_packets: 18526094
> >> dev.igb.1.mac_stats.total_pkts_recvd: 25508763
> >> dev.igb.1.mac_stats.total_pkts_txd: 16831587
> >>
> >>
> >> After(working)
> >> dev.igb.1.queue0.tx_packets: 25364597
> >> dev.igb.1.queue0.rx_packets: 18531398
> >> dev.igb.1.mac_stats.total_pkts_recvd: 25514009
> >> dev.igb.1.mac_stats.total_pkts_txd: 16836833
> >>
> >>
> >> Another interesting stat is
> >> before_notworking:dev.igb.1.interrupts.tx_queue_empty: 16646890
> >> after_notworking:dev.igb.1.interrupts.tx_queue_empty: 16647266
> >>
> >> The difference here is exactly 376 which is the number of packets that
> the device actually claims to have transmitted. It?s as though it didn?t
> see the other packets en-queued in the ring descriptor.
> >>
> >
> >Very interesting. Do you tune defaults at all? What does sysctl hw.igb
> >say? Not sure if bumping up txd would help.
> >
> >Adding Sean and Eric to throw some light.
> >
> >>
> >> I can?t do netstat just for arp as these are coming in a tunnel(Packets
> don?t? show up as arp on the interface). However, I did see the packet rate
> was about 500 packets/sec
> >>
> >
> >Cheers,
> >Hiren
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Possible transmit/stats problem in igb driver.

[Jack Vogel]

Hmmm, so I no longer have access to any of this, but I'm wondering if there
is some
other stat that might report errors or some sort that would match the
missing/bad packets?

Maybe they were considered runts or something and dropped.

Good luck :)

Jack


On Thu, Jun 2, 2016 at 2:11 PM, Sreekanth Rupavatharam  wrote:

> Inline
>
> >Apart from stats, do you see anything else going wrong? i.e. do you
> >actually see less packets (arp replies??) than expected?
>
> [SR] The packets are not going out on the wire. The tool doesn’t receive
> the packets. That’s how I started noticing the issue.
>
> >Taking your example, tx_packets is something we count in the drivers and
> >total_pkts_txd is calculated in the card and we just read it off of it
> >to report (E1000_TPT).
>
> [SR] Correct. My main question would be under what circumstance would the
> packet handed off to hardware will *not* be transmitted?. Especially
> considering there are no transmit errors or pause frames received. There
> are no dma tx failures either. That’s the baffling part. I tried another
> exercise where I used ping of various sizes going out, but that doesn’t
> seem to trigger the problem.
>
>
> >To understand your setup better, ixia is the sender and your box with
> >igb(4) is the receiver and your are sending arp requests to it.
>
> Yes, correct.
>
> >Can you post following for working (size <= 64bytes) and non-working
> >(size > 64bytes) cases for before/after?
> >
> >sysctl dev.igb | grep tx_packets
> >sysctl dev.igb | grep total_pkts_txd
> >sysctl dev.igb | grep rx_packets
> >sysctl dev.igb | grep total_pkts_recvd
>
>
> Before(not working):
> dev.igb.1.queue0.tx_packets: 24907933
> dev.igb.1.queue0.rx_packets: 18086575
> dev.igb.1.mac_stats.total_pkts_recvd: 25057359
> dev.igb.1.mac_stats.total_pkts_txd: 16647169
>
> After(not working):
> dev.igb.1.queue0.tx_packets: 24913324
> dev.igb.1.queue0.rx_packets: 18091832
> dev.igb.1.mac_stats.total_pkts_recvd: 25062618
> dev.igb.1.mac_stats.total_pkts_txd: 16647545
> >netstat -sp arp
>
> The difference is  5391 for queue0.tx_packets but for
> mac_stats.total_pkts_txd  is 376
> Everything else is matching up.
>
> Before (working)
> dev.igb.1.queue0.tx_packets: 25359165
> dev.igb.1.queue0.rx_packets: 18526094
> dev.igb.1.mac_stats.total_pkts_recvd: 25508763
> dev.igb.1.mac_stats.total_pkts_txd: 16831587
>
>
> After(working)
> dev.igb.1.queue0.tx_packets: 25364597
> dev.igb.1.queue0.rx_packets: 18531398
> dev.igb.1.mac_stats.total_pkts_recvd: 25514009
> dev.igb.1.mac_stats.total_pkts_txd: 16836833
>
>
> Another interesting stat is
> before_notworking:dev.igb.1.interrupts.tx_queue_empty: 16646890
> after_notworking:dev.igb.1.interrupts.tx_queue_empty: 16647266
>
> The difference here is exactly 376 which is the number of packets that the
> device actually claims to have transmitted. It’s as though it didn’t see
> the other packets en-queued in the ring descriptor.
>
>
> I can’t do netstat just for arp as these are coming in a tunnel(Packets
> don’t’ show up as arp on the interface). However, I did see the packet rate
> was about 500 packets/sec
>
>
> >(You can netstat -z to clear the counters between runs.)
> >
> >Cheers,
> >Hiren
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Question on sysctl tree handling

[Jack Vogel]

Hmmm,  after so many years of unloading and reloading drivers, it just
seemed like such
a convenience :)  But, instability isn't fun either, I've looked at bxe a
little, such a monstrous
driver :)

I'll give it some thought, and maybe discuss it with David. Thanks.

Cheers,

Jack


On Mon, Apr 11, 2016 at 3:09 PM, Matt Joras  wrote:

> Expanding on what mmacy said... I don't think the benefits of "easy
> reconfiguration" are worth the headaches you're going to potentially
> run into in production.
>
> bxe(4) used to do this, and it caused us a lot of problems (i.e. panics)
> at $DAY_JOB. For example, if a lagg was on top of bxe and then you
> downed bxe you could very easily hit a use-after-free since bxe free'd
> its rings while if_lagg is trying to transmit a packet.
>
> Matt Joras
>
> On Mon, Apr 11, 2016 at 2:03 PM, K. Macy  wrote:
> > You do understand that init needs to be run every time interface
> > settings are changed (TSO / PROMISC / CSUM/ etc)? Reallocating queues
> > and interrupts every time is fragile (long running systems can run low
> > on contiguous memory) and, in the common case that you're not actually
> > changing the number, gratuitous.
> >
> > Cheers.
> > -M
> >
> > On Fri, Apr 8, 2016 at 2:56 PM, Jack Vogel  wrote:
> >> LOL, why does it seem that as soon as I ask the answer hits me in the
> nose
> >> :)
> >>
> >> I found the sysctl_ctx_free call, sorry for the noise
> >>
> >> Jack
> >>
> >>
> >> On Fri, Apr 8, 2016 at 2:51 PM, Jack Vogel  wrote:
> >>
> >>>
> >>> I have a driver design where the queue/ring/irq layout is done in init
> >>> rather
> >>> than in attach, allowing easy reconfiguration. What I'm not sure about
> is
> >>> how to handle the sysctl tree during a reinit, I don't see a procedure
> to
> >>> free up things so I can restructure :(
> >>>
> >>> Am I missing something, any pointers or suggestions appreciated.
> >>>
> >>> Thanks,
> >>>
> >>> Jack
> >>>
> >>>
> >> ___
> >> freebsd-net@freebsd.org mailing list
> >> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> >> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
> > ___
> > freebsd-net@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Question on sysctl tree handling

[Jack Vogel]

Why should init have to be rerun just because TSO or CSUM is turned off or
on,
its a flag that is checked in the TX routines?

I am controlling some cases in the ioctl routine so it does the necessary
things,
but does not rerun all of init.

If its a long running system that means its unlikely be go undergoing a
config
change, yes?

BTW,  the sysctl stuff is still giving me trouble. I create a new node, and
add
stats oids to that, if a reinit happens, they can be removed, but then I
get a
warning message when the driver goes thru detach complaining that an oid
cannot be unregistered... It's proving to be frustrating :(

I appreciate the comments Kip.

Jack


On Mon, Apr 11, 2016 at 2:03 PM, K. Macy  wrote:

> You do understand that init needs to be run every time interface
> settings are changed (TSO / PROMISC / CSUM/ etc)? Reallocating queues
> and interrupts every time is fragile (long running systems can run low
> on contiguous memory) and, in the common case that you're not actually
> changing the number, gratuitous.
>
> Cheers.
> -M
>
> On Fri, Apr 8, 2016 at 2:56 PM, Jack Vogel  wrote:
> > LOL, why does it seem that as soon as I ask the answer hits me in the
> nose
> > :)
> >
> > I found the sysctl_ctx_free call, sorry for the noise
> >
> > Jack
> >
> >
> > On Fri, Apr 8, 2016 at 2:51 PM, Jack Vogel  wrote:
> >
> >>
> >> I have a driver design where the queue/ring/irq layout is done in init
> >> rather
> >> than in attach, allowing easy reconfiguration. What I'm not sure about
> is
> >> how to handle the sysctl tree during a reinit, I don't see a procedure
> to
> >> free up things so I can restructure :(
> >>
> >> Am I missing something, any pointers or suggestions appreciated.
> >>
> >> Thanks,
> >>
> >> Jack
> >>
> >>
> > ___
> > freebsd-net@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Question on sysctl tree handling

[Jack Vogel]

LOL, why does it seem that as soon as I ask the answer hits me in the nose
:)

I found the sysctl_ctx_free call, sorry for the noise

Jack


On Fri, Apr 8, 2016 at 2:51 PM, Jack Vogel  wrote:

>
> I have a driver design where the queue/ring/irq layout is done in init
> rather
> than in attach, allowing easy reconfiguration. What I'm not sure about is
> how to handle the sysctl tree during a reinit, I don't see a procedure to
> free up things so I can restructure :(
>
> Am I missing something, any pointers or suggestions appreciated.
>
> Thanks,
>
> Jack
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Question on sysctl tree handling

[Jack Vogel]

I have a driver design where the queue/ring/irq layout is done in init
rather
than in attach, allowing easy reconfiguration. What I'm not sure about is
how to handle the sysctl tree during a reinit, I don't see a procedure to
free up things so I can restructure :(

Am I missing something, any pointers or suggestions appreciated.

Thanks,

Jack
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

TSO test

[Jack Vogel]

Anyone have a 'simple' test case for TSO, something a bit
more distilled than running netperf or iperf?

Thanks,

Jack
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Intel XL710 broken link down detection?

[Jack Vogel]

LOL, Damn, what's the world coming to when you can't trust them Linux guys
:)

Jack




On Wed, Nov 11, 2015 at 8:31 AM, Ryan Stone  wrote:

> On Wed, Nov 11, 2015 at 11:18 AM, Steven Hartland 
> wrote:
>
>> Comparing this to the Linux driver which does detect the link down I've
>> discovered it actually polls the link status by default in its watchdog.
>>
>> Disabling this with "ethtool --set-priv-flags eth1 LinkPolling off" and
>> the Linux driver also fails to detect link down.
>>
>> So this seems like a firmware or even hardware bug where it should be
>> reporting down events and the Linux driver has been updated to workaround
>> the problem?
>
>
> No, apparently the Linux devs just didn't read the datasheet closely
> enough (and presumably the FreeBSD driver copied the mistake).  There is a
> mask of interrupt causes that works backwards from how one would expect;
> you mask out events that you *don't* want rather than events that you do
> want.  Both the Linux and FreeBSD drivers pass a mask of events that they
> want interrupts for (the only reason why it appears to work on link up is
> that the the AN Completed event fires when link is up, as far as I can
> tell).  Try the following patch:
>
> https://people.freebsd.org/~rstone/patches/ixl_link_int.diff
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: ixl 40G bad performance?

[Jack Vogel]

13 on a 40G interface?? I don't think that's very good for Linux either, is
this a 4x10 adapter?
Maybe elaborating on the details of the hardware, you sure you don't have a
bad PCI slot
somewhere that might be throttling everything?

Cheers,

Jack


On Sat, Oct 24, 2015 at 12:43 AM, Eggert, Lars  wrote:

> On 2015-10-23, at 23:36, Eric Joyner  wrote:
>
> I see that the sysctl does clobber the global value, but have you tried
> lowering the interval / raising the rate? You could try something like
> 10usecs, and see if that helps. We'll do some more investigation here --
> 3Gb/s on a 40Gb/s using default settings is terrible, and we shouldn't let
> that be happening.
>
>
> I played with different settings, but I've never been able to get more
> than 4Gb/s, whereas under Linux 4.2 without any special settings I get 13.
>
> See my other email on TSO/LRO not looking to be effective; that would
> certainly explain it. Plausible? Anything to try here?
>
> Lars
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: ixl 40G bad performance?

[Jack Vogel]

The 40G hardware is absolutely dependent on firmware, if you have a mismatch
for instance, it can totally bork things. So, I would work with your Intel
rep and be
sure you have the correct version for your specific hardware.

Good luck,

Jack


On Wed, Oct 21, 2015 at 5:25 AM, Eggert, Lars  wrote:

> Hi Bruce,
>
> thanks for the very detailed analysis of the ixl sysctls!
>
> On 2015-10-20, at 16:51, Bruce Evans  wrote:
> >
> > Lowering (improving) latency always lowers (unimproves) throughput by
> > increasing load.
>
> That, I also understand. But even when I back off the itr values to
> something more reasonable, throughput still remains low.
>
> With all the tweaking I have tried, I have yet to top 3 Gb/s with ixl
> cards, whereas they do ~13 Gb/s on Linux straight out of the box.
>
> Lars
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: GPL issues around OFED code in FreeBSD 9.1

[Jack Vogel]

We (meaning Intel when I was still there) raised this issue with George a
long time ago, I'm not sure
what the resolution was.

If Mellanox is the owner then they should have released the code somewhere
without any
GPL license in it, as Intel does with code they multi-license.

Jack


On Thu, Sep 3, 2015 at 12:07 PM, K. Macy  wrote:

> On Sep 3, 2015 10:33 AM, "Vijay Singh"  wrote:
> >
> > Someone told me that once the OFED code hit kernel.org the GPL is the
> only
> > license that applies. Does anyone have insights about that?
>
> That sounds bizarre since mellanox wrote the code and explicitly dual
> licensed.
>
> The problem you *do* run in to is code creep. The Linux internal interfaces
> are almost certainly totally undocumented, so a clean room reimplementation
> as they change or bugs get fixed - changing behaviors - is not possible.
> Over time the tendency is to copy and paste from Linux to OFED or the shim
> layer creating real ambiguity about provenance. In practice it's not so
> much of a problem because Linus tends to adhere to an interpretation of the
> GPLv2 that is not vendor unfriendly the way some of his lieutenants would
> prefer (see Greg's efforts to call closed source drivers derived works).
>
> Nonetheless, if you're paranoid an audit is in order.
>
> Cheers.
>
> >
> > On Mon, Aug 31, 2015 at 10:25 AM, Garrett Cooper 
> > wrote:
> >
> > >
> > > > On Aug 31, 2015, at 09:34, Hrishikesh Keremane via freebsd-hackers <
> > > freebsd-hack...@freebsd.org> wrote:
> > > >
> > > > [Sorry for cross posting]
> > > >
> > > > Hi,
> > > >
> > > > We are working on a product(FreeBSD based) that would require RDMA
> over
> > > iWARP and are considering using the OFED stack in FreeBSD 9.1.
> > > > We will be making some changes to the OFED stack to customize it to
> our
> > > requirements.
> > > >
> > > > The concern is regarding the implications of GPL licensing of OFED on
> > > our code base.
> > > > Has anyone worked with OFED in FreeBSD and/or is aware of the
> licensing
> > > issues around it?
> > > >
> > > > Thanks in advance for your help.
> > > >
> > > > Please include me in your replies as I am not subscribed to these
> lists.
> > >
> > > The OFED stack is BSD/GPLv2 dual licensed IIRC. the Mellanox import
> might
> > > have made it 100% BSD licensed though.
> > >
> > > There's freebsd-infinib...@freebsd.org as well. It's a low traffic
> list,
> > > but it might hit a better target audience in the future.
> > >
> > > Cheers,
> > > -NGie
> > > ___
> > > freebsd-net@freebsd.org mailing list
> > > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
> > >
> > ___
> > freebsd-hack...@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> > To unsubscribe, send any mail to "
> freebsd-hackers-unsubscr...@freebsd.org"
> ___
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Reg Intel Fortville IXL driver on 11-CURRENT

[Jack Vogel]

As Ryan said, its there to keep queues marked as "hung" from
getting more work scheduled on them. I really don't know what
to make of it if commenting it out is somehow improving things :)

I would suggest more careful analysis of what is going wrong
before committing anything like deleting this.

Jack


On Wed, Jun 17, 2015 at 12:17 PM, Ryan Stone  wrote:

> On Wed, Jun 17, 2015 at 7:00 AM, Lakshmi Narasimhan Sundararajan <
> lakshm...@msystechnologies.com> wrote:
>
> > [lakshmis@mau-bsd-10a ~/fortville/hol/sys/dev/ixl]$ diff -c5pt
> ixl_txrx.c
> > ixl_txrx.c.mod
> > *** ixl_txrx.c Fri Jun 12 06:56:51 2015
> > --- ixl_txrx.c.mod Fri Jun 12 06:56:33 2015
> > *** ixl_mq_start(struct ifnet *ifp, struct m
> > *** 96,112 
> > --- 96,115 
> >   } else
> >   #endif
> >   i = m->m_pkthdr.flowid % vsi->num_queues;
> >   } else
> >   i = curcpu % vsi->num_queues;
> > +
> > + #if 0
> >   /*
> >   ** This may not be perfect, but until something
> >   ** better comes along it will keep from scheduling
> >   ** on stalled queues.
> >   */
> >   if (((1 << i) & vsi->active_queues) == 0)
> >   i = ffsl(vsi->active_queues);
> > + #endif
> >
> >   que = &vsi->queues[i];
> >   txr = &que->txr;
> >
> >   err = drbr_enqueue(ifp, txr->br, m);
> > [lakshmis@mau-bsd-10a ~/fortville/hol/sys/dev/ixl]$
> >
>
> My understanding is that this code is intended to be triggered as a last
> resort.  If that code is firing regularly then the driver is not correctly
> tracking which queues are alive in the "active_queues" bitmask.
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Netmap problem with e1000e driver

[Jack Vogel]

Ah, didn't realize that, apologies, and I'll leave it to Luigi then :)

Jack


On Wed, May 27, 2015 at 2:47 PM, David DeSimone  wrote:

> Actually, Luigi has specifically requested that all users of netmap (Linux
> or BSD) use this list to field all of their questions.
>
>
> -Original Message-
> From: owner-freebsd-...@freebsd.org [mailto:owner-freebsd-...@freebsd.org]
> On Behalf Of Jack Vogel
> Sent: Wednesday, May 27, 2015 11:22 AM
> To: Oleg Prozorov
> Cc: n...@freebsd.org
> Subject: Re: Netmap problem with e1000e driver
>
> Maybe asking a Linux mailing list about a Linux driver would be more
> productive :)
>
> Jack
>
>
> On Wed, May 27, 2015 at 8:26 AM, Oleg Prozorov 
> wrote:
>
> > Hello All,
> > I am using Netmap technology in my program and have the problem :
> > when I put eth link down and then have it up netmap goes down with kernel
> > messages:
> >
> > log from dmesg:
> >
> > [ 2457.286289] irq 44: nobody cared (try booting with the "irqpoll"
> option)
> > [ 2457.286296] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G   O
> > 3.16.0-4-amd64 #1 Debian 3.16.7-ckt9-3~deb8u1
> > [ 2457.286298] Hardware name: System manufacturer System Product
> > Name/P8H61-MX R2.0, BIOS 0803 10/26/2012
> > [ 2457.286300]  880198b544c4 8150ac96 880198b54400
> > 810bd10d
> > [ 2457.286304]  880198b54400 002c 
> > 810bd631
> > [ 2457.286307]    002c
> > 
> > [ 2457.286310] Call Trace:
> > [ 2457.286312][] ? dump_stack+0x41/0x51
> > [ 2457.286324]  [] ? __report_bad_irq+0x2d/0xc0
> > [ 2457.286328]  [] ? note_interrupt+0x241/0x290
> > [ 2457.286332]  [] ? handle_irq_event_percpu+0xa1/0x190
> > [ 2457.286336]  [] ? handle_irq_event+0x38/0x60
> > [ 2457.286341]  [] ? handle_edge_irq+0x85/0x150
> > [ 2457.286347]  [] ? handle_irq+0x1d/0x30
> > [ 2457.286350]  [] ? do_IRQ+0x49/0xe0
> > [ 2457.286355]  [] ? common_interrupt+0x6d/0x6d
> > [ 2457.286356][] ?
> > __hrtimer_start_range_ns+0x1cd/0x390
> > [ 2457.286364]  [] ? cpuidle_enter_state+0x52/0xc0
> > [ 2457.286368]  [] ? cpuidle_enter_state+0x48/0xc0
> > [ 2457.286372]  [] ? cpu_startup_entry+0x2f8/0x400
> > [ 2457.286375]  [] ? start_secondary+0x20f/0x2d0
> > [ 2457.286377] handlers:
> > [ 2457.286384] [] e1000_msix_other [e1000e]
> > [ 2457.286386] Disabling IRQ #44
> >
> > before i have the next steps:
> >
> > insmod ./netmap.ko
> > insmod ./e1000e/e1000e.ko
> >
> > log from dmesg:
> >
> > [ 1645.548786] e1000e: Intel(R) PRO/1000 Network Driver - 2.3.2-k
> > [ 1645.548791] e1000e: Copyright(c) 1999 - 2014 Intel Corporation.
> > [ 1645.549056] e1000e :02:00.0: Interrupt Throttling Rate (ints/sec)
> > set to dynamic conservative mode
> > [ 1645.549089] e1000e :02:00.0: irq 42 for MSI/MSI-X
> > [ 1645.549094] e1000e :02:00.0: irq 43 for MSI/MSI-X
> > [ 1645.549098] e1000e :02:00.0: irq 44 for MSI/MSI-X
> > [ 1645.704831] 635.291734 [2720] netmap_attach success for
> eth0
> > tx 1/256 rx 1/256 queues/slots
> > [ 1645.705079] e1000e :02:00.0 eth0: registered PHC clock
> > [ 1645.705083] e1000e :02:00.0 eth0: (PCI Express:2.5GT/s:Width x1)
> > 68:05:ca:28:36:f5
> > [ 1645.705086] e1000e :02:00.0 eth0: Intel(R) PRO/1000 Network
> > Connection
> > [ 1645.705100] e1000e :02:00.0 eth0: MAC: 3, PHY: 8, PBA No:
> E46981-008
> > [ 1645.705349] e1000e :03:00.0: Interrupt Throttling Rate (ints/sec)
> > set to dynamic conservative mode
> > [ 1645.705381] e1000e :03:00.0: irq 45 for MSI/MSI-X
> > [ 1645.705386] e1000e :03:00.0: irq 46 for MSI/MSI-X
> > [ 1645.705390] e1000e :03:00.0: irq 47 for MSI/MSI-X
> > [ 1645.739490] systemd-udevd[2715]: renamed network interface eth0 to
> eth5
> > [ 1645.824716] IPv6: ADDRCONF(NETDEV_UP): eth5: link is not ready
> > [ 1645.848756] 635.435799 [2720] netmap_attach success for
> eth0
> > tx 1/256 rx 1/256 queues/slots
> > [ 1645.848851] e1000e :03:00.0 eth0: registered PHC clock
> > [ 1645.848856] e1000e :03:00.0 eth0: (PCI Express:2.5GT/s:Width x1)
> > 68:05:ca:22:19:e7
> > [ 1645.848859] e1000e :03:00.0 eth0: Intel(R) PRO/1000 Network
> > Connection
> > [ 1645.848875] e1000e :03:00.0 eth0: MAC: 3, PHY: 8, PBA No:
> E46981-008
> > [ 1645.869528] systemd-udevd[2715]: renamed network interface eth0 to
> eth6
> > [ 1645.949935] IPv6: ADDRCONF(NETDEV_UP): eth6: 

Re: Netmap problem with e1000e driver

[Jack Vogel]

Maybe asking a Linux mailing list about a Linux driver would be more
productive :)

Jack


On Wed, May 27, 2015 at 8:26 AM, Oleg Prozorov 
wrote:

> Hello All,
> I am using Netmap technology in my program and have the problem :
> when I put eth link down and then have it up netmap goes down with kernel
> messages:
>
> log from dmesg:
>
> [ 2457.286289] irq 44: nobody cared (try booting with the "irqpoll" option)
> [ 2457.286296] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G   O
> 3.16.0-4-amd64 #1 Debian 3.16.7-ckt9-3~deb8u1
> [ 2457.286298] Hardware name: System manufacturer System Product
> Name/P8H61-MX R2.0, BIOS 0803 10/26/2012
> [ 2457.286300]  880198b544c4 8150ac96 880198b54400
> 810bd10d
> [ 2457.286304]  880198b54400 002c 
> 810bd631
> [ 2457.286307]    002c
> 
> [ 2457.286310] Call Trace:
> [ 2457.286312][] ? dump_stack+0x41/0x51
> [ 2457.286324]  [] ? __report_bad_irq+0x2d/0xc0
> [ 2457.286328]  [] ? note_interrupt+0x241/0x290
> [ 2457.286332]  [] ? handle_irq_event_percpu+0xa1/0x190
> [ 2457.286336]  [] ? handle_irq_event+0x38/0x60
> [ 2457.286341]  [] ? handle_edge_irq+0x85/0x150
> [ 2457.286347]  [] ? handle_irq+0x1d/0x30
> [ 2457.286350]  [] ? do_IRQ+0x49/0xe0
> [ 2457.286355]  [] ? common_interrupt+0x6d/0x6d
> [ 2457.286356][] ?
> __hrtimer_start_range_ns+0x1cd/0x390
> [ 2457.286364]  [] ? cpuidle_enter_state+0x52/0xc0
> [ 2457.286368]  [] ? cpuidle_enter_state+0x48/0xc0
> [ 2457.286372]  [] ? cpu_startup_entry+0x2f8/0x400
> [ 2457.286375]  [] ? start_secondary+0x20f/0x2d0
> [ 2457.286377] handlers:
> [ 2457.286384] [] e1000_msix_other [e1000e]
> [ 2457.286386] Disabling IRQ #44
>
> before i have the next steps:
>
> insmod ./netmap.ko
> insmod ./e1000e/e1000e.ko
>
> log from dmesg:
>
> [ 1645.548786] e1000e: Intel(R) PRO/1000 Network Driver - 2.3.2-k
> [ 1645.548791] e1000e: Copyright(c) 1999 - 2014 Intel Corporation.
> [ 1645.549056] e1000e :02:00.0: Interrupt Throttling Rate (ints/sec)
> set to dynamic conservative mode
> [ 1645.549089] e1000e :02:00.0: irq 42 for MSI/MSI-X
> [ 1645.549094] e1000e :02:00.0: irq 43 for MSI/MSI-X
> [ 1645.549098] e1000e :02:00.0: irq 44 for MSI/MSI-X
> [ 1645.704831] 635.291734 [2720] netmap_attach success for eth0
> tx 1/256 rx 1/256 queues/slots
> [ 1645.705079] e1000e :02:00.0 eth0: registered PHC clock
> [ 1645.705083] e1000e :02:00.0 eth0: (PCI Express:2.5GT/s:Width x1)
> 68:05:ca:28:36:f5
> [ 1645.705086] e1000e :02:00.0 eth0: Intel(R) PRO/1000 Network
> Connection
> [ 1645.705100] e1000e :02:00.0 eth0: MAC: 3, PHY: 8, PBA No: E46981-008
> [ 1645.705349] e1000e :03:00.0: Interrupt Throttling Rate (ints/sec)
> set to dynamic conservative mode
> [ 1645.705381] e1000e :03:00.0: irq 45 for MSI/MSI-X
> [ 1645.705386] e1000e :03:00.0: irq 46 for MSI/MSI-X
> [ 1645.705390] e1000e :03:00.0: irq 47 for MSI/MSI-X
> [ 1645.739490] systemd-udevd[2715]: renamed network interface eth0 to eth5
> [ 1645.824716] IPv6: ADDRCONF(NETDEV_UP): eth5: link is not ready
> [ 1645.848756] 635.435799 [2720] netmap_attach success for eth0
> tx 1/256 rx 1/256 queues/slots
> [ 1645.848851] e1000e :03:00.0 eth0: registered PHC clock
> [ 1645.848856] e1000e :03:00.0 eth0: (PCI Express:2.5GT/s:Width x1)
> 68:05:ca:22:19:e7
> [ 1645.848859] e1000e :03:00.0 eth0: Intel(R) PRO/1000 Network
> Connection
> [ 1645.848875] e1000e :03:00.0 eth0: MAC: 3, PHY: 8, PBA No: E46981-008
> [ 1645.869528] systemd-udevd[2715]: renamed network interface eth0 to eth6
> [ 1645.949935] IPv6: ADDRCONF(NETDEV_UP): eth6: link is not ready
> [ 1648.795532] e1000e: eth5 NIC Link is Up 1000 Mbps Full Duplex, Flow
> Control: Rx/Tx
> [ 1648.795856] IPv6: ADDRCONF(NETDEV_CHANGE): eth5: link becomes ready
> [ 1648.843497] e1000e: eth6 NIC Link is Up 1000 Mbps Full Duplex, Flow
> Control: Rx/Tx
> [ 1648.843820] IPv6: ADDRCONF(NETDEV_CHANGE): eth6: link becomes ready
> [ 1668.060841] e1000e: eth5 NIC Link is Up 1000 Mbps Full Duplex, Flow
> Control: Rx/Tx
> [ 1668.212692] e1000e: eth6 NIC Link is Up 1000 Mbps Full Duplex, Flow
> Control: Rx/Tx
>
>
> My program used netmap desc:
> root@debian:/home/debian/Projects/bin# lsmod | grep netmap
> netmap 99228  5 e1000e
>
>
> Then even if it shows that link is up net map stops to work and only
> rmmod/insmod and restart of the netmap can bring it back to work.
>
> Could you please help me with it ?
>
> Thx a lot,
> Oleg.
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org

Re: Full 32bit flowid from igb(4)

[Jack Vogel]

I think that line was there at one time outside the RSS define, I'd have to
take a look at
the code a bit more, or maybe Eric will

Jack


On Mon, Mar 23, 2015 at 4:42 PM, hiren panchasara <
hi...@strugglingcoder.info> wrote:

> Correcting Eric's email and subject line.
>
> On 03/23/15 at 04:39P, hiren panchasara wrote:
> > sco...@freebsd.org
> > Bcc:
> > Subject: Full 32bit flowid from igb(4) [was: Re: Unbalanced LACP link]
> > Reply-To:
> > In-Reply-To: <20150319175145.gh53...@strugglingcoder.info>
> >
> > On 03/19/15 at 10:51P, hiren panchasara wrote:
> > > On 03/17/15 at 12:34P, Adrian Chadd wrote:
> > > > On 17 March 2015 at 11:33, Jason Wolfe  wrote:
> > > > > On Mon, Mar 16, 2015 at 2:43 AM, Hans Petter Selasky <
> h...@selasky.org> wrote:
> > > > >> On 03/16/15 10:37, Vitalii Duk wrote:
> > > > >>>
> > > > >>> I've changed use_flowid to 0 and it helped! But isn't it setting
> > > > >>> significant? In a description it says "Shift flowid bits to
> prevent
> > > > >>> multiqueue collisions".
> > > > >>
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> Maybe your ethernet hardware is not properly setting the m_flowid
> ...
> > > > >>
> > > > >> --HPS
> > > > >>
> > > > >
> > > > > Flip use_flowid back to 1 and try setting
> > > > > net.link.lagg.default_flowid_shift / net.link.lagg.X.flowid_shift
> to 0
> > > > > as Hiren suggested.  r260179 added this shift, which has caused us
> > > > > balancing issues with the i350/igb.
> > > > >
> > > > > https://svnweb.freebsd.org/base?view=revision&revision=260179
> > > > >
> > > > > Based on Adrian's comment about igb/ixgbe not setting the 'full
> > > > > flowid' under normal conditions, does that mean this shift should
> be 0
> > > > > by default to ensure we don't break balancing for devices that only
> > > > > set the CPU/MSIX queue?
> > > >
> > > > Or we can just see if there's anything wrong with putting the full 32
> > > > bit RSS flowid in received packets that have them.
> > >
> > > It'd be nice to have but for now I am proposing following to fix a
> known
> > > broken case because of an optimization:
> > > https://reviews.freebsd.org/D2098
> >
> > Turns out, setting flowid_shift to 0 is not the correct solution. Please
> > look at the review above for more details.
> >
> > Looking at the code, we have a way to get full flowid but it's hidden
> > under "ifdef RSS":
> > rxr->fmp->m_pkthdr.flowid = le32toh(cur->wb.lower.hi_dword.rss);
> >
> > Using just this line gives me good hash on I350 igb(4) chipset that we
> > have.
> >
> > Is it possible to expose this outside RSS? Would this break other
> > things/chips?
> >
> > Cheers,
> > Hiren
>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Adding new media types to if_media.h

[Jack Vogel]

On Fri, Feb 27, 2015 at 11:23 AM, Gleb Smirnoff  wrote:

> On Thu, Feb 26, 2015 at 08:25:59PM -0800, Adrian Chadd wrote:
> A> [snip]
> A>
> A> I think Mike's approach is good - it makes it easy to MFC to 10.2
> A> since there's extended lifecycle stuff to do there - and then we can
> A> plan out how do the "betterer" fix after it's landed and churned
> A> things.
>
> ... and we will be ought to support the "betterer" fix along with
> the "not so betterer" for a very long time.
>
> The rock on which we split in this argument is that some developers
> write their code for stable/x and then forward-port it to head,
> focused on quality of result for stable/x; while other developers
> do the opposite: write code to head, then consider or not consider
> merging it stable/x.
>
>
I think this is oversimplified. In my 10 years in this role at Intel I've
had
cases when, in response to a customer issue, I had to work from
an existing code base to solve a problem, or add support for a new
feature. But then there are other times when I've been working on a
new driver, and its been totally developed from HEAD.

It depends on what's right for the circumstance, but as I said, on this
issue we have real product/customer needs that are short term, and
the competition (Linux and Windows) is prepared to handle the new
media today, I think its in FreeBSD's interest to address this ASAP.

Jack
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Adding new media types to if_media.h

[Jack Vogel]

I agree, like I said, we have hardware coming this year that will
need support, and as we just saw 11 isn't until 2016. Eric tested
Mike's code with our needed types and it worked. If something
truly revolutionary wants to be done for 11 it can be done after
this and that not backed into the 10.X stream.

Jack


On Thu, Feb 26, 2015 at 8:25 PM, Adrian Chadd  wrote:

> [snip]
>
> I think Mike's approach is good - it makes it easy to MFC to 10.2
> since there's extended lifecycle stuff to do there - and then we can
> plan out how do the "betterer" fix after it's landed and churned
> things.
>
>
>
> -a
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1965: Add extended media types to if_media.h and ifconfig

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

BRANCH
  /head

REVISION DETAIL
  https://reviews.freebsd.org/D1965

To: erj, gnn, adrian, jfvogel
Cc: glebius, freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Adding new media types to if_media.h

[Jack Vogel]

So we have products coming soon that need to extend the media, if you have
grandiose plans in the future you can worry about things then, Linux can
handle the extended media TODAY, wouldn't it be nice to do so as well?

Also, this solution is something that can be MFC'd into 10 STABLE,
if you do something big in the 11 stream I will be that wont be possible,
so I'd say lets do this as a first step.

Jack


On Wed, Feb 25, 2015 at 2:11 PM, Gleb Smirnoff  wrote:

> On Mon, Feb 16, 2015 at 07:50:56PM -0600, Mike Karels wrote:
> M> Well, I developed the prototype as I had planned, using a 64-bit media
> M> word, and found that I got about 100 files in GENERIC that didn't
> compile;
> M> they attempted to store "media words" in an int.  My kingdom for a
> typedef.
> M> That didn't meet my goal of KPI compatibility, so I went to Plan B.
> M>
> M> Plan B is to steal an unused bit (RFU) to indicate an "extended" media
> M> type.  I then used the variant/subtype field to store the extended type.
> M> Effectively, the previously unused bit doubles the effective size of the
> M> subtype  field.  Given that the previous 5-bit field lasted us 18 years,
> M> I figured that doubling it would last a while.  I also changed the
> M> SIOGGIFMEDIA ioctl, splitting it for binary compatibility; extended
> M> types are all mapped to IFM_OTHER (31) using the old interface, but
> M> are visible using the new one.
> M>
> M> With these changes, I modified one driver (vtnet) to use an extended
> type,
> M> and the rest of GENERIC is happy.  The changes to ifconfig are also
> fairly
> M> small.  The patch is appended, where email programs will screw it up,
> M> or at ftp://ftp.karels.net/outgoing/if_media.patch.
> M>
> M> The VFAST subtype is a throw-away for testing.
> M>
> M> This seems like a reasonably pragmatic change to support the new 40 Gb/s
> M> media types until someone wants to design an improved but non-backward-
> M> compatible interface.  I think it meets the goal of suitability for
> M> back-porting; it could be MFCed.
>
> I will dare to vote against the crowd.
>
> We can't and don't plan to preserve the driver KPI for the 11 branch. The
> plan, that I hope to accomplish by 11 is to provide a driver KPI, where
> drivers do not about struct ifnet, and other network stack stuff. Of
> course, that's a huge change in KPI. But we do it for the sake to avoid
> future changes.
>
> So, all this tricks with one extra bit seem unnecessary to me. I'd suggest
> to introduce new 'struct ifmedia' with enough space, and of course put
> extra
> space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code
> to handle it, without any extended bit tricks.
>
> For the sake of userland API, save old current 'struct ifmedia' as
> 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA.
> Write a function under BURN_BRIDGES that handles OSIOCIGIFMEDIA and
> tries to convert from ifmedia to oifmedia,
>
> To summarise: the patch adds tricks to just double the ifmedia name space,
> not solving the problem forever. New API is introduced, but old limited one
> doesn't have foreseable obsolete plan, since new is tied to it. All tricks
> are performed for the sake of driver KPI stability, which isn't planned
> to be kept for this major release cycle.
>
> --
> Totus tuus, Glebius.
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1881: Allow Illumos code to co-exist with nv(9)

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1881

To: rstone, jfvogel
Cc: will, emaste, pjd, freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1883: Move libnv into the kernel and hook it into the kernel build

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1883

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1882: Add macros to make code compile in kernel

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1882

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1880: Prevent creation of an invalid nvlist

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1880

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1879: Don't allocate memory for operations that do not insert

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1879

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1878: Add function to force an nvlist into the error state

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1878

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1877: Implement asprintf in libkern

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1877

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1875: Add tests for dnv_get_*

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1875

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1876: Add tests for dnvlist_take_*

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1876

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1874: Add tests for nvlist_free* functions

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1874

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1873: Add tests for nvlist_take_*

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1873

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1872: Add test cases for nvlist_move_*

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1872

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1871: Add tests for nvlist_pack/unpack

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1871

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1870: Add tests for nvlist_clone

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1870

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1869: Tests of basic nvlist add functions

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1869

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1868: Make libnv headers includable from C++

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1868

To: rstone, jfvogel
Cc: freebsd-net, pjd
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1648: Add stubs for deprecated VC messages

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

BRANCH
  review_D1648

REVISION DETAIL
  https://reviews.freebsd.org/D1648

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Adding new media types to if_media.h

[Jack Vogel]

Nice Mike, I like it also.

Jack


On Mon, Feb 16, 2015 at 5:50 PM, Mike Karels  wrote:

> On Feb 9, gnn wrote:
>
> > On 8 Feb 2015, at 22:41, Mike Karels wrote:
>
> > > Sorry to reply to a thread after such a long delay, but I think it is
> > > unresolved, and needs more discussion.  I'd like to elaborate a bit on
> > > my goals and proposal.  I believe Adrian has newer thoughts than have
> > > been
> > > circulated here as well.
> > >
> > > The last message(s) have gone to freebsd-arch and freebsd-net.  If
> > > someone
> > > wants to pick one, we could consolidate, but this seems relevant to
> > > both.
> > >
> > > I'm going to top-post to try to summarize and extend the discussion,
> > > but the
> > > preceding emails follow for reference.
> > >
> > > To recap: the existing if_media interface is running out of steam, at
> > > least
> > > in that the "Media variant" field, with 5 bits, is going to be
> > > insufficient
> > > to express existing 40 Gb/s variants.  The if_media media type is a
> > > 32-bit
> > > int with a bunch of sub-fields for type (e.g. Ethernet),
> > > subtype/variant
> > > (e.g.  10baseT, 10base5, 1000baseT, etc), flags, and some MII-related
> > > fields.
> > >
> > > I made a proposal to extend the interface in a small way, specifically
> > > to
> > > replace the "media word" with a 64-bit int that is mostly the same,
> > > but
> > > has a new, larger variant/subtype field.  The main reason for this
> > > proposal
> > > is to maintain the driver KPI (glimpse showed me 240 inclusions of
> > > if_media.h
> > > in the kernel in 8.2).  That interface includes an initialization
> > > using a
> > > scalar value of fields ORed with each other.  It would also be easy to
> > > preserve a 32-bit user-level API/ABI that can express most of the
> > > current
> > > state, with a subtype/variant field value reserved for "other" (there
> > > is
> > > already one for "unknown", but that is not quite the same).  fwiw, I
> > > found 45 references to this user-level API in our tree, including both
> > > base and "ports"-type software, which includes libpcap, snmpd,
> > > dhclient,
> > > quagga, xorp, atm, devd, and rtsold, which argues for a
> > > backward-compatible
> > > API/ABI as well as a more-complete current interface for ifconfig at
> > > least.
> > >
> > > More generally, I see two problems with the existing if_media
> > > interface:
> > >
> > > 1. It doesn't have enough bits for all the fields, in particular,
> > > variant/
> > > subtype for Ethernet.  That is the immediate issue.
> > >
> > > 2. The interface is not sufficiently generic; it was designed around
> > > Ethernet
> > > including MII, token ring, FDDI, and a few other interface types.
> > > Some of
> > > the fields like "instance" are primarily for MII as far as I know, and
> > > are
> > > basically unused.  It is definitely not sufficient for 802.11, which
> > > has
> > > rolled its own interfaces.
> > >
> > > To solve the second problem, I think the right approach would be to
> > > reduce
> > > this interface to a truly generic one, such as media type (e.g.
> > > Ethernet),
> > > generic flags, and perhaps generic status.  Then there should be a
> > > separate
> > > media-specific interface for each type, such as Ethernet and 802.11.
> > > To a
> > > small extent, we already have that.  Solving the second, more general
> > > problem,
> > > requires a whole new driver KPI that will require surgery to every
> > > driver,
> > > which is not an exercise that I would consider.
> > >
> > > Using a separate int for each existing field, as proposed, would break
> > > the
> > > driver KPI, but would not really make the interface generic.  Trying
> > > to
> > > make a single interface with the union of all network interface
> > > requirements
> > > seems like a bad idea to me (we failed last time; the "we" is BSDi,
> > > where
> > > I was the architect when this interface was first designed).  (No, I
> > > didn't
> > > design this interface.)
> > >
> > > Solving the first problem only, I think it is preferable to preserve a
> > > compatible driver KPI, which means using a scalar value encoding what
> > > is
> > > necessary.  Although that interface is rather Ethernet-centric, that
> > > is
> > > really what it is used for.
> > >
> > > An additional, selfish goal is to make it easy to back-port drivers
> > > using
> > > the new interface to older versions (which I am quite likely to do).
> > > Preserving the KPI and general user API will be highly useful there.
> > > I'd be likely to do a 11-style version of ifconfig personally, but it
> > > might not be difficult to do in a more general way.
> > >
> > > I am willing to do a prototype for -current for evaluation.
> > >
> > > Comments, alternatives, ?
>
> > I agree with your statements above and I'd like to see the prototype.
>
> Well, I developed the prototype as I had planned, using a 64-bit media
> word, and found that I got about 100 files in GENERIC that didn't compile;
> they attempte

Re: Was Re: FreeBSD 10.1: Intel dual port 10GbE card (82599EB) second port not present?: SFP+ on intel xl710

[Jack Vogel]

I should add, that this possibility Jeff mentions involves a tool, not the
driver, and
it is not something that there is any commitment or specifics on yet.

Cheers,

Jack


On Thu, Feb 12, 2015 at 12:24 PM, Pieper, Jeffrey E <
jeffrey.e.pie...@intel.com> wrote:

> The check for supported modules in X710/XL710 is done in the FW, so
> unfortunately there is nothing we can do in the driver to add that
> functionality. There is a possibility that it could happen at some point,
> but not in the foreseeable future.
>
> Jeff
>
> -Original Message-
> From: owner-freebsd-...@freebsd.org [mailto:owner-freebsd-...@freebsd.org]
> On Behalf Of Andreas Nilsson
> Sent: Thursday, February 12, 2015 10:54 AM
> To: FreeBSD Net
> Subject: Was Re: FreeBSD 10.1: Intel dual port 10GbE card (82599EB) second
> port not present?: SFP+ on intel xl710
>
> Hello,
>
> the discussion about unsupported sfps on intel card got an interesting turn
> today: a colleague wanted to do some pps (on linux) testing on an intel
> xl710 card, but the sfps we had was not supported.
>
> Reading the previous thread, I suggested there might be a sysctl to accept
> "other" sfps. Long story short, just for fun we booted a FreeBSD 10.1
> system, and the card was recognized ( yay! ) but still "no carrier" from
> ifconfig.
>
> sysctl hw.ixl didn't show any tunables for "unsupported_sfp". Is this chip
> more picky about the tranceivers?
>
> Best regards
> Andreas Nilsson
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Double cleanup in igb_attach

[Jack Vogel]

I do not recall if I put that call in myself and, yes, it seems odd. It was
probably trying to clean up
some bad state a failed attach left things in.

If it is removed it should be thoroughly regression tested.

Jack


On Thu, Feb 12, 2015 at 10:56 AM, Sreekanth Rupavatharam <
rupav...@juniper.net> wrote:

>   Hi Jack,
> Actually, looking at the code again, it seems to me that igb_detach is not
> supposed to be called from igb_attach at all. It causes more problems than
> previously mentioned. E.g.,  in case of branching to err_late *before*
> igb_setup_interface(where the ifp is initialized), calling igb_detach there
> causes a NULL pointer access. The best way to deal with it is to take out
> the igb_detach call from the err_late: label. Thoughts? Comments?
>  -- Thanks,
>
>  Sreekanth
>
>
>
>   From: Jack Vogel 
> Date: Tuesday, January 27, 2015 at 12:42 PM
> To: Sreekanth Rupavatharam 
> Cc: hiren panchasara , "
> freebsd-net@freebsd.org" , "j...@freebsd.org" <
> j...@freebsd.org>
> Subject: Re: Double cleanup in igb_attach
>
>   Yes, I will look them over.
>
> Jack
>
>
> On Tue, Jan 27, 2015 at 12:38 PM, Sreekanth Rupavatharam <
> rupav...@juniper.net> wrote:
>
>>  Thanks jack,
>>  Now, can you please review these changes? And commit if you deem it
>> fit?
>>
>> Thanks,
>>
>>  -Sreekanth
>>
>> On Jan 27, 2015, at 12:24 PM, "Jack Vogel"  wrote:
>>
>>   E, I am one of those people :)  (jack.vo...@intel.com)
>>
>>  Jack
>>
>>
>> On Tue, Jan 27, 2015 at 12:21 PM, Sreekanth Rupavatharam <
>> rupav...@juniper.net> wrote:
>>
>>>  Definitely, but I didn't have the contact info of those people.
>>>
>>> Thanks,
>>>
>>>  -Sreekanth
>>>
>>> On Jan 27, 2015, at 12:15 PM, "Jack Vogel"  wrote:
>>>
>>>   If you want something committed to an Intel driver, asking Intel
>>> might be the
>>> courteous thing to do, don't you think?
>>>
>>> Jack
>>>
>>>
>>> On Tue, Jan 27, 2015 at 11:51 AM, Sreekanth Rupavatharam <
>>> rupav...@juniper.net> wrote:
>>>
>>>>  Hiren,
>>>> Can you help commit this?
>>>>
>>>> Index: if_igb.c
>>>>
>>>> ===
>>>>
>>>> --- if_igb.c(revision 298053)
>>>>
>>>> +++ if_igb.c(working copy)
>>>>
>>>> @@ -723,7 +723,8 @@ igb_attach(device_t dev)
>>>>
>>>>  return (0);
>>>>
>>>>
>>>>
>>>>  err_late:
>>>>
>>>> -igb_detach(dev);
>>>>
>>>> +if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>>>>
>>>> +return(error);
>>>>
>>>>  igb_free_transmit_structures(adapter);
>>>>
>>>>  igb_free_receive_structures(adapter);
>>>>
>>>>  igb_release_hw_control(adapter);
>>>>
>>>>  -- Thanks,
>>>>
>>>>  Sreekanth
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  On 1/27/15, 11:28 AM, "hiren panchasara" 
>>>> wrote:
>>>>
>>>>  + Jack
>>>> On Tue, Jan 27, 2015 at 12:00:19AM +, Sreekanth Rupavatharam wrote:
>>>>
>>>> Apologies if this is not the right forum. In igb_attach function, we
>>>> have this code.
>>>> err_late:
>>>> igb_detach(dev);
>>>>  igb_free_transmit_structures(adapter);
>>>>  igb_free_receive_structures(adapter);
>>>>  igb_release_hw_control(adapter);
>>>> err_pci:
>>>>  igb_free_pci_resources(adapter);
>>>>  if (adapter->ifp != NULL)
>>>>  if_free(adapter->ifp);
>>>>  free(adapter->mta, M_DEVBUF);
>>>>  IGB_CORE_LOCK_DESTROY(adapter);
>>>>  The problem is that igb_detach also does the same cleanup in it?s
>>>> body. Only exception is this case where it just returns EBUSY
>>>>  /* Make sure VLANS are not using driver */
>>>> if (if_vlantrunkinuse(ifp)) {
>>>> device_printf(dev,"Vlan in use, detach first\n");
>>>> return (EBUSY);
>>>> }
>>>>  I think the code in igb_attach should be changed to free up resources
>>>> only if the igb_detach returns an error. Here?s the patch for it.
>>>>  Index: if_igb.c
>>>>  ===
>>>>  --- if_igb.c (revision 298025)
>>>>  +++ if_igb.c (working copy)
>>>>  @@ -723,7 +723,8 @@ igb_attach(device_t dev)
>>>> return (0);
>>>> err_late:
>>>>  - igb_detach(dev);
>>>>  + if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>>>>  + return;
>>>> igb_free_transmit_structures(adapter);
>>>>Can anyone comment on it and tell me if my understanding is
>>>> incorrect?
>>>>
>>>>
>>>>  Seems reasonable to me at the first glance.
>>>>
>>>>  We need to call IGB_CORE_LOCK_DESTROY(adapter) before returning
>>>> though.
>>>>
>>>>  cheers,
>>>> Hiren
>>>>
>>>>
>>>
>>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: FreeBSD 10.1: Intel dual port 10GbE card (82599EB) second port not present? (Steven Hartland)

[Jack Vogel]

On Thu, Feb 12, 2015 at 6:30 AM, John Jasen  wrote:

>
> > Date: Wed, 11 Feb 2015 20:47:15 +
> > From: Steven Hartland 
> > To: freebsd-net@freebsd.org
> > Subject: Re: FreeBSD 10.1: Intel dual port 10GbE card (82599EB):
> > second port not present?
> > Message-ID: <54dbbfd3.7010...@multiplay.co.uk>
> > Content-Type: text/plain; charset=windows-1252; format=flowed
>
> 
>
> > Your problem looks like its the SFP's which are "unsupported" to which
> > the attach will fail with error EIO (5).
> >
> > You can set hw.ix.unsupported_sfp=1 in /boot/loader.conf to enable
> > unsupported SFP's but be aware you "doing so your on your own".
>
> Thanks! hw.ix.unsupported_sfp=1 corrected the immediate problem.
>
> The network port that didn't come up does not yet have an SFP module
> installed
>
> It seems the default behavior in FreeBSD is to not have the interface
> available until a SFP is inserted. While a corner case, this does make
> configuration prior to installation a bit more annoying.
>
>
> Well, exactly what use would there be in having it "available"
with no media, its like saying you want your car available with
no wheels :)  The media can make operation quite different, it
could be copper it could be fiber, the driver won't know until
there is something actually present, hence it cannot really do
"init" until then.

Jack


>
>
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Intel 82574L (em)

[Jack Vogel]

Yup, I wrote that :)

Sean, I will check around to see if anything may have changed in that
regard.

Jack


On Fri, Jan 30, 2015 at 5:00 PM, hiren panchasara <
hi...@strugglingcoder.info> wrote:

> On Fri, Jan 30, 2015 at 01:11:50PM -0800, Sean Bruno wrote:
> >
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA512
> >
> >
> http://www.intel.com/content/dam/doc/datasheet/82574l-gbe-controller-datasheet.pdf
> >
> > According to 7.1.11, this device does indeed have 2 queues for stuff and
> > or things.  So, basic RSS would be possible in something like an Atom
> box.
> >
> > I note that the em(4) driver intentionally disables this on
> > initialization.  I'm up for some science on my new shiny, soon to be
> > router box. Any reason not to default to 1 queue and allow loader.conf
> > to raise it to 2?
>
> Intel folks know better but it seems this is hartwell.
>
> em_setup_msix() in very start says:
>
> /*
> ** Setup MSI/X for Hartwell: tests have shown
> ** use of two queues to be unstable, and to
> ** provide no great gain anyway, so we simply
> ** seperate the interrupts and use a single queue.
> */
>
> Things may have changed now. I guess you can try enabling it and find out
> :-)
>
> cheers,
> Hiren
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1661: Add some security-related config parameters

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1661

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1660: Add support for mac-addr parameter

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1660

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1659: Register ixl as an SR-IOV-capable driver during attach

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

can't wait to see the infrastructure in place :)

REVISION DETAIL
  https://reviews.freebsd.org/D1659

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1658: Add sysctls for per-VF hardware counters

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

BRANCH
  review_D1658

REVISION DETAIL
  https://reviews.freebsd.org/D1658

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1657: Allow VFs to run while the PF is admin down

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

BRANCH
  review_D1657

REVISION DETAIL
  https://reviews.freebsd.org/D1657

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1656: Handle VFLR events from VFs

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

Looks good. Again, will need real world testing when we have infrastructure.

REVISION DETAIL
  https://reviews.freebsd.org/D1656

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1655: Add support for GET_STATS VC message

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

BRANCH
  review_D1655

REVISION DETAIL
  https://reviews.freebsd.org/D1655

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1654: Add support for CONFIG_PROMISCUOUS_MODE VC message

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

No problems

REVISION DETAIL
  https://reviews.freebsd.org/D1654

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1653: Add support for ADD/DEL_VLAN VC messages

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

No issues.

REVISION DETAIL
  https://reviews.freebsd.org/D1653

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1652: Add support for ADD/DEL_ETHER_ADDRESS VC messages

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

Good.

BRANCH
  review_D1652

REVISION DETAIL
  https://reviews.freebsd.org/D1652

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1651: Add support for ENABLE/DISABLE_QUEUES VC messages

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1651

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1650: Add support for CONFIG_IRQ_MAP VC message

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1650

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1649: Add support for CONFIG_VSI_QUEUES VC message

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

Don't see any problems.

REVISION DETAIL
  https://reviews.freebsd.org/D1649

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Requested Changes To] D1648: Add stubs for deprecated VC messages

[jfvogel (Jack Vogel)]

jfvogel requested changes to this revision.
jfvogel added a comment.
This revision now requires changes to proceed.

Let's not mention Linux in our comments, either say nothing or think of 
something else :)

REVISION DETAIL
  https://reviews.freebsd.org/D1648

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1647: Add support for GET_VF_RESOURCES VC message

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

seems fine

REVISION DETAIL
  https://reviews.freebsd.org/D1647

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1646: Add support for RESET_VF VC message

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1646

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1645: Add support for VERSION VC message

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

REVISION DETAIL
  https://reviews.freebsd.org/D1645

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1644: Add infrastructure for handling the VC msg channel from VFs

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

Fine

REVISION DETAIL
  https://reviews.freebsd.org/D1644

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1641: Implement PCI SR-IOV initialization methods

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

Don't see any issues, of course the proof will come in the actual use when we
have the infrastructure :)

BRANCH
  review_D1641

REVISION DETAIL
  https://reviews.freebsd.org/D1641

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1642: Implement PCI SR-IOV method to initialize individual VFs

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
This revision is now accepted and ready to land.

BRANCH
  review_D1642

REVISION DETAIL
  https://reviews.freebsd.org/D1642

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1643: Implement resetting a VF

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

See no problems.

REVISION DETAIL
  https://reviews.freebsd.org/D1643

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] [Accepted] D1640: Refactor network stack state separate from VSI state

[jfvogel (Jack Vogel)]

jfvogel accepted this revision.
jfvogel added a comment.
This revision is now accepted and ready to land.

Thanks Ryan, am happier with this.

BRANCH
  review_D1640

REVISION DETAIL
  https://reviews.freebsd.org/D1640

To: rstone, jfvogel
Cc: freebsd-net
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Double cleanup in igb_attach

[Jack Vogel]

Yes, I will look them over.

Jack


On Tue, Jan 27, 2015 at 12:38 PM, Sreekanth Rupavatharam <
rupav...@juniper.net> wrote:

>  Thanks jack,
>  Now, can you please review these changes? And commit if you deem it
> fit?
>
> Thanks,
>
>  -Sreekanth
>
> On Jan 27, 2015, at 12:24 PM, "Jack Vogel"  wrote:
>
>   E, I am one of those people :)  (jack.vo...@intel.com)
>
>  Jack
>
>
> On Tue, Jan 27, 2015 at 12:21 PM, Sreekanth Rupavatharam <
> rupav...@juniper.net> wrote:
>
>>  Definitely, but I didn't have the contact info of those people.
>>
>> Thanks,
>>
>>  -Sreekanth
>>
>> On Jan 27, 2015, at 12:15 PM, "Jack Vogel"  wrote:
>>
>>   If you want something committed to an Intel driver, asking Intel might
>> be the
>> courteous thing to do, don't you think?
>>
>> Jack
>>
>>
>> On Tue, Jan 27, 2015 at 11:51 AM, Sreekanth Rupavatharam <
>> rupav...@juniper.net> wrote:
>>
>>>  Hiren,
>>> Can you help commit this?
>>>
>>> Index: if_igb.c
>>>
>>> ===
>>>
>>> --- if_igb.c (revision 298053)
>>>
>>> +++ if_igb.c (working copy)
>>>
>>> @@ -723,7 +723,8 @@ igb_attach(device_t dev)
>>>
>>>   return (0);
>>>
>>>
>>>
>>>  err_late:
>>>
>>> - igb_detach(dev);
>>>
>>> + if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>>>
>>> + return(error);
>>>
>>>   igb_free_transmit_structures(adapter);
>>>
>>>   igb_free_receive_structures(adapter);
>>>
>>>   igb_release_hw_control(adapter);
>>>
>>>  -- Thanks,
>>>
>>>  Sreekanth
>>>
>>>
>>>
>>>
>>>
>>>
>>>  On 1/27/15, 11:28 AM, "hiren panchasara" 
>>> wrote:
>>>
>>>  + Jack
>>> On Tue, Jan 27, 2015 at 12:00:19AM +, Sreekanth Rupavatharam wrote:
>>>
>>> Apologies if this is not the right forum. In igb_attach function, we
>>> have this code.
>>> err_late:
>>> igb_detach(dev);
>>>  igb_free_transmit_structures(adapter);
>>>  igb_free_receive_structures(adapter);
>>>  igb_release_hw_control(adapter);
>>> err_pci:
>>>  igb_free_pci_resources(adapter);
>>>  if (adapter->ifp != NULL)
>>>  if_free(adapter->ifp);
>>>  free(adapter->mta, M_DEVBUF);
>>>  IGB_CORE_LOCK_DESTROY(adapter);
>>>  The problem is that igb_detach also does the same cleanup in it?s
>>> body. Only exception is this case where it just returns EBUSY
>>>  /* Make sure VLANS are not using driver */
>>> if (if_vlantrunkinuse(ifp)) {
>>> device_printf(dev,"Vlan in use, detach first\n");
>>> return (EBUSY);
>>> }
>>>  I think the code in igb_attach should be changed to free up resources
>>> only if the igb_detach returns an error. Here?s the patch for it.
>>>  Index: if_igb.c
>>>  ===
>>>  --- if_igb.c (revision 298025)
>>>  +++ if_igb.c (working copy)
>>>  @@ -723,7 +723,8 @@ igb_attach(device_t dev)
>>> return (0);
>>> err_late:
>>>  - igb_detach(dev);
>>>  + if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>>>  + return;
>>> igb_free_transmit_structures(adapter);
>>>Can anyone comment on it and tell me if my understanding is
>>> incorrect?
>>>
>>>
>>>  Seems reasonable to me at the first glance.
>>>
>>>  We need to call IGB_CORE_LOCK_DESTROY(adapter) before returning though.
>>>
>>>  cheers,
>>> Hiren
>>>
>>>
>>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Double cleanup in igb_attach

[Jack Vogel]

E, I am one of those people :)  (jack.vo...@intel.com)

Jack


On Tue, Jan 27, 2015 at 12:21 PM, Sreekanth Rupavatharam <
rupav...@juniper.net> wrote:

>  Definitely, but I didn't have the contact info of those people.
>
> Thanks,
>
>  -Sreekanth
>
> On Jan 27, 2015, at 12:15 PM, "Jack Vogel"  wrote:
>
>   If you want something committed to an Intel driver, asking Intel might
> be the
> courteous thing to do, don't you think?
>
> Jack
>
>
> On Tue, Jan 27, 2015 at 11:51 AM, Sreekanth Rupavatharam <
> rupav...@juniper.net> wrote:
>
>>  Hiren,
>> Can you help commit this?
>>
>> Index: if_igb.c
>>
>> ===
>>
>> --- if_igb.c (revision 298053)
>>
>> +++ if_igb.c (working copy)
>>
>> @@ -723,7 +723,8 @@ igb_attach(device_t dev)
>>
>>   return (0);
>>
>>
>>
>>  err_late:
>>
>> - igb_detach(dev);
>>
>> + if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>>
>> + return(error);
>>
>>   igb_free_transmit_structures(adapter);
>>
>>   igb_free_receive_structures(adapter);
>>
>>   igb_release_hw_control(adapter);
>>
>>  -- Thanks,
>>
>>  Sreekanth
>>
>>
>>
>>
>>
>>
>>  On 1/27/15, 11:28 AM, "hiren panchasara" 
>> wrote:
>>
>>  + Jack
>> On Tue, Jan 27, 2015 at 12:00:19AM +, Sreekanth Rupavatharam wrote:
>>
>> Apologies if this is not the right forum. In igb_attach function, we have
>> this code.
>> err_late:
>> igb_detach(dev);
>>  igb_free_transmit_structures(adapter);
>>  igb_free_receive_structures(adapter);
>>  igb_release_hw_control(adapter);
>> err_pci:
>>  igb_free_pci_resources(adapter);
>>  if (adapter->ifp != NULL)
>>  if_free(adapter->ifp);
>>  free(adapter->mta, M_DEVBUF);
>>  IGB_CORE_LOCK_DESTROY(adapter);
>>  The problem is that igb_detach also does the same cleanup in it?s body.
>> Only exception is this case where it just returns EBUSY
>>  /* Make sure VLANS are not using driver */
>> if (if_vlantrunkinuse(ifp)) {
>> device_printf(dev,"Vlan in use, detach first\n");
>> return (EBUSY);
>> }
>>  I think the code in igb_attach should be changed to free up resources
>> only if the igb_detach returns an error. Here?s the patch for it.
>>  Index: if_igb.c
>>  ===
>>  --- if_igb.c (revision 298025)
>>  +++ if_igb.c (working copy)
>>  @@ -723,7 +723,8 @@ igb_attach(device_t dev)
>> return (0);
>> err_late:
>>  - igb_detach(dev);
>>  + if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>>  + return;
>> igb_free_transmit_structures(adapter);
>>Can anyone comment on it and tell me if my understanding is incorrect?
>>
>>
>>  Seems reasonable to me at the first glance.
>>
>>  We need to call IGB_CORE_LOCK_DESTROY(adapter) before returning though.
>>
>>  cheers,
>> Hiren
>>
>>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Double cleanup in igb_attach

[Jack Vogel]

If you want something committed to an Intel driver, asking Intel might be
the
courteous thing to do, don't you think?

Jack


On Tue, Jan 27, 2015 at 11:51 AM, Sreekanth Rupavatharam <
rupav...@juniper.net> wrote:

>  Hiren,
> Can you help commit this?
>
> Index: if_igb.c
>
> ===
>
> --- if_igb.c (revision 298053)
>
> +++ if_igb.c (working copy)
>
> @@ -723,7 +723,8 @@ igb_attach(device_t dev)
>
>   return (0);
>
>
>
>  err_late:
>
> - igb_detach(dev);
>
> + if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>
> + return(error);
>
>   igb_free_transmit_structures(adapter);
>
>   igb_free_receive_structures(adapter);
>
>   igb_release_hw_control(adapter);
>
>  -- Thanks,
>
>  Sreekanth
>
>
>
>
>
>
>  On 1/27/15, 11:28 AM, "hiren panchasara" 
> wrote:
>
>  + Jack
> On Tue, Jan 27, 2015 at 12:00:19AM +, Sreekanth Rupavatharam wrote:
>
> Apologies if this is not the right forum. In igb_attach function, we have
> this code.
> err_late:
> igb_detach(dev);
>  igb_free_transmit_structures(adapter);
>  igb_free_receive_structures(adapter);
>  igb_release_hw_control(adapter);
> err_pci:
>  igb_free_pci_resources(adapter);
>  if (adapter->ifp != NULL)
>  if_free(adapter->ifp);
>  free(adapter->mta, M_DEVBUF);
>  IGB_CORE_LOCK_DESTROY(adapter);
>  The problem is that igb_detach also does the same cleanup in it?s body.
> Only exception is this case where it just returns EBUSY
>  /* Make sure VLANS are not using driver */
> if (if_vlantrunkinuse(ifp)) {
> device_printf(dev,"Vlan in use, detach first\n");
> return (EBUSY);
> }
>  I think the code in igb_attach should be changed to free up resources
> only if the igb_detach returns an error. Here?s the patch for it.
>  Index: if_igb.c
>  ===
>  --- if_igb.c (revision 298025)
>  +++ if_igb.c (working copy)
>  @@ -723,7 +723,8 @@ igb_attach(device_t dev)
> return (0);
> err_late:
>  - igb_detach(dev);
>  + if(igb_detach(dev) == 0) /* igb_detach did the cleanup */
>  + return;
> igb_free_transmit_structures(adapter);
>Can anyone comment on it and tell me if my understanding is incorrect?
>
>
>  Seems reasonable to me at the first glance.
>
>  We need to call IGB_CORE_LOCK_DESTROY(adapter) before returning though.
>
>  cheers,
> Hiren
>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Adding new media types to if_media.h

[Jack Vogel]

I think I'd go along with Mike, keeping it simpler seems like a good idea.

Jack


On Fri, Dec 12, 2014 at 6:39 AM, Mike Karels  wrote:
>
> > Any other thoughts, or should I start looking at seeing what I can take
> > copy from net80211?
>
> > Also adding -net, since this is pretty relevant.
>
> I had the same reaction as Adrian initially, as an int with numerous fields
> seems really messy.  However, I don't think we have the challenges of
> 802.11,
> and the only real problem is that the subtype field has run out of bits.
> And both ifconfig and the drivers are cast in the form of a scalar "media
> word", with parameters to ifmedia_add like IFM_ETHER | IFM_1000_T | IFM_FDX
> (assuming that all the bits are in a scalar).
>
> Instead, I would propose that we simply change the media word from 32 bits
> to 64, and move the subtype from its current location to a new field (e.g.
> 16 bits) in the new space.  I believe this can be KPI compatible, and it
> is relatively easy to provide a backward-compatible ABI.  There should be
> a reserved subtype for "other" that can be encoded in the existing field
> for use when the subtype can't fit in the old field.  This seems much
> easier,
> can be KPI compatible, and will make it much easier to backport drivers.
> A backport could simply define the new subtypes as "other", and the KPI
> would still be compatible.
>
> Thoughts?
>
> Mike
>
> > On Tue, Dec 9, 2014 at 10:40 AM, Adrian Chadd 
> wrote:
> > >
> > > On 9 December 2014 at 07:27, Rui Paulo  wrote:
> > > > On Dec 9, 2014, at 01:05, Eric Joyner  wrote:
> > > >>
> > > >> This is a continuation of a discussion from off the list:
> > > >>
> > > >> ixgbe needs to support devices with media types that aren't in
> > > if_media.h;
> > > >> for now those are 10GBaseKR, 10GBaseKX4, and 1000baseKX.
> Immediately,
> > > we're
> > > >> running into the issue that there is no room for all of these types
> > > under
> > > >> the IFM_ETHER category; there's only room for two more (and per John
> > > >> Baldwin, only one more if one of those two unused types is to be
> used
> > > for a
> > > >> reserved type). Long term, there are going to be tons of media
> types for
> > > >> future ethernet speeds like 25G, 50G, 100G, and etc, and ixl already
> > > >> supports media types that aren't in if_media.h, either.
> > > >>
> > > >> So, something needs to change. Does anyone have any thoughts on what
> > > should
> > > >> happen? I've thought of a few things, but I don't have an adequate
> > > grasp of
> > > >> what the pros/cons of each are:
> > > >>
> > > >> 1. Add a new media category (like IFM_ETHER) and change kernel code
> to
> > > >> treat it like the existing IFM_ETHER. This creates ~28 new media
> types
> > > we
> > > >> can use, but it may just be delaying the inevitable for a short
> time.
> > > >>
> > > >> 2. Extend media value from 32-bits to 64-bits. I don't have a good
> idea
> > > of
> > > >> what the consequences are of this on architectures that aren't
> amd64.
> > > >>
> > > >> 3. (Initially suggested by Adrian) would be to create a new media
> type
> > > >> struct (instead of using an int value) or adding an extra value to
> the
> > > >> existing ifmedia/ifmedia_entry struct for this. This sounds like the
> > > best
> > > >> solution to me, but I don't know how much effort it would take to
> > > implement
> > > >> -- does ifconfig need a lot of changing to handle this?
> > > >
> > > > ifconfig is a macro-intensive application, so maybe it's not that
> much
> > > work.
> > > >
> > > >> Thoughts? Any previous discussions worth looking at?
> > > >
> > > > Hmm, it looks like you're limited in the number of bits because of
> how
> > > the word was laid out. We can't simple remove token ring and get more
> bits
> > > for ethernet...  We could create another IFM_40GETHERNET type to
> replace
> > > token ring but that would be ugly (the IFM_TYPE() macro could handle
> this
> > > idiosyncrasy).
> > > >
> > > > I think if_media should probably be a structure with unions to store
> the
> > > subtypes.  net80211 has the same problem with MCS rates and we ended up
> > > storing them outside if_media because of this.
> > >
> > > I think solving this like how it's done in net80211 (ie, with an
> > > external structure that represents the media type details for a given
> > > media) is the right thing to do.
> > >
> > > Otherwise it'll be a path to madness in the future.
> > >
> > > The net80211 side of things is mostly extensible and I'm going to
> > > (eventually) end up using it for the 11ac rates that have shown up. I
> > > couldn't do that whilst trying to cram it into the existing ifmedia
> > > variable.
> > >
> > >
> > > -adrian
> > >
> > ___
> > freebsd-a...@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-arch
> > To unsubscribe, send any mail to "freebsd-arch-unsubscr...@freebsd.org"
>
___
f

Re: problem with 9k jumbo clusters

[Jack Vogel]

I had wanted to remove the larger cluster sizes from the driver a while
back, but for reasons
I don't remember that code change didn't happen. The new 40G ixl driver
does this, it only
uses standard clusters for anything under 2K, and above that everything
uses 4K.

I would be curious to see if this change would resolve your problem, would
you like a patch,
or are you able to hack the code yourself to do this?

Jack


On Thu, Dec 4, 2014 at 11:00 AM, Alexander V. Chernikov <
melif...@freebsd.org> wrote:

> On 04.12.2014 13:50, Yuriy Tabolin wrote:
>
>> Hi All.
>> I have a server with two Intel 10G NIC. OS FreeBSD 10.1-Release amd64.
>> Server works like NFS, samba-server and iSCSI target. Both NICs aggregated
>> into lagg device and set MTU 9014 to them. There are some tuning
>> sysctl.conf:
>> kern.maxfiles=6289601
>> kern.maxfilesperproc=5660640
>> kern.maxvnodes=3339565
>> kern.ipc.nmbclusters=12255588
>> kern.ipc.nmbjumbop=6127794
>> kern.ipc.nmbufs=78435780
>> kern.ipc.maxsockbuf=16777216
>> kern.ipc.maxsockets=6289600
>> net.inet.tcp.sendbuf_max=16777216
>> net.inet.tcp.sendspace=65536
>> net.inet.tcp.recvbuf_max=16777216
>> net.inet.tcp.recvspace=65536
>>
>> After some days of working, the errors are appearing:
>> ix1: Interface stopped DISTRIBUTING, possible flapping
>> ix0: Interface stopped DISTRIBUTING, possible flapping
>> ix0: Could not setup receive structures
>> ix1: Could not setup receive structures
>>
> Hello. It looks like
> https://lists.freebsd.org/pipermail/freebsd-net/2014-May/038630.html
> is relevant here.
>
>
>> After that errors the NICs stoped working. netstat -m shows:
>> 32881/33854/66735 mbufs in use (current/cache/total)
>> 16370/8198/24568/12255588 mbuf clusters in use (current/cache/total/max)
>> 16370/4807 mbuf+clusters out of packet secondary zone in use
>> (current/cache)
>> 0/873/873/6127794 4k (page size) jumbo clusters in use
>> (current/cache/total/max)
>> 16383/21517/37900/1815641 9k jumbo clusters in use
>> (current/cache/total/max)
>> 0/0/0/1021298 16k jumbo clusters in use (current/cache/total/max)
>> 188407K/222004K/410411K bytes allocated to network (current/cache/total)
>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
>> 0/101414306/0 requests for jumbo clusters denied (4k/9k/16k)
>> 0 requests for sfbufs denied
>> 0 requests for sfbufs delayed
>> 0 requests for I/O initiated by sendfile
>>
>> 9k jumbo clusters max is too big, but looks like system cannot allocate
>> them. There are huge number of "9k requests for jumbo clusters denied".
>> ifconfig ix down/up don't helped, reboot is needed. Thanks for any help!
>>
>>
>> --
>> Best regards,
>> Tabolin Yuriy
>> System administrator
>> Speech Technology Center
>> ___
>> freebsd-net@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>>
>>
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Problems with Intel X520-SR2

[Jack Vogel]

Good suggestions, do you ever see any 'interrupt throttled' messages? You
might
want to change the storm threshold to 0 and disable it.

I too would like to know if there are any messages when the 'hang' happens.

I don't know much about lagg, is it responsible for link events?? Its not a
normal
situation to be having so many :(

Jack


On Sun, Nov 30, 2014 at 10:01 AM, Adrian Chadd  wrote:

> Your link_irq value is way too high. I think you're exposing some
> unhandled corner case in the driver (as I had this issue when I had
> some badly cabled up ixgbe NICs) but it doesn't happen when the ixgbe
> driver isn't getting link events.
>
> My test setup at home has link_irq=1 on both sides, and it runs
> full-duplex 10GE (1 mil pps transmit/receive on each NIC) traffic for
> RSS testing for weeks at a time. I have no issues and no hiccups.
> So, I think there's two problems:
>
> * you're still seeing way too many link_irq events; and
> * i think there's some bad handling when it comes to link_irq events.
>
> I wonder if we're still clearing some of the interrupt register bits
> incorrectly in ixgbe_msix_link().
>
>
> -adrian
>
>
>
>
> -adrian
>
>
> On 30 November 2014 at 08:05, Alexander V. Chernikov
>  wrote:
> > On 30.11.2014 18:47, Marcelo Gondim wrote:
> >>
> >> Dear,
> >>
> >> Unfortunately I have more options to resolve this problem I'm having
> with
> >> Intel X520-SR2. Have we changed the X520, we exchange the optical cords,
> >> exchanged optical modules, we changed the entire server, we reduce the
> >> temperature inside the equipment, made some attempts to tunning the site
> >> calomel[1]. We spend a lot of money and do not solve the problem.
> >>
> >> What happens is that when there is a traffic above 1.2Gbps with PPS
> above
> >> 700kpps in sometimes almost daily there is a lock in two 10GbE ports the
> >> X520-SR interface. Where I am obliged to leave a script running in the
> >> background doing just that:
> >
> > What does this "lock" looks like?
> > Do you using jumbo frames?
> > Is this IPv4 or IPv4+IPv6 ?
> > Can you share "netstat -m" output?
> > Do you use ipfw dynamic states?
> > Are sure you're not hitting "net.inet.ip.fw.dyn_max=65536" ?
> >
> > dev.ix.0.queue0.no_desc_avail: 3322269
> > dev.ix.0.queue1.no_desc_avail: 5254761
> >
> > Looks suspicious. Either you're running out of mbufs due to total mbuf
> > number is small, or system is very busy sometimes.
> > What does you "top -HPSzs1" output look like?
> >
> >
> >>
> >> ifconfig ix0 down; ifconfig ix0 up; ifconfig ix1 down; ifconfig ix1 up
> >>
> >> Made it back to the interface function normally. It's already so for
> >> months and have not tried the latest driver from Intel because I do not
> see
> >> anything related to this issue.
> >>
> >> These 2-port 10GbE are my backbone linking the four cities that attend
> to
> >> our main router. One is backup to other but when the problem occurs,
> the two
> >> ports stop working and at the moment I have a break in my Internet
> >>
> >> I can only conclude that the problem is one of the things below:
> >>
> >> 1 - Intel Interface X520-SR2 has a problem with certain types of traffic
> >> and then hangs.
> >> 2 - The ixgbe driver has a bug that is causing it.
> >> 3 - Problem with FreeBSD 10.x. Not tested with FreeBSD 9.3 because it
> >> would be a regression and the equipment is very far away from me if I
> need
> >> to move me.
> >>
> >> Honestly I'm almost going on a Juniper closed solution. I would not want
> >> to do this because I love FreeBSD and I can not believe that he does not
> >> support a 2.7Gbps traffic, which is my peak traffic without getting
> having
> >> these falls. My hardware today is this:
> >>
> >> hw.machine: amd64
> >> hw.model: Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
> >> hw.ncpu: 12
> >> hw.byteorder: 1234
> >> hw.physmem: 17083641856
> >> hw.usermem: 15741001728
> >>
> >> Hardware all Intel with motherboard S2600COE [2] and with network
> >> interfaces offboard:
> >>
> >> 1x - X520-SR2 [3]
> >> 2x - I350-T2 [4]
> >>
> >> My loader.conf:
> >>
> >> loader_logo="beastie"
> >> if_lagg_load="YES"
> >> speaker_load="YES"
> >> aio_load="YES"
> >> autoboot_delay="5"
> >> net.fibs=1
> >>
> >> My sysctl.conf:
> >>
> >> net.inet.ip.forwarding=1
> >> net.inet.ip.fastforwarding=1
> >> net.inet6.ip6.forwarding=1
> >> kern.ipc.somaxconn=4096
> >> net.inet.tcp.syncookies=1
> >> net.inet.ip.redirect=1
> >> net.inet.ip.accept_sourceroute=0
> >> net.inet.ip.sourceroute=0
> >> net.inet.tcp.drop_synfin=1
> >> net.inet.udp.blackhole=1
> >> net.inet.tcp.blackhole=2
> >> security.bsd.see_other_uids=0
> >> net.inet.ip.fw.dyn_buckets=65536
> >> net.inet.ip.fw.dyn_max=65536
> >> hw.intr_storm_threshold=9000
> >> net.inet.ip.dummynet.pipe_slot_limit=800
> >> net.inet.icmp.icmplim=2000
> >>
> >> # sysctl dev.ix.
> >> dev.ix.%parent:
> >> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version -
> >> 2.5.15
> >> dev.ix.0.%driver: ix
> >> dev.ix.0.%location: slot=0 function

Re: Data Center Bridging?

[Jack Vogel]

No, I'm sorry but I've been kept very busy in development of our 40G driver,
however, now that its complete we have been taking on a variety of other
tasks,
so this is one that I can have the team look into.

Jack


On Mon, Nov 24, 2014 at 5:56 AM, Juan Mojica  wrote:

> Was there any progress made on DCB?
>
> We need a way to classify specific flows from the application and mark
> them with a priority.  Not only do we need a bit of work on the driver, but
> also a way to tag frames all the way down to the driver.  I imagine this
> would entail a socket option to mark a given flow and a mbuf flag to carry
> the request for priority classification to the driver.
>
> -Juan
>
> On Wed, Feb 6, 2013 at 5:53 AM, Randall Stewart  wrote:
>
>> Lars/Jack:
>>
>> I am pretty sure that my company would be interested in it as well.. and
>> I can
>> help out here too ;-)
>>
>> Jack: are there particular versions of Intel cards that this needs to
>>   be on (we have both igb and ix cards in my office now).. thanks.
>>
>> R
>>
>>
>> On Jan 22, 2013, at 10:43 AM, Eggert, Lars wrote:
>>
>> > Hi,
>> >
>> > on Linux, various NICs (e.g., ixgbe) support Data Center Bridging. Is
>> this also available under FreeBSD? Do *any* NICs support DCB under FreeBSD?
>> >
>> > Thanks,
>> > Lars
>> > ___
>> > freebsd-net@freebsd.org mailing list
>> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>> >
>>
>> --
>> Randall Stewart
>> 803-317-4952 (cell)
>>
>> ___
>> freebsd-net@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>>
>
>
>
> --
> Juan Mojica
> Email: jmoj...@gmail.com
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: igb Could not setup receive structures (again)

[Jack Vogel]

Yes, its strange, the mbuf resources look fine.

Can you show the dmesg record from a boot that
includes the failure please?

Jack


On Fri, Nov 21, 2014 at 9:33 AM, Gerrit Kühn 
wrote:

> On Fri, 21 Nov 2014 08:22:31 -0800 Jack Vogel  wrote
> about Re: igb Could not setup receive structures (again):
>
> JV> After you get the error do a `netstat -m` and see what the state of the
> JV> 9K jumbo pool is, for that is the size you would be using.
>
> Hm...
>
> ---
> root@mclane:~ # netstat -m
> 20472/33783/54255 mbufs in use (current/cache/total)
> 20461/31381/51842/1014856 mbuf clusters in use (current/cache/total/max)
> 20461/29764 mbuf+clusters out of packet secondary zone in use
> (current/cache)
> 0/191/191/507428 4k (page size) jumbo clusters in use
> (current/cache/total/max)
> 0/3425/3425/150349 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/84571 16k jumbo clusters in use (current/cache/total/max)
> 46040K/102796K/148836K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> 0/10/0 requests for jumbo clusters denied (4k/9k/16k)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
>
> root@mclane:~ # ifconfig igb1 mtu 9000
>
> root@mclane:~ # netstat -m
> 16373/37882/54255 mbufs in use (current/cache/total)
> 16369/35473/51842/1014856 mbuf clusters in use (current/cache/total/max)
> 16369/33856 mbuf+clusters out of packet secondary zone in use
> (current/cache)
> 0/191/191/507428 4k (page size) jumbo clusters in use
> (current/cache/total/max)
> 0/3425/3425/150349 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/84571 16k jumbo clusters in use (current/cache/total/max)
> 36831K/112005K/148836K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> 0/12/0 requests for jumbo clusters denied (4k/9k/16k)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
>
> ---
>
>
> JV> Depending on the specific device type you may have up to 8 rings
> JV> per interface, and with a ring size of 1K...
> JV> It does seem like you should have enough, but maybe something else
> JV> in your system is using the pool? As I said, look at netstat, it
> JV> should give you the truth, and then adjust the allocated size to fit
> JV> your needs.
>
> I must admit that I cannot make much of this output... looks all fine to
> me?
>
>
> cu
>   Gerrit
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: igb Could not setup receive structures (again)

[Jack Vogel]

The driver does not calculate what it needs, it just keeps taking :) Hmmm,
well it would
be difficult to calculate a real total, as each instance of the driver (one
per port) has no
idea about other instances. I suppose there could be a display of the total
needed per
port?

Jack


On Fri, Nov 21, 2014 at 12:34 PM, David DeSimone 
wrote:

> Would it be possible for the driver to report how many clusters it
> calculated that it needs, whenever it runs into this memory shortage during
> attach?  That way an administrator might have some idea how much to
> increase their tunables in order to meet the driver's requirements.
>
> As it is, it's merely a guessing game or requires digging into driver
> source to come up with the right numbers.
>
>
> -Original Message-
> From: owner-freebsd-...@freebsd.org [mailto:owner-freebsd-...@freebsd.org]
> On Behalf Of Jack Vogel
> Sent: Friday, November 21, 2014 10:23 AM
> To: Gerrit Kühn
> Cc: FreeBSD Net
> Subject: Re: igb Could not setup receive structures (again)
>
> The message is pretty straightforward, its only cause is the inability
> of the driver to get the necessary clusters to populate the RX rings.
>
> After you get the error do a `netstat -m` and see what the state of the
> 9K jumbo pool is, for that is the size you would be using.
>
> Depending on the specific device type you may have up to 8 rings
> per interface, and with a ring size of 1K...
>
> It does seem like you should have enough, but maybe something else
> in your system is using the pool? As I said, look at netstat, it should
> give
> you the truth, and then adjust the allocated size to fit your needs.
>
> Jack
>
>
> On Fri, Nov 21, 2014 at 2:48 AM, Gerrit Kühn 
> wrote:
>
> > Hi all,
> >
> > I get the error message above when trying to go for jumbo frames (mtu
> > 9000) on a system here. I found a lot of comments on this, but they all
> > state that this is due to a too low setting for mbufs in FreeBSD8/9
> > settings.
> > However, I have 10-stable here, and the settings look rather high to me:
> >
> > kern.ipc.nmbclusters: 1014856
> > kern.ipc.maxmbufmem: 8313700352
> > kern.ipc.nmbufs: 6495090
> > kern.ipc.nmbjumbop: 507428
> > kern.ipc.nmbjumbo9: 451047
> > kern.ipc.nmbjumbo16: 338284
> >
> > FreeBSD mclane 10.0-STABLE FreeBSD 10.0-STABLE #5 r261710: Mon Feb 10
> > 16:55:29 CET 2014 r...@mclane.rt.aei.uni-hannover.de:
> /usr/obj/usr/src/sys/GENERIC
> > amd64
> >
> >
> > Is this still too low? I have a 6core machine with 6 igb interfaces, and
> I
> > cannot even get one of these set to mtu 9000 without getting "Could not
> > setup receive structures".
> >
> >
> > cu
> >   Gerrit
> > ___
> > freebsd-net@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
> >
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
> This email message is intended for the use of the person to whom it has
> been sent, and may contain information that is confidential or legally
> protected. If you are not the intended recipient or have received this
> message in error, you are not authorized to copy, distribute, or otherwise
> use this message or its attachments. Please notify the sender immediately
> by return e-mail and permanently delete this message and any attachments.
> Verio Inc. makes no warranty that this email is error or virus free. Thank
> you.
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: igb Could not setup receive structures (again)

[Jack Vogel]

The message is pretty straightforward, its only cause is the inability
of the driver to get the necessary clusters to populate the RX rings.

After you get the error do a `netstat -m` and see what the state of the
9K jumbo pool is, for that is the size you would be using.

Depending on the specific device type you may have up to 8 rings
per interface, and with a ring size of 1K...

It does seem like you should have enough, but maybe something else
in your system is using the pool? As I said, look at netstat, it should give
you the truth, and then adjust the allocated size to fit your needs.

Jack


On Fri, Nov 21, 2014 at 2:48 AM, Gerrit Kühn 
wrote:

> Hi all,
>
> I get the error message above when trying to go for jumbo frames (mtu
> 9000) on a system here. I found a lot of comments on this, but they all
> state that this is due to a too low setting for mbufs in FreeBSD8/9
> settings.
> However, I have 10-stable here, and the settings look rather high to me:
>
> kern.ipc.nmbclusters: 1014856
> kern.ipc.maxmbufmem: 8313700352
> kern.ipc.nmbufs: 6495090
> kern.ipc.nmbjumbop: 507428
> kern.ipc.nmbjumbo9: 451047
> kern.ipc.nmbjumbo16: 338284
>
> FreeBSD mclane 10.0-STABLE FreeBSD 10.0-STABLE #5 r261710: Mon Feb 10
> 16:55:29 CET 2014 
> r...@mclane.rt.aei.uni-hannover.de:/usr/obj/usr/src/sys/GENERIC
> amd64
>
>
> Is this still too low? I have a 6core machine with 6 igb interfaces, and I
> cannot even get one of these set to mtu 9000 without getting "Could not
> setup receive structures".
>
>
> cu
>   Gerrit
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: [Bug 193053] ixgbe(4) IXGBE_LEGACY_TX + ALTQ path broken

[Jack Vogel]

LEGACY is not there just because of ALTQ, but rather because there
have been significant customers of Intel that use the driver in OS versions
older than 8 that do not have the required interface.

Jack... going back to my sabbatical now :)



On Thu, Oct 2, 2014 at 12:22 AM, Ermal Luçi  wrote:

> In pfSense the driver has been modified to compile a hybrid mode.
> Meaning have activated both LEGACY and new transmit queue model.
>
> It works correctly and avoids the problems of recompiling with ALTQ.
> It also solves the  problem on having performance impacts when ALTQ is not
> in use.
>
> There are even some drivers in the tree that do this by default already.
>
> From my analysis there should not be an LEGACY define but just mask all of
> this behind ALTQ definition if needed.
>
> On Wed, Oct 1, 2014 at 8:28 PM,  wrote:
>
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193053
> >
> > --- Comment #9 from ncrog...@gmail.com ---
> > Any chance we can get some kind of resolution on this before 10.1? It
> seems
> > like simply upgrading the ixgbe driver with the latest from Intel fixes
> the
> > problem. If that is not possible, the changes in the patch I posted are
> > still
> > working for me in production
> >
> > --
> > You are receiving this mail because:
> > You are the assignee for the bug.
> > ___
> > freebsd-net@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
> >
>
>
>
> --
> Ermal
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: ixgbe i2c interface

[Jack Vogel]

Thanks Eric !  I'll commit it tomorrow since Eric approves :)

Jack



On Sat, Aug 16, 2014 at 3:27 PM, Eric Joyner  wrote:

> I unofficially approve of it!
>
> ---
> Eric Joyner
> On Aug 16, 2014 12:21 PM, "Alexander V. Chernikov" <
> melif...@yandex-team.ru> wrote:
>
>> Hello Jack!
>>
>> Can you please commit (or let me commit) the following one-liner?
>>
>> ___
>> freebsd-net@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Question on rx queue in ixgbe driver

[Jack Vogel]

HEAD and TAIL are actually hw registers, the driver as it is configured
these
days never (and cannot) modify HEAD, There is an option to use the HEAD
register as a method of managing the RX side (called Head Writeback), but
in ixgbe this is not used, rather we rely on the DD bit of a descriptor to
be
written back by the hardware to indicate an operation is complete.

The span between HEAD and TAIL is the extent of usable buffers for the hw.
In the starting state having them equal indicates a clean slate, however
once
operating TAIL will trail HEAD, the range between them being "uncleaned"

The next_to_check and next_to_refresh indices are used by the driver to
manage
the ring. I designed them to allow a seperated operation of cleaning and of
refreshing
the buffers. In old drivers these two were always in lock step, and you
could never
send a packet to the stack until you FIRST did the mbuf allocation, then if
that
failed for any reason the packet was actually dropped in order to keep that
precious
mbuf cluster.  Now refresh can be deferred as many as 8 while cleaning.

These two indices are each exclusively controlled by those two operations,
and its
only refresh, by writing the TAIL register, that actually limits what the
hw engine
can do, since it will not go beyond TAIL.

There is no "should be" on those indices, they are set up so things operate
correctly, and they do :)

Hope this clarifies things somewhat?

Jack




On Wed, Jul 23, 2014 at 5:37 AM, Xu Zhe  wrote:

> Hi,
>
> I am reading ixgbe driver of Freebsd and got some problems, which are
> described below.
>
> (1) Why rxd_tail does not equals to rxd_head?
>
> Here, rxd_tail/rxd_head is the value of
> dev.ix.0.queue0.rxd_tail/dev.ix.0.queue0.rxd_head from sysctl (take the
> first
> queue of ix0 as example).
>
> Actually, in most cases, rxd_head - rxd_tail == 1.
>
> Refers to the code, these values are actually next_to_refresh/next_to_check
> for each receive queue (though the sysctl implementation is read directly
> from
> the hardware registers I suppose). Why next_to_refresh is always one
> smaller
> than next_to_check (of course, when the latter is 0, the former is 2047
> possibly, which is the size of rx ring - 1)? In my point of view, this
> means
> that we will always have one tiny mbuf (which is pointed by
> next_to_refresh)
> that is already checked (passed up to upper network stack) but not
> refreshed
> (not prepared for the next receive). It does not make sense? Or I missed
> anything important?
>
> (2) The init value of rxd_tail
>
> Even if (1) has no problem, when ixgbe device is inited, rxd_tail (or say,
> next_to_refresh) is set to zero (in function ixgbe_setup_receive_ring). I
> think it should be rxr->num_desc - 1.
>
> This should not matter much in the latest ixgbe driver, but it might cause
> old
> driver (ixgbe 2.5.8 at least) to double init the rx_ring descriptors (both
> in
> ixgbe_setup_receive_ring when ixgbe init up, and the first entry of
> ixgbe_refresh_mbufs of the first interrupt come). This is a case I met in
> my
> test environment.
>
> ==
>
> Looking forward to any of your replies to help clarify my thoughts. Thanks
> in
> advance.
>
> Peter
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing

[Jack Vogel]

I had missed the fact that Alex turned this off in the Linux driver, sounds
to me like its the right thing to do for FreeBSD also.

Jack



On Mon, Jul 14, 2014 at 5:44 PM, Adrian Chadd  wrote:

> Hi,
>
> Whilst digging into UDP receive side scaling on the intel ixgbe(4)
> NIC, I stumbled across how it hashes traffic between IP fragmented
> traffic and non IP-fragmented traffic.
>
> Here's how it surfaced:
>
> * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and
> TCP/UDP (4-tuple);
> * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple
> and comes into queue A;
> * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple
> and comes into queue B.
>
> So if there's a mix of small and large datagrams, we'll end up with
> some packets coming in via queue A and some by queue B. In normal
> operation that'll result in out of order packets.
>
> For the RSS stuff I'm working on it means that some packets will match
> the PCBGROUP setup and some won't. By default UDP configures a 2-tuple
> hash so it expects packets to come in hashed appropriately. But that
> only matches for large frames. For small frames it'll be hashed via
> the 4-tuple and it won't match.
>
> The ip reassembly code doesn't recalculate the flowid/flowtype once
> it's finished. It'd be nice to do that before further processing so it
> can be placed in the right netisr.
>
> So there's a couple of semi-overlapping issues:
>
> * Right now we could get TCP and UDP frames out of order. I'd like to
> at least have ixgbe(4) hash on the 2-tuple for UDP rather than the
> 4-tuple. That fixes that silly corner case. It's not likely going to
> show up except for things like forwarding workloads. Maybe people
> doing memcached work, I'm not sure.
>
> * Whether or not to calculate the flowid/flowtype in ip_reass() (or
> maybe in the netisr input path, in case there's no flowid assigned) so
> work is better distributed;
>
> * .. then if we do that, we could do 4-tuple UDP hashing again and
> we'd just recalculate for any large frames.
>
> Here's what happened with Linux and ixgbe in 2010 on this topic:
>
> http://comments.gmane.org/gmane.linux.network/166687
>
> What do people think?
>
>
> -a
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Network Intel X520-SR2 stopping

[Jack Vogel]

What does a netstat -m show, I noticed you show no_desc counts on
all your queues, perhaps you don't have enough mbufs/clusters available?
Does your message log show any events or messages of significance?

I'm not sure about the module compatibility, Jeff would be better positioned
to answer that.

Jack



On Fri, Jul 4, 2014 at 12:38 AM, Marcelo Gondim 
wrote:

> Hi all,
>
> Both modules are 850nm. Could there be some incompatibility between the
> Datacom XFP optical module and the SFP + Intel X520-SR2 optical module?
> I ask this because I replaced all the hardware and optical cords and
> nothing worked. Datacom is a DM4100.
>
> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version -
> 2.5.15
> dev.ix.0.%driver: ix
> dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0
> dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086
> subdevice=0x7a11 class=0x02
> dev.ix.0.%parent: pci131
> dev.ix.0.fc: 3
> dev.ix.0.enable_aim: 1
> dev.ix.0.advertise_speed: 0
> dev.ix.0.dropped: 0
> dev.ix.0.mbuf_defrag_failed: 0
> dev.ix.0.watchdog_events: 0
> dev.ix.0.link_irq: 61846
> dev.ix.0.queue0.interrupt_rate: 8
> dev.ix.0.queue0.irqs: 2240081588
> dev.ix.0.queue0.txd_head: 1687
> dev.ix.0.queue0.txd_tail: 1687
> dev.ix.0.queue0.tso_tx: 5
> dev.ix.0.queue0.no_tx_dma_setup: 0
> dev.ix.0.queue0.no_desc_avail: 2171
> dev.ix.0.queue0.tx_packets: 4721544662
> dev.ix.0.queue0.rxd_head: 223
> dev.ix.0.queue0.rxd_tail: 222
> dev.ix.0.queue0.rx_packets: 3939926170
> dev.ix.0.queue0.rx_bytes: 708714319872
> dev.ix.0.queue0.rx_copies: 1428958853
> dev.ix.0.queue0.lro_queued: 0
> dev.ix.0.queue0.lro_flushed: 0
> dev.ix.0.queue1.interrupt_rate: 10
> dev.ix.0.queue1.irqs: 2164304165
> dev.ix.0.queue1.txd_head: 24
> dev.ix.0.queue1.txd_tail: 24
> dev.ix.0.queue1.tso_tx: 0
> dev.ix.0.queue1.no_tx_dma_setup: 0
> dev.ix.0.queue1.no_desc_avail: 241832
> dev.ix.0.queue1.tx_packets: 4995476933
> dev.ix.0.queue1.rxd_head: 1161
> dev.ix.0.queue1.rxd_tail: 1160
> dev.ix.0.queue1.rx_packets: 3901327164
> dev.ix.0.queue1.rx_bytes: 691440627148
> dev.ix.0.queue1.rx_copies: 1408409383
> dev.ix.0.queue1.lro_queued: 0
> dev.ix.0.queue1.lro_flushed: 0
> dev.ix.0.queue2.interrupt_rate: 8
> dev.ix.0.queue2.irqs: 2167993136
> dev.ix.0.queue2.txd_head: 1329
> dev.ix.0.queue2.txd_tail: 1329
> dev.ix.0.queue2.tso_tx: 0
> dev.ix.0.queue2.no_tx_dma_setup: 0
> dev.ix.0.queue2.no_desc_avail: 190120
> dev.ix.0.queue2.tx_packets: 5013202508
> dev.ix.0.queue2.rxd_head: 2039
> dev.ix.0.queue2.rxd_tail: 2038
> dev.ix.0.queue2.rx_packets: 3955460159
> dev.ix.0.queue2.rx_bytes: 743382421188
> dev.ix.0.queue2.rx_copies: 1422295822
> dev.ix.0.queue2.lro_queued: 0
> dev.ix.0.queue2.lro_flushed: 0
> dev.ix.0.queue3.interrupt_rate: 71428
> dev.ix.0.queue3.irqs: 2139498119
> dev.ix.0.queue3.txd_head: 673
> dev.ix.0.queue3.txd_tail: 673
> dev.ix.0.queue3.tso_tx: 0
> dev.ix.0.queue3.no_tx_dma_setup: 0
> dev.ix.0.queue3.no_desc_avail: 94226
> dev.ix.0.queue3.tx_packets: 5301360114
> dev.ix.0.queue3.rxd_head: 416
> dev.ix.0.queue3.rxd_tail: 415
> dev.ix.0.queue3.rx_packets: 3951345010
> dev.ix.0.queue3.rx_bytes: 723655881546
> dev.ix.0.queue3.rx_copies: 1424750061
> dev.ix.0.queue3.lro_queued: 0
> dev.ix.0.queue3.lro_flushed: 0
> dev.ix.0.queue4.interrupt_rate: 10
> dev.ix.0.queue4.irqs: 2027199532
> dev.ix.0.queue4.txd_head: 764
> dev.ix.0.queue4.txd_tail: 764
> dev.ix.0.queue4.tso_tx: 0
> dev.ix.0.queue4.no_tx_dma_setup: 0
> dev.ix.0.queue4.no_desc_avail: 174621
> dev.ix.0.queue4.tx_packets: 5250099331
> dev.ix.0.queue4.rxd_head: 780
> dev.ix.0.queue4.rxd_tail: 779
> dev.ix.0.queue4.rx_packets: 3898505370
> dev.ix.0.queue4.rx_bytes: 680413268286
> dev.ix.0.queue4.rx_copies: 1415917068
> dev.ix.0.queue4.lro_queued: 0
> dev.ix.0.queue4.lro_flushed: 0
> dev.ix.0.queue5.interrupt_rate: 62500
> dev.ix.0.queue5.irqs: 2076140170
> dev.ix.0.queue5.txd_head: 553
> dev.ix.0.queue5.txd_tail: 553
> dev.ix.0.queue5.tso_tx: 23
> dev.ix.0.queue5.no_tx_dma_setup: 0
> dev.ix.0.queue5.no_desc_avail: 178058
> dev.ix.0.queue5.tx_packets: 5266432651
> dev.ix.0.queue5.rxd_head: 1870
> dev.ix.0.queue5.rxd_tail: 1869
> dev.ix.0.queue5.rx_packets: 3876348699
> dev.ix.0.queue5.rx_bytes: 684015705094
> dev.ix.0.queue5.rx_copies: 1433886240
> dev.ix.0.queue5.lro_queued: 0
> dev.ix.0.queue5.lro_flushed: 0
> dev.ix.0.queue6.interrupt_rate: 10
> dev.ix.0.queue6.irqs: 2106459361
> dev.ix.0.queue6.txd_head: 181
> dev.ix.0.queue6.txd_tail: 181
> dev.ix.0.queue6.tso_tx: 0
> dev.ix.0.queue6.no_tx_dma_setup: 0
> dev.ix.0.queue6.no_desc_avail: 2112
> dev.ix.0.queue6.tx_packets: 4786334585
> dev.ix.0.queue6.rxd_head: 1165
> dev.ix.0.queue6.rxd_tail: 1164
> dev.ix.0.queue6.rx_packets: 3963283094
> dev.ix.0.queue6.rx_bytes: 707926383609
> dev.ix.0.queue6.rx_copies: 1443393267
> dev.ix.0.queue6.lro_queued: 0
> dev.ix.0.queue6.lro_flushed: 0
> dev.ix.0.queue7.interrupt_rate: 62500
> dev.ix.0.queue7.irqs: 2046097590
> dev.ix.0.queue7.txd_head: 1928
> dev.

Re: Network Intel X520-SR2 stopping

[Jack Vogel]

Is only one port a problem? When it gets into the state can you
do a sysctl dev.ix.X..

Jack



On Tue, Jul 1, 2014 at 6:48 PM, Marcelo Gondim 
wrote:

> Hi all,
>
> I'm having problems with a 10GbE Intel X520-SR2 interface. After a running
> time, the interface does not send or receive more data. I am obliged to
> make a down and up the interface for it to return to work. Have changed the
> interface, optical cords, optical modules and problem continues.
>
> (root@rt01)[~]# netstat -inh
> NameMtu Network   Address  Ipkts Ierrs Idrop Opkts
> Oerrs  Coll
> igb0   1.5K   00:1e:67:9b:03:e10 0 00
> 0 0
> igb1   1.5K   00:1e:67:9b:03:e2  19M 0 0  27M
> 0 0
> igb1  - 177.xx.xxx.0/ 177.xx.xxx.1   15K - -  14K
> - -
> igb1  - fe80::21e:67f fe80::21e:67ff:fe2 - -4
> - -
> igb1  - 2804::cad 2804::cade:b1  14K - -  12K
> - -
> igb2   1.5K   00:1e:67:9b:03:e3  17M  5.8K 0  23M
> 0 0
> igb2  - 177.xx.xxx.40 177.xx.xxx.41  10K - -  12K
> - -
> igb2  - fe80::21e:67f fe80::21e:67ff:fe0 - -1
> - -
> igb2  - 2804::bad 2804::bad:b1f  14K - -  14K
> - -
> igb3   1.5K   00:1e:67:9b:03:e4  11K 0 0 441M
> 0 0
> igb3  - 186.xxx.x.148 186.xxx.x.150  10K - -  94K
> - -
> igb3  - fe80::21e:67f fe80::21e:67ff:fe0 - -1
> - -
> igb4   1.5K   00:1b:21:7b:ee:6c 5.8G  5.5K 0 194K
> 0 0
> igb4  - 159.xx.xx.96/ 159.xx.xx.98   63K - -  18K
> - -
> igb4  - fe80::21b:21f fe80::21b:21ff:fe0 - -0
> - -
> igb4  - 2804:xxx: 2804:xxx::ffd  17K - -  16K
> - -
> igb5   1.5K   00:1b:21:7b:ee:6d 6.9G  8.0K 0 9.1G
> 0 0
> igb5  - 64.xxx.xxx.68 64.xxx.xxx.70  28K - - 5.5M
> - -
> igb5  - fe80::21b:21f fe80::21b:21ff:fe0 - -  135
> - -
> igb5  - 2001:xxx:2001 2001:xxx:2001:100  17K - - 125K
> - -
> igb6   1.5K   00:1b:21:7b:ee:98 4.3G 0 0 3.8G
> 0 0
> igb6  - fe80::21b:21f fe80::21b:21ff:fe0 - -0
> - -
> igb7   1.5K   00:1b:21:7b:ee:990 0 00
> 0 0
> *ix01.5K  00:1b:21:89:25:28 -1.5   118  1.5T 0.1T
> 0 0*
> ix0   - fe80::21b:21f fe80::21b:21ff:fe0 - -0
> - -
> ix11.5K  00:1b:21:89:25:290 0 00
> 0 0
> pflog   32K   0 0 00
> 0 0
> pfsyn  1.5K   0 0 00
> 0 0
> lo0 16K 52K 0 0  52K
> 0 0
> lo0   - ::1/128   ::1  0 - -   41
> - -
> lo0   - fe80::1%lo0/6 fe80::1%lo0  0 - -0
> - -
> lo0   - 127.0.0.0/8   127.0.0.1  48K - -  52K
> - -
> disc0   64K   0 0 00
> 0 0
>
> # uname -a
> FreeBSD rt01.xx.xxx.xx 10.0-STABLE FreeBSD 10.0-STABLE #9 r267839: Tue
> Jun 24 21:06:39 BRT 2014 r...@rt01.xx.xxx.xx:/usr/obj/usr/src/sys/GONDIM
>  amd64
> You have new mail in /var/mail/root
>
> # netstat -ihw 1
> input(Total)   output
>packets  errs idrops  bytespackets  errs  bytes colls
>   813K 0 0   522M   832K 9   615M 0
>   812K 0 0   523M   831K 9   616M 0
>   811K 0 0   521M   829K 6   614M 0
>   807K 0 0   519M   826K15   615M 0
>   794K 0 0   514M   814K 7   610M 0
>   799K 0 0   520M   816K 7   612M 0
>   804K 0 0   523M   822K14   613M 0
>   794K 0 0   518M   813K14   610M 0
>   779K 0 0   510M   799K 8   601M 0
>   790K 0 0   514M   809K 7   603M 0
>   818K 0 0   525M   835K 7   617M 0
>   810K 0 0   522M   827K12   614M 0
>   794K 0 0   514M   812K13   603M 0
>   794K 0 0   518M   813K13   608M 0
>   790K 0 0   516M   810K 5   606M 0
>   807K 0 0   525M   824K 9   614M 0
>   808K 0 0   529M   827K10   619M 0
>   805K 0 0   524M   823K 7   614M 0
>   804K 0 0   526M   821K 8   615M 0
>   799K 0 0   520M   816K 6   612M 0
>   803K 0 0   522M   819K11   610M

Re: Regarding Netmap in VM

[Jack Vogel]

Ya, it might be nice to do a bunch of cleanup like that,  maybe once the
i40e
release happens I'll have some time to look into that.

Jack



On Tue, May 27, 2014 at 2:49 AM, Luigi Rizzo  wrote:

>
>
>
> On Tue, May 27, 2014 at 5:00 AM, Jack Vogel  wrote:
>
>> I don't think we've ever tried this at Intel either. I had meant to check
>> the ixv code to
>> see if it had your code in it Luigi but got distracted, if it doesn't
>> then it obviously can't
>> work for the case of a VM.
>>
>
> ixv.c does not contain the netmap hooks though it might be
> relatively straightforward to add them, possibly reusing
> the same netmap hooks and support functions used in ixgbe.c
>
> As a matter of fact, there seems to be a large amount
> of duplicated code between ixv.c and ixgbe.c,
> so i wonder whether it makes sense to remove the duplication and
> use just a single copy of the common parts ?
>
> cheers
> luigi
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Regarding Netmap in VM

[Jack Vogel]

I don't think we've ever tried this at Intel either. I had meant to check
the ixv code to
see if it had your code in it Luigi but got distracted, if it doesn't then
it obviously can't
work for the case of a VM.

I think with the new 40G VF driver, when that gets committed it might be
desirable to
have netmap support.

And, although we don't have SRIOV yet... that should be changing before
long :)

Jack



On Mon, May 26, 2014 at 10:06 AM, Luigi Rizzo  wrote:

> On Thu, May 22, 2014 at 09:28:45AM +0530, Prashant Upadhyaya wrote:
> > Hi,
> >
> > Suppose I am on x86 with Intel 82599 NIC.
> > Now I spawn a VM (running FreeBSD as guest OS with Netmap support)
> > For better I/O performance, I have two main choices --
> >
> > 1. Pass the 82599 NIC as a PCI passthrough device into the VM
> > 2. Use SRIOV VF of 82599 into the VM
> >
> > Question is, will Netmap be able to utilize both the above environments
> > when I run the userspace application in the guest OS in the VM, or will
> > there be any issues.
>
> if i remember well we do not have sriov vf support yet in freebsd
> so you should follow the pci-passthrough approach.
>
> It should probably work, i am not 100% sure who is in charge of
> programming the IOMMU for the guest os when accessing a device
> through pci passthrough. If this part is missing, the symptoms
> you should see is that packets will have all bytes set to 0x00
>
> cheers
> luigi
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: 9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF

[Jack Vogel]

Nick,

I'm very busy with some critical internal deadlines, I will look at this
when I can
come up for air, but please be patient.

Jack



On Mon, May 12, 2014 at 4:56 PM, Nick Rogers  wrote:

> On Fri, May 2, 2014 at 1:49 PM, Nick Rogers  wrote:
> > Hello,
> >
> > I am hoping someone can help me debug a kernel panic I have been
> experiencing
> > on one of my production systems. The system is a PF+ALTQ
> firewall/gateway with
> > about 1k simultaneous devices behind it at any given time, pushing no
> more
> > than 100Mb/s of traffic. I have obtained a crash dump and tried to debug
> it
> > with kgdb, but am still at a loss.
> >
> > I have completely replaced the hardware to rule out issues with
> > disk/memory/etc, and it appears to be a kernel issue, likely with e1000
> driver
> > and/or PF.
> >
> > The panic seems to happen during times of heavier use, but the frequency
> is
> > not very predictable (anywhere from a few times a day to a once a week),
> so I
> > feel like its some kind of strange traffic pattern that I am unable to
> > pinpoint.
> >
> > I am using a slightly custom kernel based on GENERIC, mainly to bring in
> ALTQ
> > and some other options.
> >
> > It may be worth noting that I also have IGB_LEGACY_TX set for the e1000
> > driver, although I do not believe that should affect em(4).
> >
> > Hoping someone can be of assistance in debugging this problem. I am
> willing to
> > test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if
> that
> > is advisable. Unfortunately I cannot try 10-STABLE.
> >
> > Thanks,
> >
> > -Nick Rogers
> >
> >
> > uname -v
> > FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014
> > root@fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS
> >
> >
> >
> > Here is the kernel conf...
> >
> ..
> > include GENERIC
> >
> > ident RGNETS
> >
> > makeoptions MODULES_OVERRIDE=""
> >
> > options DEVICE_POLLING
> >
> > device crypto
> > device cryptodev
> >
> > options VLAN_ARRAY
> >
> > device amdtemp
> >
> > # PF
> > device pf
> > device pflog
> > options ALTQ
> > options ALTQ_CBQ# Class Bases Queuing (CBQ)
> > options ALTQ_RED# Random Early Detection (RED)
> > options ALTQ_RIO# RED In/Out
> > options ALTQ_HFSC   # Hierarchical Packet Scheduler (HFSC)
> > options ALTQ_PRIQ   # Priority Queuing (PRIQ)
> > options ALTQ_NOPCC  # Required for SMP build
> >
> > # PPPoE
> > options NETGRAPH
> > options NETGRAPH_ETHER
> > options NETGRAPH_PPPOE
> > options NETGRAPH_SOCKET
> >
> > # IPsec
> > device enc
> > options IPSEC
> > options IPSEC_FILTERTUNNEL
> > options IPSEC_NAT_T
> >
> ..
> >
> >
> > The crash dump
> >
> ..
> >
> > GNU gdb 6.1.1 [FreeBSD]
> > Copyright 2004 Free Software Foundation, Inc.
> > GDB is free software, covered by the GNU General Public License, and you
> are
> > welcome to change it and/or distribute copies of it under certain
> conditions.
> > Type "show copying" to see the conditions.
> > There is absolutely no warranty for GDB.  Type "show warranty" for
> details.
> > This GDB was configured as "amd64-marcel-freebsd"...
> >
> > Unread portion of the kernel message buffer:
> >
> >
> > Fatal trap 12: page fault while in kernel mode
> > cpuid = 5; apic id = 05
> > fault virtual address = 0x10
> > fault code = supervisor read data, page not present
> > instruction pointer = 0x20:0x8033d350
> > stack pointer= 0x28:0xff83545384b0
> > frame pointer= 0x28:0xff83545384c0
> > code segment = base 0x0, limit 0xf, type 0x1b
> > = DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags = interrupt enabled, resume, IOPL = 0
> > current process = 12 (irq262: em2:rx 0)
> > trap number = 12
> > panic: page fault
> > cpuid = 5
> > KDB: stack backtrace:
> > #0 0x80956836 at kdb_backtrace+0x66
> > #1 0x8091c40e at panic+0x1ce
> > #2 0x80d31e70 at trap_fatal+0x290
> > #3 0x80d321d1 at trap_pfault+0x211
> > #4 0x80d327d3 at trap+0x363
> > #5 0x80d1b9d3 at calltrap+0x8
> > #6 0x8034872d at pf_test_rule+0x17ed
> > #7 0x8034ba12 at pf_test+0x1032
> > #8 0x8035112b at pf_check_in+0x2b
> > #9 0x809e952e at pfil_run_hooks+0x9e
> > #10 0x80a5286a at ip_input+0x2ea
> > #11 0x809e8858 at netisr_dispatch_src+0x218
> > #12 0x809df93d at ether_demux+0x14d
> > #13 0x809dfc1e at ether_nh_input+0x1fe
> > #14 0x809e8858 at netisr_dispatch_src+0x218
> > #15 0x809df85f at ether_demux+0x6f
> > #16 0x809dfc1e at ether_nh_input+0x1fe
> > #17 0x809e8858 at netisr_dispatch_src+0x218
> > Uptime: 17d7h20m59s
> > Dumping 2932 out of 12256 MB: (CTRL-C to abort)
> > ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
> >
> > Re

Re: 9.2 ixgbe tx queue hang

[Jack Vogel]

Your 4K mbuf pool is not being used, make sure you increase the size once
you are
using that or you'll just be having the same issue with a different pool.

Oh, and that patch was against the code in HEAD, it might need some manual
hacking
if you're using anything older.

Not sure what you mean about memory allocation in 10, this change is not 10
specific, its
something I intended on doing and it just slipped between the cracks.

Jack



On Thu, Mar 20, 2014 at 3:32 PM, Christopher Forgeron
wrote:

> I agree, performance is noticeably worse with TSO off, but I thought it
> would be a good step in troubleshooting. I'm glad you're a regular reader
> of the list, so I don't have to settle for slow performance. :-)
>
> I'm applying your patch now, I think it will fix it - but I'll report in
> after it's run iometer for the night regardless.
>
> On another note: What's so different about memory allocation in 10 that is
> making this an issue?
>
>
>
>
> On Thu, Mar 20, 2014 at 7:24 PM, Jack Vogel  wrote:
>
>> I strongly discourage anyone from disabling TSO on 10G, its necessary to
>> get the
>> performance one wants to see on the hardware.
>>
>> Here is a patch to do what i'm talking about:
>>
>> *** ixgbe.cFri Jan 10 18:12:20 2014
>> --- ixgbe.jfv.cThu Mar 20 23:04:15 2014
>> *** ixgbe_init_locked(struct adapter *adapte
>> *** 1140,1151 
>>   */
>>   if (adapter->max_frame_size <= 2048)
>>   adapter->rx_mbuf_sz = MCLBYTES;
>> - else if (adapter->max_frame_size <= 4096)
>> - adapter->rx_mbuf_sz = MJUMPAGESIZE;
>> - else if (adapter->max_frame_size <= 9216)
>> - adapter->rx_mbuf_sz = MJUM9BYTES;
>>   else
>> ! adapter->rx_mbuf_sz = MJUM16BYTES;
>>
>>   /* Prepare receive descriptors and buffers */
>>   if (ixgbe_setup_receive_structures(adapter)) {
>> --- 1140,1147 
>>   */
>>   if (adapter->max_frame_size <= 2048)
>>   adapter->rx_mbuf_sz = MCLBYTES;
>>   else
>> ! adapter->rx_mbuf_sz = MJUMPAGESIZE;
>>
>>   /* Prepare receive descriptors and buffers */
>>   if (ixgbe_setup_receive_structures(adapter)) {
>>
>>
>>
>>
>>
>>
>> On Thu, Mar 20, 2014 at 3:12 PM, Christopher Forgeron <
>> csforge...@gmail.com> wrote:
>>
>>> Hi Jack,
>>>
>>>  I'm on ixgbe 2.5.15
>>>
>>>  I see a few other threads about using MJUMPAGESIZE instead of
>>> MJUM9BYTES.
>>>
>>>  If you have a patch you'd like me to test, I'll compile it in and let
>>> you know. I was just looking at Garrett's if_em.c patch and thinking about
>>> applying it to ixgbe..
>>>
>>>  As it stands I seem to not be having the problem now that I have
>>> disabled TSO on ix0, but I still need more test runs to confirm - Which is
>>> also in line (i think) with what you are all saying.
>>>
>>>
>>>
>>>
>>> On Thu, Mar 20, 2014 at 7:00 PM, Jack Vogel  wrote:
>>>
>>>> What he's saying is that the driver should not be using 9K mbuf
>>>> clusters, I thought
>>>> this had been changed but I see the code in HEAD is still using the
>>>> larger clusters
>>>> when you up the mtu.  I will put it on my list to change with the next
>>>> update to HEAD.
>>>>
>>>>
>>>> What version of ixgbe are you using?
>>>>
>>>> Jack
>>>>
>>>>
>>>>
>>>> On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron <
>>>> csforge...@gmail.com> wrote:
>>>>
>>>>> I have found this:
>>>>>
>>>>> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html
>>>>>
>>>>> I think what you're saying is that;
>>>>> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster
>>>>> - modern NIC drivers can gather 9000 bytes of data from various memory
>>>>> locations
>>>>> - The fact that I'm seeing 9k jumbo clusters is showing me that my
>>>>> driver
>>>>> is trying to allocate 9k of contiguous space, and it's failing.
>>>>>
>>>>> Please correct me if I'm off here, I'd love to unders

Re: 9.2 ixgbe tx queue hang

[Jack Vogel]

I strongly discourage anyone from disabling TSO on 10G, its necessary to
get the
performance one wants to see on the hardware.

Here is a patch to do what i'm talking about:

*** ixgbe.cFri Jan 10 18:12:20 2014
--- ixgbe.jfv.cThu Mar 20 23:04:15 2014
*** ixgbe_init_locked(struct adapter *adapte
*** 1140,1151 
  */
  if (adapter->max_frame_size <= 2048)
  adapter->rx_mbuf_sz = MCLBYTES;
- else if (adapter->max_frame_size <= 4096)
- adapter->rx_mbuf_sz = MJUMPAGESIZE;
- else if (adapter->max_frame_size <= 9216)
- adapter->rx_mbuf_sz = MJUM9BYTES;
  else
! adapter->rx_mbuf_sz = MJUM16BYTES;

  /* Prepare receive descriptors and buffers */
  if (ixgbe_setup_receive_structures(adapter)) {
--- 1140,1147 
  */
  if (adapter->max_frame_size <= 2048)
  adapter->rx_mbuf_sz = MCLBYTES;
  else
! adapter->rx_mbuf_sz = MJUMPAGESIZE;

  /* Prepare receive descriptors and buffers */
  if (ixgbe_setup_receive_structures(adapter)) {






On Thu, Mar 20, 2014 at 3:12 PM, Christopher Forgeron
wrote:

> Hi Jack,
>
>  I'm on ixgbe 2.5.15
>
>  I see a few other threads about using MJUMPAGESIZE instead of MJUM9BYTES.
>
>  If you have a patch you'd like me to test, I'll compile it in and let you
> know. I was just looking at Garrett's if_em.c patch and thinking about
> applying it to ixgbe..
>
>  As it stands I seem to not be having the problem now that I have disabled
> TSO on ix0, but I still need more test runs to confirm - Which is also in
> line (i think) with what you are all saying.
>
>
>
>
> On Thu, Mar 20, 2014 at 7:00 PM, Jack Vogel  wrote:
>
>> What he's saying is that the driver should not be using 9K mbuf clusters,
>> I thought
>> this had been changed but I see the code in HEAD is still using the
>> larger clusters
>> when you up the mtu.  I will put it on my list to change with the next
>> update to HEAD.
>>
>>
>> What version of ixgbe are you using?
>>
>> Jack
>>
>>
>>
>> On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron <
>> csforge...@gmail.com> wrote:
>>
>>> I have found this:
>>>
>>> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html
>>>
>>> I think what you're saying is that;
>>> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster
>>> - modern NIC drivers can gather 9000 bytes of data from various memory
>>> locations
>>> - The fact that I'm seeing 9k jumbo clusters is showing me that my driver
>>> is trying to allocate 9k of contiguous space, and it's failing.
>>>
>>> Please correct me if I'm off here, I'd love to understand more.
>>>
>>>
>>> On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman <
>>> woll...@hergotha.csail.mit.edu> wrote:
>>>
>>> > In article
>>> > ,
>>> > csforge...@gmail.com writes:
>>> >
>>> > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k)
>>> >
>>> > This is going to screw you.  You need to make sure that no NIC driver
>>> > ever allocates 9k jumbo pages -- unless you are using one of those
>>> > mythical drivers that can't do scatter/gather DMA on receive, which
>>> > you don't appear to be.
>>> >
>>> > These failures occur when the driver is trying to replenish its
>>> > receive queue, but is unable to allocate three *physically* contiguous
>>> > pages of RAM to construct the 9k jumbo cluster (of which the remaining
>>> > 3k is simply wasted).  This happens on any moderately active server,
>>> > once physical memory gets checkerboarded with active single pages,
>>> > particularly with ZFS where those pages are wired in kernel memory and
>>> > so can't be evicted.
>>> >
>>> > -GAWollman
>>> >
>>> ___
>>> freebsd-net@freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>>>
>>
>>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: 9.2 ixgbe tx queue hang

[Jack Vogel]

What he's saying is that the driver should not be using 9K mbuf clusters, I
thought
this had been changed but I see the code in HEAD is still using the larger
clusters
when you up the mtu.  I will put it on my list to change with the next
update to HEAD.


What version of ixgbe are you using?

Jack



On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron
wrote:

> I have found this:
>
> http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html
>
> I think what you're saying is that;
> - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster
> - modern NIC drivers can gather 9000 bytes of data from various memory
> locations
> - The fact that I'm seeing 9k jumbo clusters is showing me that my driver
> is trying to allocate 9k of contiguous space, and it's failing.
>
> Please correct me if I'm off here, I'd love to understand more.
>
>
> On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman <
> woll...@hergotha.csail.mit.edu> wrote:
>
> > In article
> > ,
> > csforge...@gmail.com writes:
> >
> > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k)
> >
> > This is going to screw you.  You need to make sure that no NIC driver
> > ever allocates 9k jumbo pages -- unless you are using one of those
> > mythical drivers that can't do scatter/gather DMA on receive, which
> > you don't appear to be.
> >
> > These failures occur when the driver is trying to replenish its
> > receive queue, but is unable to allocate three *physically* contiguous
> > pages of RAM to construct the 9k jumbo cluster (of which the remaining
> > 3k is simply wasted).  This happens on any moderately active server,
> > once physical memory gets checkerboarded with active single pages,
> > particularly with ZFS where those pages are wired in kernel memory and
> > so can't be evicted.
> >
> > -GAWollman
> >
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Intel 10 gig cards mtu 9000

[Jack Vogel]

There's only one reason RX structures fail, and that's insufficient mbuf
pool.
You will find the driver probably uses the 9K mbuf pool in that driver, so
look
at how many queues it wants to set up, how big your ring is, and do the
math.

Jack



On Mon, Mar 17, 2014 at 6:34 AM, Wayne Hotmail  wrote:

> I am unable to add 2 Intel 10 gig nics to my Freebsd 9.1 at 9000 MTU. I
> tried to add them last night and was only able to set only a max of 4000
> MTU.  I get an error telling me kernel: ix1: Could not setup receive
> structures and the interface will not pass traffic.
>
> I found these changes on the Intel readme but this did solve my issue.
> kern.ipc.nmbclusters=262144
>  kern.ipc.nmbjumbop=262144
>
> There seems to be some calculation that needs to be done with multiple
> CPUs. I am running 12 cpus what tweak do I need to make so I can run my
> MTUs at 9000 on my 1 NIC with no vlans and 1 NIC with 3 vlans on it.
>
> Wayne
>
>
> ___
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: 9.2 ixgbe tx queue hang (was: Network loss)

[Jack Vogel]

I suppose to be sure where the issue really occurs it would be best if both
pseudo drivers
were out of the picture.  Once we see if it still occurs we can take the
next step.

Regards,

Jack



On Thu, Mar 6, 2014 at 12:58 PM, Markus Gebert
wrote:

>
> On 06.03.2014, at 19:33, Jack Vogel  wrote:
>
> > You did not make it explicit before, but I noticed in your dtrace info
> that
> > you are using
> > lagg, its been the source of lots of problems, so take it out of the
> setup
> > and see if this
> > queue problem still happens please.
> >
> > Jack
>
> Well, last year when upgrading another batch of servers (same hardware) to
> 9.2, we tried find a solution to this network problem, and we eliminated
> lagg where we had used it before, which did not help at all. That's why I
> didn't mention it explicitly.
>
> My point is, I can confirm that 9.2 has network problems on this same
> hardware with or without lagg, so it's unlikely that removing it will bring
> immediate success. OTOH, I didn't have this tx queue theory back then, so I
> cannot be sure that what we saw then without lagg, and what we see now with
> lagg, really are the same problem.
>
> I guess, for the sake of simplicity I will remove lagg on these new
> systems. But before I do that, to save time, I wanted to ask wether I
> should remove vlan interfaces too? While that didn't help either last year,
> my guess is that I should take them out of the picture, unless you say
> otherwise.
>
> Thanks for looking into this.
>
>
> Markus
>
>
>
> > On Thu, Mar 6, 2014 at 2:24 AM, Markus Gebert <
> markus.geb...@hostpoint.ch>wrote:
> >
> >> (creating a new thread, because I'm no longer sure this is related to
> >> Johan's thread that I originally used to discuss this)
> >>
> >> On 27.02.2014, at 18:02, Jack Vogel  wrote:
> >>
> >>> I would make SURE that you have enough mbuf resources of whatever size
> >> pool
> >>> that you are
> >>> using (2, 4, 9K), and I would try the code in HEAD if you had not.
> >>>
> >>> Jack
> >>
> >> Jack, we've upgraded some other systems on which I get more time to
> debug
> >> (no impact for customers). Although those systems use the nfsclient
> too, I
> >> no longer think that NFS is the source of the problem (hence the new
> >> thread). I think it's the ixgbe driver and/or card. When our problem
> >> occurs, it looks like it's a single tx queue that gets stuck somehow
> (its
> >> buf_ring remains full).
> >>
> >> I tracked ping using dtrace to determine the source of ENOBUFS it
> returns
> >> every few packets when things get weird:
> >>
> >> # dtrace -n 'fbt:::return / arg1 == ENOBUFS && execname == "ping" / {
> >> stack(); }'
> >> dtrace: description 'fbt:::return ' matched 25476 probes
> >> CPU IDFUNCTION:NAME
> >> 26   7730ixgbe_mq_start:return
> >>  if_lagg.ko`lagg_transmit+0xc4
> >>  kernel`ether_output_frame+0x33
> >>  kernel`ether_output+0x4fe
> >>  kernel`ip_output+0xd74
> >>  kernel`rip_output+0x229
> >>  kernel`sosend_generic+0x3f6
> >>  kernel`kern_sendit+0x1a3
> >>  kernel`sendit+0xdc
> >>  kernel`sys_sendto+0x4d
> >>  kernel`amd64_syscall+0x5ea
> >>  kernel`0x80d35667
> >>
> >>
> >>
> >> The only way ixgbe_mq_start could return ENOBUFS would be when
> >> drbr_enqueue() encouters a full tx buf_ring. Since a new ping packet
> >> probably has no flow id, it should be assigned to a queue based on
> curcpu,
> >> which made me try to pin ping to single cpus to check wether it's always
> >> the same tx buf_ring that reports being full. This turned out to be
> true:
> >>
> >> # cpuset -l 0 ping 10.0.4.5
> >> PING 10.0.4.5 (10.0.4.5): 56 data bytes
> >> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.347 ms
> >> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.135 ms
> >>
> >> # cpuset -l 1 ping 10.0.4.5
> >> PING 10.0.4.5 (10.0.4.5): 56 data bytes
> >> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.184 ms
> >> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.232 ms
> >>
> >> # cpuset -l 2 ping 10.0.4.5
> >> PING 10.0.4.5 (10.0.4.5): 56 data

Re: 9.2 ixgbe tx queue hang (was: Network loss)

[Jack Vogel]

You did not make it explicit before, but I noticed in your dtrace info that
you are using
lagg, its been the source of lots of problems, so take it out of the setup
and see if this
queue problem still happens please.

Jack



On Thu, Mar 6, 2014 at 2:24 AM, Markus Gebert wrote:

> (creating a new thread, because I'm no longer sure this is related to
> Johan's thread that I originally used to discuss this)
>
> On 27.02.2014, at 18:02, Jack Vogel  wrote:
>
> > I would make SURE that you have enough mbuf resources of whatever size
> pool
> > that you are
> > using (2, 4, 9K), and I would try the code in HEAD if you had not.
> >
> > Jack
>
> Jack, we've upgraded some other systems on which I get more time to debug
> (no impact for customers). Although those systems use the nfsclient too, I
> no longer think that NFS is the source of the problem (hence the new
> thread). I think it's the ixgbe driver and/or card. When our problem
> occurs, it looks like it's a single tx queue that gets stuck somehow (its
> buf_ring remains full).
>
> I tracked ping using dtrace to determine the source of ENOBUFS it returns
> every few packets when things get weird:
>
> # dtrace -n 'fbt:::return / arg1 == ENOBUFS && execname == "ping" / {
> stack(); }'
> dtrace: description 'fbt:::return ' matched 25476 probes
> CPU IDFUNCTION:NAME
>  26   7730ixgbe_mq_start:return
>   if_lagg.ko`lagg_transmit+0xc4
>   kernel`ether_output_frame+0x33
>   kernel`ether_output+0x4fe
>   kernel`ip_output+0xd74
>   kernel`rip_output+0x229
>   kernel`sosend_generic+0x3f6
>   kernel`kern_sendit+0x1a3
>   kernel`sendit+0xdc
>   kernel`sys_sendto+0x4d
>   kernel`amd64_syscall+0x5ea
>   kernel`0x80d35667
>
>
>
> The only way ixgbe_mq_start could return ENOBUFS would be when
> drbr_enqueue() encouters a full tx buf_ring. Since a new ping packet
> probably has no flow id, it should be assigned to a queue based on curcpu,
> which made me try to pin ping to single cpus to check wether it's always
> the same tx buf_ring that reports being full. This turned out to be true:
>
> # cpuset -l 0 ping 10.0.4.5
> PING 10.0.4.5 (10.0.4.5): 56 data bytes
> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.347 ms
> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.135 ms
>
> # cpuset -l 1 ping 10.0.4.5
> PING 10.0.4.5 (10.0.4.5): 56 data bytes
> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.184 ms
> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.232 ms
>
> # cpuset -l 2 ping 10.0.4.5
> PING 10.0.4.5 (10.0.4.5): 56 data bytes
> ping: sendto: No buffer space available
> ping: sendto: No buffer space available
> ping: sendto: No buffer space available
> ping: sendto: No buffer space available
> ping: sendto: No buffer space available
>
> # cpuset -l 3 ping 10.0.4.5
> PING 10.0.4.5 (10.0.4.5): 56 data bytes
> 64 bytes from 10.0.4.5: icmp_seq=0 ttl=255 time=0.130 ms
> 64 bytes from 10.0.4.5: icmp_seq=1 ttl=255 time=0.126 ms
> [...snip...]
>
> The system has 32 cores, if ping runs on cpu 2, 10, 18 or 26, which use
> the third tx buf_ring, ping reliably return ENOBUFS. If ping is run on any
> other cpu using any other tx queue, it runs without any packet loss.
>
> So, when ENOBUFS is returned, this is not due to an mbuf shortage, it's
> because the buf_ring is full. Not surprisingly, netstat -m looks pretty
> normal:
>
> # netstat -m
> 38622/11823/50445 mbufs in use (current/cache/total)
> 32856/11642/44498/132096 mbuf clusters in use (current/cache/total/max)
> 32824/6344 mbuf+clusters out of packet secondary zone in use
> (current/cache)
> 16/3906/3922/66048 4k (page size) jumbo clusters in use
> (current/cache/total/max)
> 0/0/0/33024 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/16512 16k jumbo clusters in use (current/cache/total/max)
> 75431K/41863K/117295K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> 0/0/0 sfbufs in use (current/peak/max)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
> 0 calls to protocol drain routines
>
> In the meantime I've checked the commit log of the ixgbe driver in HEAD
> and besides there are little differences between HEAD and 9.2, I don't see
> a co

Re: igb(4) panic: already enqueue

[Jack Vogel]

Good, and I'll get the map changes in after the weekend.

Jack



On Sat, Oct 12, 2013 at 3:50 AM, Konstantin Belousov wrote:

> On Wed, Oct 09, 2013 at 10:56:25AM -0700, Jack Vogel wrote:
> > Give the new driver I just committed to HEAD a try to verify/falsify a
> fix
> > please.
> >
> Updated driver seemingly fixed my issue with the drdb panic, thank you.
>
> The tx busdma map leakage is still there.
> > Regards,
> >
> > Jack
> >
> >
> >
> > On Wed, Oct 9, 2013 at 10:23 AM, hiren panchasara <
> > hiren.panchas...@gmail.com> wrote:
> >
> > > Jack,
> > > I am also seeing similar panics at $work on a couple weeks old
> STABLE-9.
> > >
> > > Can you please look into this issue?
> > >
> > > cheers,
> > > Hiren
> > >
> > > 1) HP DL360e Gen8, 2 x Xeon E5-2430 2.20GHz
> > >
> > > panic: buf=0xfe002810d700 already enqueue at 995 prod=997 cons=995
> > > cpuid = 17
> > > KDB: stack backtrace:
> > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
> > > 0xff868637b030
> > > kdb_backtrace() at kdb_backtrace+0x37/frame 0xff868637b0f0
> > > panic() at panic+0x1d8/frame 0xff868637b1f0
> > > igb_mq_start() at igb_mq_start+0x1cb/frame 0xff868637b240
> > > ether_output_frame() at ether_output_frame+0x33/frame
> 0xff868637b260
> > > ether_output() at ether_output+0x52d/frame 0xff868637b2f0
> > > ip_output() at ip_output+0xe38/frame 0xff868637b3e0
> > > tcp_output() at tcp_output+0x122c/frame 0xff868637b5a0
> > > tcp_do_segment() at tcp_do_segment+0x306c/frame 0xff868637b6c0
> > > tcp_input() at tcp_input+0x909/frame 0xff868637b7f0
> > > ip_input() at ip_input+0xbd/frame 0xff868637b840
> > > netisr_dispatch_src() at netisr_dispatch_src+0x152/frame
> 0xff868637b8a0
> > > ether_demux() at ether_demux+0x17d/frame 0xff868637b8d0
> > > ether_nh_input() at ether_nh_input+0x208/frame 0xff868637b910
> > > netisr_dispatch_src() at netisr_dispatch_src+0x152/frame
> 0xff868637b970
> > > igb_rxeof() at igb_rxeof+0x394/frame 0xff868637b9e0
> > > igb_handle_que() at igb_handle_que+0x9b/frame 0xff868637ba20
> > > taskqueue_run_locked() at taskqueue_run_locked+0x93/frame
> > > 0xff868637ba80
> > > taskqueue_thread_loop() at taskqueue_thread_loop+0x3e/frame
> > > 0xff868637baa0
> > > fork_exit() at fork_exit+0x135/frame 0xff868637baf0
> > > fork_trampoline() at fork_trampoline+0xe/frame 0xff868637baf0
> > >
> > > 2) HP DL160 G6, 2 x Xeon E5620 2.40GHz
> > >
> > > panic: buf=0xfe01b6334700 already enqueue at 42 prod=43 cons=42
> > > cpuid = 0
> > > KDB: stack backtrace:
> > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
> > > 0xff800037a620
> > > kdb_backtrace() at kdb_backtrace+0x37/frame 0xff800037a6e0
> > > panic() at panic+0x1d8/frame 0xff800037a7e0
> > > igb_mq_start() at igb_mq_start+0x1cb/frame 0xff800037a830
> > > ether_output_frame() at ether_output_frame+0x33/frame
> 0xff800037a850
> > > ether_output() at ether_output+0x52d/frame 0xff800037a8e0
> > > ip_output() at ip_output+0xe38/frame 0xff800037a9d0
> > > syncache_respond() at syncache_respond+0x462/frame 0xff800037aa90
> > > syncache_timer() at syncache_timer+0xdf/frame 0xff800037aac0
> > > softclock() at softclock+0x2c6/frame 0xff800037ab60
> > > intr_event_execute_handlers() at
> > > intr_event_execute_handlers+0x6a/frame 0xff800037ab90
> > > ithread_loop() at ithread_loop+0xac/frame 0xff800037abe0
> > > fork_exit() at fork_exit+0x135/frame 0xff800037ac30
> > > fork_trampoline() at fork_trampoline+0xe/frame 0xff800037ac30
> > >
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: igb(4) panic: already enqueue

[Jack Vogel]

Give the new driver I just committed to HEAD a try to verify/falsify a fix
please.

Regards,

Jack



On Wed, Oct 9, 2013 at 10:23 AM, hiren panchasara <
hiren.panchas...@gmail.com> wrote:

> Jack,
> I am also seeing similar panics at $work on a couple weeks old STABLE-9.
>
> Can you please look into this issue?
>
> cheers,
> Hiren
>
> 1) HP DL360e Gen8, 2 x Xeon E5-2430 2.20GHz
>
> panic: buf=0xfe002810d700 already enqueue at 995 prod=997 cons=995
> cpuid = 17
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
> 0xff868637b030
> kdb_backtrace() at kdb_backtrace+0x37/frame 0xff868637b0f0
> panic() at panic+0x1d8/frame 0xff868637b1f0
> igb_mq_start() at igb_mq_start+0x1cb/frame 0xff868637b240
> ether_output_frame() at ether_output_frame+0x33/frame 0xff868637b260
> ether_output() at ether_output+0x52d/frame 0xff868637b2f0
> ip_output() at ip_output+0xe38/frame 0xff868637b3e0
> tcp_output() at tcp_output+0x122c/frame 0xff868637b5a0
> tcp_do_segment() at tcp_do_segment+0x306c/frame 0xff868637b6c0
> tcp_input() at tcp_input+0x909/frame 0xff868637b7f0
> ip_input() at ip_input+0xbd/frame 0xff868637b840
> netisr_dispatch_src() at netisr_dispatch_src+0x152/frame 0xff868637b8a0
> ether_demux() at ether_demux+0x17d/frame 0xff868637b8d0
> ether_nh_input() at ether_nh_input+0x208/frame 0xff868637b910
> netisr_dispatch_src() at netisr_dispatch_src+0x152/frame 0xff868637b970
> igb_rxeof() at igb_rxeof+0x394/frame 0xff868637b9e0
> igb_handle_que() at igb_handle_que+0x9b/frame 0xff868637ba20
> taskqueue_run_locked() at taskqueue_run_locked+0x93/frame
> 0xff868637ba80
> taskqueue_thread_loop() at taskqueue_thread_loop+0x3e/frame
> 0xff868637baa0
> fork_exit() at fork_exit+0x135/frame 0xff868637baf0
> fork_trampoline() at fork_trampoline+0xe/frame 0xff868637baf0
>
> 2) HP DL160 G6, 2 x Xeon E5620 2.40GHz
>
> panic: buf=0xfe01b6334700 already enqueue at 42 prod=43 cons=42
> cpuid = 0
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
> 0xff800037a620
> kdb_backtrace() at kdb_backtrace+0x37/frame 0xff800037a6e0
> panic() at panic+0x1d8/frame 0xff800037a7e0
> igb_mq_start() at igb_mq_start+0x1cb/frame 0xff800037a830
> ether_output_frame() at ether_output_frame+0x33/frame 0xff800037a850
> ether_output() at ether_output+0x52d/frame 0xff800037a8e0
> ip_output() at ip_output+0xe38/frame 0xff800037a9d0
> syncache_respond() at syncache_respond+0x462/frame 0xff800037aa90
> syncache_timer() at syncache_timer+0xdf/frame 0xff800037aac0
> softclock() at softclock+0x2c6/frame 0xff800037ab60
> intr_event_execute_handlers() at
> intr_event_execute_handlers+0x6a/frame 0xff800037ab90
> ithread_loop() at ithread_loop+0xac/frame 0xff800037abe0
> fork_exit() at fork_exit+0x135/frame 0xff800037ac30
> fork_trampoline() at fork_trampoline+0xe/frame 0xff800037ac30
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Exposing sysctls for ixgbe

[Jack Vogel]

Good with me Hiren

Jack



On Fri, Oct 4, 2013 at 2:55 PM, hiren panchasara  wrote:

> I am going to commit this change in the weekend.
>
> Let me know if there are any objections.
>
> Thanks,
> Hiren
>
>
>
> On Mon, Sep 30, 2013 at 8:51 AM, hiren panchasara wrote:
>
>> + jfv
>>
>>
>> On Wed, Sep 25, 2013 at 5:35 PM, Adrian Chadd  wrote:
>>
>>> please cc jfv and get them into his driver. :)
>>>
>>> -a
>>>
>>>
>>> On 24 September 2013 23:53, hiren panchasara  wrote:
>>>
 Any comments?

 cheers,
 Hiren

 On Sun, Sep 22, 2013 at 12:01 PM, hiren panchasara >>> >wrote:

 > $ sysctl hw.igb
 > hw.igb.rxd: 4096
 > hw.igb.txd: 4096
 > hw.igb.enable_aim: 1
 > hw.igb.enable_msix: 1
 > hw.igb.max_interrupt_rate: 8000
 > hw.igb.buf_ring_size: 4096
 > hw.igb.header_split: 0
 > hw.igb.num_queues: 1
 > hw.igb.rx_process_limit: 100
 > $ sysctl hw.ix
 > sysctl: unknown oid 'hw.ix': No such file or directory
 >
 > I thought it would be nice to have these things exposed. So I copied
 them
 > from igb:
 > http://people.freebsd.org/~hiren/ixgbe_sysctls.txt
 >
 > Changes for if_igb.c is to expose correct auto-tuned value for a
 running
 > system for "hw.igb.num_queues", which is not the case right now.
 >
 > Thanks to markj@ for help/pointers.
 >
 > Please let me know if the diffs look okay.
 >
 > cheers,
 > Hiren
 >
 ___
 freebsd-net@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-net
 To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

>>>
>>>
>>
>
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Network stack changes

[Jack Vogel]

Very interesting material Alexander, only had time to glance at it now,
will look in more
depth later, thanks!

Jack



On Wed, Aug 28, 2013 at 11:30 AM, Alexander V. Chernikov <
melif...@yandex-team.ru> wrote:

> Hello list!
>
> There is a lot constantly raising  discussions related to networking stack
> performance/changes.
>
> I'll try to summarize current problems and possible solutions from my
> point of view.
> (Generally this is one problem: stack is slooow**,
> but we need to know why and what to do).
>
> Let's start with current IPv4 packet flow on a typical router:
> http://static.ipfw.ru/images/**freebsd_ipv4_flow.png
>
> (I'm sorry I can't provide this as text since Visio don't have any
> 'ascii-art' exporter).
>
> Note that we are using process-to-completion model, e.g. process any
> packet in ISR until it is either
> consumed by L4+ stack or dropped or put to egress NIC queue.
>
> (There is also deferred ISR model implemented inside netisr but it does
> not change much:
> it can help to do more fine-grained hashing (for GRE or other similar
> traffic), but
> 1) it uses per-packet mutex locking which kills all performance
> 2) it currently does not have _any_ hashing functions (see absence of
> flags in `netstat -Q`)
> People using 
> http://static.ipfw.ru/patches/**netisr_ip_flowid.diff(or
>  modified PPPoe/GRE version)
> report some profit, but without fixing (1) it can't help much
> )
>
> So, let's start:
>
> 1) Ixgbe uses mutex to protect each RX ring which is perfectly fine since
> there is nearly no contention
> (the only thing that can happen is driver reconfiguration which is rare
> and, more signifficant, we do this once
> for the batch of packets received in given interrupt). However, due to
> some (im)possible deadlocks current code
> does per-packet ring unlock/lock (see ixgbe_rx_input()).
> There was a discussion ended with nothing: http://lists.freebsd.org/**
> pipermail/freebsd-net/2012-**October/033520.html
>
> 1*) Possible BPF users. Here we have one rlock if there are any readers
> present
> (and mutex for any matching packets, but this is more or less OK.
> Additionally, there is WIP to implement multiqueue BPF
> and there is chance that we can reduce lock contention there). There is
> also an "optimize_writers" hack permitting applications
> like CDP to use BPF as writers but not registering them as receivers
> (which implies rlock)
>
> 2/3) Virtual interfaces (laggs/vlans over lagg and other simular
> constructions).
> Currently we simply use rlock to make s/ix0/lagg0/ and, what is much more
> funny - we use complex vlan_hash with another rlock to
> get vlan interface from underlying one.
>
> This is definitely not like things should be done and this can be changed
> more or less easily.
>
> There are some useful terms/techniques in world of software/hardware
> routing: they have clear 'control plane' and 'data plane' separation.
> Former one is for dealing control traffic (IGP, MLD, IGMP snooping, lagg
> hellos, ARP/NDP, etc..) and some data traffic (packets with TTL=1, with
> options, destined to hosts without ARP/NDP record, and similar). Latter one
> is done in hardware (or effective software implementation).
> Control plane is responsible to provide data for efficient data plane
> operations. This is the point we are missing nearly everywhere.
>
> What I want to say is: lagg is pure control-plane stuff and vlan is nearly
> the same. We can't apply this approach to complex cases like
> lagg-over-vlans-over-vlans-**over-(pppoe_ng0-and_wifi0)
> but we definitely can do this for most common setups like (igb* or ix* in
> lagg with or without vlans on top of lagg).
>
> We already have some capabilities like VLANHWFILTER/VLANHWTAG, we can add
> some more. We even have per-driver hooks to program HW filtering.
>
> One small step to do is to throw packet to vlan interface directly (P1),
> proof-of-concept(working in production):
> http://lists.freebsd.org/**pipermail/freebsd-net/2013-**April/035270.html
>
> Another is to change lagg packet accounting: http://lists.freebsd.org/**
> pipermail/svn-src-all/2013-**April/067570.html
> Again, this is more like HW boxes do (aggregate all counters including
> errors) (and I can't imagine what real error we can get from _lagg_).
>
> 4) If we are router, we can do either slooow ip_input() -> ip_forward() ->
> ip_output() cycle or use optimized ip_fastfwd() which falls back to 'slow'
> path for multicast/options/local traffic (e.g. works exactly like 'data
> plane' part).
> (Btw, we can consider net.inet.ip.fastforwarding to be turned on by
> default at least for non-IPSEC kernels)
>
> Here we have to