Re: netmap: traffic distribution
On Fri, Sep 27, 2013 at 8:16 PM, chintu hetam wrote: > As far as i know, flow director is Intel terminology it addresses both RSS > and RFS. I think FreeBSD implementation is RFS. > > Luigi, you touched upon SW de-multiplexer, i would like to know why it's > necessary. the sw demux is only necessary if you expect to have traffic with heavy processing requirements for which you cannot easily guarantee that the RSS more or less balances the queues. Since the number of RSS queues is small, unbalance is not unlikely; in software you could in principle demux each flow to its own queue and then have one worker process per thread process queues. Note though that scheduling may become complicated if you care for CPU locality, but if the sw demux only reads buffers, then the situation is not too bad. > let say i have 82599 ixgbe driver (RSS enabled)configured with 5 tuple hash. > My application reads from netmap queue 0-7(1-8), for testing purpose i will > define tuples in such a way that i know for sure that each packet will be > filtered to specific hw queue(0-7). Now is it safe to assume that in netmap > mode in user space i will receive the packet as in ixgbe hw queue? yes. > > Michio, reason i asked for performance values > http://arxiv.org/ftp/arxiv/papers/1106/1106.0443.pdf The problem that the paper describes is due to the incorrect implementation (at least according to the description give in the paper) of the Flow Director (FD) feature in linux. The hardware demux (RSS or RFS) per se does not cause reordering unless it is misued (as described in the paper). Basically when FD is active, the RFS entry in the NIC is generated by looking at the core used for the transmit side of a flow, and (according to what the paper says; not sure if it has been fixed since), the association is updated dynamically and not generated just once at the beginning of the flow. So if the userspace thread floats between CPUs, the RFS entry keepfloating, and incoming packets for the same flow can be delivered to different queues. cheers luigi -- -+--- Prof. Luigi RIZZO, ri...@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/. Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -+--- ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: netmap: traffic distribution
As far as i know, flow director is Intel terminology it addresses both RSS and RFS. I think FreeBSD implementation is RFS. Luigi, you touched upon SW de-multiplexer, i would like to know why it's necessary. let say i have 82599 ixgbe driver (RSS enabled)configured with 5 tuple hash. My application reads from netmap queue 0-7(1-8), for testing purpose i will define tuples in such a way that i know for sure that each packet will be filtered to specific hw queue(0-7). Now is it safe to assume that in netmap mode in user space i will receive the packet as in ixgbe hw queue? Michio, reason i asked for performance values http://arxiv.org/ftp/arxiv/papers/1106/1106.0443.pdf I would like to test the accuracy of RFS with HW RSS in netmap mode.. cheers, r2r On Fri, Sep 27, 2013 at 2:59 AM, hiren panchasara < hiren.panchas...@gmail.com> wrote: > > > > On Thu, Sep 26, 2013 at 2:27 PM, chintu hetam wrote: > >> Hiren, >> >> https://www.kernel.org/doc/Documentation/networking/scaling.txt must >> read to understand nuances of each of this features. None of this >> techniques are used for mostly none other than performance reasons. >> > > Thanks for the link. > So, RFS (Receive Flow Steering) is equivalent to "flow director" mentioned > in FreeBSD's ixgbe drivers? > >> >> Michio, personally i am interested to know performance results in netmap >> mode with RFS patch you just mentioned. >> > Takuya/Luigi might have some numbers. > > Thanks, > Hiren > > > > ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Adding Flow Director sysctls to ixgbe(4) (was: netmap: traffic distribution)
On Fri, Sep 27, 2013 at 1:58 AM, Takuya ASADA wrote: > 2013/9/27 Adrian Chadd > >> On 27 September 2013 00:43, hiren panchasara >> wrote: >> >> >>> Takuya, >>> >>> I see a lot of responses/comments on proposed changes. Was anything >>> decided >>> at the end of it? As far as I can tell, its still not committed to the >>> tree. >>> >> >> I'd rather see an ioctl API for that chipset and then have a separate >> tool program it for now. >> > > Ah, like cxgbetool and cxgbe? (it has device specific tool and ioctls) > http://svnweb.freebsd.org/base/head/tools/tools/cxgbetool/ > Something like this for ixgbe would be nice to start with, imo. Cheers, Hiren > http://svnweb.freebsd.org/base/head/sys/dev/cxgb/ > > >> So, how bout we hack that up? :) >> > > Sound's interesting ;-) > Could you tell me more detail about your idea? > > ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: netmap: traffic distribution
I think you meant "reply-all" :-) On Fri, Sep 27, 2013 at 7:53 AM, chintu hetam wrote: > As far as i know, flow director is Intel terminology it addresses both RSS > and RFS. I think FreeBSD implementation is RFS. > > Luigi, you touched upon SW de-multiplexer, i would like to know why it's > necessary. > let say i have 82599 ixgbe driver (RSS enabled)configured with 5 tuple > hash. My application reads from netmap queue 0-7(1-8), i know for sure that > each hash will be filtered to specific hw queue(0-7), is it safe to assume > netmap will provide packets in same order. > > Michio, reason i asked for performance values > http://arxiv.org/ftp/arxiv/papers/1106/1106.0443.pdf > I would like to test the accuracy of RFS,RSS and others in netmap mode.. > > Thanks > Hardik > > > On Fri, Sep 27, 2013 at 2:59 AM, hiren panchasara < > hiren.panchas...@gmail.com> wrote: > >> >> >> >> On Thu, Sep 26, 2013 at 2:27 PM, chintu hetam wrote: >> >>> Hiren, >>> >>> https://www.kernel.org/doc/Documentation/networking/scaling.txt must >>> read to understand nuances of each of this features. None of this >>> techniques are used for mostly none other than performance reasons. >>> >> >> Thanks for the link. >> So, RFS (Receive Flow Steering) is equivalent to "flow director" >> mentioned in FreeBSD's ixgbe drivers? >> >>> >>> Michio, personally i am interested to know performance results in netmap >>> mode with RFS patch you just mentioned. >>> >> Takuya/Luigi might have some numbers. >> >> Thanks, >> Hiren >> >> >> >> > ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Adding Flow Director sysctls to ixgbe(4)
On 27 September 2013 03:09, Julian Elischer wrote: > On 9/27/13 4:53 PM, Adrian Chadd wrote: > >> I don't care about whether there's a generic API right now. I'd rather see >> it done as a staged thing, but _not_ sysctls. >> >> Having sysctls to add/remove entries from things is just plain evil. >> >> I'd rather instead come up with a device specific ioctl API for this for >> now w/ a userland tool for each particular chip. Then once we all get a >> bit >> more experience doing this, a unified API can be proposed. >> > > that makes it worse > If you want to put a device specific sysctl/ioctl set out there then have > a device INDEPENDENT > tool that knows how to handle the devices we have modified and when we > have enough examples we can change the > ioctl/sysctl interface to a generic one without changing the interface > people are using in their scripts. > I agree that's the eventual goal, sure. But there's not necessarily a clear-cut set of shared behaviour that a generic API could actually use. That's why I'm a fan of doing it in a couple of stages - get the device-specific stuff into the tree, get that stuff debugged, then sit down in the future and figure out what the sane intersection of this. The shared API may be partly "shared overlapping behaviour"and may partly be "have a capabilities api that defines which parts of the behaviour is supported by the NIC." Thanks, -adrian ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Adding Flow Director sysctls to ixgbe(4)
On 9/27/13 4:53 PM, Adrian Chadd wrote: I don't care about whether there's a generic API right now. I'd rather see it done as a staged thing, but _not_ sysctls. Having sysctls to add/remove entries from things is just plain evil. I'd rather instead come up with a device specific ioctl API for this for now w/ a userland tool for each particular chip. Then once we all get a bit more experience doing this, a unified API can be proposed. that makes it worse If you want to put a device specific sysctl/ioctl set out there then have a device INDEPENDENT tool that knows how to handle the devices we have modified and when we have enough examples we can change the ioctl/sysctl interface to a generic one without changing the interface people are using in their scripts. -adrian On 27 September 2013 01:49, Takuya ASADA wrote: Hi, I think there were many comment which says "don't use device specific sysctls, we should have more generic interface to configure NIC filter, like Linux's ethtool". And I heard same idea in last BSDCan, but there's still no proposal of "generic interface for NIC filter", I suppose. So, honestly I haven't good idea to merge the change. Is there any people who has good idea to handle this? Or, just merge this patch would be fine? 2013/9/27 hiren panchasara On Thu, Sep 26, 2013 at 10:38 AM, hiren panchasara < hiren.panchas...@gmail.com> wrote: On Thu, Sep 26, 2013 at 12:58 AM, Michio Honda wrote: Hi, The handiest way to try flexible flow distribution is using Flow Director. I've confirmed that the patch posted to this list two years ago works with netmap/ixgbe. http://freebsd.1045724.n5.nabble.com/Adding-Flow-Director-sysctls-to-ixgbe-4-td4769489.html Thanks a lot for the link, Michio! It seems this work is yet not committed?!? Takuya, I see a lot of responses/comments on proposed changes. Was anything decided at the end of it? As far as I can tell, its still not committed to the tree. Thanks, Hiren ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: kern/172113: [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type
Synopsis: [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type State-Changed-From-To: open->patched State-Changed-By: pluknet State-Changed-When: Fri Sep 27 09:00:31 UTC 2013 State-Changed-Why: The fix committed to HEAD (r254002), stable/9 (r254003), releng/9.2 (r254009). http://www.freebsd.org/cgi/query-pr.cgi?pr=172113 ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: kern/172113: [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type
I forward these mails for archives to easier track what was committed for this PR. -- Forwarded message -- From: Jack F Vogel Date: 6 August 2013 22:00 Subject: svn commit: r254002 - head/sys/dev/e1000 To: src-committ...@freebsd.org, svn-src-...@freebsd.org, svn-src-h...@freebsd.org Author: jfv Date: Tue Aug 6 18:00:53 2013 New Revision: 254002 URL: http://svnweb.freebsd.org/changeset/base/254002 Log: When the igb driver is static there are cases when early interrupts occur, resulting in a panic in refresh_mbufs, to prevent this add a check in the interrupt handler for DRV_RUNNING. MFC after: 1 day (critical for 9.2) Modified: head/sys/dev/e1000/if_igb.c Modified: head/sys/dev/e1000/if_igb.c == --- head/sys/dev/e1000/if_igb.c Tue Aug 6 17:22:06 2013(r254001) +++ head/sys/dev/e1000/if_igb.c Tue Aug 6 18:00:53 2013(r254002) @@ -1572,6 +1572,10 @@ igb_msix_que(void *arg) u32 newitr = 0; boolmore_rx; + /* Ignore spurious interrupts */ + if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) + return; + E1000_WRITE_REG(&adapter->hw, E1000_EIMC, que->eims); ++que->irqs; -- Forwarded message -- From: Jack F Vogel Date: 7 August 2013 01:16 Subject: svn commit: r254009 - releng/9.2/sys/dev/e1000 To: src-committ...@freebsd.org, svn-src-...@freebsd.org, svn-src-rel...@freebsd.org Author: jfv Date: Tue Aug 6 21:16:00 2013 New Revision: 254009 URL: http://svnweb.freebsd.org/changeset/base/254009 Log: When the igb driver is static there are cases when early interrupts occur, resulting in a panic in refresh_mbufs, to prevent this add a check to the interrupt handler for DRV_RUNNING. Approved by: re Modified: releng/9.2/sys/dev/e1000/if_igb.c Directory Properties: releng/9.2/sys/ (props changed) releng/9.2/sys/dev/ (props changed) releng/9.2/sys/dev/e1000/ (props changed) Modified: releng/9.2/sys/dev/e1000/if_igb.c == --- releng/9.2/sys/dev/e1000/if_igb.c Tue Aug 6 21:01:38 2013 (r254008) +++ releng/9.2/sys/dev/e1000/if_igb.c Tue Aug 6 21:16:00 2013 (r254009) @@ -1572,6 +1572,10 @@ igb_msix_que(void *arg) u32 newitr = 0; boolmore_rx; + /* Ignore spurious interrupts */ + if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) + return; + E1000_WRITE_REG(&adapter->hw, E1000_EIMC, que->eims); ++que->irqs; ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Adding Flow Director sysctls to ixgbe(4) (was: netmap: traffic distribution)
2013/9/27 Adrian Chadd > On 27 September 2013 00:43, hiren panchasara > wrote: > > >> Takuya, >> >> I see a lot of responses/comments on proposed changes. Was anything >> decided >> at the end of it? As far as I can tell, its still not committed to the >> tree. >> > > I'd rather see an ioctl API for that chipset and then have a separate tool > program it for now. > Ah, like cxgbetool and cxgbe? (it has device specific tool and ioctls) http://svnweb.freebsd.org/base/head/tools/tools/cxgbetool/ http://svnweb.freebsd.org/base/head/sys/dev/cxgb/ > So, how bout we hack that up? :) > Sound's interesting ;-) Could you tell me more detail about your idea? ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Adding Flow Director sysctls to ixgbe(4) (was: netmap: traffic distribution)
I don't care about whether there's a generic API right now. I'd rather see it done as a staged thing, but _not_ sysctls. Having sysctls to add/remove entries from things is just plain evil. I'd rather instead come up with a device specific ioctl API for this for now w/ a userland tool for each particular chip. Then once we all get a bit more experience doing this, a unified API can be proposed. -adrian On 27 September 2013 01:49, Takuya ASADA wrote: > Hi, > > I think there were many comment which says "don't use device specific > sysctls, we should have more generic interface to configure NIC filter, > like Linux's ethtool". > And I heard same idea in last BSDCan, but there's still no proposal of > "generic interface for NIC filter", I suppose. > So, honestly I haven't good idea to merge the change. > Is there any people who has good idea to handle this? > > Or, just merge this patch would be fine? > > > 2013/9/27 hiren panchasara > > > > > > > > > On Thu, Sep 26, 2013 at 10:38 AM, hiren panchasara < > > hiren.panchas...@gmail.com> wrote: > > > >> > >> > >> > >> On Thu, Sep 26, 2013 at 12:58 AM, Michio Honda >wrote: > >> > >>> Hi, > >>> > >>> The handiest way to try flexible flow distribution is using Flow > >>> Director. > >>> I've confirmed that the patch posted to this list two years ago works > >>> with netmap/ixgbe. > >>> > >>> > http://freebsd.1045724.n5.nabble.com/Adding-Flow-Director-sysctls-to-ixgbe-4-td4769489.html > >>> > >> > >> Thanks a lot for the link, Michio! > >> > >> It seems this work is yet not committed?!? > >> > > > > Takuya, > > > > I see a lot of responses/comments on proposed changes. Was anything > > decided at the end of it? As far as I can tell, its still not committed > to > > the tree. > > > > Thanks, > > Hiren > > > > > ___ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" > ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Adding Flow Director sysctls to ixgbe(4) (was: netmap: traffic distribution)
Hi, I think there were many comment which says "don't use device specific sysctls, we should have more generic interface to configure NIC filter, like Linux's ethtool". And I heard same idea in last BSDCan, but there's still no proposal of "generic interface for NIC filter", I suppose. So, honestly I haven't good idea to merge the change. Is there any people who has good idea to handle this? Or, just merge this patch would be fine? 2013/9/27 hiren panchasara > > > > On Thu, Sep 26, 2013 at 10:38 AM, hiren panchasara < > hiren.panchas...@gmail.com> wrote: > >> >> >> >> On Thu, Sep 26, 2013 at 12:58 AM, Michio Honda wrote: >> >>> Hi, >>> >>> The handiest way to try flexible flow distribution is using Flow >>> Director. >>> I've confirmed that the patch posted to this list two years ago works >>> with netmap/ixgbe. >>> >>> http://freebsd.1045724.n5.nabble.com/Adding-Flow-Director-sysctls-to-ixgbe-4-td4769489.html >>> >> >> Thanks a lot for the link, Michio! >> >> It seems this work is yet not committed?!? >> > > Takuya, > > I see a lot of responses/comments on proposed changes. Was anything > decided at the end of it? As far as I can tell, its still not committed to > the tree. > > Thanks, > Hiren > > ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: kern/172113: [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type
The following reply was made to PR kern/172113; it has been noted by GNATS. From: Eugene Grosbein To: bug-follo...@freebsd.org Cc: Jack Vogel , George Neville-Neil , John Baldwin Subject: Re: kern/172113: [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4): m_getjcl: invalid cluster type Date: Fri, 27 Sep 2013 15:31:58 +0700 Hi! Audit-Trail of http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/172113 has more than enough patches to fix the problem. Would someone be so kind to commit one of them so 9.2-RELEASE won't be broken out-of-the-box for noted Supermicro's? ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Adding Flow Director sysctls to ixgbe(4) (was: netmap: traffic distribution)
On 27 September 2013 00:43, hiren panchasara wrote: > Takuya, > > I see a lot of responses/comments on proposed changes. Was anything decided > at the end of it? As far as I can tell, its still not committed to the > tree. > I'd rather see an ioctl API for that chipset and then have a separate tool program it for now. So, how bout we hack that up? :) -adrian ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Adding Flow Director sysctls to ixgbe(4) (was: netmap: traffic distribution)
On Thu, Sep 26, 2013 at 10:38 AM, hiren panchasara < hiren.panchas...@gmail.com> wrote: > > > > On Thu, Sep 26, 2013 at 12:58 AM, Michio Honda wrote: > >> Hi, >> >> The handiest way to try flexible flow distribution is using Flow Director. >> I've confirmed that the patch posted to this list two years ago works >> with netmap/ixgbe. >> >> http://freebsd.1045724.n5.nabble.com/Adding-Flow-Director-sysctls-to-ixgbe-4-td4769489.html >> > > Thanks a lot for the link, Michio! > > It seems this work is yet not committed?!? > Takuya, I see a lot of responses/comments on proposed changes. Was anything decided at the end of it? As far as I can tell, its still not committed to the tree. Thanks, Hiren ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
