Re: cxgbe and netmap
On 02.01.2015 19:53, Luigi Rizzo wrote: On Fri, Jan 02, 2015 at 06:57:50PM +0300, Alexander V. Chernikov wrote: Hello list! FreeBSD has netmap support for chelsio T5 cards, which is amazing. The great thing about implementation is that you can play with traffic-generating applications without affecting main OS interface, which has always been a problem for Intel cards. However, this approach (having additional netmap-only ifp) turns to be a bit problematic for netmap-based networking elements participating in routing. In Intel case you can configure all your interfaces, run routing daemon, run netmap application and punt all to-host traffic to kernel via host pipes. for clarity, please call this host netmap PORT, not pipe, as the latter can be confused with dummynet pipes. Ok, understood. It looks like I can't do this using current implementation: mac addresses are different for main/netmap interfaces so I can't run routing daemon on main interface (or sub-interfaces). I also can't run routing daemon on top of ncxgbe* interface since it appears to ignore non-netmap-derived traffic.. Maybe navdeep did not implement the host side for ncxgbe ? This should be a relatively trivial thing to do. Otherwise, for the time being, you could try the following hack: - create a tap interface, say tap*, and give it the same MAC as ncxgbe*. You will use only the host port for tap* - open both tap* and ncxgbe* in netmap mode, make sure to set the same flags, mode (promisc etc.), mtu on both; - run the routing daemon on top of tap* - use the tap*'s host netmap port to send up traffic coming from ncxgbe* directed to the host (and vice versa, inject netmap packets coming from tap*'s host netmap port into ncxgbe*'s netmap rings. Oh. vlans on top of tap inteface :) Ok, thanks for the suggestion, I'll try to do this. cheers luigi Is it possible to make ncxgbe* interfaces behave more like ordinary ones? ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: netmap pipes
On Sat, Jan 03, 2015 at 06:42:07PM +0100, Luigi Rizzo wrote:
On Sat, Jan 03, 2015 at 09:32:55PM +0400, Slawa Olhovchenkov wrote:
On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote:
On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote:
Can anybody explain netmap pipes (more then netmap(4))?
How use it?
How it works?
pipes works over existing network adapter in netmap mode?
Or indepened (can I create netmap pipe named 'some_strange_name')?
What purpose of master and slave?
think of pipes as regular netmap ports connected back to back,
(or as two ports on a VALE switch, if you like).
They are unrelated to network devices (though the name indicates
how they share memory, but forget that for the time being);
are created using the strange names like valeX:Y{0 and valeX:Y}0
(which I realize are not yet in the manpage, sorry);
can I use names other then valeX:Y? for example 'inside0'?
no, the basename has to be netmap:fooX (where foo is some existing
ethernet device) or valeX:Y
Can I transfer through netmap pipes packet with non-ethernet structure?
and you have master and slave because need to name both endpoints.
I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for
example), how I can use this? Writing to master replicated to all
slaves? Writing to any slaves reading from master? Or unidirected?
a pipe has only two endpoints.
What differens between master and slave?
Can I write to slave and read from master?
Can I first create slave second master?
If you want a full n-port switch use a VALE switch.
Hmm.
NIOCREGIF
binds the port named in nr_name to the file descriptor. For a phys-
ical device this also switches it into netmap mode, disconnecting
it from the host stack. Multiple file descriptors can be bound to
the same port, with proper synchronization left to the user.
NIOCREGIF can also bind a file descriptor to one endpoint of a
netmap pipe, consisting of two netmap ports with a crossover con-
nection. A netmap pipe share the same memory space of the parent
port, and is meant to enable configuration where a master process
acts as a dispatcher towards slave processes.
To enable this function, the nr_arg1 field of the structure can be
used as a hint to the kernel to indicate how many pipes we expect
to use, and reserve extra space in the memory region.
What this talk about?
I can create multiple pipe pairs over existing netmap interface?
What right way to use this?
NIOCREGIF(ix0, NR_REG_ONE_NIC, nr_arg1=16, nr_ringid=0)
NIOCREGIF(ix0, NR_REG_PIPE_MASTER, nr_ringid=0..15)
NIOCREGIF(ix0, NR_REG_PIPE_SLAVE, nr_ringid=0..15)
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: cxgbe and netmap
On 02.01.2015 21:46, Navdeep Parhar wrote: On Fri, Jan 02, 2015 at 06:57:50PM +0300, Alexander V. Chernikov wrote: Hello list! FreeBSD has netmap support for chelsio T5 cards, which is amazing. The great thing about implementation is that you can play with traffic-generating applications without affecting main OS interface, which has always been a problem for Intel cards. However, this approach (having additional netmap-only ifp) turns to be a bit problematic for netmap-based networking elements participating in routing. In Intel case you can configure all your interfaces, run routing daemon, run netmap application and punt all to-host traffic to kernel via host pipes. It looks like I can't do this using current implementation: mac addresses are different for main/netmap interfaces so I can't run routing daemon on main interface (or sub-interfaces). I also can't run routing daemon on top of ncxgbe* interface since it appears to ignore non-netmap-derived traffic.. Is it possible to make ncxgbe* interfaces behave more like ordinary ones? Yes, I need to write a simple transmit and receive handler for the non-netmap traffic on the ncxgbe/ncxl interfaces. This is a bit complicated because the normal rx runs in a mode where 1 rx buffer does not always equal 1 rx frame. Now that netmap is in GENERIC, it may be best to carve out a separate cxgbe_netmap module that can be loaded by those who want to use netmap on top of cxgbe/cxl hardware. So no more magic 'n' interfaces by default (some people were caught by surprise at the sudden appearance of the 'n' interfaces on HEAD), and fully functional 'n' interfaces as soon as you load the additional module. What do you and other netmap users think? I'm open to taking this Having loadable netmap support would be great - this approach looks much more flexible. driver's netmap support in whatever direction the users want it to go. Regards, Navdeep ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: Issue with forwarding when creates new interface [was USB Tethering and forwarding]
Hi, I can also replicate this behaviour on 10.1-RELEASE by simply creating an additional vlan interface. It affects IPv4 and IPv6 forwarding. This is taken from a test setup of FreeBSD boxes running Quagga as BGP routers - but with a default GENERIC kernel. This machine has 2x ixgbe, 4x igb and 2x bce physical interfaces, with a cloned lo1 and vlan0. [root@test1 ~]# uname -a FreeBSD test1.prtsystems.ltd.uk 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014 r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 1 [root@test1 ~]# ifconfig vlan1 create [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 0 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 0 I haven't tried using 10.0 as a router, so don't know if this crept in between 10.0 and 10.1 or 9 and 10. Paul. On 03/01/2015 13:12, wishmaster wrote: Hi, I have been seeing strange behavior of my system lately. After creating new interface the system variable net.inet.ip.forwarding becomes 0. E.g. manually load if_ral kernel module, then rel0 interface appears and net.inet.ip.forwarding becomes 0. Previously this happened when I attached smartphone with USB tethering is on. May be this is VIMAGE-related... Any ideas? Below my original first post. Hi, list. Server works as router for small network and some services in the jails. When I connect Android-based smartphone and attempt to use USB Tethering, the net.inet.ip.forwarding becomes 0 and I must change it to 1 every time. Is this normal behavior? FreeBSD server.local 10.1-STABLE FreeBSD 10.1-STABLE #1 r275636: Mon Dec 22 11:05:33 EET 2014 wishmaster@server.local:/usr/obj/usr/src/sys/SMS i386 Kernel has been compiled with VIMAGE Cheers, Vitaliy ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: netmap pipes
On Sat, Jan 03, 2015 at 09:32:55PM +0400, Slawa Olhovchenkov wrote:
On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote:
On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote:
Can anybody explain netmap pipes (more then netmap(4))?
How use it?
How it works?
pipes works over existing network adapter in netmap mode?
Or indepened (can I create netmap pipe named 'some_strange_name')?
What purpose of master and slave?
think of pipes as regular netmap ports connected back to back,
(or as two ports on a VALE switch, if you like).
They are unrelated to network devices (though the name indicates
how they share memory, but forget that for the time being);
are created using the strange names like valeX:Y{0 and valeX:Y}0
(which I realize are not yet in the manpage, sorry);
can I use names other then valeX:Y? for example 'inside0'?
no, the basename has to be netmap:fooX (where foo is some existing
ethernet device) or valeX:Y
and you have master and slave because need to name both endpoints.
I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for
example), how I can use this? Writing to master replicated to all
slaves? Writing to any slaves reading from master? Or unidirected?
a pipe has only two endpoints.
If you want a full n-port switch use a VALE switch.
cheers
luigi
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: Issue with forwarding when creates new interface [was USB Tethering and forwarding]
On 1/3/2015 9:19 AM, Paul Thornton wrote:
Hi,
I can also replicate this behaviour on 10.1-RELEASE by simply creating
an additional vlan interface. It affects IPv4 and IPv6 forwarding.
Strange, I dont see that on RELENG_10
0{marble}# ifconfig em2 up
0{marble}# ifconfig em2.3 create 1.1.1.2/24
0{marble}# sysctl -a | grep forwarding
net.inet.ip.forwarding: 1
net.inet.ip.fastforwarding: 0
net.inet6.ip6.forwarding: 0
0{marble}# ifconfig vlan4 create 2.2.2.2 vlan 4 vlandev em2
0{marble}# sysctl -a | grep forwarding
net.inet.ip.forwarding: 1
net.inet.ip.fastforwarding: 0
net.inet6.ip6.forwarding: 0
0{marble}#
do you set forwarding via just /etc/sysctl.conf or in /etc/rc.conf via
ipv6_gateway_enable and gateway_enable. I seem to recall some discussion
about there being a difference. Perhaps devd is calling something that
then fiddles with the setting ignoring whats in sysctl.conf ?
---Mike
--
---
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada http://www.tancsa.com/
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: netmap pipes
On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote:
On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote:
Can anybody explain netmap pipes (more then netmap(4))?
How use it?
How it works?
pipes works over existing network adapter in netmap mode?
Or indepened (can I create netmap pipe named 'some_strange_name')?
What purpose of master and slave?
think of pipes as regular netmap ports connected back to back,
(or as two ports on a VALE switch, if you like).
They are unrelated to network devices (though the name indicates
how they share memory, but forget that for the time being);
are created using the strange names like valeX:Y{0 and valeX:Y}0
(which I realize are not yet in the manpage, sorry);
can I use names other then valeX:Y? for example 'inside0'?
and you have master and slave because need to name both endpoints.
I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for
example), how I can use this? Writing to master replicated to all
slaves? Writing to any slaves reading from master? Or unidirected?
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Issue with forwarding when creates new interface [was USB Tethering and forwarding]
Hi, I have been seeing strange behavior of my system lately. After creating new interface the system variable net.inet.ip.forwarding becomes 0. E.g. manually load if_ral kernel module, then rel0 interface appears and net.inet.ip.forwarding becomes 0. Previously this happened when I attached smartphone with USB tethering is on. May be this is VIMAGE-related... Any ideas? Below my original first post. Hi, list. Server works as router for small network and some services in the jails. When I connect Android-based smartphone and attempt to use USB Tethering, the net.inet.ip.forwarding becomes 0 and I must change it to 1 every time. Is this normal behavior? FreeBSD server.local 10.1-STABLE FreeBSD 10.1-STABLE #1 r275636: Mon Dec 22 11:05:33 EET 2014 wishmaster@server.local:/usr/obj/usr/src/sys/SMS i386 Kernel has been compiled with VIMAGE Cheers, Vitaliy ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: netmap pipes
On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote:
Can anybody explain netmap pipes (more then netmap(4))?
How use it?
How it works?
pipes works over existing network adapter in netmap mode?
Or indepened (can I create netmap pipe named 'some_strange_name')?
What purpose of master and slave?
think of pipes as regular netmap ports connected back to back,
(or as two ports on a VALE switch, if you like).
They are unrelated to network devices (though the name indicates
how they share memory, but forget that for the time being);
are created using the strange names like valeX:Y{0 and valeX:Y}0
(which I realize are not yet in the manpage, sorry);
and you have master and slave because need to name both endpoints.
cheers
luigi
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
netmap pipes
Can anybody explain netmap pipes (more then netmap(4))? How use it? How it works? pipes works over existing network adapter in netmap mode? Or indepened (can I create netmap pipe named 'some_strange_name')? What purpose of master and slave? ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: host pipes and netmap 'emulated mode'
On 02.01.2015 19:45, Luigi Rizzo wrote: On Fri, Jan 02, 2015 at 06:36:00PM +0300, Alexander V. Chernikov wrote: Hello list. It looks like it is impossible to use host pipes and emulated netmap mode in some cases. For example, if you're doing something like what traditional router do: packet processing, with kernel-visible logical interfaces, routing daemon running there, you can easily get a panic like this: #0 0x8094aa76 at kdb_backtrace+0x66 #1 0x809104ee at panic+0x1ce #2 0x80cf9660 at trap_fatal+0x290 #3 0x80cf99c1 at trap_pfault+0x211 #4 0x80cf9f89 at trap+0x329 #5 0x80ce30d3 at calltrap+0x8 #6 0x809d3b5f at ether_demux+0x6f #7 0x809d3f34 at ether_nh_input+0x204 #8 0x809dd6d8 at netisr_dispatch_src+0x218 #9 0x8061b2b5 at netmap_send_up+0x35 #10 0x8061b3d7 at netmap_txsync_to_host+0x97 #11 0x8061b400 at netmap_txsync_to_host_compat+0x10 #12 0x8061de8c at netmap_poll+0x2fc #13 0x807f2313 at devfs_poll_f+0x63 #14 0x8095ea3d at sys_poll+0x35d #15 0x80cf8e0a at amd64_syscall+0x5ea #16 0x80ce33b7 at Xfast_syscall+0xf7 Uptime: 4m21s The problem here is the following: netmap changes if_input() for the logical network interface and always assumes that generic_rx_handler() is called with netmap-enabled ifp (e.g. original inteface). Unfortunately, there are cases where we have different ifp passed to if_input handler. This particular case is triggered by (*ifp-if_input)(ifv-ifv_ifp, m); line, where ifp represents netmap-enabled NIC, and ifv-ifv_ifp represents vlan subinterface. Then, generic_rx_handler() tries to looking NA/GNA structure but fails since vlan subinterface is not netmap-enabled. So, it looks like that we need a way to call original if_input() but I can't imagine (good) one/ Surely we can put a check in generic_rx_handler() to make sure that NA(ifp) is NULL -- this is already a relatively expensive code path so the extra checks won't harm. Yes, but it would be great if we can recover/call original input procedure instead of silently dropping frame But I am a bit unclear on how you trigger this error, can you give me more details ? The offending instruction (*ifp-if_input)(ifv-ifv_ifp, m) is in vlan_input(), so it looks like you are setting the parent interface in netmap mode, and (looking at the trace) sending packets to the host port. Yes. Basically, this is netmap router case - we configure vlan interface, bridge interfaces, etc using kernel interfaces, and propagate some (or all) configuration to netmap application. It somehow processes fast path traffic and sends control plane traffic to host via host pipe to handle routing daemon updates, icmp, fragments, etc.. So i suppose the error path is when netmap_send_up() calls the original input handler, NA(ifp)-if_input [i should rename the field to something else] which is vlan_input(). I think a proper fix is to make vlan_input() netmap aware, and call NA(ifp)-if_input if the interface is in netmap mode. Well, than we will have to make bridge code netmap aware, netgraph and tunnel after that. It seems this is pretty invasive way of doing things. Otherwise, if vlan_input() is the only case where if_input() is called with a different ifp, _and_ the vlan (child) interface has a reference to the parent (ifp-parent, though i don't know where this is), we can tweak generic_rx_handler() so that it calls NA(ifp-parent)-if_input in case NA(ifp) is null. We will have to add different code for all type of virtual interfaces, which seems to be better approach. There is also an option like having per-vnet ifindex-based array with vlan (and other objects) tracking to get ifp quickly. What we really lacks here is set_if_input_func() method which can call some eventhandler so it can be done more generic way (so, adding glebius@ here) cheers luigi ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: netmap pipes
On Sat, Jan 03, 2015 at 09:57:21PM +0400, Slawa Olhovchenkov wrote:
On Sat, Jan 03, 2015 at 06:42:07PM +0100, Luigi Rizzo wrote:
On Sat, Jan 03, 2015 at 09:32:55PM +0400, Slawa Olhovchenkov wrote:
On Sat, Jan 03, 2015 at 06:21:39PM +0100, Luigi Rizzo wrote:
On Sat, Jan 03, 2015 at 08:40:35PM +0400, Slawa Olhovchenkov wrote:
Can anybody explain netmap pipes (more then netmap(4))?
How use it?
How it works?
pipes works over existing network adapter in netmap mode?
Or indepened (can I create netmap pipe named 'some_strange_name')?
What purpose of master and slave?
think of pipes as regular netmap ports connected back to back,
(or as two ports on a VALE switch, if you like).
They are unrelated to network devices (though the name indicates
how they share memory, but forget that for the time being);
are created using the strange names like valeX:Y{0 and valeX:Y}0
(which I realize are not yet in the manpage, sorry);
can I use names other then valeX:Y? for example 'inside0'?
no, the basename has to be netmap:fooX (where foo is some existing
ethernet device) or valeX:Y
Can I transfer through netmap pipes packet with non-ethernet structure?
yes. A pipe will not look at the packet's headers or content.
and you have master and slave because need to name both endpoints.
I create pipe with many slaves (NIOCREGIF with nr_arg1=16, for
example), how I can use this? Writing to master replicated to all
slaves? Writing to any slaves reading from master? Or unidirected?
a pipe has only two endpoints.
What differens between master and slave?
there is no difference between master and slave,
you just need two names and a way to relate them.
Can I write to slave and read from master?
pipes are bidirectional and blocking. So you can write
on one and read from the other, in any order.
Can I first create slave second master?
You can create the endpoints in any order (internally,
they are both created at the same time).
If you want a full n-port switch use a VALE switch.
Hmm.
NIOCREGIF
binds the port named in nr_name to the file descriptor. For a phys-
ical device this also switches it into netmap mode, disconnecting
it from the host stack. Multiple file descriptors can be bound to
the same port, with proper synchronization left to the user.
NIOCREGIF can also bind a file descriptor to one endpoint of a
netmap pipe, consisting of two netmap ports with a crossover con-
nection. A netmap pipe share the same memory space of the parent
port, and is meant to enable configuration where a master process
acts as a dispatcher towards slave processes.
To enable this function, the nr_arg1 field of the structure can be
used as a hint to the kernel to indicate how many pipes we expect
to use, and reserve extra space in the memory region.
What this talk about?
This is a 'power user' feature which maybe is not what you need
(and at the moment I don't have time to explain in more detail
or update the manpage).
Pipes share memory with the netmap port (VALE port or NIC) with the
same basename, and since memory allocation occurs at once, on the
first open you need to tell the OS how many pipes need to share
memory with the same port -- that is the role of nr_arg1.
cheers
luigi
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
Re: Issue with forwarding when creates new interface [was USB Tethering and forwarding]
Hi, On 03/01/2015 18:06, Mike Tancsa wrote: do you set forwarding via just /etc/sysctl.conf or in /etc/rc.conf via ipv6_gateway_enable and gateway_enable. I seem to recall some discussion about there being a difference. Perhaps devd is calling something that then fiddles with the setting ignoring whats in sysctl.conf ? That seems to be what is happening. In the earlier post, I was just setting the three sysctls in /etc/sysctl.conf - and observed that forwarding went away if an interface was added. Doing it by setting fastforwarding only in sysctl.conf, and setting both gateway_enables to yes in rc.conf fixes the problem: [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 1 [root@test1 ~]# ifconfig vlan1 create [root@test1 ~]# sysctl -a | grep forwarding net.inet.ip.forwarding: 1 net.inet.ip.fastforwarding: 1 net.inet6.ip6.forwarding: 1 That's quite ... odd, to sat the least. I can't see anything in devd.conf which would relate to a new interface being created, but that doesn't mean that there isn't some magic functionality in there. Paul. ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org
