Re: [ovs-discuss] IPsec error
On Thu, Apr 15, 2021 at 4:00 PM Mariana Cruz wrote: > > Yes, probably I made that mistake because I remember using that command, > probably because my first try didn’t worked. I used this tutorial > (https://docs.openvswitch.org/en/latest/tutorials/ipsec/) I think it didn’t > worked but I can try again... Then I tried doing something else, that was > when I probably ran that command. > Can I follow the steps in Fedora of that tutorial or if not can you please > help me and tell me what and how is the right way to install OVS and IPSec in > CentOS 8? Like a link with this information? > I would recommend to redo everything from scratch and make sure ./configure is invoked consistently. > Thank you, > Mariana Cruz > > On 15 Apr 2021, at 21:46, Ansis wrote: > > On Thu, Apr 15, 2021 at 3:29 PM Mariana Cruz > wrote: > > > Hello, > > > Thank you for your reply. Unfortunately my screen is cut but i managed > > to see what was on the other half of the error message (see attach). > > > Have you mixed and matched pieces of OVS that were built with > ./configure --prefix=/usr/local and ./configure --prefix=/? > > This may have happened accidentally if you tried to build some bits > yourself and installed other from Centos repo. > > > > > Waiting for your reply. > > > > Thanks in advance, > > > Mariana Cruz > > > On 15/04/21 20:10, Ansis wrote: > > On Thu, Apr 15, 2021 at 1:37 PM Mariana Cruz > > wrote: > > Hello, > > > I am trying to install and start the openvswitch-ipsec.service in my > > CentOS 8, but it generates some errors (see attach file), so the IPsec > > tunnel doesn't work. > > > Can you please help? > > Can you post the full line of "No such file or directory: > > '/usr/local"? It was truncated in your screenshot. > > Thanks a lot. > > > Mariana Cruz > > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] IPsec error
On Thu, Apr 15, 2021 at 3:29 PM Mariana Cruz wrote: > > Hello, > > Thank you for your reply. Unfortunately my screen is cut but i managed > to see what was on the other half of the error message (see attach). Have you mixed and matched pieces of OVS that were built with ./configure --prefix=/usr/local and ./configure --prefix=/? This may have happened accidentally if you tried to build some bits yourself and installed other from Centos repo. > > Waiting for your reply. > > > Thanks in advance, > > Mariana Cruz > > On 15/04/21 20:10, Ansis wrote: > > On Thu, Apr 15, 2021 at 1:37 PM Mariana Cruz > > wrote: > >> Hello, > >> > >> I am trying to install and start the openvswitch-ipsec.service in my > >> CentOS 8, but it generates some errors (see attach file), so the IPsec > >> tunnel doesn't work. > >> > >> Can you please help? > > Can you post the full line of "No such file or directory: > > '/usr/local"? It was truncated in your screenshot. > >> Thanks a lot. > >> > >> Mariana Cruz > >> > >> ___ > >> discuss mailing list > >> disc...@openvswitch.org > >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] IPsec error
On Thu, Apr 15, 2021 at 1:37 PM Mariana Cruz wrote: > > Hello, > > I am trying to install and start the openvswitch-ipsec.service in my > CentOS 8, but it generates some errors (see attach file), so the IPsec > tunnel doesn't work. > > Can you please help? Can you post the full line of "No such file or directory: '/usr/local"? It was truncated in your screenshot. > > Thanks a lot. > > Mariana Cruz > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] IPsec tunnel - swordfish issue
On Mon, Apr 6, 2020 at 1:22 PM Majcher Wojciech (STUD)
wrote:
>
> Hi,
>
> I've tried to establish ipsec tunnel according to OvS IPsec tutorial. On one
> side of the tunnel i use Fedora 31 OS and StrongSwan IKE daemon.
>
> I am getting strongswan service error:
>
> strongswan.service - strongSwan IPsec IKEv1/IKEv2 daemon using ipsec.conf
>Loaded: loaded (/usr/lib/systemd/system/strongswan.service; disabled;
> vendor preset: disabled)
>Active: inactive (dead)
>
> Apr 06 20:19:49 fedora.wojtek strongswan[3177]: 00[CFG]
> /etc/strongswan/strongswan.d/charon.conf:4: syntax error, unexpected .,
> expecting : or '{' or '=' [.]
> Apr 06 20:19:49 fedora.wojtek strongswan[3177]: 00[CFG] invalid config file
> '/etc/strongswan/strongswan.conf'
> Apr 06 20:19:49 fedora.wojtek strongswan[3177]: 00[LIB] abort initialization
> due to invalid configuration
> Apr 06 20:19:49 fedora.wojtek strongswan[3177]: charon has quit: integrity
> test of libstrongswan failed
> Apr 06 20:19:49 fedora.wojtek ipsec_starter[3177]: charon has quit: integrity
> test of libstrongswan failed
> Apr 06 20:19:49 fedora.wojtek strongswan[3177]: charon refused to be started
> Apr 06 20:19:49 fedora.wojtek ipsec_starter[3177]: charon refused to be
> started
> Apr 06 20:19:49 fedora.wojtek strongswan[3177]: ipsec starter stopped
> Apr 06 20:19:49 fedora.wojtek ipsec_starter[3177]: ipsec starter stopped
> Apr 06 20:19:49 fedora.wojtek systemd[1]: strongswan.service: Succeeded.
>
>
> charon.conf:
>
> # Generated by ovs-monitor-ipsec...do not modify by hand!
>
>
> charon.plugins.kernel-netlink.set_proto_port_transport_sa = yes
Is the line line #4 that is causing the issue the one above?
If yes, then I am wondering if that option has been removed
set_proto_port_transport_sa option in later versions. Can you simply
remove it and reload strongswan with "ipsec restart" to see if the
issue went away?
> charon.plugins.kernel-netlink.xfrm_ack_expires = 10
> charon.load_modular = yes
> charon.plugins.gcm.load = yes
>
> strongswan.conf:
>
> # strongswan.conf - strongSwan configuration file
> #
> # Refer to the strongswan.conf(5) manpage for details
> #
> # Configuration changes should be made in the included files
>
> charon {
> load_modular = yes
> plugins {
> include strongswan.d/charon/*.conf
> }
> }
>
> include strongswan.d/*.conf
>
>
> OvS:
>
> openvswitch-ipsec.x86_64
>2.12.0-1.fc31
> openvswitch.x86_64
> 2.12.0-1.fc31
>
> StrongSwan:
>
> strongswan.x86_64
>5.7.2-3.fc31
>
> Is it the StrongSwan service issue ? The tutorial is for fedora 27 and
> StrongSwan (>= v5.3.5).
>
> Best Regards,
> Wojtek
>
>
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVS-IPsec - Network going down.
On Tue, 26 Nov 2019 at 04:24, Rajak, Vishal wrote: > > Hi, > > > > We are trying to bring up IPsec over vxlan between two nodes of openstack > cluster in our lab environment. Since what you described below does *much more* than set up trivial ipsec vxlan tunnel setup, then: 1. have you tried to set up same thing with plain vxlan setup and succeed? I think you are jumping too early in conclusions that you have IPsec problem here. 2. have you tried to set up simple ipsec vxlan and succeed without setting up OpenFlow controller, fail-mode=safe and patch ports? A simple IPsec set up really does not need that. 3. Also, did you check ovs-monitor-ipsec.log and libreswan files for errors? > > > > Note: There are only 2 nodes in cluster(Compute and controller). > > Following are the steps followed to bring-up ovs with ipsec > > Link: http://docs.openvswitch.org/en/latest/tutorials/ipsec/ > > Commands used on Controller node: (IP – 10.2.2.1) > > a. dnf install python2-openvswitch libreswan \ > "kernel-devel-uname-r == $(uname -r)" > > b. yum install python-openvswitch - to install > python-openvswitch-2.11.0-4.el7.x86_64 as it has support for ipsec. > > c. Download openvswitch 2.11 version rpms and put it in on the server > d. Install openvswitch rpms in the server > > ex- rpm -ivh openvswitch-ipsec-2.11.0-4.el7.x86_64.rpm -- to install > ovs-ipsec rpm > > e. iptables -A INPUT -p esp -j ACCEPT > f. iptables -A INPUT -p udp --dport 500 -j ACCEPT Do you have firewalld running? If so then using iptables is not the right way to ACCEPT traffic. You have to use firewalld-cmd. > > g. cp -r /usr/share/openvswitch/ /usr/local/share/ I don't understand why you have to do the command above. Are you mixing self built packages with official packages that have different path prefixes? > h. systemctl start openvswitch-ipsec.service > > I. ovs-vsctl add-port br-ex ipsec_vxlan -- set interface ipsec_vxlan > type=vxlan options:remote_ip=10.2.2.2 options:psk=swordfish > > > > Commands used on compute node: (IP -10.2.2.2) > > Link: https://devinpractice.com/2016/10/18/open-vswitch-introduction-part-1/ > > a. ovs-vsctl add-br br-ex > > b. ip link set br-ex up > > c. ovs-vsctl add-port br-ex enp1s0f1 > > d. ip addr del 10.2.2.2/24 dev enp1s0f1 > > e. ip addr add 10.2.2.2/24 dev br-ex > > f. ip route add default via 10.2.2.254 dev br-ex > > g. Same step as done for controller node above for ipsec configuration. > > After bringing up ipsec in compute node the connectivity for all 10.2.2.0 > network went down. Commands a-g don't have anything to do with IPsec. I see how network connectivity could go down there at the time you move IP address from enp1s0f1 to br-ex. > > Following are the steps followed to resolve the issue of 10.2.2.0.network > down due to creation of bridge in compute node > > Replicated the output of ovs-vsctl show in compute node by comparing the > output of controller node by executing following commands > > 1. ovs-vsctl set-controller br-ex tcp:127.0.0.1:6633 > > 2. ovs-vsctl – set Bridge br-ex fail-mode=secure > > After running the above 2 commands the network connectivity to outside > network from compute node went down and other servers came up. > > 3. ovs-vsctl add-port br-ex phy-br-ex – set interface phy-br-ex type=patch > options:peer=int-br-ex > > 4. ovs-vsctl add-port br-int int-br-ex – set interface int-br-ex > type=patch options:peer=phy-br-ex Is phy-br-ex the physical bridge and int-br-ex the integration bridge? If so then it seems odd that you are trying to connect them with patch port and use IPsec (or for that matter even plain) tunneling. > > After running the above 2 commands also the compute node was > not reachable to outside network. > > Compared the files in network-script in both compute node and controller node > and found some difference. > > Compute node didn’t had ifcfg-br-ex file, so added it in compute node. > Made some changes in ifcfg-enp1s0f1 file after comparing it with the same > file present in controller node. > > d. Restarted the network service. > > e. After restarting the network service the changes which was made in > ovs-vsctl was removed and only the bridge br-ex which was created on physical > interface remained. > > f. The compute node started ping the outer network as well. > > g. Ran command to establish ipsec-vsxlan. > > ovs-vsctl add-port br-ex ipsec_vxlan -- set interface ipsec_vxlan > type=vxlan options:remote_ip=10.2.2.1 options:psk=swordfish > > h. After port was added for ipsec-vxlan the network again went down. > > I. Removed the ipsec-vxlan port. > > j. Now the compute node has bridge over physical interface and its > pinging to outside network as well. > > k. Tried pinging from VM in compute node to VM in controlled node. Ping > didn’t work. > >1. Removed the VM form compute node and tried creating ano
Re: [ovs-discuss] Issue porting openvswitch-ipsec on XCP-ng
On Mon, 9 Sep 2019 at 02:36, Benjamin wrote: > > Hello everyone, > > I'm Benjamin, a french developer working at Vates (the editor of XCP-ng > a XenServer fork). > I've been working in the network area of XCP-ng in order to create a SDN > Controller controlling openvswitch on several hosts. > > Everything is working great as for now! > > I am using openvswitch v2.11.0. > However I'm trying to add IPSEC support into XCP-ng and I'm facing an issue. > > I've successfully installed libreswan version 3.26, and the > openvswitch-ipsec service from rhel and the python script ovs-monitor-ipsec. > I'm using Pre-Shared Key for IPSEC. > > When I attempt to create tunnels, everything seems to go smoothly: > - there's no error in ovs-vswitchd.log nor in ovs-monitor-ipsec.log > - ovs-appctl -t ovs-monitor-ipsec tunnels/show shows me the tunnels with > correct configurations and active connections. Share your ovs-appctl output here. > > But there's no traffic passing on the tunnels created by openvswitch and > since there's no helpful log I don't know how to investigate the issue. > I hoped you could point me in the right direction. Did the plain tunnel work in your setup? E.g. if you are using geneve with ipsec simply try plain geneve. > > Here's what appears in ovs-vswitchd.log after tunnels creation: > > 2019-09-09T08:16:49.311Z|00018|tunnel(handler7)|WARN|receive tunnel port > not found > (pkt_mark=0x1,udp,tun_id=0x3,tun_src=192.168.5.28,tun_dst=192.168.5.27,tun_ipv6_src=::,tun_ipv6_dst=::,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=0,tun_ttl=64,tun_flags=key,in_port=4,vlan_tci=0x,dl_src=b2:bc:3c:29:bd:fd,dl_dst=ff:ff:ff:ff:ff:ff,nw_src=0.0.0.0,nw_dst=255.255.255.255,nw_tos=16,nw_ecn=0,nw_ttl=128,tp_src=68,tp_dst=67) > 2019-09-09T08:16:49.311Z|00019|ofproto_dpif_upcall(handler7)|INFO|Dropped > 1 log messages in last 214 seconds (most recently, 214 seconds ago) due > to excessive rate > 2019-09-09T08:16:49.311Z|00020|ofproto_dpif_upcall(handler7)|INFO|received > packet on unassociated datapath port 4 > 2019-09-09T08:16:49.914Z|3|tunnel(revalidator6)|WARN|receive tunnel > port not found > (pkt_mark=0x1,udp,tun_id=0x3,tun_src=192.168.5.28,tun_dst=192.168.5.27,tun_ipv6_src=::,tun_ipv6_dst=::,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=0,tun_ttl=64,tun_flags=key,in_port=4,vlan_tci=0x,dl_src=b2:bc:3c:29:bd:fd,dl_dst=ff:ff:ff:ff:ff:ff,nw_src=0.0.0.0,nw_dst=255.255.255.255,nw_tos=16,nw_ecn=0,nw_ttl=128,tp_src=68,tp_dst=67) > > There's plenty of errors like this after the tunnels are created and I > attempt to ping through the tunnels. > > Does that ring a bell to anyone? IIRC, I have seen that "receive tunnel port not found" in following cases: 1. skb mark not being set and ovs user space is confused which tunnel is that (this is specific to IPsec). 2. openvswitch kernel module does not support particular tunnel flavor even in plain (this is not specific to IPsec). Probably checking ofport value for tunnel in OVSDB's Interface table would help to pinpoint which case it is. > > Do not hesitate to ask me anything that can help debug this issue. > > Thank you, > Benjamin Reis > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Multiple IPSEC bridges
On Fri, 13 Sep 2019 at 01:26, Benjamin wrote: > > Hello, > > Is it possible to have multiple bridges using IPSEC/GRE tunnels with > same configuration? IIRC, it is not possible to create multiple IPsec tunnels of same flavor (in your case GRE) between the same two endpoints. This limitation kinda comes from Linux IPsec stack. While ip-xfrm man page mentions that it is possible for IPsec stack to match on GRE key, there is not way to match on Geneve, VXLAN, STT and other protocols the same way. So since we could not implement this in uniform manner across all transport protocols, then we did not bother to implement that for GRE either. > For now, creating one works fine but as soon as I create a 2nd none > works, there's no active connections and no error in logs. > I'm using `options:key` to allow having multiple GRE tunnels with same > configuration. > > Thanks > Benjamin Reis > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Ipsec tunnel is not encrypted
On Fri, 5 Jul 2019 at 12:26, wrote:
>
> I try to create a Ipsec tunnel between 2 hosts. The tunnel was create
> and i can communicate between hosts. But, when i capture packets using
> tcpdump, i see that the traffic is not encrypted.
>
> My topology:
>
> +--+ +--+
> | vm0 | 10.250.204.11/24| vm1 |
> 10.250.204.21/24
> +--+ +--+
> (vm_port0) (vm_port0)
> | |
> | |
> | |
> | |
> 10.250.204.10/24 10.250.204.20/24
> +--+ +--+
> |remibr0| |remibr0|
> +--+ +---+
> | eth1 |--| eth1 |
> +--+ +---+
> 10.16.0.138/16 10.16.0.247/16
>
> The commands that i run:
>
> ovs-vsctl add-br remibr0
> ovs-vsctl add-port remibr0 vxlan0 -- set Interface vxlan0 type=vxlan
> options:remote_ip=10.16.0.247 options:psk=test123
> ovs-vsctl add-port remibr0 vi0 -- set Interface vi0 type=internal
> ifconfig vi0 10.250.204.20/24 up
>
> My ovs-vsctl show:
>
> Bridge "remibr0"
> Port "vxlan0"
> Interface "vxlan0"
> type: vxlan
> options: {key="test123", remote_ip="10.16.0.247"}
As Ben pointed out, use OVS 2.11 or later (`ps -Af | grep
ovs-monitor-ipsec` is the ultimate test to see if you have the OVS's
IPsec daemon running. Without it, IPsec integration will not work).
Also, `ovs-vsctl show` command does not correctly represent the output
you should see after executing the commands you mentioned few lines
higher - options:psk is not set but the options:key is set. You must
set options:psk for IPsec.
> Port "sw1-p1"
> Interface "sw1-p1"
> Port "remibr0"
> Interface "remibr0"
> type: internal
> ovs_version: "2.10.1"
>
> Someone knows if i messed up in some steep or i'm confused about concepts?
>
> Thanks!
>
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID changes
On Tue, 23 Apr 2019 at 09:30, Jaime Caamaño Ruiz wrote: > > Hello > > So the non root owned log directory (and run directory) is shared > between non root OVS processes and root OVN processes. Doesn't this > raise some security concerns? As a "security concern" you mean something among the lines where one of ovs-* processes running under openvswitch user would go ahead and create a file with its owner that later one of ovn processes would blindly reuse without checking that it actually belongs to root:root? > > Also, since logrotate rotates the logs with the OVS user, it may fail > to some extent to rotate the root owned OVN log files. Consequences > vary but in it's current state some logging might be lost. This could > be improved but I would guess that the approprioate thing to do is to > etiher use a different log directory for OVN or make its processes run > with the OVS user also. Has any of this been considered? Can you give a more concrete example? I believe logrotate is running under root and should be able to rotate everything? > > BR > Jaime. > > > -Original Message- > From: Jaime Caamaño Ruiz > Reply-to: jcaam...@suse.com > To: Ansis Atteka > Cc: ovs-discuss@openvswitch.org, Ben Pfaff > Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID > changes > Date: Wed, 17 Apr 2019 12:52:32 +0200 > > > You also need to chown /var/log/openvswitch.*.log files. > > OVS seems to be already handling this. I dont know the details but I > guess that before dropping capabilities, OVS chowns these by itself. > > > However, > > what about other daemons, like ovn? Do they share run time dir? > > Haven't looked into this in a while and don't have a setup to check > > myself. > > The permisions of the openvswitch run directory are already being > handled by the service unit file. There is read access to files created > there by OVS and AFAIK what it ultimately makes it a no problem for OVN > is that it still runs as root with the provided unit files. If anyone > changes this to a user different than the openvswitch user, then that > is an already existing issue. Am I missing something here? > > Agree with you other considerations. > > BR > Jaime. > > -Original Message- > From: Ansis Atteka > To: jcaam...@suse.de > Cc: ovs-discuss@openvswitch.org, Ben Pfaff > Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID > changes > Date: Tue, 16 Apr 2019 18:21:48 -0700 > > On Tue, 16 Apr 2019 at 17:20, Jaime Caamaño Ruiz > wrote: > > > > My intention was doing it at systemd unit (prestart) for file conf.db > > (and .conf.db.~lock~ that stays around) only. > > You also need to chown /var/log/openvswitch.*.log files. > > > > I was not thinking on absolutely fool proof mechanism, among other > > things because the admin might have customized the location for the > > database. Also, updating and restarting the service are things that > > would usually be monitored. So best effort. > > make sure that best effort solution informs admin in case of error so > that he does not have to go file by file to find the offending one. > > > > > At systemd unit prestart the service is stopped and OVS nor nobody > > should really be messing with the database files owned by OVS. If the > > spec file is changing things back to root unappropriately then that > > should be changed too. > > Yes. > > > > > The runtime /run/openvswitch directory is also controlled by systemd > > and at least on my system is removed if the service is not active, > > either by graceful or ungraceful exit. > > I somewhat agree with this, because back in those days we were using > SystemV where the init.d managed lifecycle of run directory.. However, > what about other daemons, like ovn? Do they share run time dir? > Haven't looked into this in a while and don't have a setup to check > myself. > > > > > I guess we will need to look for specific issues. Any hint to find > > that > > patch? > > 1. test upgrade to your patched OVS and also from your patched OVS to > another future version > 2. if you change existing spec files that are shared with on Fedora or > RHEL/CentOS make sure you don't break anything for them. > 3. prefer consistency w.r.t. default behavior across deb and rpm > packages (if you change user by default consider to do that for all > platforms) > 4. besides database you will also need to set right access bits to log > files. On my Ubuntu it is "-rw-r- 1 root adm " which means: > 4.1. you have to chown it; OR > 4.2. add openvswitch
Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID changes
On Tue, 16 Apr 2019 at 17:20, Jaime Caamaño Ruiz wrote: > > My intention was doing it at systemd unit (prestart) for file conf.db > (and .conf.db.~lock~ that stays around) only. You also need to chown /var/log/openvswitch.*.log files. > > I was not thinking on absolutely fool proof mechanism, among other > things because the admin might have customized the location for the > database. Also, updating and restarting the service are things that > would usually be monitored. So best effort. make sure that best effort solution informs admin in case of error so that he does not have to go file by file to find the offending one. > > At systemd unit prestart the service is stopped and OVS nor nobody > should really be messing with the database files owned by OVS. If the > spec file is changing things back to root unappropriately then that > should be changed too. Yes. > > The runtime /run/openvswitch directory is also controlled by systemd > and at least on my system is removed if the service is not active, > either by graceful or ungraceful exit. I somewhat agree with this, because back in those days we were using SystemV where the init.d managed lifecycle of run directory.. However, what about other daemons, like ovn? Do they share run time dir? Haven't looked into this in a while and don't have a setup to check myself. > > I guess we will need to look for specific issues. Any hint to find that > patch? 1. test upgrade to your patched OVS and also from your patched OVS to another future version 2. if you change existing spec files that are shared with on Fedora or RHEL/CentOS make sure you don't break anything for them. 3. prefer consistency w.r.t. default behavior across deb and rpm packages (if you change user by default consider to do that for all platforms) 4. besides database you will also need to set right access bits to log files. On my Ubuntu it is "-rw-r- 1 root adm " which means: 4.1. you have to chown it; OR 4.2. add openvswitch user to adm group and chmod that file 5. test logrotation. Maybe you can use a one forced logrotation invocation to change the ownrer. 6. check that other stuff like ovs-monitor-ipsec is not affected. It stil need to talk with StrongSwan or libreswan that best to my knowledge still runs as root. 7. if you use USER= then: 7.1. For Linux datapath make sure you retain CAP_NET_ADMIN Linux Capabilities 7.2. I don't know if there are special privileges you need to retain for DPDK. And in future possibly AF_XDP. 8. I haven't looked in a while but check if ovn (that has its own Systemd service file) shares run dir with OVS > > BR > Jaime. > > -Original Message- > From: Ansis Atteka > To: jcaam...@suse.de > Cc: ovs-discuss@openvswitch.org, Ben Pfaff > Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID > changes > Date: Tue, 16 Apr 2019 16:11:44 -0700 > > On Tue, 16 Apr 2019 at 15:36, Jaime Caamaño Ruiz > wrote: > > > > > On any given system, I would expect ovsdb-server to run as the same > > > user > > > every time. Is there a reason to sometimes use a different user? > > > > Well, changing from root which was the only supported option in the > > past and current default to a different user (like the suggested > > openvswitch), this one time, might be somewhat common. And the > > scenario > > that I am looking at is suporting a pianless upgrade through it. > > Actually debian packages still by default use "root" user. Not > "openvswitch". > > Long time ago there was an attempt to change the default user across > all different flavor packages to openvswitch (you can lookup the patch > on mailing list). However, as you noticed upgrades are tricky. Hence > packages built from our debian/rules and rhel/openvswitch.spec.in > files still use "root" as default user. The packages built with > rhel/openvswitch-fedora.spec.in are an exception where the user indeed > is "openvswitch". Unless you passed --without libcapng flag to > rpmbuild invocation. Then the user would still be "root". > > Here are the difficulties with automating the change of file ownership: > 1. from which context to change ownership? As chown must be invoked > from something that runs under root then there is not much choice - > either package installation time (%post scriptlet) or daemon startup > time (assuming you are not using Systemd's USER= feature in spec file) > or something that admin explicitly has to invoke from bash as "root". > 1.1. if it is package installation time then you can't update user > gracefully at daemon init time. It will be possible only at > installation time. > 1.2. if it is installation time then you can run in
Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID changes
On Tue, 16 Apr 2019 at 16:11, Ansis Atteka wrote: > > On Tue, 16 Apr 2019 at 15:36, Jaime Caamaño Ruiz wrote: > > > > > On any given system, I would expect ovsdb-server to run as the same > > > user > > > every time. Is there a reason to sometimes use a different user? > > > > Well, changing from root which was the only supported option in the > > past and current default to a different user (like the suggested > > openvswitch), this one time, might be somewhat common. And the scenario > > that I am looking at is suporting a pianless upgrade through it. > > Actually debian packages still by default use "root" user. Not "openvswitch". > > Long time ago there was an attempt to change the default user across > all different flavor packages to openvswitch (you can lookup the patch > on mailing list). However, as you noticed upgrades are tricky. Hence > packages built from our debian/rules and rhel/openvswitch.spec.in > files still use "root" as default user. The packages built with > rhel/openvswitch-fedora.spec.in are an exception where the user indeed > is "openvswitch". Unless you passed --without libcapng flag to > rpmbuild invocation. Then the user would still be "root". > > Here are the difficulties with automating the change of file ownership: > 1. from which context to change ownership? As chown must be invoked > from something that runs under root then there is not much choice - > either package installation time (%post scriptlet) or daemon startup > time (assuming you are not using Systemd's USER= feature in spec file) typo: I meant Systemd *service (not spec) file. Albeit I guess with *.service files when using USER= you can still specify Exec= and ExecStartPre= where one runs as confined user and the other runs as root that could chown() files. One step back .. I am wondering if user downgrade should rather be done through systemd and not with --user flags. Haven't looked into this too closely, but for some of my other non-OVS projects the Systemd user downgrade seems to be working just fine. > or something that admin explicitly has to invoke from bash as "root". > 1.1. if it is package installation time then you can't update user > gracefully at daemon init time. It will be possible only at > installation time. > 1.2. if it is installation time then you can run into weird race typo: instead of "installation time" I meant "daemon startup time" > conditions where some code that was executed under root (as package > %post scriptlets) or previous instance of OVS that was still running > as root could change ownership back to root for some files and cause > unexpected "permission denied" issues (the patch I mentioned was > affected with these rare race conditions) > 2. for which files to change ownership? What happens if Unix domain > socket file remained on fileystem with "root" permissions when OVS was > abruptly killed? What if someone else put a file under ovs directories > with non-root user - should you blindly chown that too? What if %post > scriptlet that runs under root non-atomically created a file and in > next line chown() it back - is there a window for race? Should you > follow directories/symlinks recursively? Changing user for conf.db > file is not enough... > > I am not saying it is impossible to change user gracefully on > upgrades. I am saying that code may have to be reorganized quite a bit > to eliminate the issues I raised above. > > Alternatively, for Centos, RHEL and Fedora one can use SElinux to > confine processes and leave them running as root (instead of running > them under limited privilege user). I believe on SUSE (and Debian) > once could use AppArmor to achieve the same results. At least for > SElinux it was a little bit easier to make upgrade seamless than with > Linux user confinement approach. > > > > > So if that makes sense to you, I will go ahead with the patch. > > > > BR > > Jaime. > > > > -Original Message- > > From: Ben Pfaff > > To: jcaam...@suse.de > > Cc: ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID > > changes > > Date: Tue, 16 Apr 2019 15:25:24 -0700 > > > > On Tue, Apr 16, 2019 at 11:32:21PM +0200, Jaime Caamaño Ruiz wrote: > > > When sysconfig OVS_USER_ID is changed to a different user, it > > > requires > > > a manual ownership change of the OVS conf.db database if existing or > > > otherwise ovsdb-server will fail to (re)start. I was wondering if I > > > am > > > missing any particular reason why this is change of ow
Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID changes
On Tue, 16 Apr 2019 at 15:36, Jaime Caamaño Ruiz wrote: > > > On any given system, I would expect ovsdb-server to run as the same > > user > > every time. Is there a reason to sometimes use a different user? > > Well, changing from root which was the only supported option in the > past and current default to a different user (like the suggested > openvswitch), this one time, might be somewhat common. And the scenario > that I am looking at is suporting a pianless upgrade through it. Actually debian packages still by default use "root" user. Not "openvswitch". Long time ago there was an attempt to change the default user across all different flavor packages to openvswitch (you can lookup the patch on mailing list). However, as you noticed upgrades are tricky. Hence packages built from our debian/rules and rhel/openvswitch.spec.in files still use "root" as default user. The packages built with rhel/openvswitch-fedora.spec.in are an exception where the user indeed is "openvswitch". Unless you passed --without libcapng flag to rpmbuild invocation. Then the user would still be "root". Here are the difficulties with automating the change of file ownership: 1. from which context to change ownership? As chown must be invoked from something that runs under root then there is not much choice - either package installation time (%post scriptlet) or daemon startup time (assuming you are not using Systemd's USER= feature in spec file) or something that admin explicitly has to invoke from bash as "root". 1.1. if it is package installation time then you can't update user gracefully at daemon init time. It will be possible only at installation time. 1.2. if it is installation time then you can run into weird race conditions where some code that was executed under root (as package %post scriptlets) or previous instance of OVS that was still running as root could change ownership back to root for some files and cause unexpected "permission denied" issues (the patch I mentioned was affected with these rare race conditions) 2. for which files to change ownership? What happens if Unix domain socket file remained on fileystem with "root" permissions when OVS was abruptly killed? What if someone else put a file under ovs directories with non-root user - should you blindly chown that too? What if %post scriptlet that runs under root non-atomically created a file and in next line chown() it back - is there a window for race? Should you follow directories/symlinks recursively? Changing user for conf.db file is not enough... I am not saying it is impossible to change user gracefully on upgrades. I am saying that code may have to be reorganized quite a bit to eliminate the issues I raised above. Alternatively, for Centos, RHEL and Fedora one can use SElinux to confine processes and leave them running as root (instead of running them under limited privilege user). I believe on SUSE (and Debian) once could use AppArmor to achieve the same results. At least for SElinux it was a little bit easier to make upgrade seamless than with Linux user confinement approach. > > So if that makes sense to you, I will go ahead with the patch. > > BR > Jaime. > > -Original Message- > From: Ben Pfaff > To: jcaam...@suse.de > Cc: ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID > changes > Date: Tue, 16 Apr 2019 15:25:24 -0700 > > On Tue, Apr 16, 2019 at 11:32:21PM +0200, Jaime Caamaño Ruiz wrote: > > When sysconfig OVS_USER_ID is changed to a different user, it > > requires > > a manual ownership change of the OVS conf.db database if existing or > > otherwise ovsdb-server will fail to (re)start. I was wondering if I > > am > > missing any particular reason why this is change of ownership is not > > automatically being handled through the service unit file as it is > > being done with other items. > > On any given system, I would expect ovsdb-server to run as the same > user > every time. Is there a reason to sometimes use a different user? > > Or, perhaps you are saying that, just before invoking ovsdb-server, the > service unit should chown (or whatever) the database file to the user > that it is going to use to invoke ovsdb-server. That might be > reasonable, but no one has thought to do it yet. Do you want to submit > a patch? > > Thanks, > > Ben. > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] ipsec_gre not working in OpenWRT
I am trying to enable ipsec_gre tunnelling between two openwrt. [ANSIS}:Is the iptables rule that sets skb_mark to 1 for incoming ESP packets in effect? Are you using ufw or firewall-cmd or some other iptables pipeline where following invocation "debian/openvswitch-ipsec.init:iptables -D INPUT -t mangle $1 -j MARK --set-mark 1/1 || return 0" does not work for your case? ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Understand GRE IPSEC tunnels in 2.5/2.6/2.7
On Thu, Feb 16, 2017 at 3:40 AM, Keith Holleman wrote: > > Ansis, > > Thanks for the quick reply. I have looked at your repository but am not > really at a point right now where I can move and try it out. Looks like you > are again proposing moving from Racoon to Strongswan which is a good thing > in my opinion. > > Since you didn't answer all of the questions directly, I'll relay what I > found by looking at your repo and let me know if I'm off base. As best I > can tell you have you proposed again to still use the mark but have given > the user control over which bit they can use. However, I see that mark used > in the scripts you have proposed but I see no changes to set that mark > to/from the tunnels in other code. Therefore, and I think you eluded to > this, it is up to the user to ensure that the mark/bit is set properly prior > to leaving OVS via a IPSEC-GRE tunnel port in order to be encrypted > properly. Is that right - nothing will automatically set this bit? Thanks, that is a bug and I will remove the iptables rule that currently installs skb mark. I think that it should be up to administrator to set ingress skb mark because otherwise ovs-monitor-ipsec would have to know how to do that with raw iptables, ufw and potentially also with firewalld. I missed this because to reduce development test cycle I was just just scp'īng ovs-monitor-ipsec daemon and started it directly. > > How about for incoming packets that have been decrypted? Does the XFRM > policy set the mark on those packets after decryption? Or is your what No XFRM never sets mark on its own. Though, it retains previously set mark. > your iptables rules in debian/openvswitch-ipsec.init is attempting to do? > That however still seem to be hardcoded to bit 1/1. And the GRE tunnel code > checks that the mark is set and therefore IPSEC-GRE tunnels only decap GRE > packets that have come through the IPSec tunnel? In other words, someone > can't just inject unencrypted GRE packets to the host and make it look like > they arrived on the tunnel? > > I still am not sure I have the whole path laid out clearly. You discovered a bug in my code hence probably the new implementation proposal was not fully clear yet. Thanks and I will update the code! > > -K > > On Sat, Feb 11, 2017 at 3:33 PM, Ansis Atteka wrote: >> >> On Fri, Feb 10, 2017 at 6:41 PM, Keith Holleman via discuss >> wrote: >> > >> > I am trying to understand a few things about the implementation of GRE >> > over >> > IPSec tunnels across some of the more recent OVS releases. As I >> > understand >> > it, 2.5 had support for it but in 2.6 the IPSec scripts (and effectively >> > the >> > support) were pulled out in 2.6 due to concerns around mark usage. I >> > have >> > also seen patches proposed that migrated from racoon to strongswan but I >> > don't think they were ever fully accepted. So I have three questions: >> > >> > 1) What is the plan for IPSec support in 2.7 and/or future releases? >> >> Here is my private ovs IPsec repository - >> https://github.com/ansisatteka/ovs-ipsec.git that I recently rebased >> on latest OVS. Any feedback would be greatly appreciated. It is for >> sure not production ready but should work on Ubuntu. >> >> >> > 2) Is there any plan to move away from racoon to something else? The >> > patch >> > proposed a while ago didn't seem to have any comments or debate against >> > it >> > that I could find. >> >> Racoon development seems to have stalled. Also there were few features >> lacking in racoon that were available in strongSwan. >> >> > 3) Can someone explain a bit of the mark behavior? First as it exists >> > in >> > 2.5? Apparently part of the problem with it was the fact that OVS >> > expects >> > to use the LSB of the mark for it's own purposes and it will conflict >> > with >> > other uses of the mark. I think this is problem I have in places where >> > the >> > mark is used for other behaviors. Is it that after the ESP packet >> > arrives >> > and is transformed back into the GRE packet, the mark is set to 0x1 by >> > the >> > xfrm policy and then the OVS layer is expecting that when decap'ing the >> > GRE >> > packet? And it is in this manner that a GRE-IPSec packet (with mark >> > 0x1) is >> > distinguished from a regular GRE packet (with mark 0x0)? But once this >> > tunnel port is found and the GRE packet is decapped - what is the state >> > of >> > the m
Re: [ovs-discuss] Understand GRE IPSEC tunnels in 2.5/2.6/2.7
On Fri, Feb 10, 2017 at 6:41 PM, Keith Holleman via discuss wrote: > > I am trying to understand a few things about the implementation of GRE over > IPSec tunnels across some of the more recent OVS releases. As I understand > it, 2.5 had support for it but in 2.6 the IPSec scripts (and effectively the > support) were pulled out in 2.6 due to concerns around mark usage. I have > also seen patches proposed that migrated from racoon to strongswan but I > don't think they were ever fully accepted. So I have three questions: > > 1) What is the plan for IPSec support in 2.7 and/or future releases? Here is my private ovs IPsec repository - https://github.com/ansisatteka/ovs-ipsec.git that I recently rebased on latest OVS. Any feedback would be greatly appreciated. It is for sure not production ready but should work on Ubuntu. > 2) Is there any plan to move away from racoon to something else? The patch > proposed a while ago didn't seem to have any comments or debate against it > that I could find. Racoon development seems to have stalled. Also there were few features lacking in racoon that were available in strongSwan. > 3) Can someone explain a bit of the mark behavior? First as it exists in > 2.5? Apparently part of the problem with it was the fact that OVS expects > to use the LSB of the mark for it's own purposes and it will conflict with > other uses of the mark. I think this is problem I have in places where the > mark is used for other behaviors. Is it that after the ESP packet arrives > and is transformed back into the GRE packet, the mark is set to 0x1 by the > xfrm policy and then the OVS layer is expecting that when decap'ing the GRE > packet? And it is in this manner that a GRE-IPSec packet (with mark 0x1) is > distinguished from a regular GRE packet (with mark 0x0)? But once this > tunnel port is found and the GRE packet is decapped - what is the state of > the mark, connmark/connstate? I may have some of it wrong but tried to > learn and trace it out through iptables using the "TRACE" action, by messing > with the mark, and by looking at OVS logs. > > And is there an agreed upon way to to fix this mark usage when returning the > IPSec support to OVS? If so what does that behavior look like. You can check the proposed skb mark usage in my private repository. Let me know what you think. The problem with previous IPsec implementation was that it was impossible to use skb mark with overlays and hence it was removed. > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] systemd issue
On Tue, Jan 24, 2017 at 8:06 PM, Muminul Islam Russell wrote: > Thanks a lot for detailed information. > > > [root@localhost ~]# ps -AZf | egrep ovs > system_u:system_r:openvswitch_t:s0 root 3079 1 0 10:07 ? > 00:00:00 ovsdb-server: monitoring pid 3080 (healthy) > system_u:system_r:openvswitch_t:s0 root 3080 3079 0 10:07 ? > 00:00:00 ovsdb-server /etc/openvswitch/conf.db -vconsole:emer > -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock > --private-key=db:Open_vSwitch,SSL,private_key > --certificate=db:Open_vSwitch,SSL,certificate > --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --no-chdir > --log-file=/var/log/openvswitch/ovsdb-server.log > --pidfile=/var/run/openvswitch/ovsdb-server.pid --detach --monitor > system_u:system_r:openvswitch_t:s0 root 3089 1 0 10:07 ? > 00:00:00 ovs-vswitchd: monitoring pid 3090 (healthy) > system_u:system_r:openvswitch_t:s0 root 3090 3089 0 10:07 ? > 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer > -vsyslog:err -vfile:info --mlockall --no-chdir > --log-file=/var/log/openvswitch/ovs-vswitchd.log > --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor As can be seen from the above `ps -Z` output both ovs-vswitchd and ovsbd-server processes are running under the expected openvswitch_t type; also it is apparent that there are no extra ovs-* process running under unconfined type preventing these ones to start properly. So it the pid file with wrong SElinux label must be a remnant from one of the previous times you started openvswitch. Do you have a reason to suspect that in the past there was at least single time where you started openvswitch in any other way than via systemctl? If yes, than that is probably the problem and just change SElinux context for /var/run/openvswitch via restorecon command to fix the issue. > unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root 3104 2492 > 0 10:08 pts/0 00:00:00 grep -E --color=auto ovs > > --- > Muminul > > On Tue, Jan 24, 2017 at 7:06 AM, Ansis Atteka wrote: >> On Mon, Jan 23, 2017 at 9:22 PM, Muminul Islam Russell >> wrote: >>> Hi Ansis, >>> >>> Thanks. I am newbie to this technology. Could you please tell me how >>> can I use wrong unconfined type while creating the directory >>> manually. >> >> I would recommend you to think about Mandatory Access Control >> (SElinux) in analogical way as you already think about Discretionary >> Access Control (ie directory and file ownership by Linux Users) - same >> caveats apply to both of them. >> >> My guess would be that you got into this non working state by starting >> ovs-* processes directly from command line (e.g. something like >> ./ovs-vswitchd ...). This caused ovs-* processes to start under >> unconfined type and hence all the unix domain sockets and files >> created by them were also created under unconfined type. And now, >> later on, you are attempting to start ovs-vswitchd correctly via >> systemd where this time these processes bootstrap under the SELinux >> openvswitch type and hence they can't anymore clean up remnants >> created by previous ovs_ process instances that were running under >> unconfined type. To confirm this theory can you copy paste output of >> "ps -AZf | egrep ovs" command? >> >> To get out of this situation you need to relabel these files back to >> openvswitch_* type by running restorecon command. >> >> >> >>> >>> Here is the output that you requested. >>> [root@localhost ~]# ls -Z /var/run/openvswitch/ >>> srwx--. root root unconfined_u:object_r:var_run_t:s0 br0.mgmt >>> srwx--. root root unconfined_u:object_r:var_run_t:s0 br0.snoop >>> srwx--. root root unconfined_u:object_r:var_run_t:s0 db.sock >>> srwx--. root root unconfined_u:object_r:var_run_t:s0 >>> ovsdb-server.2593.ctl >>> -rw-r--r--. root root unconfined_u:object_r:var_run_t:s0 ovsdb-server.pid >>> srwx--. root root unconfined_u:object_r:var_run_t:s0 >>> ovs-vswitchd.2605.ctl >>> -rw-r--r--. root root unconfined_u:object_r:var_run_t:s0 ovs-vswitchd.pid >>> [root@localhost ~]# >>> >>> Thanks, >>> Muminul >>> >>> On Mon, Jan 23, 2017 at 10:46 AM, Ansis Atteka wrote: >>>> On Fri, Jan 20, 2017 at 3:48 PM, Muminul Islam Russell >>>> wrote: >>>>> Thanks for the clarification. >>>>> >>>>> When I change selinux mode to permissive it goes through. I am >>>>> wondering if there is a way >>>>> to resolve this issue while selinux in enforcing
Re: [ovs-discuss] systemd issue
On Mon, Jan 23, 2017 at 9:22 PM, Muminul Islam Russell wrote: > Hi Ansis, > > Thanks. I am newbie to this technology. Could you please tell me how > can I use wrong unconfined type while creating the directory > manually. I would recommend you to think about Mandatory Access Control (SElinux) in analogical way as you already think about Discretionary Access Control (ie directory and file ownership by Linux Users) - same caveats apply to both of them. My guess would be that you got into this non working state by starting ovs-* processes directly from command line (e.g. something like ./ovs-vswitchd ...). This caused ovs-* processes to start under unconfined type and hence all the unix domain sockets and files created by them were also created under unconfined type. And now, later on, you are attempting to start ovs-vswitchd correctly via systemd where this time these processes bootstrap under the SELinux openvswitch type and hence they can't anymore clean up remnants created by previous ovs_ process instances that were running under unconfined type. To confirm this theory can you copy paste output of "ps -AZf | egrep ovs" command? To get out of this situation you need to relabel these files back to openvswitch_* type by running restorecon command. > > Here is the output that you requested. > [root@localhost ~]# ls -Z /var/run/openvswitch/ > srwx--. root root unconfined_u:object_r:var_run_t:s0 br0.mgmt > srwx--. root root unconfined_u:object_r:var_run_t:s0 br0.snoop > srwx--. root root unconfined_u:object_r:var_run_t:s0 db.sock > srwx--. root root unconfined_u:object_r:var_run_t:s0 ovsdb-server.2593.ctl > -rw-r--r--. root root unconfined_u:object_r:var_run_t:s0 ovsdb-server.pid > srwx--. root root unconfined_u:object_r:var_run_t:s0 ovs-vswitchd.2605.ctl > -rw-r--r--. root root unconfined_u:object_r:var_run_t:s0 ovs-vswitchd.pid > [root@localhost ~]# > > Thanks, > Muminul > > On Mon, Jan 23, 2017 at 10:46 AM, Ansis Atteka wrote: >> On Fri, Jan 20, 2017 at 3:48 PM, Muminul Islam Russell >> wrote: >>> Thanks for the clarification. >>> >>> When I change selinux mode to permissive it goes through. I am >>> wondering if there is a way >>> to resolve this issue while selinux in enforcing mode. >> >> This could be something as trivial as: >> 1. deleting /var/run/openvswitch directory and/or all its contents >> that were properly taggerd with one of openvswitch type >> 2. manually recreating this directory under wrong unconfined type. >> >> >> Can you post output of `ls -Z` command for /var/run/openvswitch >> directory and also all its contents to provide or disprove the theory >> I have above? >> >>> >>> Thanks, >>> Muminul >>> >>> On Fri, Jan 20, 2017 at 3:35 PM, Ben Pfaff wrote: >>>> On Fri, Jan 20, 2017 at 03:08:39PM -0800, Muminul Islam Russell wrote: >>>>> Hi, >>>>> >>>>> I am using 2.3.1 version and having issue with starting openvswitch >>>>> service with systemd. >>>>> >>>>> [root@localhost ~]# systemctl status openvswitch >>>>> >>>>> Jan 20 15:00:54 localhost systemd[1]: Starting LSB: Open vSwitch switch... >>>>> Jan 20 15:00:54 localhost openvswitch[3196]: Starting ovsdb-server >>>>> ovsdb-server: /var/run/openvswitch/ovsdb-server.pid: pidfile check >>>>> failed (Permission denied), aborting >>>> >>>> ovsdb-server tried to check whether it was already running, by reading >>>> its own pidfile, but it couldn't read it due to a "permission denied" >>>> error. >>> ___ >>> discuss mailing list >>> disc...@openvswitch.org >>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] systemd issue
On Fri, Jan 20, 2017 at 3:48 PM, Muminul Islam Russell wrote: > Thanks for the clarification. > > When I change selinux mode to permissive it goes through. I am > wondering if there is a way > to resolve this issue while selinux in enforcing mode. This could be something as trivial as: 1. deleting /var/run/openvswitch directory and/or all its contents that were properly taggerd with one of openvswitch type 2. manually recreating this directory under wrong unconfined type. Can you post output of `ls -Z` command for /var/run/openvswitch directory and also all its contents to provide or disprove the theory I have above? > > Thanks, > Muminul > > On Fri, Jan 20, 2017 at 3:35 PM, Ben Pfaff wrote: >> On Fri, Jan 20, 2017 at 03:08:39PM -0800, Muminul Islam Russell wrote: >>> Hi, >>> >>> I am using 2.3.1 version and having issue with starting openvswitch >>> service with systemd. >>> >>> [root@localhost ~]# systemctl status openvswitch >>> >>> Jan 20 15:00:54 localhost systemd[1]: Starting LSB: Open vSwitch switch... >>> Jan 20 15:00:54 localhost openvswitch[3196]: Starting ovsdb-server >>> ovsdb-server: /var/run/openvswitch/ovsdb-server.pid: pidfile check >>> failed (Permission denied), aborting >> >> ovsdb-server tried to check whether it was already running, by reading >> its own pidfile, but it couldn't read it due to a "permission denied" >> error. > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Replacing IPsec-GRE tunnel ports
On Wed, Nov 23, 2016 at 12:29 AM, Bolesław Tokarski wrote: > Hi, > > I would love this to be a problem of the testing tool. It does not seem to > be the case, though. I ran iperf3 in the default mode, which is TCP, and it > was the same command tested on OVS with VLAN tagging and without it - it > achieved 870Mbps without VLANs. > > Ansis, your wireshark suggestion was in the right direction - although I did > test outgoing packets with tcpdump already, but these were ESP, so not > saying a lot. Now I made a test of the traffic flowing through the internal > interface, and indeed there's some weird going on. 1. For the bad case did you see ESP packets getting fragmented? The PCAP file you attached only has iperf packets so I can't tell that. 2. Also, you did not explicitly mention if packet capture was gathered on sender (10.100.0.3) or receiver (10.100.0.4). However, I would be inclined to guess that you ran tcpdump on receiver (10.100.0.4), because of latency pattern in TCP three-way handshake. First of all before troubleshooting any iperf TCP performance issues I would recommend you to do several iperf UDP tests with -b flag, because TCP flow control introduces a lot of variables that I have to speculate about. Run this UDP test couple times and try to guess "optimal" target bandwidth when drops are still close to 0% and also keep attention to packet reordering. Now getting back to the TCP packet capture that you sent over to me... what I see in Wirehark's "TCP stream graph analysis tool" is: 1. that TCP data segments are received in bursts that are consistently separated by ~0.25s dormant intervals. Since the packet capture was gathered on receiver and not on the sender it could mean two things: 1.1. Either the TCP ACK from receiver to sender was delayed for one reason or another. Hence, TCP flow control kicked in and slowed down the data send rate on sender; OR 1.2. Either TCP data segments in 0.25s burst-rate fashion were delayed from sender to receiver. Since receiver did not receive any data it could not acknowledge it and tell sender to send packets at higher rate. This is more likely scenario (see point #2). 2. There is almost always one TCP segment from the next burst of TCP data segments that appears prematurely in previous burst. This makes me think that sender actually did send out more data except it was queued somewhere (see point #1.2). 3. There are bunch of out-of-order TCP segments within the "burst" as well. I would be interested to find out if UDP test would confirm the same packet reordering. 4. can you monitor "ovs-dpctl show" stats in tight loop and see if upcalls to ovs-vswitchd increase in 0.25 second pattern as well? This would prove or disprove if OVS is queuing packets and introduces this 0.25s delay. > > The untagged traffic was a smooth flow of TCP sends and ACKs, the tagged > traffic is more interesting. I get a significant number of losses, > retransmissions, TCP out-of-order notes, there's even an RST near the end. > Packets are marked as 'don't fragment'. MTU on the interface is 1394, > raising it to 1420 makes the traffic flatten out to 0, lowering it does not > seem to make a difference. > > I am attaching the packet dump from the capped communication, the non-VLAN > comms produced 200MB packet capture during the same 2s, so not that sexy to > transfer over email. > > I am trying to bring up two Ubuntu 15.04 VMs, this version has OVS 2.3.2, > but an older kernel, 3.19, I'll try to see on which other environment I can > or can not reproduce the problem. I'm afraid I'm not capable enough to > dismantle the kernel and see which code path does one traffic go through and > the other does not. > > Best regards, > Bolesław Tokarski > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Replacing IPsec-GRE tunnel ports
On Tue, Nov 22, 2016 at 10:18 AM, Bolesław Tokarski wrote: > Hi, Ansis, > > Huge thanks for looking into this. I did some investigations to try to > replicate what the script is doing - I managed to set an ipsec transport > connection between hosts up (suddenly it was much more obvious with > strongswan compared to racoon), and configured OVS to use gre tunnel between > the hosts. I only failed with packet marking, removed it for the time being > and I got communication running this way. > > Anyway, the 1Mbps problem re-appeared for VLANs, so indeed it's not the > script's fault. The remaining parts are in the kernel, so I started to look > into this topic but... well, that's a deep thing. > > To my amazement, I just tested the same setup with Racoon and VLANs, and > bandwidth is also capped at 1Mbps. I started to wonder what changed, > reverted to previous kernel (there was an upgrade in the time between), > reverted to the very same package and config used at that point, and I still > see I'm capped at 1Mbps. My best guess is that I never tested the bandwidth > of the previous setup with VLANs, either. That, something more subtle, or > I'm crazy. I would recommend to get packet capture for the good case (no VLANs) and bad case (VLANs) and draw more conclusions: 1) if there is something else gong on like IP fragmentation; AND 2) see if packet was dropped on the sender or receiver. In the wireshark you can use "compare two packet captures" for this or you could also look in counters of `ip -s xfrm state` or `netstat -s` if they correlate with drops. Also, by an chance is this iperf UDP test with 64k byte datagrams? That would be the usual suspect for 1000x performance regression, if I had to guess. > > So, I need to back off from my statement that it used to work on Racoon. > > Anyway. OpenSUSE 42.1, kernel 4.1.34, OVS 2.3.3, strognswan 5.2.2. The > /etc/ipsec.conf looks like this: > > config setup > uniqueids=no > > conn %default > keyingtries=%forever > type=transport > keyexchange=ikev2 > auto=add > ike=aes128gcm12-aesxcbc-modp1024 > esp=aes128gcm12-modp1024 > mobike=no > > conn gre4-1 > left=3.3.3.3 > right=4.4.4.4 > rightcert=ovs-4.4.4.4.pem > leftcert=/etc/openvswitch/cert.pem > > the other party has a symmetrical config. > > # ip xfrm policy > src 4.4.4.4/32 dst 3.3.3.3/32 > dir in priority 2819 ptype main > tmpl src 0.0.0.0 dst 0.0.0.0 > proto esp reqid 1 mode transport > src 3.3.3.3/32 dst 4.4.4.4/32 > dir out priority 2819 ptype main > tmpl src 0.0.0.0 dst 0.0.0.0 > proto esp reqid 1 mode transport > src 0.0.0.0/0 dst 0.0.0.0/0 > socket in priority 0 ptype main > src 0.0.0.0/0 dst 0.0.0.0/0 > socket out priority 0 ptype main > src 0.0.0.0/0 dst 0.0.0.0/0 > socket in priority 0 ptype main > src 0.0.0.0/0 dst 0.0.0.0/0 > socket out priority 0 ptype main > src ::/0 dst ::/0 > socket in priority 0 ptype main > src ::/0 dst ::/0 > socket out priority 0 ptype main > src ::/0 dst ::/0 > socket in priority 0 ptype main > src ::/0 dst ::/0 > socket out priority 0 ptype main > > # ip xfrm state > src 4.4.4.4 dst 3.3.3.3 > proto esp spi 0xc1436dcb reqid 1 mode transport > replay-window 32 > aead rfc4106(gcm(aes)) 0xsomekey 96 > anti-replay context: seq 0x0, oseq 0xa48d, bitmap 0x > sel src 4.4.4.4/32 dst 3.3.3.3/32 > src 3.3.3.3 dst 4.4.4.4 > proto esp spi 0xca5b77f8 reqid 1 mode transport > replay-window 32 > aead rfc4106(gcm(aes)) 0xsomekey 96 > anti-replay context: seq 0xb6b7, oseq 0x0, bitmap 0x > sel src 3.3.3.3/32 dst 4.4.4.4/32 > > I'll try to replicate that problem on a Ubuntu VM. > > Best regards, > Bolesław Tokarski > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Replacing IPsec-GRE tunnel ports
On Sat, Nov 19, 2016 at 1:31 PM, Bolesław Tokarski wrote: > Hi, Ansis, > > I've got a performance problem with my setup - I've got 1Mbps speed reported > by iperf3, while the physical link is 1Gbps. As you seem to be the only one > using StrongSwan with OVS around, you are my only hope :] > > So - two physical hosts, running OpenSUSE 42.1, with kernel 4.1.34, > StrongSwan 5.2.2, OVS 2.3.3 (with your strongswan ovs patch). Tunnel > established with ipsec_gre port on ipv4, connection established with > StrongSwan, MTU on interfaces set to 1400. iperf3 between host interfaces. > > I managed to narrow down the scope of the 1Mbps speed issue to StrongSwan > IPsec+GRE+OVS+VLAN tagging. Without VLAN tags I achieve 885Mbps. With > Racoon+GRE+OVS+VLAN tagging I also get ~870-880 Mbps. Can you provide output of "ip xfrm state" and "ip xfrm policy" for: 1. the bad performance case: StrongSwan IPsec+GRE+OVS+VLAN tagging 2. the good performance case: Racoon+GRE+OVS+VLAN tagging As I said it is the XFRM module doing the actual packet forwarding. It seems as if racoon and strongSwan configures something differently there. > > Did you happen to test StrongSwan with OVS and VLAN tagging with regards to > performance? I am quite sure I *DID NOT* test VLANs and that there could be a bug. > > I can provide you with more details if you'd like to replicate the issue, > but perhaps that's something you have seen already? > > Best regards, > Bolesław Tokarski > > > 2016-11-19 13:55 GMT+01:00 Bolesław Tokarski : >> >> Hi, Ansis, >> >> I tried your openvswitch-ipsec patch for strongswan on my current OVS 2.3 >> installation. Although I found it was written somewhere between 2.3 and 2.4, >> it was relatively easy to adapt it to run on 2.3.3. As I needed only >> gre_ipsec, I did not need to care this is only implemented from 2.4+. >> >> I tested it on OpenSUSE 42.1, strongswan 5.2.2, ovs 2.3.3 (with my patches >> and the strongswan patch). >> >> There's a minor bug in the patch, it wraps >> 'charon.plugins.kernel-netlink.xfrm_ack_expires = 10' in some garbage in >> front. The script generated /etc/ipsec.d/certs/ovs-$portname.pem, while I >> think strongswan expected that to be an IP address (unless I stumbled upon a >> cert issue I fixed later, details below). Also, I found it was required to >> specify 'local_ip' in ovs-vsctl, as strongswan fails to find the tunnel >> policy otherwise. >> >> I found StrongSwan to be very picky regarding certificates. I needed to >> specify an IP address in subjAltNames in the certificate. >> >> Beside that - the support you wrote looks like it works decently. I had >> some issues with racoon, with it not catching a certificate change, and >> failing desperately on one connection. Since it seems Strongswan is >> maintained better, it seems to be a good alternative. >> >> Thank you for this nice piece of code :) >> >> Regards, >> Bolesław Tokarski >> > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Replacing IPsec-GRE tunnel ports
On Thu, Nov 17, 2016 at 4:04 AM, Bolesław Tokarski wrote: > Hi, > > Thank you very much for the answer. > >> > >> > I am yet to come across a good guide on how to set up an OVS IPsec-GRE >> > tunnel port alternative. Most guides are either for site-to-site IPsec >> > tunnels, or for OVS GRE tunnels. >> >> Such guides in details wold be on strongSwan, racoon, OpenSwan or >> libreswan project sites. > > > Well, I did see a number of guides on setting up tunnels, not so much on > putting the traffic forward to an OVS port. I saw what ends up in > ipsec.conf, but I believe the traffic going the the ipsec tunnel ends up on > a Unix socket and gets directed to ovs-monitor-ipsec or so... I might fully > get the image, though. No, the ovs-monitor-ipsec daemon is not doing the actual IPsec forwarding, ovs-monitor-ipsec just configures strongSwan. Also, strongSwan is not doing the actual IPsec forwarding - strongSwan just configures XFRM module in Linux kernel. It is XFRM module in Linux Kernel that does the actual IPsec forwarding. > >> >> However, if you are interested you can take a >> peek at this link - >> https://www.mail-archive.com/dev@openvswitch.org/msg46915.html - and >> extract what the ovs-monitor-ipsec daemon would set in ipsec.conf and >> ipsec.secrets file. > > > I saw the patch on the mailing list before. I am experiencing some issues > with racoon, it does not seem to handle SA expiry too well. I had a number > of situations where I needed to recreate the OVS ports for it to catch up. > How's StrongSwan doing? I guess you're using it in production? I haven't followed racoon lately. Though strongSwan also has its own bugs. However, most of those bugs that I have had encountered in strongSwan were either already fixed in the latest strongSwan release and it took some time for me to root cause them; OR strongSwan maintainers fixed those bugs for me after I reported them on their bug tracker. Also, it is a pity that for I still have to edit ipsec.conf file instead of being able to use Python Vici API that they provide. At least this is still the case for Ubuntu 16.10. And, no, I would not dare to say that I am using strongSwan in production, because the patch I pointed you to *is not* even up-streamed. > >> >> If you are ok to skip this particular OVS 2.7 version, then I plan to >> reintroduce ovs-monitor-ipsec daemon in the next one. It was abruptly >> removed because it was decided that ovs-monitor-ipsec can't have a >> hard coded bit of skb_mark because it interferes with OpenFlow >> skb_mark match. > > > Good to hear that. The ovs-monitor-ipsec daemon was quite easy to use and I > even preferred to add OpenSUSE support to it than to set the tunnels up > manually, which sounds bizarre, but hey - it worked. > > Best regards, > Bolesław Tokarski ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Replacing IPsec-GRE tunnel ports
On Tue, Nov 15, 2016 at 12:46 AM, Bolesław Tokarski wrote: > Hi, > > I found that IPsec - GRE tunnel-ports support was deprecated in OVS 2.6 and > removed from 2.7/master recently > (https://mail.openvswitch.org/pipermail/ovs-git/2016-September/018774.html) I think that the title of that patch is slightly misleading. It should have been simply "Remove ovs-monitor-ipsec from OVS" and not "Allow external IPsec tunnel management", because external IPsec tunnel management was already possible before that patch by simply not installing openvswitch-ipsec package and letting administrator to populate IPsec configuration files manually. > > I am yet to come across a good guide on how to set up an OVS IPsec-GRE > tunnel port alternative. Most guides are either for site-to-site IPsec > tunnels, or for OVS GRE tunnels. Such guides in details wold be on strongSwan, racoon, OpenSwan or libreswan project sites. However, if you are interested you can take a peek at this link - https://www.mail-archive.com/dev@openvswitch.org/msg46915.html - and extract what the ovs-monitor-ipsec daemon would set in ipsec.conf and ipsec.secrets file. If you are ok to skip this particular OVS 2.7 version, then I plan to reintroduce ovs-monitor-ipsec daemon in the next one. It was abruptly removed because it was decided that ovs-monitor-ipsec can't have a hard coded bit of skb_mark because it interferes with OpenFlow skb_mark match. ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
