For my test, I was using the ovs-sandbox. If you look in
tutorial/ovs-sandbox, it creates the certificates as follows:
if [ "$HAVE_OPENSSL" = yes ]; then
OVS_PKI="run ovs-pki --dir=$sandbox/pki --log=$sandbox/ovs-pki.log"
$OVS_PKI init
$OVS_PKI req+sign ovnsb switch
$OVS_PKI req+sign ovnnb switch
for i in $(seq $n_controllers); do
$OVS_PKI -u req+sign chassis-$i switch
done
fi
In my case, I have one chassis, so it creates a chassis-1-cert.pem and
chassis-1-privkey.pem file. The CN for the chassis-1-cert.pem file is
"chassis-1".
Then later, when starting ovn-controller, it passes the certificate
information as command line arguments. Specifically it passes "-p
$sandbox/chassis-1-privkey.pem -c $sandbox/chassis-1-cert.pem -C
$sandbox/pki/switchca/cacert.pem". I'm not actually sure if this is
relevant when performing the RBAC tests, but I figured I'd share it anyway.
When I attempt to use the chassis-1-cert to add an Encap with
chassis-name "1234" I get the failure that I pasted earlier. Just to be
on the safe side, I did a completely clean clone of OVS just to be sure
there wasn't something fishy going on, and it still worked for me.
Just in case there was some sort of mixup in the code between the RBAC
role name and client ID of the certificate, in the sandbox I did the
following:
$ ovs-pki --dir=sandbox/pki --log=sandbox/ovs-pki.log req+sign
ovn-controller switch
ovn-controller-req.pem Fri Sep 14 08:24:23 EDT 2018
fingerprint 8f14e9a57a8162b53cb11adf473466587280cad5
$ ovn-sbctl --id=@test97c983e4 --db=ssl:127.0.0.1:6642
--private-key=ovn-controller-privkey.pem
--certificate=ovn-controller-cert.pem
--ca-cert=sandbox/pki/switchca/cacert.pem create encap
chassis_name='1234' ip='1234' options:csum="true" type="geneve"
2018-09-14T12:25:21Z|00002|sbctl|WARN|row id "@test97c983e4" was created
but no reference to it was inserted, so it will not actually appear in
the database
2018-09-14T12:25:21Z|00003|ovsdb_idl|WARN|transaction error:
{"details":"RBAC rules for client \"ovn-controller
id:8403c1a3-2320-4e9f-8889-963cbde0f71c\" role \"ovn-controller\"
prohibit row insertion into table \"Encap\".","error":"permission error"}
I still get the same error even when I create a certificate with CN
"ovn-controller".
One possibility we haven't considered is that the warning about the row
being created with no reference prints *before* the transaction error.
It is possible that something in your scenario is causing the command to
fail before it even reaches the RBAC insertion trigger. I'm not sure
what it would be, but it's something to consider.
It may be worth running under gdb and putting breakpoints in
ovsdb/rbac.c in the ovsdb_rbac_insert() function. This way you can see
if this function is even being run in the first place. If it is running,
you can see if it is allowing the transaction and why. Or you could use
good old printf-style debugging if you're more comfortable with that :)
On 09/13/2018 07:10 PM, Feng, Zhenhua wrote:
Hi Mark,
I try to set
ovn-sbctl set-connection role=ovn-controller pssl:6642
But it did not change the results. Also for your second and third points, see inline comments below
“””Aside from that, double-check the output of `ovn-sbctl list connection`
and ensure that the connection has what you expect. Ensure that the
"role" column is actually set to "ovn-controller".”””
Yes my role is set to von-controller (see the below results)
# ovn-sbctl list connection
_uuid : c7689162-62d6-4d54-85cb-61748f70e101
external_ids : {}
inactivity_probe : []
is_connected : false
max_backoff : []
other_config : {}
read_only : false
role : ovn-controller
status : {sec_since_connect="0", sec_since_disconnect="0"}
target : "pssl:6642:[central_node_ip]"
My certificate subject line is like this
Subject: C=US, ST=CA, O=Open vSwitch, OU=Open vSwitch certifier,
CN=ovn-controller id:21c0ee80-2f67-4dd1-b734-165ba6b15461
So the CN in the certificate (ovn-controller) will definitely not match the
chassis_name (“1234”) field used in the ovn-sbctl create command.
By further debugging, I’m suspecting that how I generate and distribute the
certificate / keys may be worth some attention here:
I’m using the built-in ovn PKI, I did
1) Generate the main controllerca/ and switchca/ PKI
ovs-pki -B 1024 init
2) use the switch PKI to generate the private key, certificate for ovn
controllers
ovs-pki -B 1024 req+sign ovn-controller switch
3) copy the private key and certificate generated in step 2 into the testing
hypervisor
Here , how does ovs-pki knows what CN to use in the generated certificate? In
my case it is “ovn-controller” as provided in the command. But should it be set
to a real switch/chassis_name on the target hypervisor? The generic name
ovn-controller does not make much sense to me.
“ Subject: C=US, ST=CA, O=Open vSwitch, OU=Open vSwitch certifier,
CN=ovn-controller id:21c0ee80-2f67-4dd1-b734-165ba6b15461”
Did you see any issue here? It would also be very helpful if you can share how
you setup your PKI for your ssl and rbac environment for the comparison .
Thanks a lot !
Zhenhua
On 9/13/18, 12:40 PM, "Mark Michelson" <mmich...@redhat.com> wrote:
Hello,
I set up a similar scenario using the OVS sandbox. I started the OVN
sandbox and then issued the command:
ovn-sbctl --id=@test97c983e4 --db=ssl:127.0.0.1:6642
--private-key=sandbox/chassis-1-privkey.pem
--certificate=sandbox/chassis-1-cert.pem
--ca-cert=sandbox/pki/switchca/cacert.pem create encap
chassis_name='1234' ip='1234' options:csum="true" type="geneve"
It's the same command you used but with the db IP address filled in and
the certificate files changed to the ones installed by the sandbox. This
is what I see:
2018-09-13T19:05:50Z|00002|sbctl|WARN|row id "@test97c983e4" was created
but no reference to it was inserted, so it will not actually appear in
the database
2018-09-13T19:05:50Z|00003|ovsdb_idl|WARN|transaction error:
{"details":"RBAC rules for client \"chassis-1\" role \"ovn-controller\"
prohibit row insertion into table \"Encap\".","error":"permission error"}
It starts the same as what you see but then also has a permission error
on the Encap table.
I think your issue is that you used "ssl" in your `ovn-sbctl
set-connection` command instead of "pssl". Try using
ovn-sbctl set-connection role=ovn-controller pssl:6642
or
ovn-sbctl set-connection role=ovn-controller pssl:6642:[central_node_ip]
Note that with pssl, you put the port before the IP address. If you
leave out the IP address, then OVN binds to the wildcard address.
Aside from that, double-check the output of `ovn-sbctl list connection`
and ensure that the connection has what you expect. Ensure that the
"role" column is actually set to "ovn-controller".
Also, double-check the client certificate you are using. The way RBAC
works is that it looks at the CN field on the client certificate and
sets that as the ID of the stream. The RBAC-permission Encap table says
to make sure that the chassis-name in the Encap record matches the ID of
the stream. So if you are creating the Encap using a client certificate
with CN set to "1234" then it will succeed.
Hopefully one of the above recommendations will fix the problem.
Mark!
On 09/11/2018 09:52 PM, Feng, Zhenhua wrote:
> Hi
>
> This is my first question here so please kindly let me know if I need
> to add any information to make the question actually answerable. : )
>
> Basically, I have a working OVN test environment with one central node
> (where nb and sb db runs) and 2 hypervisor nodes (ovn-controllers). SSL
> has already been enabled for connection to south bound db. Hypervisors
> connect to sb db via SSL.
>
> To enable rbac on south bound connection I’ve issue the following
> command on central node
>
> /ovn-sbctl set-connection role=ovn-controller ssl:[central_node_ip]:6642/
>
> The role, permission and connection table looks as follow:
>
> /#ovn-sbctl list rbac-role/
>
> /_uuid : cfba4fb4-2073-4639-8da4-f43e3336583d/
>
> /name : ovn-controller/
>
> /permissions : {Chassis=6b444b75-a3dd-4482-8399-91db41eacbe8,
> Encap=4e57bd31-f326-4c52-9814-963e92cd39cd,
> MAC_Binding=e8c3f556-2f98-4b15-b40a-5a444903902a,
> Port_Binding=9adcc428-8910-4e49-bff5-a82ec1dad586} /
>
> //
>
> /# ovn-sbctl list rbac-permission/
>
> /_uuid : 9adcc428-8910-4e49-bff5-a82ec1dad586/
>
> /authorization : [""]/
>
> /insert_delete : false/
>
> /table : Port_Binding/
>
> /update : [chassis]/
>
> //
>
> /_uuid : 4e57bd31-f326-4c52-9814-963e92cd39cd/
>
> /authorization : [chassis_name]/
>
> /insert_delete : true/
>
> /table : Encap/
>
> /update : [ip, options, type]/
>
> //
>
> /_uuid : 6b444b75-a3dd-4482-8399-91db41eacbe8/
>
> /authorization : [name]/
>
> /insert_delete : true/
>
> /table : Chassis/
>
> /update : [encaps, external_ids, nb_cfg,
> vtep_logical_switches]/
>
> //
>
> /_uuid : e8c3f556-2f98-4b15-b40a-5a444903902a/
>
> /authorization : [""]/
>
> /insert_delete : true/
>
> /table : MAC_Binding/
>
> /update : [datapath, ip, logical_port, mac]/
>
> Both the role and permission table come with OVN by default.
>
> I tested the following scenario for rbac:
>
> Add one more row in the encap table with a random chassis_name “1234”
> from one of the hypervisors. (The command issued and results are as
> below). I’m expecting the command to fail because of rbac permission on
> encap table which authorize based on chassis_name. But rom the output,
> it appears to have succeeded.
>
> # ovn-sbctl --id=@test97c983e4 --db=ssl:[central_node_ip]:6642
> --private-key=/etc/openvswitch/ovn-controller-privkey.pem
> --certificate=/etc/openvswitch/ovn-controller-cert.pem
> --ca-cert=/etc/openvswitch/cacert.pem create encap chassis_name='1234'
> ip='1234' options:csum="true" type="geneve"
>
> 2018-09-11T21:58:27Z|00002|sbctl|WARN|row id "@test97c983e4" was created
> but no reference to it was inserted, so it will not actually appear in
> the database
>
> I cant find many documents that actually shows how to setup and test
> rbac for ovn other than the official doc
>
>
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.openvswitch.org%2Fen%2Flatest%2Ftopics%2Frole-based-access-control%2F&data=02%7C01%7Czhenhfeng%40ebay.com%7Cf953c098a6244260ad0108d619b0aeaa%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C636724644009586959&sdata=ZCW5J8ZlaCWU%2FhWuVBUOwMAlz1Yf6gid%2BX8bGJNONwc%3D&reserved=0
>
> Can someone help to tell what’s going on here?
>
> Am I configured rbac wrong or use the wrong test parameters?
>
> Thank you very much!
>
> Zhenhua
>
>
>
> _______________________________________________
> discuss mailing list
> disc...@openvswitch.org
>
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-discuss&data=02%7C01%7Czhenhfeng%40ebay.com%7Cf953c098a6244260ad0108d619b0aeaa%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C636724644009586959&sdata=BP%2FEPmOgBve5sxmrH4kfgkWf0qROPCocJr20VwcM0Ck%3D&reserved=0
>
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
